2024-04-17T04:22:21.949 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T04:22:21.949 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000 --name teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi --verbose --description rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} -- /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/orig.config.yaml 2024-04-17T04:22:22.000 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T04:22:22.084 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000 branch: squid description: rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: ceph-qa@ceph.com first_in_suite: false job_id: '7659000' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: debug rgw: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw d3n l1 datacache persistent path: /tmp/rgw_datacache/ rgw d3n l1 datacache size: 10737418240 rgw d3n l1 local datacache enabled: true rgw enable ops log: true rgw max chunk size: 4194304 rgw obj stripe size: 6291456 rgw s3 auth use sts: true rgw sts key: abcdefghijklmnop rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 rgw override bucket index max shards: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: 461a84e6fbbf089011eafd13b98169fb9e9b8cc9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 461a84e6fbbf089011eafd13b98169fb9e9b8cc9 rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 compression type: random datacache: true datacache_path: /tmp/rgw_datacache ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast inline data: false storage classes: LUKEWARM, FROZEN s3tests: accounts: iam alt root: RGW99999999999999999 iam root: RGW88888888888888888 s3 main: RGW11111111111111111 s3 tenant: RGW33333333333333333 force-branch: ceph-squid slow_backend: true storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 3597 sha1: 461a84e6fbbf089011eafd13b98169fb9e9b8cc9 sleep_before_teardown: 0 subset: 0/1 suite: rgw suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 41c500dd60c93c8d83053b72cffc3dd1a952a336 targets: smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC+yz+tDPlkQjjbMRn3xAZhK5N+VOMBKaH64bHOAUTAGGaLPAgA17cnBwc6d1kNP1Ykyi6t9SWDZvp5vwZJxKLc= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLipxCHy1pCsU745cAdHzITMYpbdtcaqlJswFTMKVxe3yg9Jd2rj6c+vI3v5PDaXqLk74gpu1memTOkr/Ex6wfY= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-datacache.sh env: RGW_DATACACHE_PATH: /tmp/rgw_datacache - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - cls/test_cls_user.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-squid rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-squid force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_21:40:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T04:22:22.084 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa; will attempt to use it 2024-04-17T04:22:22.085 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks 2024-04-17T04:22:22.085 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T04:22:22.087 INFO:teuthology.task.internal:Checking packages... 2024-04-17T04:22:22.111 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '461a84e6fbbf089011eafd13b98169fb9e9b8cc9' 2024-04-17T04:22:22.111 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T04:22:22.112 INFO:teuthology.packaging:ref: None 2024-04-17T04:22:22.112 INFO:teuthology.packaging:tag: None 2024-04-17T04:22:22.112 INFO:teuthology.packaging:branch: squid 2024-04-17T04:22:22.112 INFO:teuthology.packaging:sha1: 461a84e6fbbf089011eafd13b98169fb9e9b8cc9 2024-04-17T04:22:22.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-17T04:22:22.420 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2363-g41c500dd-1jammy 2024-04-17T04:22:22.422 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T04:22:22.431 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T04:22:22.432 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T04:22:22.468 INFO:teuthology.task.internal:Saving configuration 2024-04-17T04:22:22.487 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T04:22:22.498 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T04:22:22.527 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 04:16:19.046875', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGi/Fge2Nw8Y6ZEEgIub1VMPsvGgzBbkFE7Vby8/5Wlv'} 2024-04-17T04:22:22.549 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 04:16:19.045559', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOf5KzUXRqvpWdc8xE2R6gsL3KzE3QDp6gFpFvOnDZyQ'} 2024-04-17T04:22:22.549 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T04:22:22.559 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-17T04:22:22.559 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-17T04:22:22.559 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T04:22:22.673 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f18b232f280>, signals=[15]) 2024-04-17T04:22:22.673 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T04:22:22.683 INFO:teuthology.task.internal:Opening connections... 2024-04-17T04:22:22.683 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:22:22.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:22:22.766 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:22:22.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:22:22.842 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T04:22:22.853 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-04-17T04:22:22.858 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-04-17T04:22:22.858 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Ubuntu" 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="22.04" 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_CODENAME=jammy 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:ID=ubuntu 2024-04-17T04:22:22.906 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE=debian 2024-04-17T04:22:22.907 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-17T04:22:22.907 INFO:teuthology.orchestra.run.smithi088.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-17T04:22:22.907 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-17T04:22:22.907 INFO:teuthology.orchestra.run.smithi088.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-17T04:22:22.907 INFO:teuthology.orchestra.run.smithi088.stdout:UBUNTU_CODENAME=jammy 2024-04-17T04:22:22.907 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-04-17T04:22:22.930 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-04-17T04:22:22.936 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-04-17T04:22:22.936 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="22.04" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=jammy 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-17T04:22:22.985 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=jammy 2024-04-17T04:22:22.986 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-04-17T04:22:23.005 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T04:22:23.022 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T04:22:23.045 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T04:22:23.045 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-04-17T04:22:23.048 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-04-17T04:22:23.052 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T04:22:23.080 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T04:22:23.081 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-04-17T04:22:23.096 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-04-17T04:22:23.115 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T04:22:23.172 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T04:22:23.194 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-17T04:22:23.194 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-17T04:22:23.194 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T04:22:23.195 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T04:22:23.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:22:23.199 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-04-17T04:22:23.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:22:23.200 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-04-17T04:22:23.247 INFO:teuthology.orchestra.run.smithi088.stdout:5.15.0-56-generic 2024-04-17T04:22:23.247 INFO:teuthology.task.kernel:Running kernel on smithi088: 5.15.0-56-generic 2024-04-17T04:22:23.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get clean 2024-04-17T04:22:23.250 INFO:teuthology.orchestra.run.smithi154.stdout:5.15.0-56-generic 2024-04-17T04:22:23.250 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.15.0-56-generic 2024-04-17T04:22:23.251 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2024-04-17T04:22:23.372 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get update 2024-04-17T04:22:23.398 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-04-17T04:22:23.517 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T04:22:23.552 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-17T04:22:23.589 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-17T04:22:23.622 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-17T04:22:23.625 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-17T04:22:23.663 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T04:22:23.745 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-17T04:22:23.748 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-17T04:22:23.803 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-17T04:22:23.857 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-17T04:22:23.864 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-17T04:22:23.867 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-17T04:22:23.867 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-17T04:22:23.892 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-17T04:22:23.893 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-17T04:22:23.897 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T04:22:23.897 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-17T04:22:23.908 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-17T04:22:23.915 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-17T04:22:23.917 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-17T04:22:23.917 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-17T04:22:23.922 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-17T04:22:23.928 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-17T04:22:23.930 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-17T04:22:23.931 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-17T04:22:23.935 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-17T04:22:23.936 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-17T04:22:23.936 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-17T04:22:23.936 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-17T04:22:23.937 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-17T04:22:23.937 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-17T04:22:23.951 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-17T04:22:23.951 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-17T04:22:23.955 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-17T04:22:23.957 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-17T04:22:23.961 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T04:22:23.961 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-17T04:22:23.971 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-17T04:22:23.972 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-17T04:22:23.972 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-17T04:22:23.972 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-17T04:22:23.978 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-17T04:22:23.979 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-17T04:22:23.980 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-17T04:22:23.980 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-17T04:22:23.991 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-17T04:22:24.015 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-17T04:22:24.042 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-17T04:22:24.098 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-17T04:22:24.238 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,562 kB] 2024-04-17T04:22:24.324 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-17T04:22:24.491 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-17T04:22:24.503 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-17T04:22:24.504 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-17T04:22:24.547 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [611 kB] 2024-04-17T04:22:24.585 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-17T04:22:24.586 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-17T04:22:24.594 INFO:teuthology.orchestra.run.smithi088.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T04:22:24.595 INFO:teuthology.orchestra.run.smithi088.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-17T04:22:24.620 INFO:teuthology.orchestra.run.smithi088.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-17T04:22:24.683 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [298 kB] 2024-04-17T04:22:24.702 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-17T04:22:24.702 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,702 kB] 2024-04-17T04:22:24.735 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.7 kB] 2024-04-17T04:22:24.736 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [286 kB] 2024-04-17T04:22:24.741 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-17T04:22:24.796 INFO:teuthology.orchestra.run.smithi088.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-17T04:22:24.907 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,075 kB] 2024-04-17T04:22:25.193 INFO:teuthology.orchestra.run.smithi088.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-17T04:22:25.199 INFO:teuthology.orchestra.run.smithi088.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-17T04:22:25.200 INFO:teuthology.orchestra.run.smithi088.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-17T04:22:25.214 INFO:teuthology.orchestra.run.smithi088.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-17T04:22:25.216 INFO:teuthology.orchestra.run.smithi088.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-17T04:22:25.542 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-17T04:22:25.579 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-04-17T04:22:25.592 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-17T04:22:25.594 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-17T04:22:25.594 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.7 kB] 2024-04-17T04:22:25.596 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.1 kB] 2024-04-17T04:22:25.597 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-17T04:22:25.598 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-17T04:22:25.624 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-17T04:22:25.628 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-17T04:22:25.792 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-17T04:22:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-17T04:22:25.978 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-17T04:22:26.039 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-17T04:22:26.068 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-17T04:22:29.422 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 12.9 MB in 2s (7,145 kB/s) 2024-04-17T04:22:30.307 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 12.9 MB in 3s (4,967 kB/s) 2024-04-17T04:22:30.756 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:22:30.774 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T04:22:30.837 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:22:31.034 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-04-17T04:22:31.035 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-04-17T04:22:31.206 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:22:31.206 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:22:31.206 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:22:31.207 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:22:31.207 INFO:teuthology.orchestra.run.smithi088.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-17T04:22:31.207 INFO:teuthology.orchestra.run.smithi088.stdout: python2.7-minimal 2024-04-17T04:22:31.207 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:22:31.208 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2024-04-17T04:22:31.208 INFO:teuthology.orchestra.run.smithi088.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T04:22:31.208 INFO:teuthology.orchestra.run.smithi088.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-17T04:22:31.209 INFO:teuthology.orchestra.run.smithi088.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-17T04:22:31.210 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2024-04-17T04:22:31.210 INFO:teuthology.orchestra.run.smithi088.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-17T04:22:31.210 INFO:teuthology.orchestra.run.smithi088.stdout:Recommended packages: 2024-04-17T04:22:31.210 INFO:teuthology.orchestra.run.smithi088.stdout: thermald 2024-04-17T04:22:31.265 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2024-04-17T04:22:31.265 INFO:teuthology.orchestra.run.smithi088.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T04:22:31.265 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-17T04:22:31.266 INFO:teuthology.orchestra.run.smithi088.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-17T04:22:31.267 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages will be upgraded: 2024-04-17T04:22:31.267 INFO:teuthology.orchestra.run.smithi088.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-17T04:22:31.470 INFO:teuthology.orchestra.run.smithi088.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T04:22:31.470 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 113 MB of archives. 2024-04-17T04:22:31.470 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-17T04:22:31.470 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-17T04:22:31.703 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:22:31.720 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T04:22:31.775 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:22:31.958 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-04-17T04:22:31.958 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-04-17T04:22:32.111 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:22:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:22:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:22:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:22:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-17T04:22:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: python2.7-minimal 2024-04-17T04:22:32.112 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:22:32.113 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-04-17T04:22:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T04:22:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-generic linux-image-5.15.0-102-generic 2024-04-17T04:22:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-5.15.0-102-generic linux-modules-extra-5.15.0-102-generic 2024-04-17T04:22:32.114 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-04-17T04:22:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-17T04:22:32.114 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2024-04-17T04:22:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: thermald 2024-04-17T04:22:32.191 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-04-17T04:22:32.191 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-5.15.0-102 linux-headers-5.15.0-102-generic 2024-04-17T04:22:32.191 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.15.0-102-generic linux-modules-5.15.0-102-generic 2024-04-17T04:22:32.192 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-extra-5.15.0-102-generic 2024-04-17T04:22:32.193 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2024-04-17T04:22:32.193 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-17T04:22:32.373 INFO:teuthology.orchestra.run.smithi154.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T04:22:32.373 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 113 MB of archives. 2024-04-17T04:22:32.373 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-17T04:22:32.373 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-102-generic amd64 5.15.0-102.112 [22.7 MB] 2024-04-17T04:22:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-17T04:22:33.476 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-17T04:22:33.798 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-102-generic amd64 5.15.0-102.112 [11.5 MB] 2024-04-17T04:22:34.144 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-102-generic amd64 5.15.0-102.112 [63.9 MB] 2024-04-17T04:22:35.738 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-17T04:22:35.738 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-17T04:22:35.739 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-17T04:22:36.334 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.102.99 [1,700 B] 2024-04-17T04:22:36.334 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.102.99 [2,512 B] 2024-04-17T04:22:36.334 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102 all 5.15.0-102.112 [12.4 MB] 2024-04-17T04:22:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-17T04:22:36.443 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-17T04:22:36.825 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 113 MB in 5s (21.9 MB/s) 2024-04-17T04:22:36.990 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-17T04:22:37.136 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-102-generic amd64 5.15.0-102.112 [2,877 kB] 2024-04-17T04:22:37.264 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.102.99 [2,360 B] 2024-04-17T04:22:37.657 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 113 MB in 5s (22.4 MB/s) 2024-04-17T04:22:37.863 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-5.15.0-102-generic. 2024-04-17T04:22:39.102 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-17T04:22:39.108 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:39.339 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:39.964 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-17T04:22:39.968 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../0-linux-modules-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:40.099 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:41.340 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-17T04:22:41.361 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:41.440 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:41.876 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-17T04:22:41.896 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:41.925 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:42.292 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-image-5.15.0-102-generic. 2024-04-17T04:22:42.313 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../1-linux-image-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:42.393 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:42.854 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-102-generic. 2024-04-17T04:22:42.875 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:42.904 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:48.599 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T04:22:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T04:22:48.992 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T04:22:49.113 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T04:22:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-17T04:22:49.455 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-17T04:22:49.499 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T04:22:49.546 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../3-linux-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T04:22:49.665 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T04:22:50.014 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T04:22:50.177 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T04:22:50.964 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.15.0-102. 2024-04-17T04:22:50.986 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../5-linux-headers-5.15.0-102_5.15.0-102.112_all.deb ... 2024-04-17T04:22:51.022 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T04:22:54.872 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-17T04:22:54.909 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:54.947 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:57.043 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.15.0-102-generic. 2024-04-17T04:22:57.081 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../6-linux-headers-5.15.0-102-generic_5.15.0-102.112_amd64.deb ... 2024-04-17T04:22:57.118 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:57.734 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T04:22:57.857 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T04:22:58.215 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T04:22:58.329 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:58.454 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-17T04:22:58.583 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:22:59.896 INFO:teuthology.orchestra.run.smithi088.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-17T04:22:59.896 INFO:teuthology.orchestra.run.smithi088.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-17T04:22:59.896 INFO:teuthology.orchestra.run.smithi088.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-17T04:22:59.896 INFO:teuthology.orchestra.run.smithi088.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:00.051 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:00.729 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.102.99_amd64.deb ... 2024-04-17T04:23:00.887 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-generic (5.15.0.102.99) over (5.15.0.56.54) ... 2024-04-17T04:23:01.262 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.15.0-102 (5.15.0-102.112) ... 2024-04-17T04:23:01.392 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:01.444 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:01.540 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-generic (5.15.0.102.99) ... 2024-04-17T04:23:01.663 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:02.829 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-17T04:23:02.938 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-17T04:23:02.999 INFO:teuthology.orchestra.run.smithi154.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-17T04:23:02.999 INFO:teuthology.orchestra.run.smithi154.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-17T04:23:02.999 INFO:teuthology.orchestra.run.smithi154.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-102-generic 2024-04-17T04:23:02.999 INFO:teuthology.orchestra.run.smithi154.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:03.064 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:03.126 INFO:teuthology.orchestra.run.smithi088.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-17T04:23:03.126 INFO:teuthology.orchestra.run.smithi088.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:03.148 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:04.541 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-extra-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-generic (5.15.0.102.99) ... 2024-04-17T04:23:06.619 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-generic (5.15.0.102.99) ... 2024-04-17T04:23:06.745 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for linux-image-5.15.0-102-generic (5.15.0-102.112) ... 2024-04-17T04:23:06.830 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-17T04:23:06.830 INFO:teuthology.orchestra.run.smithi154.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:17.093 INFO:teuthology.orchestra.run.smithi088.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-17T04:23:17.093 INFO:teuthology.orchestra.run.smithi088.stdout:Sourcing file `/etc/default/grub' 2024-04-17T04:23:17.113 INFO:teuthology.orchestra.run.smithi088.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T04:23:17.180 INFO:teuthology.orchestra.run.smithi088.stdout:Generating grub configuration file ... 2024-04-17T04:23:17.511 INFO:teuthology.orchestra.run.smithi088.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T04:23:17.531 INFO:teuthology.orchestra.run.smithi088.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:17.929 INFO:teuthology.orchestra.run.smithi088.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T04:23:17.936 INFO:teuthology.orchestra.run.smithi088.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T04:23:18.013 INFO:teuthology.orchestra.run.smithi088.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T04:23:18.019 INFO:teuthology.orchestra.run.smithi088.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T04:23:18.181 INFO:teuthology.orchestra.run.smithi088.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T04:23:18.181 INFO:teuthology.orchestra.run.smithi088.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T04:23:18.181 INFO:teuthology.orchestra.run.smithi088.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T04:23:18.228 INFO:teuthology.orchestra.run.smithi088.stdout:done 2024-04-17T04:23:18.727 DEBUG:teuthology.orchestra.run.smithi088:> dpkg -s linux-image-generic 2024-04-17T04:23:18.752 INFO:teuthology.orchestra.run.smithi088.stdout:Package: linux-image-generic 2024-04-17T04:23:18.752 INFO:teuthology.orchestra.run.smithi088.stdout:Status: install ok installed 2024-04-17T04:23:18.752 INFO:teuthology.orchestra.run.smithi088.stdout:Priority: optional 2024-04-17T04:23:18.752 INFO:teuthology.orchestra.run.smithi088.stdout:Section: kernel 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Installed-Size: 21 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: amd64 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Source: linux-meta 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Version: 5.15.0.102.99 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Recommends: thermald 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout:Description: Generic Linux kernel image 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout: This package will always depend on the latest generic kernel image 2024-04-17T04:23:18.753 INFO:teuthology.orchestra.run.smithi088.stdout: available. 2024-04-17T04:23:18.753 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-17T04:23:18.754 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T04:23:18.754 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T04:23:18.754 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-17T04:23:18.754 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T04:23:18.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get clean 2024-04-17T04:23:18.916 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get update 2024-04-17T04:23:19.041 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T04:23:19.054 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T04:23:19.075 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T04:23:19.158 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T04:23:20.314 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:23:20.330 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T04:23:20.389 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:23:20.578 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-04-17T04:23:20.578 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-04-17T04:23:20.732 INFO:teuthology.orchestra.run.smithi088.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:23:20.733 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:23:20.770 INFO:teuthology.orchestra.run.smithi088.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T04:23:20.772 DEBUG:teuthology.orchestra.run.smithi088:> dpkg -s linux-image-generic 2024-04-17T04:23:20.789 INFO:teuthology.orchestra.run.smithi088.stdout:Package: linux-image-generic 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Status: install ok installed 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Priority: optional 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Section: kernel 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Installed-Size: 21 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: amd64 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Source: linux-meta 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Version: 5.15.0.102.99 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T04:23:20.790 INFO:teuthology.orchestra.run.smithi088.stdout:Recommends: thermald 2024-04-17T04:23:20.791 INFO:teuthology.orchestra.run.smithi088.stdout:Description: Generic Linux kernel image 2024-04-17T04:23:20.791 INFO:teuthology.orchestra.run.smithi088.stdout: This package will always depend on the latest generic kernel image 2024-04-17T04:23:20.791 INFO:teuthology.orchestra.run.smithi088.stdout: available. 2024-04-17T04:23:20.791 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-04-17T04:23:20.838 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.pmyWf6xW69 2024-04-17T04:23:20.838 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pmyWf6xW69 2024-04-17T04:23:20.867 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-17T04:23:20.868 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub' 2024-04-17T04:23:20.891 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T04:23:20.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.pmyWf6xW69 2024-04-17T04:23:20.913 INFO:teuthology.orchestra.run.smithi154.stdout:Generating grub configuration file ... 2024-04-17T04:23:21.006 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.pmyWf6xW69 is 10KB 2024-04-17T04:23:21.019 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.pmyWf6xW69 2024-04-17T04:23:21.025 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-17T04:23:21.078 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:23:21.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-17T04:23:21.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-17T04:23:21.144 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-17T04:23:21.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo update-grub 2024-04-17T04:23:21.186 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T04:23:21.219 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:21.554 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T04:23:21.559 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T04:23:21.625 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T04:23:21.630 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T04:23:21.785 INFO:teuthology.orchestra.run.smithi154.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T04:23:21.785 INFO:teuthology.orchestra.run.smithi154.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T04:23:21.785 INFO:teuthology.orchestra.run.smithi154.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T04:23:21.827 INFO:teuthology.orchestra.run.smithi154.stdout:done 2024-04-17T04:23:22.306 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 21 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.15.0.102.99 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T04:23:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T04:23:22.338 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2024-04-17T04:23:22.338 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2024-04-17T04:23:22.338 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2024-04-17T04:23:22.338 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2024-04-17T04:23:22.338 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-102-generic 2024-04-17T04:23:22.338 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T04:23:22.338 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T04:23:22.338 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-17T04:23:22.338 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T04:23:22.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2024-04-17T04:23:22.376 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub' 2024-04-17T04:23:22.377 INFO:teuthology.orchestra.run.smithi088.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T04:23:22.379 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-04-17T04:23:22.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-04-17T04:23:22.550 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T04:23:22.556 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:22.626 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T04:23:22.627 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T04:23:22.633 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T04:23:22.654 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T04:23:22.839 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T04:23:22.845 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T04:23:22.888 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T04:23:22.893 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T04:23:23.052 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T04:23:23.052 INFO:teuthology.orchestra.run.smithi088.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T04:23:23.052 INFO:teuthology.orchestra.run.smithi088.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T04:23:23.064 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-04-17T04:23:23.075 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2024-04-17T04:23:23.967 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:23:23.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-17T04:23:24.039 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:23:24.241 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-04-17T04:23:24.241 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-04-17T04:23:24.395 INFO:teuthology.orchestra.run.smithi154.stdout:linux-image-generic is already the newest version (5.15.0.102.99). 2024-04-17T04:23:24.395 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:23:24.395 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:23:24.395 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:23:24.395 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:23:24.395 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:23:24.396 INFO:teuthology.orchestra.run.smithi154.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:23:24.396 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:23:24.435 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-17T04:23:24.436 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2024-04-17T04:23:24.490 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 21 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.15.0.102.99 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-102), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.15.0-102-generic, linux-modules-extra-5.15.0-102-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2024-04-17T04:23:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2024-04-17T04:23:24.492 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-04-17T04:23:24.534 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.pbaSdhO4Uf 2024-04-17T04:23:24.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pbaSdhO4Uf 2024-04-17T04:23:24.587 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.pbaSdhO4Uf 2024-04-17T04:23:24.705 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.pbaSdhO4Uf is 10KB 2024-04-17T04:23:24.720 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.pbaSdhO4Uf 2024-04-17T04:23:24.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-17T04:23:24.780 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:23:24.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-17T04:23:24.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-17T04:23:24.864 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-102-generic 2024-04-17T04:23:24.864 DEBUG:teuthology.orchestra.run.smithi154:> sudo update-grub 2024-04-17T04:23:26.192 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub' 2024-04-17T04:23:26.194 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-17T04:23:26.197 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-04-17T04:23:26.387 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.15.0-102-generic 2024-04-17T04:23:26.393 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.15.0-102-generic 2024-04-17T04:23:26.676 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-17T04:23:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-17T04:23:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-17T04:23:26.730 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-17T04:23:26.851 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-17T04:23:26.851 INFO:teuthology.orchestra.run.smithi154.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-17T04:23:26.851 INFO:teuthology.orchestra.run.smithi154.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-17T04:23:26.880 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-04-17T04:23:26.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-04-17T04:23:53.102 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-17T04:23:53.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:23:53.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:23:56.886 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-17T04:23:56.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:23:56.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:24:11.551 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-17T04:24:15.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-04-17T04:24:20.554 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:24:20.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:24:23.618 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-17T04:24:24.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:24:24.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:24:27.458 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-04-17T04:24:35.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:24:35.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:24:38.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-17T04:24:39.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:24:39.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:24:53.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:24:53.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:25:25.581 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-17T04:25:26.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:25:26.154 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-102-generic"... 2024-04-17T04:25:26.155 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-04-17T04:25:26.199 INFO:teuthology.orchestra.run.smithi088.stdout:5.15.0-102-generic 2024-04-17T04:25:26.199 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-17T04:25:26.200 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T04:25:26.200 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-17T04:25:27.201 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-17T04:25:27.201 DEBUG:teuthology.orchestra.run.smithi088:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T04:25:27.543 INFO:teuthology.orchestra.run.smithi088.stdout:ttyS1 2024-04-17T04:25:27.565 DEBUG:teuthology.parallel:result is None 2024-04-17T04:25:39.462 DEBUG:teuthology.orchestra.remote:timed out 2024-04-17T04:25:54.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:25:54.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:25:54.736 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-17T04:25:55.366 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:25:55.366 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-102-generic"... 2024-04-17T04:25:55.366 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-04-17T04:25:55.414 INFO:teuthology.orchestra.run.smithi154.stdout:5.15.0-102-generic 2024-04-17T04:25:55.414 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-102-generic vs 5.15.0-102-generic 2024-04-17T04:25:55.414 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T04:25:55.414 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-17T04:25:56.415 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-17T04:25:56.416 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T04:25:56.785 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-04-17T04:25:56.815 DEBUG:teuthology.parallel:result is None 2024-04-17T04:25:56.815 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T04:25:56.822 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T04:25:56.822 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T04:25:56.825 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T04:25:56.831 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T04:25:56.843 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T04:25:56.921 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T04:25:56.921 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T04:25:56.924 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T04:25:56.954 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T04:25:56.993 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T04:25:56.993 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T04:25:56.996 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T04:25:57.014 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T04:25:57.017 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T04:25:57.023 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T04:25:57.025 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T04:25:57.026 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T04:25:57.068 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T04:25:57.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T04:25:57.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T04:25:57.087 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T04:25:57.145 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T04:25:57.145 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T04:25:57.148 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T04:25:57.153 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T04:25:57.199 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T04:25:57.247 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:25:57.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T04:25:57.329 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T04:25:57.335 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T04:25:57.383 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:25:57.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T04:25:57.445 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-04-17T04:25:57.448 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-04-17T04:25:57.529 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T04:25:57.537 INFO:teuthology.task.internal:Starting timer... 2024-04-17T04:25:57.537 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T04:25:57.598 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T04:25:57.645 DEBUG:teuthology.task.selinux:Excluding smithi088: OS 'ubuntu' does not support SELinux 2024-04-17T04:25:57.646 DEBUG:teuthology.task.selinux:Excluding smithi154: OS 'ubuntu' does not support SELinux 2024-04-17T04:25:57.646 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T04:25:57.646 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-17T04:25:57.647 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T04:25:57.647 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T04:25:57.696 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T04:25:57.713 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T04:25:57.714 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi088.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T04:32:58.803 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-04-17T04:32:58.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:32:58.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:32:58.888 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-17T04:32:58.962 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-17T04:32:58.963 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:32:58.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T04:32:59.041 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-17T04:32:59.118 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-17T04:32:59.119 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T04:32:59.130 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T04:32:59.130 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T04:32:59.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T04:32:59.133 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T04:32:59.133 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Command line: ntpd -gq 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: ---------------------------------------------------- 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: ntp-4 is maintained by Network Time Foundation, 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: corporation. Support and training for ntp-4 are 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: available at https://www.nwtime.org/support 2024-04-17T04:32:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: ---------------------------------------------------- 2024-04-17T04:32:59.161 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: proto: precision = 0.046 usec (-24) 2024-04-17T04:32:59.162 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: basedate set to 2022-02-04 2024-04-17T04:32:59.162 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: gps base set to 2022-02-06 (week 2196) 2024-04-17T04:32:59.162 INFO:teuthology.orchestra.run.smithi088.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T04:32:59.162 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T04:32:59.162 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: restrict ::: KOD does nothing without LIMITED. 2024-04-17T04:32:59.163 INFO:teuthology.orchestra.run.smithi088.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listen and drop on 0 v6wildcard [::]:123 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listen normally on 2 lo 127.0.0.1:123 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listen normally on 3 enp3s0f1 172.21.15.88:123 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listen normally on 4 lo [::1]:123 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d42d%5]:123 2024-04-17T04:32:59.164 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:32:59 ntpd[17183]: Listening on routing socket on fd #22 for interface updates 2024-04-17T04:32:59.185 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-17T04:32:59.185 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Command line: ntpd -gq 2024-04-17T04:32:59.185 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: ---------------------------------------------------- 2024-04-17T04:32:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: ntp-4 is maintained by Network Time Foundation, 2024-04-17T04:32:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-17T04:32:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: corporation. Support and training for ntp-4 are 2024-04-17T04:32:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: available at https://www.nwtime.org/support 2024-04-17T04:32:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: ---------------------------------------------------- 2024-04-17T04:32:59.187 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: proto: precision = 0.044 usec (-24) 2024-04-17T04:32:59.187 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: basedate set to 2022-02-04 2024-04-17T04:32:59.187 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: gps base set to 2022-02-06 (week 2196) 2024-04-17T04:32:59.187 INFO:teuthology.orchestra.run.smithi154.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T04:32:59.187 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-17T04:32:59.188 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: restrict ::: KOD does nothing without LIMITED. 2024-04-17T04:32:59.188 INFO:teuthology.orchestra.run.smithi154.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listen and drop on 0 v6wildcard [::]:123 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listen normally on 2 lo 127.0.0.1:123 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listen normally on 3 enp3s0f1 172.21.15.154:123 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listen normally on 4 lo [::1]:123 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7761%5]:123 2024-04-17T04:32:59.189 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:32:59 ntpd[17196]: Listening on routing socket on fd #22 for interface updates 2024-04-17T04:33:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:00 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:00.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:00 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:01 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:02 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:02.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:02 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:02 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:02 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:02.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:02 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:02.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:02 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:02.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:02 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:03.160 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:03 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:03.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:03 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:03.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:03 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:03.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:03 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:03.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:03 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:03.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:03 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:03.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:03.188 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:03 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:04.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:04 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:04.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:04 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:04.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:04 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:04.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:04 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:04.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:04 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:04.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:04 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:04.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:04 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:05.160 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:05 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:05.160 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:05 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:05.160 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:05 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:05.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:05 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:05.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:05 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:05.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:05 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:05.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:05 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:06.161 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:06.162 INFO:teuthology.orchestra.run.smithi088.stdout:17 Apr 04:33:06 ntpd[17183]: ntpd: time slew -0.000907 s 2024-04-17T04:33:06.162 INFO:teuthology.orchestra.run.smithi088.stdout:ntpd: time slew -0.000907s 2024-04-17T04:33:06.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-17T04:33:06.162 INFO:teuthology.orchestra.run.smithi088.stderr:17 Apr 04:33:06 ntpd[17183]: can't open /var/log/ntpstats/loopstats.20240417: Permission denied 2024-04-17T04:33:06.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:06.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:06.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-17T04:33:06.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240417: Permission denied 2024-04-17T04:33:06.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-17T04:33:06.186 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240417: Permission denied 2024-04-17T04:33:06.187 INFO:teuthology.orchestra.run.smithi154.stdout:17 Apr 04:33:06 ntpd[17196]: ntpd: time slew -0.002164 s 2024-04-17T04:33:06.187 INFO:teuthology.orchestra.run.smithi154.stdout:ntpd: time slew -0.002164s 2024-04-17T04:33:06.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-17T04:33:06.187 INFO:teuthology.orchestra.run.smithi154.stderr:17 Apr 04:33:06 ntpd[17196]: can't open /var/log/ntpstats/loopstats.20240417: Permission denied 2024-04-17T04:33:06.220 INFO:teuthology.orchestra.run.smithi088.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T04:33:06.220 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================== 2024-04-17T04:33:06.220 INFO:teuthology.orchestra.run.smithi088.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.220 INFO:teuthology.orchestra.run.smithi088.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.220 INFO:teuthology.orchestra.run.smithi088.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.220 INFO:teuthology.orchestra.run.smithi088.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.246 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T04:33:06.246 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2024-04-17T04:33:06.246 INFO:teuthology.orchestra.run.smithi154.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.247 INFO:teuthology.orchestra.run.smithi154.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.247 INFO:teuthology.orchestra.run.smithi154.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.247 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-17T04:33:06.247 INFO:teuthology.run_tasks:Running task install... 2024-04-17T04:33:06.257 DEBUG:teuthology.task.install:project ceph 2024-04-17T04:33:06.258 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '461a84e6fbbf089011eafd13b98169fb9e9b8cc9'}} 2024-04-17T04:33:06.258 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '461a84e6fbbf089011eafd13b98169fb9e9b8cc9'} 2024-04-17T04:33:06.258 INFO:teuthology.task.install:Using flavor: default 2024-04-17T04:33:06.263 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T04:33:06.264 INFO:teuthology.task.install:extra packages: [] 2024-04-17T04:33:06.264 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-key list | grep Ceph 2024-04-17T04:33:06.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-key list | grep Ceph 2024-04-17T04:33:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-17T04:33:06.343 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-17T04:33:06.352 INFO:teuthology.orchestra.run.smithi088.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-17T04:33:06.352 INFO:teuthology.orchestra.run.smithi088.stdout:uid [ unknown] Ceph.com (release key) 2024-04-17T04:33:06.353 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-17T04:33:06.353 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-04-17T04:33:06.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=461a84e6fbbf089011eafd13b98169fb9e9b8cc9 2024-04-17T04:33:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-17T04:33:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph.com (release key) 2024-04-17T04:33:06.371 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-17T04:33:06.371 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-04-17T04:33:06.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=461a84e6fbbf089011eafd13b98169fb9e9b8cc9 2024-04-17T04:33:06.520 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default/ 2024-04-17T04:33:06.520 INFO:teuthology.task.install.deb:Package version is 19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:33:06.578 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default/ 2024-04-17T04:33:06.579 INFO:teuthology.task.install.deb:Package version is 19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:33:06.638 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:33:06.638 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-17T04:33:06.652 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2024-04-17T04:33:06.688 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:33:06.688 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-17T04:33:06.702 DEBUG:teuthology.orchestra.run.smithi088:> sudo apt-get update 2024-04-17T04:33:06.822 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T04:33:06.836 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T04:33:06.857 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T04:33:06.870 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T04:33:06.870 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-17T04:33:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-17T04:33:06.906 INFO:teuthology.orchestra.run.smithi088.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-17T04:33:06.923 INFO:teuthology.orchestra.run.smithi088.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy InRelease 2024-04-17T04:33:06.934 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy InRelease 2024-04-17T04:33:06.937 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-17T04:33:06.944 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-17T04:33:06.966 INFO:teuthology.orchestra.run.smithi088.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-17T04:33:06.970 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-17T04:33:06.987 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,416 B] 2024-04-17T04:33:07.007 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-17T04:33:07.014 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-17T04:33:07.045 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,416 B] 2024-04-17T04:33:07.087 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-17T04:33:07.169 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 28.9 kB in 0s (85.4 kB/s) 2024-04-17T04:33:07.232 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 28.9 kB in 0s (81.2 kB/s) 2024-04-17T04:33:08.135 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:33:08.138 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:33:08.151 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2287-g461a84e6-1jammy cephadm=19.0.0-2287-g461a84e6-1jammy ceph-mds=19.0.0-2287-g461a84e6-1jammy ceph-mgr=19.0.0-2287-g461a84e6-1jammy ceph-common=19.0.0-2287-g461a84e6-1jammy ceph-fuse=19.0.0-2287-g461a84e6-1jammy ceph-test=19.0.0-2287-g461a84e6-1jammy ceph-volume=19.0.0-2287-g461a84e6-1jammy radosgw=19.0.0-2287-g461a84e6-1jammy python3-rados=19.0.0-2287-g461a84e6-1jammy python3-rgw=19.0.0-2287-g461a84e6-1jammy python3-cephfs=19.0.0-2287-g461a84e6-1jammy python3-rbd=19.0.0-2287-g461a84e6-1jammy libcephfs2=19.0.0-2287-g461a84e6-1jammy libcephfs-dev=19.0.0-2287-g461a84e6-1jammy librados2=19.0.0-2287-g461a84e6-1jammy librbd1=19.0.0-2287-g461a84e6-1jammy rbd-fuse=19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:33:08.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2287-g461a84e6-1jammy cephadm=19.0.0-2287-g461a84e6-1jammy ceph-mds=19.0.0-2287-g461a84e6-1jammy ceph-mgr=19.0.0-2287-g461a84e6-1jammy ceph-common=19.0.0-2287-g461a84e6-1jammy ceph-fuse=19.0.0-2287-g461a84e6-1jammy ceph-test=19.0.0-2287-g461a84e6-1jammy ceph-volume=19.0.0-2287-g461a84e6-1jammy radosgw=19.0.0-2287-g461a84e6-1jammy python3-rados=19.0.0-2287-g461a84e6-1jammy python3-rgw=19.0.0-2287-g461a84e6-1jammy python3-cephfs=19.0.0-2287-g461a84e6-1jammy python3-rbd=19.0.0-2287-g461a84e6-1jammy libcephfs2=19.0.0-2287-g461a84e6-1jammy libcephfs-dev=19.0.0-2287-g461a84e6-1jammy librados2=19.0.0-2287-g461a84e6-1jammy librbd1=19.0.0-2287-g461a84e6-1jammy rbd-fuse=19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:33:08.210 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:33:08.217 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:33:08.407 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-04-17T04:33:08.407 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-04-17T04:33:08.412 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-04-17T04:33:08.413 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-04-17T04:33:08.592 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:33:08.593 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:33:08.593 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:33:08.593 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:33:08.593 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:33:08.593 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:33:08.593 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-17T04:33:08.595 INFO:teuthology.orchestra.run.smithi088.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-17T04:33:08.596 INFO:teuthology.orchestra.run.smithi088.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet zip 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-17T04:33:08.597 INFO:teuthology.orchestra.run.smithi088.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-17T04:33:08.598 INFO:teuthology.orchestra.run.smithi088.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-17T04:33:08.598 INFO:teuthology.orchestra.run.smithi088.stdout:Recommended packages: 2024-04-17T04:33:08.598 INFO:teuthology.orchestra.run.smithi088.stdout: btrfs-tools 2024-04-17T04:33:08.607 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:33:08.607 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:33:08.607 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:33:08.607 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:33:08.608 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:33:08.608 INFO:teuthology.orchestra.run.smithi154.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:33:08.608 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:33:08.608 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-04-17T04:33:08.608 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-17T04:33:08.610 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-17T04:33:08.611 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet zip 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2024-04-17T04:33:08.612 INFO:teuthology.orchestra.run.smithi154.stdout: btrfs-tools 2024-04-17T04:33:08.695 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2024-04-17T04:33:08.695 INFO:teuthology.orchestra.run.smithi088.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-17T04:33:08.695 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-17T04:33:08.696 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-17T04:33:08.696 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-17T04:33:08.696 INFO:teuthology.orchestra.run.smithi088.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-17T04:33:08.696 INFO:teuthology.orchestra.run.smithi088.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T04:33:08.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-17T04:33:08.698 INFO:teuthology.orchestra.run.smithi088.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-17T04:33:08.715 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-04-17T04:33:08.715 INFO:teuthology.orchestra.run.smithi154.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-17T04:33:08.715 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-17T04:33:08.715 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-17T04:33:08.715 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-17T04:33:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-17T04:33:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-17T04:33:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-17T04:33:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-17T04:33:08.764 INFO:teuthology.orchestra.run.smithi088.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T04:33:08.764 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 179 MB of archives. 2024-04-17T04:33:08.764 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-17T04:33:08.764 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-17T04:33:08.779 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T04:33:08.779 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 179 MB of archives. 2024-04-17T04:33:08.779 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-17T04:33:08.779 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-17T04:33:08.857 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2287-g461a84e6-1jammy [3,546 kB] 2024-04-17T04:33:08.865 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-17T04:33:08.867 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-17T04:33:08.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-17T04:33:08.886 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-17T04:33:08.888 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-17T04:33:08.905 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-17T04:33:08.916 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2287-g461a84e6-1jammy [3,546 kB] 2024-04-17T04:33:08.958 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-17T04:33:08.964 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-17T04:33:08.977 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-17T04:33:08.981 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-17T04:33:08.982 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-17T04:33:08.982 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-17T04:33:08.983 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-17T04:33:08.986 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-17T04:33:08.991 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-17T04:33:08.993 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-17T04:33:08.995 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-17T04:33:08.997 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-17T04:33:09.000 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-17T04:33:09.004 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-17T04:33:09.005 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-17T04:33:09.006 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-17T04:33:09.015 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-17T04:33:09.016 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-17T04:33:09.017 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-17T04:33:09.019 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-17T04:33:09.019 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-17T04:33:09.020 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-17T04:33:09.021 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-17T04:33:09.023 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-17T04:33:09.023 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-17T04:33:09.024 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-17T04:33:09.025 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-17T04:33:09.025 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-17T04:33:09.032 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-17T04:33:09.034 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-17T04:33:09.037 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-17T04:33:09.037 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-17T04:33:09.037 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-17T04:33:09.039 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-17T04:33:09.039 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-17T04:33:09.051 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-17T04:33:09.053 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-17T04:33:09.053 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-17T04:33:09.053 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-17T04:33:09.054 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-17T04:33:09.056 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-17T04:33:09.057 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-17T04:33:09.057 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-17T04:33:09.060 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-17T04:33:09.060 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-17T04:33:09.061 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-17T04:33:09.061 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-17T04:33:09.062 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-17T04:33:09.071 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-17T04:33:09.072 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-17T04:33:09.072 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-17T04:33:09.075 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-17T04:33:09.075 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-17T04:33:09.076 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-17T04:33:09.077 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-17T04:33:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-17T04:33:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-17T04:33:09.091 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2287-g461a84e6-1jammy [3,218 kB] 2024-04-17T04:33:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-17T04:33:09.092 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-17T04:33:09.093 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-17T04:33:09.096 INFO:teuthology.orchestra.run.smithi088.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-17T04:33:09.097 INFO:teuthology.orchestra.run.smithi088.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-17T04:33:09.097 INFO:teuthology.orchestra.run.smithi088.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-17T04:33:09.103 INFO:teuthology.orchestra.run.smithi088.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-17T04:33:09.103 INFO:teuthology.orchestra.run.smithi088.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-17T04:33:09.104 INFO:teuthology.orchestra.run.smithi088.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-17T04:33:09.105 INFO:teuthology.orchestra.run.smithi088.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-17T04:33:09.106 INFO:teuthology.orchestra.run.smithi088.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-17T04:33:09.110 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-17T04:33:09.111 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-17T04:33:09.112 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-17T04:33:09.115 INFO:teuthology.orchestra.run.smithi088.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-17T04:33:09.116 INFO:teuthology.orchestra.run.smithi088.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-17T04:33:09.117 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-17T04:33:09.117 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-17T04:33:09.128 INFO:teuthology.orchestra.run.smithi088.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-17T04:33:09.129 INFO:teuthology.orchestra.run.smithi088.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-17T04:33:09.131 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-17T04:33:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-17T04:33:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-17T04:33:09.132 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-17T04:33:09.133 INFO:teuthology.orchestra.run.smithi088.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-17T04:33:09.134 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-17T04:33:09.135 INFO:teuthology.orchestra.run.smithi088.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-17T04:33:09.137 INFO:teuthology.orchestra.run.smithi154.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-17T04:33:09.138 INFO:teuthology.orchestra.run.smithi154.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-17T04:33:09.161 INFO:teuthology.orchestra.run.smithi154.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-17T04:33:09.162 INFO:teuthology.orchestra.run.smithi154.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-17T04:33:09.166 INFO:teuthology.orchestra.run.smithi154.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-17T04:33:09.167 INFO:teuthology.orchestra.run.smithi154.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-17T04:33:09.168 INFO:teuthology.orchestra.run.smithi154.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-17T04:33:09.188 INFO:teuthology.orchestra.run.smithi088.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-17T04:33:09.189 INFO:teuthology.orchestra.run.smithi088.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-17T04:33:09.189 INFO:teuthology.orchestra.run.smithi154.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-17T04:33:09.190 INFO:teuthology.orchestra.run.smithi088.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-17T04:33:09.190 INFO:teuthology.orchestra.run.smithi088.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-17T04:33:09.191 INFO:teuthology.orchestra.run.smithi088.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-17T04:33:09.192 INFO:teuthology.orchestra.run.smithi154.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2287-g461a84e6-1jammy [711 kB] 2024-04-17T04:33:09.215 INFO:teuthology.orchestra.run.smithi154.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2287-g461a84e6-1jammy [356 kB] 2024-04-17T04:33:09.230 INFO:teuthology.orchestra.run.smithi154.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2287-g461a84e6-1jammy [32.8 kB] 2024-04-17T04:33:09.231 INFO:teuthology.orchestra.run.smithi154.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2287-g461a84e6-1jammy [180 kB] 2024-04-17T04:33:09.233 INFO:teuthology.orchestra.run.smithi088.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-17T04:33:09.236 INFO:teuthology.orchestra.run.smithi088.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-17T04:33:09.237 INFO:teuthology.orchestra.run.smithi154.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2287-g461a84e6-1jammy [64.6 kB] 2024-04-17T04:33:09.237 INFO:teuthology.orchestra.run.smithi088.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-17T04:33:09.239 INFO:teuthology.orchestra.run.smithi154.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2287-g461a84e6-1jammy [330 kB] 2024-04-17T04:33:09.250 INFO:teuthology.orchestra.run.smithi154.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2287-g461a84e6-1jammy [6,777 kB] 2024-04-17T04:33:09.260 INFO:teuthology.orchestra.run.smithi088.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-17T04:33:09.260 INFO:teuthology.orchestra.run.smithi088.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-17T04:33:09.273 INFO:teuthology.orchestra.run.smithi154.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-17T04:33:09.274 INFO:teuthology.orchestra.run.smithi154.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-17T04:33:09.276 INFO:teuthology.orchestra.run.smithi154.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-17T04:33:09.277 INFO:teuthology.orchestra.run.smithi154.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-17T04:33:09.278 INFO:teuthology.orchestra.run.smithi154.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-17T04:33:09.279 INFO:teuthology.orchestra.run.smithi088.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-17T04:33:09.280 INFO:teuthology.orchestra.run.smithi088.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-17T04:33:09.280 INFO:teuthology.orchestra.run.smithi088.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-17T04:33:09.281 INFO:teuthology.orchestra.run.smithi088.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-17T04:33:09.281 INFO:teuthology.orchestra.run.smithi088.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-17T04:33:09.282 INFO:teuthology.orchestra.run.smithi088.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-17T04:33:09.286 INFO:teuthology.orchestra.run.smithi088.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-17T04:33:09.303 INFO:teuthology.orchestra.run.smithi154.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-17T04:33:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-17T04:33:09.307 INFO:teuthology.orchestra.run.smithi088.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-17T04:33:09.308 INFO:teuthology.orchestra.run.smithi088.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-17T04:33:09.309 INFO:teuthology.orchestra.run.smithi088.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-17T04:33:09.322 INFO:teuthology.orchestra.run.smithi154.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-17T04:33:09.322 INFO:teuthology.orchestra.run.smithi154.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-17T04:33:09.324 INFO:teuthology.orchestra.run.smithi154.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-17T04:33:09.325 INFO:teuthology.orchestra.run.smithi154.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-17T04:33:09.325 INFO:teuthology.orchestra.run.smithi154.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-17T04:33:09.325 INFO:teuthology.orchestra.run.smithi154.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-17T04:33:09.326 INFO:teuthology.orchestra.run.smithi088.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-17T04:33:09.329 INFO:teuthology.orchestra.run.smithi088.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-17T04:33:09.330 INFO:teuthology.orchestra.run.smithi088.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-17T04:33:09.331 INFO:teuthology.orchestra.run.smithi088.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-17T04:33:09.336 INFO:teuthology.orchestra.run.smithi088.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-17T04:33:09.337 INFO:teuthology.orchestra.run.smithi088.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-17T04:33:09.340 INFO:teuthology.orchestra.run.smithi088.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-17T04:33:09.344 INFO:teuthology.orchestra.run.smithi154.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-17T04:33:09.345 INFO:teuthology.orchestra.run.smithi154.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-17T04:33:09.349 INFO:teuthology.orchestra.run.smithi154.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-17T04:33:09.352 INFO:teuthology.orchestra.run.smithi088.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-17T04:33:09.352 INFO:teuthology.orchestra.run.smithi154.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-17T04:33:09.352 INFO:teuthology.orchestra.run.smithi088.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-17T04:33:09.353 INFO:teuthology.orchestra.run.smithi088.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-17T04:33:09.354 INFO:teuthology.orchestra.run.smithi154.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-17T04:33:09.355 INFO:teuthology.orchestra.run.smithi154.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-17T04:33:09.370 INFO:teuthology.orchestra.run.smithi154.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-17T04:33:09.373 INFO:teuthology.orchestra.run.smithi154.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-17T04:33:09.374 INFO:teuthology.orchestra.run.smithi154.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-17T04:33:09.375 INFO:teuthology.orchestra.run.smithi154.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-17T04:33:09.385 INFO:teuthology.orchestra.run.smithi088.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-17T04:33:09.386 INFO:teuthology.orchestra.run.smithi088.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-17T04:33:09.399 INFO:teuthology.orchestra.run.smithi088.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-17T04:33:09.400 INFO:teuthology.orchestra.run.smithi088.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-17T04:33:09.400 INFO:teuthology.orchestra.run.smithi088.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-17T04:33:09.405 INFO:teuthology.orchestra.run.smithi088.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-17T04:33:09.406 INFO:teuthology.orchestra.run.smithi088.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-17T04:33:09.422 INFO:teuthology.orchestra.run.smithi154.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-17T04:33:09.423 INFO:teuthology.orchestra.run.smithi154.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-17T04:33:09.428 INFO:teuthology.orchestra.run.smithi154.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-17T04:33:09.428 INFO:teuthology.orchestra.run.smithi154.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-17T04:33:09.429 INFO:teuthology.orchestra.run.smithi154.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-17T04:33:09.429 INFO:teuthology.orchestra.run.smithi154.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-17T04:33:09.453 INFO:teuthology.orchestra.run.smithi154.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-17T04:33:09.454 INFO:teuthology.orchestra.run.smithi154.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-17T04:33:09.464 INFO:teuthology.orchestra.run.smithi154.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-17T04:33:09.464 INFO:teuthology.orchestra.run.smithi154.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-17T04:33:09.482 INFO:teuthology.orchestra.run.smithi154.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2287-g461a84e6-1jammy [112 kB] 2024-04-17T04:33:09.483 INFO:teuthology.orchestra.run.smithi154.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-17T04:33:09.484 INFO:teuthology.orchestra.run.smithi154.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2287-g461a84e6-1jammy [466 kB] 2024-04-17T04:33:09.486 INFO:teuthology.orchestra.run.smithi154.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-17T04:33:09.487 INFO:teuthology.orchestra.run.smithi154.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-17T04:33:09.498 INFO:teuthology.orchestra.run.smithi154.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2287-g461a84e6-1jammy [25.6 MB] 2024-04-17T04:33:10.498 INFO:teuthology.orchestra.run.smithi154.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2287-g461a84e6-1jammy [5,084 kB] 2024-04-17T04:33:10.599 INFO:teuthology.orchestra.run.smithi154.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2287-g461a84e6-1jammy [244 kB] 2024-04-17T04:33:10.606 INFO:teuthology.orchestra.run.smithi154.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2287-g461a84e6-1jammy [124 kB] 2024-04-17T04:33:10.610 INFO:teuthology.orchestra.run.smithi154.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2287-g461a84e6-1jammy [1,525 kB] 2024-04-17T04:33:10.662 INFO:teuthology.orchestra.run.smithi154.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2287-g461a84e6-1jammy [6,246 kB] 2024-04-17T04:33:10.712 INFO:teuthology.orchestra.run.smithi088.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2287-g461a84e6-1jammy [3,218 kB] 2024-04-17T04:33:10.894 INFO:teuthology.orchestra.run.smithi154.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2287-g461a84e6-1jammy [22.7 MB] 2024-04-17T04:33:11.499 INFO:teuthology.orchestra.run.smithi088.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2287-g461a84e6-1jammy [711 kB] 2024-04-17T04:33:11.653 INFO:teuthology.orchestra.run.smithi088.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2287-g461a84e6-1jammy [356 kB] 2024-04-17T04:33:11.730 INFO:teuthology.orchestra.run.smithi088.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2287-g461a84e6-1jammy [32.8 kB] 2024-04-17T04:33:11.737 INFO:teuthology.orchestra.run.smithi088.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2287-g461a84e6-1jammy [180 kB] 2024-04-17T04:33:11.765 INFO:teuthology.orchestra.run.smithi088.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2287-g461a84e6-1jammy [64.6 kB] 2024-04-17T04:33:11.779 INFO:teuthology.orchestra.run.smithi088.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2287-g461a84e6-1jammy [330 kB] 2024-04-17T04:33:11.847 INFO:teuthology.orchestra.run.smithi088.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2287-g461a84e6-1jammy [6,777 kB] 2024-04-17T04:33:11.866 INFO:teuthology.orchestra.run.smithi154.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2287-g461a84e6-1jammy [14.0 kB] 2024-04-17T04:33:11.867 INFO:teuthology.orchestra.run.smithi154.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2287-g461a84e6-1jammy [869 kB] 2024-04-17T04:33:11.893 INFO:teuthology.orchestra.run.smithi154.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2287-g461a84e6-1jammy [2,436 kB] 2024-04-17T04:33:11.937 INFO:teuthology.orchestra.run.smithi154.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2287-g461a84e6-1jammy [778 kB] 2024-04-17T04:33:11.947 INFO:teuthology.orchestra.run.smithi154.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2287-g461a84e6-1jammy [145 kB] 2024-04-17T04:33:11.949 INFO:teuthology.orchestra.run.smithi154.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2287-g461a84e6-1jammy [2,295 kB] 2024-04-17T04:33:12.038 INFO:teuthology.orchestra.run.smithi154.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2287-g461a84e6-1jammy [8,625 kB] 2024-04-17T04:33:12.424 INFO:teuthology.orchestra.run.smithi154.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2287-g461a84e6-1jammy [14.2 kB] 2024-04-17T04:33:12.424 INFO:teuthology.orchestra.run.smithi154.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2287-g461a84e6-1jammy [50.7 MB] 2024-04-17T04:33:12.680 INFO:teuthology.orchestra.run.smithi088.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2287-g461a84e6-1jammy [112 kB] 2024-04-17T04:33:12.682 INFO:teuthology.orchestra.run.smithi088.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2287-g461a84e6-1jammy [466 kB] 2024-04-17T04:33:12.730 INFO:teuthology.orchestra.run.smithi088.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2287-g461a84e6-1jammy [25.6 MB] 2024-04-17T04:33:15.055 INFO:teuthology.orchestra.run.smithi088.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2287-g461a84e6-1jammy [5,084 kB] 2024-04-17T04:33:15.316 INFO:teuthology.orchestra.run.smithi088.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2287-g461a84e6-1jammy [244 kB] 2024-04-17T04:33:15.330 INFO:teuthology.orchestra.run.smithi154.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2287-g461a84e6-1jammy [121 kB] 2024-04-17T04:33:15.331 INFO:teuthology.orchestra.run.smithi088.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2287-g461a84e6-1jammy [124 kB] 2024-04-17T04:33:15.332 INFO:teuthology.orchestra.run.smithi154.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2287-g461a84e6-1jammy [38.9 kB] 2024-04-17T04:33:15.334 INFO:teuthology.orchestra.run.smithi154.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2287-g461a84e6-1jammy [13.4 MB] 2024-04-17T04:33:15.335 INFO:teuthology.orchestra.run.smithi088.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2287-g461a84e6-1jammy [1,525 kB] 2024-04-17T04:33:15.429 INFO:teuthology.orchestra.run.smithi088.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2287-g461a84e6-1jammy [6,246 kB] 2024-04-17T04:33:15.884 INFO:teuthology.orchestra.run.smithi154.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2287-g461a84e6-1jammy [91.4 kB] 2024-04-17T04:33:15.912 INFO:teuthology.orchestra.run.smithi088.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2287-g461a84e6-1jammy [22.7 MB] 2024-04-17T04:33:16.419 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 179 MB in 7s (25.1 MB/s) 2024-04-17T04:33:16.585 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-17T04:33:16.684 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-17T04:33:16.690 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-17T04:33:16.764 INFO:teuthology.orchestra.run.smithi088.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2287-g461a84e6-1jammy [14.0 kB] 2024-04-17T04:33:16.765 INFO:teuthology.orchestra.run.smithi088.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2287-g461a84e6-1jammy [869 kB] 2024-04-17T04:33:16.775 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T04:33:16.781 INFO:teuthology.orchestra.run.smithi088.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2287-g461a84e6-1jammy [2,436 kB] 2024-04-17T04:33:16.838 INFO:teuthology.orchestra.run.smithi088.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2287-g461a84e6-1jammy [778 kB] 2024-04-17T04:33:16.858 INFO:teuthology.orchestra.run.smithi088.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2287-g461a84e6-1jammy [145 kB] 2024-04-17T04:33:16.861 INFO:teuthology.orchestra.run.smithi088.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2287-g461a84e6-1jammy [2,295 kB] 2024-04-17T04:33:16.942 INFO:teuthology.orchestra.run.smithi088.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2287-g461a84e6-1jammy [8,625 kB] 2024-04-17T04:33:17.120 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-17T04:33:17.146 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-17T04:33:17.186 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T04:33:17.222 INFO:teuthology.orchestra.run.smithi088.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2287-g461a84e6-1jammy [14.2 kB] 2024-04-17T04:33:17.222 INFO:teuthology.orchestra.run.smithi088.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2287-g461a84e6-1jammy [50.7 MB] 2024-04-17T04:33:17.475 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-17T04:33:17.501 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-17T04:33:17.533 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T04:33:17.987 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-17T04:33:18.013 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T04:33:18.062 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:33:18.407 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-17T04:33:18.433 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T04:33:18.464 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:33:18.826 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-17T04:33:18.852 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T04:33:18.893 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:33:19.157 INFO:teuthology.orchestra.run.smithi088.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2287-g461a84e6-1jammy [121 kB] 2024-04-17T04:33:19.160 INFO:teuthology.orchestra.run.smithi088.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2287-g461a84e6-1jammy [38.9 kB] 2024-04-17T04:33:19.161 INFO:teuthology.orchestra.run.smithi088.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2287-g461a84e6-1jammy [13.4 MB] 2024-04-17T04:33:19.254 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-17T04:33:19.280 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-17T04:33:19.321 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T04:33:19.527 INFO:teuthology.orchestra.run.smithi088.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/squid/461a84e6fbbf089011eafd13b98169fb9e9b8cc9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2287-g461a84e6-1jammy [91.4 kB] 2024-04-17T04:33:19.607 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librados2. 2024-04-17T04:33:19.634 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../007-librados2_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:19.665 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librados2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:20.079 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 179 MB in 11s (16.6 MB/s) 2024-04-17T04:33:20.102 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librbd1. 2024-04-17T04:33:20.128 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../008-librbd1_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:20.168 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librbd1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:20.190 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-17T04:33:20.285 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-17T04:33:20.290 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-17T04:33:20.364 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T04:33:20.630 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs2. 2024-04-17T04:33:20.655 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:20.658 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-17T04:33:20.685 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-17T04:33:20.696 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:20.725 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T04:33:20.973 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rados. 2024-04-17T04:33:20.988 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:21.032 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rados (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:21.078 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-17T04:33:21.105 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-17T04:33:21.145 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T04:33:21.326 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-17T04:33:21.352 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:21.393 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-argparse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:21.540 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-17T04:33:21.566 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T04:33:21.612 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cephfs. 2024-04-17T04:33:21.615 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:33:21.627 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:21.662 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cephfs (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:21.982 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-17T04:33:22.001 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-17T04:33:22.007 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:22.028 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T04:33:22.048 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:22.068 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:33:22.423 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-17T04:33:22.449 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-17T04:33:22.489 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:33:22.499 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-17T04:33:22.525 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-17T04:33:22.634 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T04:33:22.809 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-17T04:33:22.824 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-17T04:33:22.859 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T04:33:22.962 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-prettytable. 2024-04-17T04:33:22.988 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-17T04:33:23.029 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-17T04:33:23.179 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librados2. 2024-04-17T04:33:23.205 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../007-librados2_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:23.245 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librados2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:23.265 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rbd. 2024-04-17T04:33:23.280 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:23.323 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rbd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-17T04:33:23.702 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-17T04:33:23.723 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librbd1. 2024-04-17T04:33:23.743 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T04:33:23.749 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../008-librbd1_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:23.789 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librbd1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:24.029 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-17T04:33:24.055 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-17T04:33:24.095 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T04:33:24.209 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcephfs2. 2024-04-17T04:33:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:24.259 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcephfs2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:24.437 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-17T04:33:24.452 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-17T04:33:24.490 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T04:33:24.511 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rados. 2024-04-17T04:33:24.534 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:24.578 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rados (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:24.818 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package lua5.1. 2024-04-17T04:33:24.844 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-17T04:33:24.885 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-17T04:33:24.898 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-17T04:33:24.923 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:24.964 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-ceph-argparse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:25.217 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cephfs. 2024-04-17T04:33:25.230 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package lua-any. 2024-04-17T04:33:25.242 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:25.256 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-17T04:33:25.296 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-17T04:33:25.299 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cephfs (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:25.541 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package zip. 2024-04-17T04:33:25.556 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-17T04:33:25.591 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking zip (3.0-12build2) ... 2024-04-17T04:33:25.594 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-17T04:33:25.609 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:25.644 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:25.869 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package unzip. 2024-04-17T04:33:25.895 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-17T04:33:25.927 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-17T04:33:25.972 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-17T04:33:25.998 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-17T04:33:26.038 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T04:33:26.331 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package luarocks. 2024-04-17T04:33:26.333 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-prettytable. 2024-04-17T04:33:26.357 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-17T04:33:26.359 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-17T04:33:26.398 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-17T04:33:26.399 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-17T04:33:26.635 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rbd. 2024-04-17T04:33:26.658 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:26.727 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rbd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:26.781 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librgw2. 2024-04-17T04:33:26.807 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../025-librgw2_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:26.847 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librgw2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:27.089 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-17T04:33:27.114 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-17T04:33:27.155 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T04:33:27.383 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rgw. 2024-04-17T04:33:27.410 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:27.441 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-17T04:33:27.441 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:27.467 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-17T04:33:27.599 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T04:33:27.794 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-17T04:33:27.821 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T04:33:28.002 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-17T04:33:28.028 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-17T04:33:28.077 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T04:33:28.155 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libradosstriper1. 2024-04-17T04:33:28.182 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:28.222 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libradosstriper1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:28.422 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua5.1. 2024-04-17T04:33:28.449 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-17T04:33:28.488 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-17T04:33:28.617 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-common. 2024-04-17T04:33:28.644 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:28.684 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua-any. 2024-04-17T04:33:28.860 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-17T04:33:28.908 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-17T04:33:29.202 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package zip. 2024-04-17T04:33:29.229 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-17T04:33:29.277 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking zip (3.0-12build2) ... 2024-04-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package unzip. 2024-04-17T04:33:29.632 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-17T04:33:29.672 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-17T04:33:30.092 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package luarocks. 2024-04-17T04:33:30.119 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-17T04:33:30.131 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-base. 2024-04-17T04:33:30.158 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:30.158 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-17T04:33:30.215 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-base (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:30.491 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package librgw2. 2024-04-17T04:33:30.506 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../025-librgw2_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:30.541 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking librgw2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:30.802 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-17T04:33:30.829 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-17T04:33:30.868 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-17T04:33:31.035 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rgw. 2024-04-17T04:33:31.050 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:31.093 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:31.171 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cheroot. 2024-04-17T04:33:31.198 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-17T04:33:31.238 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T04:33:31.438 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-17T04:33:31.504 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T04:33:31.574 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-17T04:33:31.601 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-17T04:33:31.641 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-17T04:33:31.807 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libradosstriper1. 2024-04-17T04:33:31.834 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:31.873 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libradosstriper1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:31.944 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-17T04:33:31.970 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-17T04:33:32.010 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-17T04:33:32.210 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-common. 2024-04-17T04:33:32.237 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:32.268 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:32.271 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-17T04:33:32.287 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-17T04:33:32.322 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-17T04:33:32.624 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempora. 2024-04-17T04:33:32.650 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-17T04:33:32.691 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-17T04:33:33.169 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-portend. 2024-04-17T04:33:33.195 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-17T04:33:33.235 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-17T04:33:33.555 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-17T04:33:33.582 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-17T04:33:33.621 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-17T04:33:33.740 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-base. 2024-04-17T04:33:33.767 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:33.816 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-base (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:33.900 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-17T04:33:33.926 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-17T04:33:33.966 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-17T04:33:34.303 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-natsort. 2024-04-17T04:33:34.330 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-17T04:33:34.370 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-17T04:33:34.377 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-17T04:33:34.404 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-17T04:33:34.444 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-17T04:33:34.690 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-logutils. 2024-04-17T04:33:34.705 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cheroot. 2024-04-17T04:33:34.717 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-17T04:33:34.721 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-17T04:33:34.755 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T04:33:34.756 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-17T04:33:35.017 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-17T04:33:35.033 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-17T04:33:35.041 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-17T04:33:35.057 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-17T04:33:35.067 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-17T04:33:35.091 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-17T04:33:35.354 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-mako. 2024-04-17T04:33:35.378 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-17T04:33:35.380 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-17T04:33:35.404 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-17T04:33:35.420 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T04:33:35.444 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-17T04:33:35.747 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-17T04:33:35.757 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-17T04:33:35.774 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-17T04:33:35.784 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-17T04:33:35.813 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-17T04:33:35.824 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-17T04:33:36.116 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-tempora. 2024-04-17T04:33:36.126 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-17T04:33:36.143 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-17T04:33:36.153 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-17T04:33:36.183 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-17T04:33:36.193 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-17T04:33:36.444 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-portend. 2024-04-17T04:33:36.459 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-17T04:33:36.462 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webob. 2024-04-17T04:33:36.478 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-17T04:33:36.494 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-17T04:33:36.512 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-17T04:33:36.822 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-17T04:33:36.848 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-17T04:33:36.857 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-waitress. 2024-04-17T04:33:36.884 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-17T04:33:36.888 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-17T04:33:36.926 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T04:33:37.174 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-17T04:33:37.201 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-17T04:33:37.241 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-17T04:33:37.243 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempita. 2024-04-17T04:33:37.270 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-17T04:33:37.310 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T04:33:37.613 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-paste. 2024-04-17T04:33:37.640 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-17T04:33:37.679 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T04:33:37.844 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-natsort. 2024-04-17T04:33:37.871 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-17T04:33:37.911 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-17T04:33:38.015 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-17T04:33:38.058 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T04:33:38.172 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-logutils. 2024-04-17T04:33:38.187 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-17T04:33:38.230 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-17T04:33:38.361 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-17T04:33:38.390 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-17T04:33:38.427 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-17T04:33:38.533 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-17T04:33:38.560 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-17T04:33:38.600 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-17T04:33:38.730 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webtest. 2024-04-17T04:33:38.757 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-17T04:33:38.797 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-17T04:33:38.902 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-mako. 2024-04-17T04:33:38.929 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-17T04:33:38.969 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T04:33:39.058 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pecan. 2024-04-17T04:33:39.084 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-17T04:33:39.116 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T04:33:39.298 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-17T04:33:39.324 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-17T04:33:39.364 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-17T04:33:39.421 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-certifi. 2024-04-17T04:33:39.448 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-17T04:33:39.487 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-17T04:33:39.633 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-17T04:33:39.648 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-17T04:33:39.683 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-17T04:33:39.816 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-idna. 2024-04-17T04:33:39.843 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-17T04:33:39.882 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-17T04:33:39.969 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-webob. 2024-04-17T04:33:39.995 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-17T04:33:40.052 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-17T04:33:40.194 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-urllib3. 2024-04-17T04:33:40.221 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-17T04:33:40.260 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T04:33:40.397 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-waitress. 2024-04-17T04:33:40.423 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-17T04:33:40.467 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T04:33:40.572 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests. 2024-04-17T04:33:40.599 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-17T04:33:40.638 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T04:33:40.758 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-tempita. 2024-04-17T04:33:40.784 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-17T04:33:40.825 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T04:33:40.966 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-17T04:33:40.993 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-17T04:33:41.033 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T04:33:41.127 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-paste. 2024-04-17T04:33:41.154 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-17T04:33:41.194 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T04:33:41.327 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-17T04:33:41.343 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:41.377 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:41.523 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-17T04:33:41.549 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-17T04:33:41.589 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T04:33:41.766 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-17T04:33:41.792 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:41.832 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:41.891 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-17T04:33:41.918 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-17T04:33:41.958 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-17T04:33:42.786 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-17T04:33:42.812 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-17T04:33:43.002 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-17T04:33:43.078 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-webtest. 2024-04-17T04:33:43.093 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-17T04:33:43.136 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-17T04:33:43.305 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-openssl. 2024-04-17T04:33:43.332 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-17T04:33:43.473 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pecan. 2024-04-17T04:33:43.499 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-17T04:33:43.539 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T04:33:43.553 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-17T04:33:43.902 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-certifi. 2024-04-17T04:33:43.916 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr. 2024-04-17T04:33:43.929 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-17T04:33:43.943 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:43.968 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-17T04:33:43.983 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:44.297 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-idna. 2024-04-17T04:33:44.323 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-17T04:33:44.353 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mon. 2024-04-17T04:33:44.363 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-17T04:33:44.380 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:44.419 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mon (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:44.658 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-urllib3. 2024-04-17T04:33:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-17T04:33:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T04:33:45.027 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-requests. 2024-04-17T04:33:45.040 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-osd. 2024-04-17T04:33:45.042 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-17T04:33:45.066 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:45.077 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T04:33:45.106 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-osd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:45.347 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-17T04:33:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-17T04:33:45.405 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T04:33:45.683 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-17T04:33:45.698 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:45.733 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:46.121 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-17T04:33:46.147 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:46.176 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph. 2024-04-17T04:33:46.187 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:46.203 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../066-ceph_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:46.243 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:46.507 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-17T04:33:46.528 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-fuse. 2024-04-17T04:33:46.533 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-17T04:33:46.544 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:46.573 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-17T04:33:46.578 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:46.851 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-openssl. 2024-04-17T04:33:46.866 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-17T04:33:46.901 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-17T04:33:46.932 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mds. 2024-04-17T04:33:46.959 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:46.998 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mds (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:47.229 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr. 2024-04-17T04:33:47.255 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:47.295 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:47.393 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephadm. 2024-04-17T04:33:47.420 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../069-cephadm_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:47.460 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:47.690 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mon. 2024-04-17T04:33:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:47.757 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mon (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:47.779 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-17T04:33:47.805 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-17T04:33:47.846 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-17T04:33:48.190 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-17T04:33:48.217 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T04:33:48.257 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T04:33:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-osd. 2024-04-17T04:33:48.370 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:48.410 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-osd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:49.328 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-babel. 2024-04-17T04:33:49.356 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T04:33:49.395 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T04:33:49.447 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph. 2024-04-17T04:33:49.474 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../066-ceph_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:49.513 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:49.723 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jinja2. 2024-04-17T04:33:49.750 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-17T04:33:49.789 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T04:33:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-fuse. 2024-04-17T04:33:49.867 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:49.907 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:50.159 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-17T04:33:50.187 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:50.226 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:50.252 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mds. 2024-04-17T04:33:50.278 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:50.310 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mds (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:50.562 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-17T04:33:50.589 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-17T04:33:50.629 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-17T04:33:50.705 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package cephadm. 2024-04-17T04:33:50.731 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../069-cephadm_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:33:50.771 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:50.948 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-routes. 2024-04-17T04:33:50.975 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-17T04:33:51.015 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T04:33:51.107 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-17T04:33:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-17T04:33:51.174 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-17T04:33:51.352 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-17T04:33:51.378 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:51.418 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:51.510 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-17T04:33:51.525 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T04:33:51.568 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T04:33:52.598 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-babel. 2024-04-17T04:33:52.625 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-17T04:33:52.664 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T04:33:52.698 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-17T04:33:52.714 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-17T04:33:52.765 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T04:33:52.992 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-jinja2. 2024-04-17T04:33:53.019 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-17T04:33:53.092 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T04:33:53.269 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-joblib. 2024-04-17T04:33:53.295 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-17T04:33:53.327 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T04:33:53.495 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-17T04:33:53.522 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:53.562 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:53.680 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-17T04:33:53.707 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-17T04:33:53.747 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-17T04:33:53.915 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-17T04:33:53.942 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-17T04:33:53.981 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-17T04:33:54.250 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn. 2024-04-17T04:33:54.267 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-routes. 2024-04-17T04:33:54.277 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-17T04:33:54.295 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-17T04:33:54.316 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T04:33:54.334 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T04:33:54.654 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-17T04:33:54.681 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:54.720 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-17T04:33:55.038 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:55.154 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:56.017 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-17T04:33:56.033 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-17T04:33:56.067 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T04:33:56.124 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cachetools. 2024-04-17T04:33:56.151 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-17T04:33:56.190 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-17T04:33:56.521 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-17T04:33:56.561 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-17T04:33:56.570 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-joblib. 2024-04-17T04:33:56.597 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-17T04:33:56.659 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T04:33:56.889 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-17T04:33:56.916 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-17T04:33:56.955 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T04:33:57.015 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-17T04:33:57.042 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-17T04:33:57.081 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-17T04:33:57.300 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rsa. 2024-04-17T04:33:57.327 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-17T04:33:57.366 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-17T04:33:57.367 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-sklearn. 2024-04-17T04:33:57.395 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-17T04:33:57.425 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T04:33:57.670 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-google-auth. 2024-04-17T04:33:57.697 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-17T04:33:57.736 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-17T04:33:58.056 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-17T04:33:58.083 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-17T04:33:58.084 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-17T04:33:58.112 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:58.123 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T04:33:58.443 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-websocket. 2024-04-17T04:33:58.452 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:58.470 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-17T04:33:58.509 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-17T04:33:58.805 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-17T04:33:58.831 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-17T04:33:58.871 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T04:33:59.414 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-cachetools. 2024-04-17T04:33:59.430 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-17T04:33:59.464 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-17T04:33:59.510 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-17T04:33:59.526 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:33:59.560 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:33:59.750 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-17T04:33:59.778 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-17T04:33:59.818 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-17T04:33:59.820 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-17T04:33:59.837 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-17T04:33:59.879 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T04:34:00.179 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-17T04:34:00.206 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-17T04:34:00.224 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-17T04:34:00.245 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T04:34:00.251 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T04:34:00.551 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package jq. 2024-04-17T04:34:00.558 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-rsa. 2024-04-17T04:34:00.579 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T04:34:00.586 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-17T04:34:00.624 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-17T04:34:00.871 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package socat. 2024-04-17T04:34:00.886 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-17T04:34:00.895 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-google-auth. 2024-04-17T04:34:00.911 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-17T04:34:00.929 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-17T04:34:00.945 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-17T04:34:01.248 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-17T04:34:01.249 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package xmlstarlet. 2024-04-17T04:34:01.265 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-17T04:34:01.275 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-17T04:34:01.314 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T04:34:01.332 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-17T04:34:01.626 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-websocket. 2024-04-17T04:34:01.653 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-17T04:34:01.686 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-test. 2024-04-17T04:34:01.692 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-17T04:34:01.702 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:01.744 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-test (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:02.004 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-17T04:34:02.031 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-17T04:34:02.070 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T04:34:02.709 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-17T04:34:02.736 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:34:02.775 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:03.095 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-17T04:34:03.122 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-17T04:34:03.161 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T04:34:03.456 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-17T04:34:03.483 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T04:34:03.514 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T04:34:03.917 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package jq. 2024-04-17T04:34:03.944 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-17T04:34:04.149 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-volume. 2024-04-17T04:34:04.177 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:34:04.223 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-volume (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:04.236 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package socat. 2024-04-17T04:34:04.252 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-17T04:34:04.294 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-17T04:34:04.589 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package xmlstarlet. 2024-04-17T04:34:04.616 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-17T04:34:04.664 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-17T04:34:04.777 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-17T04:34:04.805 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:04.853 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs-dev (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:05.018 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-test. 2024-04-17T04:34:05.045 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:05.085 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-test (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:05.239 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-17T04:34:05.266 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-17T04:34:05.313 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T04:34:05.702 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-17T04:34:05.729 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-17T04:34:05.768 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-17T04:34:06.130 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package nvme-cli. 2024-04-17T04:34:06.158 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-17T04:34:06.196 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T04:34:06.627 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package pkg-config. 2024-04-17T04:34:06.655 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-17T04:34:06.693 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T04:34:06.938 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-17T04:34:06.954 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-17T04:34:06.996 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-17T04:34:07.397 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package ceph-volume. 2024-04-17T04:34:07.416 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-17T04:34:07.424 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2287-g461a84e6-1jammy_all.deb ... 2024-04-17T04:34:07.444 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-17T04:34:07.472 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking ceph-volume (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:07.482 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-17T04:34:07.759 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-17T04:34:07.760 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastescript. 2024-04-17T04:34:07.786 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:07.788 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-17T04:34:07.825 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcephfs-dev (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:07.827 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-17T04:34:08.122 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pluggy. 2024-04-17T04:34:08.128 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-17T04:34:08.138 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-17T04:34:08.155 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-17T04:34:08.180 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-17T04:34:08.194 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T04:34:08.467 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-psutil. 2024-04-17T04:34:08.494 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-17T04:34:08.533 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-17T04:34:08.541 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-17T04:34:08.568 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-17T04:34:08.615 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-17T04:34:08.886 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-py. 2024-04-17T04:34:08.914 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-17T04:34:08.952 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-17T04:34:09.061 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package nvme-cli. 2024-04-17T04:34:09.088 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-17T04:34:09.136 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T04:34:09.281 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pygments. 2024-04-17T04:34:09.308 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-17T04:34:09.347 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T04:34:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package pkg-config. 2024-04-17T04:34:09.602 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-17T04:34:09.641 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T04:34:09.877 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-17T04:34:09.905 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-17T04:34:09.918 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-17T04:34:09.934 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-17T04:34:09.943 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-17T04:34:10.002 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-17T04:34:10.254 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-toml. 2024-04-17T04:34:10.282 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-17T04:34:10.321 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-17T04:34:10.422 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-17T04:34:10.449 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-17T04:34:10.488 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-17T04:34:10.624 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pytest. 2024-04-17T04:34:10.652 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-17T04:34:10.690 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T04:34:10.808 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pastescript. 2024-04-17T04:34:10.835 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-17T04:34:10.882 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-17T04:34:11.044 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplejson. 2024-04-17T04:34:11.071 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-17T04:34:11.110 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-17T04:34:11.261 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pluggy. 2024-04-17T04:34:11.289 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-17T04:34:11.335 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-17T04:34:11.338 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-17T04:34:11.354 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-17T04:34:11.396 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-17T04:34:11.714 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-psutil. 2024-04-17T04:34:11.741 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-17T04:34:11.788 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-17T04:34:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-py. 2024-04-17T04:34:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-17T04:34:12.159 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package radosgw. 2024-04-17T04:34:12.183 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-17T04:34:12.186 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../116-radosgw_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:12.225 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking radosgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:12.477 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pygments. 2024-04-17T04:34:12.494 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-17T04:34:12.535 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T04:34:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-17T04:34:13.009 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-17T04:34:13.037 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package rbd-fuse. 2024-04-17T04:34:13.048 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-17T04:34:13.064 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:13.103 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking rbd-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:13.392 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-toml. 2024-04-17T04:34:13.407 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-17T04:34:13.420 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-17T04:34:13.459 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-17T04:34:13.649 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T04:34:13.761 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-pytest. 2024-04-17T04:34:13.776 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T04:34:13.789 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-17T04:34:13.828 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T04:34:14.036 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-17T04:34:14.182 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-simplejson. 2024-04-17T04:34:14.210 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-17T04:34:14.248 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-17T04:34:14.361 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-17T04:34:14.517 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-17T04:34:14.545 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-17T04:34:14.584 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-17T04:34:14.778 INFO:teuthology.orchestra.run.smithi154.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-17T04:34:14.863 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:15.279 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package radosgw. 2024-04-17T04:34:15.307 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../116-radosgw_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:15.345 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking radosgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:15.577 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user cephadm....done 2024-04-17T04:34:15.725 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T04:34:16.010 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-17T04:34:16.132 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package rbd-fuse. 2024-04-17T04:34:16.160 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2287-g461a84e6-1jammy_amd64.deb ... 2024-04-17T04:34:16.198 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking rbd-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:16.270 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-17T04:34:16.396 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-17T04:34:16.503 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-17T04:34:16.647 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-17T04:34:16.753 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-17T04:34:16.879 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-17T04:34:16.881 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T04:34:16.991 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-17T04:34:17.140 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-17T04:34:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T04:34:17.479 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-17T04:34:17.653 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-17T04:34:17.902 INFO:teuthology.orchestra.run.smithi088.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-17T04:34:17.929 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-17T04:34:17.964 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:18.197 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-17T04:34:18.424 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-17T04:34:18.430 INFO:teuthology.orchestra.run.smithi088.stdout:Adding system user cephadm....done 2024-04-17T04:34:18.578 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-17T04:34:18.658 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-17T04:34:18.855 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-17T04:34:18.909 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-17T04:34:19.097 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-17T04:34:19.160 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-argparse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:19.199 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-17T04:34:19.433 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-17T04:34:19.528 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T04:34:19.654 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T04:34:19.684 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-17T04:34:19.780 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T04:34:19.810 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-17T04:34:19.906 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-17T04:34:20.111 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-17T04:34:20.149 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-17T04:34:20.464 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-17T04:34:20.525 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-17T04:34:20.555 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-17T04:34:20.619 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T04:34:20.731 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-17T04:34:20.745 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-17T04:34:20.982 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-17T04:34:21.096 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T04:34:21.197 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-17T04:34:21.217 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-17T04:34:21.448 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T04:34:21.460 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-17T04:34:21.716 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-17T04:34:21.719 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-17T04:34:21.970 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-ceph-argparse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:22.042 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up zip (3.0-12build2) ... 2024-04-17T04:34:22.177 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T04:34:22.213 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-17T04:34:22.414 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-17T04:34:22.540 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-17T04:34:22.666 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-17T04:34:22.811 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T04:34:22.909 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-17T04:34:23.046 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T04:34:23.164 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-17T04:34:23.283 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-17T04:34:23.289 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T04:34:23.339 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-17T04:34:23.395 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-17T04:34:23.864 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-17T04:34:23.933 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-17T04:34:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-17T04:34:24.176 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T04:34:24.216 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-17T04:34:24.476 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-17T04:34:24.536 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T04:34:24.793 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up zip (3.0-12build2) ... 2024-04-17T04:34:24.812 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T04:34:24.919 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-17T04:34:25.172 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-17T04:34:25.415 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-17T04:34:25.533 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:25.580 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-17T04:34:25.839 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-17T04:34:25.843 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-idna (3.3-1) ... 2024-04-17T04:34:25.965 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-17T04:34:26.091 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-17T04:34:26.186 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T04:34:26.392 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-17T04:34:26.776 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-17T04:34:27.027 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-17T04:34:27.299 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T04:34:27.396 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-17T04:34:27.551 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:34:27.672 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-17T04:34:27.685 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-17T04:34:27.945 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T04:34:27.998 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-17T04:34:28.071 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-17T04:34:28.197 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T04:34:28.241 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-17T04:34:28.367 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:28.482 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-17T04:34:28.668 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-idna (3.3-1) ... 2024-04-17T04:34:28.724 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-17T04:34:28.984 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:34:28.986 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-17T04:34:29.110 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-17T04:34:29.436 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-17T04:34:29.687 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-17T04:34:29.938 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-17T04:34:30.159 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-17T04:34:30.214 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-17T04:34:30.410 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:34:30.474 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-17T04:34:30.544 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-17T04:34:30.733 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T04:34:30.812 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-17T04:34:30.868 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-17T04:34:30.938 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-17T04:34:31.039 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-17T04:34:31.144 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T04:34:31.253 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T04:34:31.324 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-17T04:34:31.538 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-17T04:34:31.567 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-17T04:34:31.818 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:34:31.839 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-17T04:34:31.944 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-17T04:34:32.090 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T04:34:32.191 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-17T04:34:32.253 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-17T04:34:32.317 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T04:34:32.414 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-17T04:34:32.513 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-17T04:34:32.660 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T04:34:32.772 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-17T04:34:32.778 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T04:34:33.032 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-17T04:34:33.155 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-17T04:34:33.291 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-17T04:34:33.397 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-17T04:34:33.507 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-17T04:34:33.551 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-17T04:34:33.676 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-17T04:34:33.758 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:34:33.892 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T04:34:33.944 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-17T04:34:34.045 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-17T04:34:34.210 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T04:34:34.330 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-17T04:34:34.511 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T04:34:34.640 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-17T04:34:34.779 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-17T04:34:34.880 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-17T04:34:34.890 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-17T04:34:35.008 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-17T04:34:35.134 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-17T04:34:35.139 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-17T04:34:35.223 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-17T04:34:35.407 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T04:34:35.452 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-17T04:34:35.554 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-17T04:34:35.777 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-17T04:34:35.938 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-17T04:34:36.154 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-17T04:34:36.181 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-17T04:34:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-17T04:34:36.430 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T04:34:36.558 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-17T04:34:36.709 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-17T04:34:36.774 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T04:34:36.901 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librados2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:37.027 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-17T04:34:37.027 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:37.153 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-17T04:34:37.328 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-17T04:34:37.489 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:37.598 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libradosstriper1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:37.612 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-17T04:34:37.708 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-17T04:34:37.730 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-17T04:34:37.973 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-17T04:34:37.993 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librbd1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:38.119 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-modules-core (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:38.224 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-17T04:34:38.270 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:38.482 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T04:34:38.482 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T04:34:38.569 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-17T04:34:38.938 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs-dev (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:38.945 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-17T04:34:39.064 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rados (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:39.190 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librgw2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:39.264 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-17T04:34:39.316 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T04:34:39.599 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-17T04:34:39.725 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librados2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:39.851 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.002 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-17T04:34:40.364 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcephfs2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.515 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libradosstriper1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.544 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rbd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.665 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-17T04:34:40.670 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up rbd-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.796 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.914 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cephfs (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:40.958 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librbd1 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:41.015 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:41.076 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-modules-core (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:41.177 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:41.374 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T04:34:41.375 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-17T04:34:41.543 INFO:teuthology.orchestra.run.smithi154.stdout:Adding group ceph....done 2024-04-17T04:34:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcephfs-dev (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:41.920 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user ceph....done 2024-04-17T04:34:42.013 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rados (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:42.080 INFO:teuthology.orchestra.run.smithi154.stdout:Setting system user ceph properties....done 2024-04-17T04:34:42.096 INFO:teuthology.orchestra.run.smithi154.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-17T04:34:42.164 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up librgw2 (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:42.230 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-17T04:34:42.315 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-17T04:34:42.561 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-17T04:34:42.993 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-test (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:43.111 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up radosgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:43.538 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T04:34:43.538 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T04:34:43.577 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rbd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:43.703 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up rbd-fuse (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:43.837 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-rgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:43.988 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-cephfs (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:44.013 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-base (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:44.139 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-common (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:44.366 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-17T04:34:44.732 INFO:teuthology.orchestra.run.smithi088.stdout:Adding group ceph....done 2024-04-17T04:34:44.848 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mds (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:45.064 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T04:34:45.064 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T04:34:45.109 INFO:teuthology.orchestra.run.smithi088.stdout:Adding system user ceph....done 2024-04-17T04:34:45.311 INFO:teuthology.orchestra.run.smithi088.stdout:Setting system user ceph properties....done 2024-04-17T04:34:45.330 INFO:teuthology.orchestra.run.smithi088.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-17T04:34:45.468 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-17T04:34:45.533 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T04:34:45.747 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T04:34:45.804 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-17T04:34:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-osd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:46.290 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-test (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:46.433 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up radosgw (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:46.532 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T04:34:46.533 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T04:34:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T04:34:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-17T04:34:47.011 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:47.137 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:47.272 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mon (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-base (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:47.475 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T04:34:47.475 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T04:34:47.807 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-17T04:34:47.989 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:48.124 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:48.250 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-dashboard (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:48.264 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mds (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:48.401 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-volume (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:48.505 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T04:34:48.505 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-17T04:34:48.746 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T04:34:48.838 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T04:34:48.966 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:48.974 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T04:34:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T04:34:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-17T04:34:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-osd (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:49.932 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T04:34:49.932 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-17T04:34:50.116 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T04:34:50.494 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:50.654 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:50.781 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T04:34:50.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-04-17T04:34:50.834 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2024-04-17T04:34:51.011 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2024-04-17T04:34:51.011 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2024-04-17T04:34:51.139 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mon (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:51.175 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:34:51.175 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:34:51.175 INFO:teuthology.orchestra.run.smithi154.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:34:51.175 INFO:teuthology.orchestra.run.smithi154.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:34:51.175 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:34:51.176 INFO:teuthology.orchestra.run.smithi154.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:34:51.176 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:34:51.177 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2024-04-17T04:34:51.177 INFO:teuthology.orchestra.run.smithi154.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-04-17T04:34:51.177 INFO:teuthology.orchestra.run.smithi154.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-04-17T04:34:51.177 INFO:teuthology.orchestra.run.smithi154.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-04-17T04:34:51.177 INFO:teuthology.orchestra.run.smithi154.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-04-17T04:34:51.178 INFO:teuthology.orchestra.run.smithi154.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-04-17T04:34:51.179 INFO:teuthology.orchestra.run.smithi154.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-04-17T04:34:51.179 INFO:teuthology.orchestra.run.smithi154.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-04-17T04:34:51.179 INFO:teuthology.orchestra.run.smithi154.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-04-17T04:34:51.216 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-04-17T04:34:51.217 INFO:teuthology.orchestra.run.smithi154.stdout: s3cmd 2024-04-17T04:34:51.274 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 35 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T04:34:51.274 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 10.4 MB of archives. 2024-04-17T04:34:51.274 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-04-17T04:34:51.274 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-04-17T04:34:51.294 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-04-17T04:34:51.294 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-04-17T04:34:51.298 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-04-17T04:34:51.338 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-04-17T04:34:51.355 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-04-17T04:34:51.355 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T04:34:51.356 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-17T04:34:51.357 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-04-17T04:34:51.391 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-04-17T04:34:51.415 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-04-17T04:34:51.416 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-04-17T04:34:51.417 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-04-17T04:34:51.468 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-04-17T04:34:51.469 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-04-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-04-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-04-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-04-17T04:34:51.488 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-04-17T04:34:51.488 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-04-17T04:34:51.490 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-04-17T04:34:51.491 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-04-17T04:34:51.496 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-04-17T04:34:51.498 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-04-17T04:34:51.510 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-04-17T04:34:51.511 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-04-17T04:34:51.511 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-04-17T04:34:51.512 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-04-17T04:34:51.513 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-04-17T04:34:51.529 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-04-17T04:34:51.533 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-04-17T04:34:51.535 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-04-17T04:34:51.549 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-04-17T04:34:51.549 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-04-17T04:34:51.569 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-04-17T04:34:51.569 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-04-17T04:34:51.570 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-04-17T04:34:51.865 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-cephadm (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:52.001 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 10.4 MB in 0s (30.0 MB/s) 2024-04-17T04:34:52.016 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:52.089 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libapache-pom-java. 2024-04-17T04:34:52.117 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-mgr-dashboard (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:52.170 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173890 files and directories currently installed.) 2024-04-17T04:34:52.176 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-04-17T04:34:52.229 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libapache-pom-java (18-1) ... 2024-04-17T04:34:52.251 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up ceph-volume (19.0.0-2287-g461a84e6-1jammy) ... 2024-04-17T04:34:52.490 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-04-17T04:34:52.506 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-04-17T04:34:52.549 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-04-17T04:34:52.595 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T04:34:52.714 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T04:34:52.776 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-04-17T04:34:52.792 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-04-17T04:34:52.826 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-04-17T04:34:52.866 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T04:34:53.071 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcdi-api-java. 2024-04-17T04:34:53.098 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-04-17T04:34:53.137 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-04-17T04:34:53.398 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcommons-cli-java. 2024-04-17T04:34:53.414 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-04-17T04:34:53.456 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-04-17T04:34:53.709 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcommons-parent-java. 2024-04-17T04:34:53.737 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-04-17T04:34:53.775 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-04-17T04:34:54.003 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcommons-io-java. 2024-04-17T04:34:54.019 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-04-17T04:34:54.061 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-04-17T04:34:54.175 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-17T04:34:54.330 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-04-17T04:34:54.358 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-04-17T04:34:54.397 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-04-17T04:34:54.708 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-04-17T04:34:54.723 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-04-17T04:34:54.766 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-04-17T04:34:54.835 INFO:teuthology.orchestra.run.smithi088.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T04:34:54.837 DEBUG:teuthology.orchestra.run.smithi088:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-04-17T04:34:54.893 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:34:55.019 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjsr305-java. 2024-04-17T04:34:55.047 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-04-17T04:34:55.085 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-04-17T04:34:55.101 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-04-17T04:34:55.102 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-04-17T04:34:55.283 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:34:55.283 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:34:55.283 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:34:55.283 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:34:55.283 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:34:55.284 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:34:55.284 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-04-17T04:34:55.285 INFO:teuthology.orchestra.run.smithi088.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-04-17T04:34:55.286 INFO:teuthology.orchestra.run.smithi088.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-04-17T04:34:55.287 INFO:teuthology.orchestra.run.smithi088.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-04-17T04:34:55.313 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libguava-java. 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-04-17T04:34:55.324 INFO:teuthology.orchestra.run.smithi088.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-04-17T04:34:55.325 INFO:teuthology.orchestra.run.smithi088.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-04-17T04:34:55.325 INFO:teuthology.orchestra.run.smithi088.stdout: s3cmd 2024-04-17T04:34:55.329 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-04-17T04:34:55.371 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libguava-java (29.0-6) ... 2024-04-17T04:34:55.384 INFO:teuthology.orchestra.run.smithi088.stdout:0 upgraded, 35 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T04:34:55.384 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 10.4 MB of archives. 2024-04-17T04:34:55.384 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-04-17T04:34:55.384 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-04-17T04:34:55.404 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-04-17T04:34:55.405 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-04-17T04:34:55.408 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-04-17T04:34:55.447 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-04-17T04:34:55.465 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-04-17T04:34:55.467 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-04-17T04:34:55.500 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-04-17T04:34:55.523 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-04-17T04:34:55.524 INFO:teuthology.orchestra.run.smithi088.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-04-17T04:34:55.525 INFO:teuthology.orchestra.run.smithi088.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-04-17T04:34:55.575 INFO:teuthology.orchestra.run.smithi088.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-04-17T04:34:55.576 INFO:teuthology.orchestra.run.smithi088.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-04-17T04:34:55.593 INFO:teuthology.orchestra.run.smithi088.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-04-17T04:34:55.593 INFO:teuthology.orchestra.run.smithi088.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-04-17T04:34:55.594 INFO:teuthology.orchestra.run.smithi088.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-04-17T04:34:55.594 INFO:teuthology.orchestra.run.smithi088.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-04-17T04:34:55.595 INFO:teuthology.orchestra.run.smithi088.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-04-17T04:34:55.598 INFO:teuthology.orchestra.run.smithi088.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-04-17T04:34:55.598 INFO:teuthology.orchestra.run.smithi088.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-04-17T04:34:55.604 INFO:teuthology.orchestra.run.smithi088.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-04-17T04:34:55.617 INFO:teuthology.orchestra.run.smithi088.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-04-17T04:34:55.617 INFO:teuthology.orchestra.run.smithi088.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-04-17T04:34:55.618 INFO:teuthology.orchestra.run.smithi088.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-04-17T04:34:55.619 INFO:teuthology.orchestra.run.smithi088.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-04-17T04:34:55.620 INFO:teuthology.orchestra.run.smithi088.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-04-17T04:34:55.620 INFO:teuthology.orchestra.run.smithi088.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-04-17T04:34:55.637 INFO:teuthology.orchestra.run.smithi088.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-04-17T04:34:55.640 INFO:teuthology.orchestra.run.smithi088.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-04-17T04:34:55.642 INFO:teuthology.orchestra.run.smithi088.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-04-17T04:34:55.657 INFO:teuthology.orchestra.run.smithi088.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-04-17T04:34:55.657 INFO:teuthology.orchestra.run.smithi088.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-04-17T04:34:55.675 INFO:teuthology.orchestra.run.smithi088.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-04-17T04:34:55.675 INFO:teuthology.orchestra.run.smithi088.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-04-17T04:34:55.675 INFO:teuthology.orchestra.run.smithi088.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-04-17T04:34:55.799 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libaopalliance-java. 2024-04-17T04:34:55.827 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-04-17T04:34:55.891 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-04-17T04:34:56.102 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 10.4 MB in 0s (30.2 MB/s) 2024-04-17T04:34:56.143 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libguice-java. 2024-04-17T04:34:56.171 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-04-17T04:34:56.181 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libapache-pom-java. 2024-04-17T04:34:56.209 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-04-17T04:34:56.260 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173890 files and directories currently installed.) 2024-04-17T04:34:56.266 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-04-17T04:34:56.313 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libapache-pom-java (18-1) ... 2024-04-17T04:34:56.599 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-04-17T04:34:56.615 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-04-17T04:34:56.631 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-04-17T04:34:56.648 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-04-17T04:34:56.657 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-04-17T04:34:56.690 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-04-17T04:34:56.893 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-04-17T04:34:56.909 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-04-17T04:34:56.934 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjansi-native-java. 2024-04-17T04:34:56.943 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-04-17T04:34:56.962 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-04-17T04:34:57.000 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-04-17T04:34:57.171 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcdi-api-java. 2024-04-17T04:34:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-04-17T04:34:57.228 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjansi-java. 2024-04-17T04:34:57.237 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-04-17T04:34:57.244 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-04-17T04:34:57.278 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjansi-java (1.18-1) ... 2024-04-17T04:34:57.497 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmaven-parent-java. 2024-04-17T04:34:57.498 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcommons-cli-java. 2024-04-17T04:34:57.514 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-04-17T04:34:57.525 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-04-17T04:34:57.557 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-04-17T04:34:57.564 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-04-17T04:34:57.800 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-04-17T04:34:57.809 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcommons-parent-java. 2024-04-17T04:34:57.827 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-04-17T04:34:57.837 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-04-17T04:34:57.866 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-04-17T04:34:57.876 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-04-17T04:34:58.128 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcommons-io-java. 2024-04-17T04:34:58.152 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-04-17T04:34:58.155 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-04-17T04:34:58.180 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-04-17T04:34:58.195 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-04-17T04:34:58.227 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-04-17T04:34:58.447 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-04-17T04:34:58.463 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-04-17T04:34:58.464 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-04-17T04:34:58.480 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-04-17T04:34:58.496 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-04-17T04:34:58.522 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-04-17T04:34:58.808 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-04-17T04:34:58.824 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-04-17T04:34:58.842 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-04-17T04:34:58.858 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-04-17T04:34:58.858 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-04-17T04:34:58.901 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-04-17T04:34:59.094 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjsr305-java. 2024-04-17T04:34:59.121 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-04-17T04:34:59.160 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-04-17T04:34:59.170 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-04-17T04:34:59.198 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-04-17T04:34:59.237 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-04-17T04:34:59.413 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libguava-java. 2024-04-17T04:34:59.429 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-04-17T04:34:59.472 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libguava-java (29.0-6) ... 2024-04-17T04:34:59.489 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-04-17T04:34:59.505 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-04-17T04:34:59.564 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-04-17T04:34:59.817 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-04-17T04:34:59.833 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-04-17T04:34:59.875 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-04-17T04:34:59.933 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libaopalliance-java. 2024-04-17T04:34:59.960 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-04-17T04:34:59.999 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-04-17T04:35:00.119 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-04-17T04:35:00.147 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-04-17T04:35:00.178 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-04-17T04:35:00.438 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-04-17T04:35:00.454 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-04-17T04:35:00.497 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-04-17T04:35:00.749 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libslf4j-java. 2024-04-17T04:35:00.765 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-04-17T04:35:00.807 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-04-17T04:35:00.829 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libguice-java. 2024-04-17T04:35:00.857 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-04-17T04:35:00.904 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-04-17T04:35:01.086 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsisu-inject-java. 2024-04-17T04:35:01.114 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-04-17T04:35:01.153 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-04-17T04:35:01.308 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-04-17T04:35:01.325 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-04-17T04:35:01.367 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-04-17T04:35:01.422 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-04-17T04:35:01.440 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-04-17T04:35:01.599 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-04-17T04:35:01.636 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjansi-native-java. 2024-04-17T04:35:01.652 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-04-17T04:35:01.694 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-04-17T04:35:01.877 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmaven3-core-java. 2024-04-17T04:35:01.894 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-04-17T04:35:01.935 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-04-17T04:35:01.997 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libjansi-java. 2024-04-17T04:35:02.025 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-04-17T04:35:02.072 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libjansi-java (1.18-1) ... 2024-04-17T04:35:02.323 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libwagon-file-java. 2024-04-17T04:35:02.351 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-04-17T04:35:02.357 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmaven-parent-java. 2024-04-17T04:35:02.374 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-04-17T04:35:02.389 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-04-17T04:35:02.416 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-04-17T04:35:02.625 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-04-17T04:35:02.653 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-04-17T04:35:02.668 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-04-17T04:35:02.683 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-04-17T04:35:02.696 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-04-17T04:35:02.735 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-04-17T04:35:03.021 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-04-17T04:35:03.049 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-04-17T04:35:03.088 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-04-17T04:35:03.136 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package maven. 2024-04-17T04:35:03.164 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-04-17T04:35:03.211 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking maven (3.6.3-5) ... 2024-04-17T04:35:03.326 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-04-17T04:35:03.339 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-04-17T04:35:03.382 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-04-17T04:35:03.523 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-magic. 2024-04-17T04:35:03.550 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-04-17T04:35:03.589 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-04-17T04:35:03.694 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-04-17T04:35:03.710 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-04-17T04:35:03.752 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-04-17T04:35:03.867 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package s3cmd. 2024-04-17T04:35:03.895 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-04-17T04:35:03.933 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-04-17T04:35:04.030 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-04-17T04:35:04.059 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-04-17T04:35:04.105 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-04-17T04:35:04.296 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-04-17T04:35:04.391 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-04-17T04:35:04.419 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-04-17T04:35:04.422 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-04-17T04:35:04.458 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-04-17T04:35:04.549 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-04-17T04:35:04.674 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-04-17T04:35:04.710 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-04-17T04:35:04.726 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-04-17T04:35:04.768 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-04-17T04:35:04.792 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-04-17T04:35:04.901 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-04-17T04:35:05.002 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-04-17T04:35:05.038 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-04-17T04:35:05.066 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-04-17T04:35:05.113 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-04-17T04:35:05.137 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-04-17T04:35:05.263 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-04-17T04:35:05.389 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-04-17T04:35:05.440 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-04-17T04:35:05.469 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-04-17T04:35:05.515 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libapache-pom-java (18-1) ... 2024-04-17T04:35:05.515 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-04-17T04:35:05.641 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-04-17T04:35:05.767 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-04-17T04:35:05.818 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libslf4j-java. 2024-04-17T04:35:05.847 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-04-17T04:35:05.884 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-04-17T04:35:05.885 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-04-17T04:35:05.994 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-04-17T04:35:06.103 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-04-17T04:35:06.180 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libsisu-inject-java. 2024-04-17T04:35:06.197 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-04-17T04:35:06.239 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-04-17T04:35:06.363 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-04-17T04:35:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-04-17T04:35:06.583 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-04-17T04:35:06.612 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-04-17T04:35:06.658 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-04-17T04:35:06.741 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-04-17T04:35:06.892 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmaven-parent-java (31-2) ... 2024-04-17T04:35:06.986 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libmaven3-core-java. 2024-04-17T04:35:07.010 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-04-17T04:35:07.018 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcommons-parent-java (43-1) ... 2024-04-17T04:35:07.052 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-04-17T04:35:07.135 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-04-17T04:35:07.253 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up s3cmd (2.2.0-1) ... 2024-04-17T04:35:07.432 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libwagon-file-java. 2024-04-17T04:35:07.460 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-04-17T04:35:07.498 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-04-17T04:35:07.563 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-04-17T04:35:07.689 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-04-17T04:35:07.751 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-04-17T04:35:07.767 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-04-17T04:35:07.809 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-04-17T04:35:07.815 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libguava-java (29.0-6) ... 2024-04-17T04:35:07.941 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-04-17T04:35:08.067 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-04-17T04:35:08.176 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-04-17T04:35:08.237 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package maven. 2024-04-17T04:35:08.265 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-04-17T04:35:08.285 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-04-17T04:35:08.303 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking maven (3.6.3-5) ... 2024-04-17T04:35:08.403 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libguice-java (4.2.3-2) ... 2024-04-17T04:35:08.529 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjansi-java (1.18-1) ... 2024-04-17T04:35:08.590 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package python3-magic. 2024-04-17T04:35:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-04-17T04:35:08.658 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-04-17T04:35:08.658 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-04-17T04:35:08.781 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-04-17T04:35:08.907 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up maven (3.6.3-5) ... 2024-04-17T04:35:08.943 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package s3cmd. 2024-04-17T04:35:08.971 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-04-17T04:35:09.009 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-04-17T04:35:09.181 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-04-17T04:35:09.244 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T04:35:09.397 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-04-17T04:35:09.523 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-04-17T04:35:09.649 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-04-17T04:35:09.775 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-04-17T04:35:09.926 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-04-17T04:35:10.094 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-04-17T04:35:10.245 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-04-17T04:35:10.404 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-04-17T04:35:10.539 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-04-17T04:35:10.553 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T04:35:10.559 DEBUG:teuthology.parallel:result is None 2024-04-17T04:35:10.665 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-04-17T04:35:10.791 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libapache-pom-java (18-1) ... 2024-04-17T04:35:10.985 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-04-17T04:35:11.136 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-04-17T04:35:11.287 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-04-17T04:35:11.446 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-04-17T04:35:11.589 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-04-17T04:35:11.849 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-04-17T04:35:11.966 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-04-17T04:35:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-04-17T04:35:12.185 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmaven-parent-java (31-2) ... 2024-04-17T04:35:12.303 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcommons-parent-java (43-1) ... 2024-04-17T04:35:12.428 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-04-17T04:35:12.555 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up s3cmd (2.2.0-1) ... 2024-04-17T04:35:12.856 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-04-17T04:35:12.982 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-04-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libguava-java (29.0-6) ... 2024-04-17T04:35:13.268 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-04-17T04:35:13.410 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-04-17T04:35:13.561 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-04-17T04:35:13.712 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-04-17T04:35:13.872 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libguice-java (4.2.3-2) ... 2024-04-17T04:35:13.998 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libjansi-java (1.18-1) ... 2024-04-17T04:35:14.124 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-04-17T04:35:14.250 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-04-17T04:35:14.376 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up maven (3.6.3-5) ... 2024-04-17T04:35:14.666 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-04-17T04:35:14.747 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T04:35:16.138 INFO:teuthology.orchestra.run.smithi088.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-17T04:35:16.143 DEBUG:teuthology.parallel:result is None 2024-04-17T04:35:16.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=461a84e6fbbf089011eafd13b98169fb9e9b8cc9 2024-04-17T04:35:16.366 DEBUG:teuthology.orchestra.run.smithi088:> dpkg-query -W -f '${Version}' ceph 2024-04-17T04:35:16.385 INFO:teuthology.orchestra.run.smithi088.stdout:19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:35:16.386 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:35:16.386 INFO:teuthology.task.install:The correct ceph version 19.0.0-2287-g461a84e6-1jammy is installed. 2024-04-17T04:35:16.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=461a84e6fbbf089011eafd13b98169fb9e9b8cc9 2024-04-17T04:35:16.605 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-query -W -f '${Version}' ceph 2024-04-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:35:16.625 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2287-g461a84e6-1jammy 2024-04-17T04:35:16.625 INFO:teuthology.task.install:The correct ceph version 19.0.0-2287-g461a84e6-1jammy is installed. 2024-04-17T04:35:16.627 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T04:35:16.627 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:16.627 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T04:35:16.641 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:16.642 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T04:35:16.680 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T04:35:16.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:16.681 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T04:35:16.696 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T04:35:16.753 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:16.753 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T04:35:16.767 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T04:35:16.823 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T04:35:16.824 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:16.824 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T04:35:16.836 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T04:35:16.892 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:16.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T04:35:16.906 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T04:35:16.959 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T04:35:16.959 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:16.959 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T04:35:16.971 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T04:35:17.024 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:17.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T04:35:17.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T04:35:17.090 INFO:teuthology.run_tasks:Running task ceph... 2024-04-17T04:35:17.172 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-17T04:35:17.172 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2024-04-17T04:35:17.174 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-04-17T04:35:17.187 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-17T04:35:17.188 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T04:35:17.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T04:35:17.242 INFO:tasks.ceph:Creating extra log directories... 2024-04-17T04:35:17.242 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T04:35:17.282 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T04:35:17.308 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-17T04:35:17.308 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-17T04:35:17.308 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000', 'branch': 'squid', 'description': 'rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7659000', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': '461a84e6fbbf089011eafd13b98169fb9e9b8cc9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '461a84e6fbbf089011eafd13b98169fb9e9b8cc9'}}, 'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'datacache': True, 'datacache_path': '/tmp/rgw_datacache', 'ec-data-pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'frontend': 'beast', 'inline data': False, 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 main': 'RGW11111111111111111', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-squid', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 3597, 'sha1': '461a84e6fbbf089011eafd13b98169fb9e9b8cc9', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'rgw', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '41c500dd60c93c8d83053b72cffc3dd1a952a336', 'targets': {'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC+yz+tDPlkQjjbMRn3xAZhK5N+VOMBKaH64bHOAUTAGGaLPAgA17cnBwc6d1kNP1Ykyi6t9SWDZvp5vwZJxKLc=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLipxCHy1pCsU745cAdHzITMYpbdtcaqlJswFTMKVxe3yg9Jd2rj6c+vI3v5PDaXqLk74gpu1memTOkr/Ex6wfY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '461a84e6fbbf089011eafd13b98169fb9e9b8cc9'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-datacache.sh']}, 'env': {'RGW_DATACACHE_PATH': '/tmp/rgw_datacache'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-squid', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-squid', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-16_21:40:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-17T04:35:17.308 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T04:35:17.338 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T04:35:17.344 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T04:35:17.387 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T04:35:17.400 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:17.400 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T04:35:17.445 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T04:35:17.446 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-17 04:29:40.589367967 +0000 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-17 04:29:40.585368003 +0000 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-17 04:29:40.585368003 +0000 2024-04-17T04:35:17.494 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2024-04-17T04:35:17.495 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T04:35:17.548 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-17T04:35:17.548 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-17T04:35:17.548 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00044699 s, 1.1 MB/s 2024-04-17T04:35:17.549 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T04:35:17.599 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-04-17T04:35:17.645 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T04:35:17.645 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:17.645 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-17T04:35:17.645 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:17.645 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-17 04:29:41.365361035 +0000 2024-04-17T04:35:17.646 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-17 04:29:41.105363357 +0000 2024-04-17T04:35:17.646 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-17 04:29:41.105363357 +0000 2024-04-17T04:35:17.646 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2024-04-17T04:35:17.646 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T04:35:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-17T04:35:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-17T04:35:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000407324 s, 1.3 MB/s 2024-04-17T04:35:17.701 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T04:35:17.750 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-17 04:29:41.617358783 +0000 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-17 04:29:41.613358820 +0000 2024-04-17T04:35:17.801 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-17 04:29:41.613358820 +0000 2024-04-17T04:35:17.802 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2024-04-17T04:35:17.802 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T04:35:17.855 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-17T04:35:17.855 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-17T04:35:17.855 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000354168 s, 1.4 MB/s 2024-04-17T04:35:17.856 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T04:35:17.906 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-04-17T04:35:17.953 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T04:35:17.953 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:17.953 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-04-17T04:35:17.953 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:17.953 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-17 04:29:42.369352068 +0000 2024-04-17T04:35:17.954 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-17 04:29:42.125354246 +0000 2024-04-17T04:35:17.954 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-17 04:29:42.125354246 +0000 2024-04-17T04:35:17.954 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2024-04-17T04:35:17.954 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T04:35:18.006 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-17T04:35:18.006 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-17T04:35:18.007 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000288751 s, 1.8 MB/s 2024-04-17T04:35:18.008 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T04:35:18.054 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:18.055 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:18.055 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T04:35:18.061 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T04:35:18.061 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-04-17T04:35:18.108 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T04:35:18.108 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:18.108 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-04-17T04:35:18.108 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:18.109 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-17 04:29:42.678932457 +0000 2024-04-17T04:35:18.109 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-17 04:29:42.410934850 +0000 2024-04-17T04:35:18.109 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-17 04:29:42.410934850 +0000 2024-04-17T04:35:18.109 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-04-17T04:35:18.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T04:35:18.163 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-17T04:35:18.163 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-17T04:35:18.163 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000444709 s, 1.2 MB/s 2024-04-17T04:35:18.164 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T04:35:18.213 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-04-17T04:35:18.259 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-17 04:29:43.154928204 +0000 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-17 04:29:42.914930349 +0000 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-17 04:29:42.914930349 +0000 2024-04-17T04:35:18.260 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-04-17T04:35:18.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T04:35:18.314 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-17T04:35:18.314 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-17T04:35:18.314 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000384388 s, 1.3 MB/s 2024-04-17T04:35:18.315 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T04:35:18.365 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-04-17T04:35:18.412 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T04:35:18.412 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:18.412 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-04-17T04:35:18.412 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:18.412 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-17 04:29:43.654923739 +0000 2024-04-17T04:35:18.412 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-17 04:29:43.390926097 +0000 2024-04-17T04:35:18.413 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-17 04:29:43.390926097 +0000 2024-04-17T04:35:18.413 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-04-17T04:35:18.413 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T04:35:18.466 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-17T04:35:18.466 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-17T04:35:18.466 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000347648 s, 1.5 MB/s 2024-04-17T04:35:18.467 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T04:35:18.517 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-04-17T04:35:18.563 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 933 Links: 1 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-17 04:29:43.906921488 +0000 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-17 04:29:43.902921524 +0000 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-17 04:29:43.902921524 +0000 2024-04-17T04:35:18.564 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-04-17T04:35:18.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T04:35:18.618 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-17T04:35:18.618 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-17T04:35:18.618 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000384977 s, 1.3 MB/s 2024-04-17T04:35:18.619 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T04:35:18.669 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:18.669 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T04:35:18.669 INFO:tasks.ceph:Generating config... 2024-04-17T04:35:18.670 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw d3n l1 datacache size = 10737418240 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw d3n l1 local datacache enabled = True 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw enable ops log = True 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw max chunk size = 4194304 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw obj stripe size = 6291456 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw s3 auth use sts = True 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw sts key = abcdefghijklmnop 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-04-17T04:35:18.671 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-17T04:35:18.672 INFO:tasks.ceph:[client] setuser = ceph 2024-04-17T04:35:18.672 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-17T04:35:18.672 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-17T04:35:18.672 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-04-17T04:35:18.672 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-04-17T04:35:18.672 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-04-17T04:35:18.672 INFO:tasks.ceph:[global] rgw override bucket index max shards = 1 2024-04-17T04:35:18.672 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-17T04:35:18.672 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-17T04:35:18.672 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-17T04:35:18.672 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-17T04:35:18.672 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-17T04:35:18.672 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-04-17T04:35:18.672 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-17T04:35:18.672 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-17T04:35:18.672 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-17T04:35:18.673 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-17T04:35:18.674 INFO:tasks.ceph:Setting up mon.a... 2024-04-17T04:35:18.674 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-17T04:35:18.708 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.keyring 2024-04-17T04:35:18.726 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-17T04:35:18.804 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T04:35:18.864 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.88'), ('mon.c', '[v2:172.21.15.88:3301,v1:172.21.15.88:6790]'), ('mon.b', '172.21.15.154')] 2024-04-17T04:35:18.865 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.88,[v2:172.21.15.88:3301,v1:172.21.15.88:6790],172.21.15.154', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': True, 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-17T04:35:18.866 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:18.866 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T04:35:18.914 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.88 --addv c '[v2:172.21.15.88:3301,v1:172.21.15.88:6790]' --add b 172.21.15.154 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:18.999 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:35:18.999 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:35:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: generated fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:setting min_mon_release = quincy 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 0 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:min_mon_release 17 (quincy) 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:election_strategy: 1 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:35:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-17T04:35:19.001 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T04:35:19.049 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5b040342-9999-436c-bdde-ed9b9762bed0... 2024-04-17T04:35:19.050 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T04:35:19.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2024-04-17T04:35:19.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.120 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.121 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.88,[v2:172.21.15.88:3301,v1:172.21.15.88:6790],172.21.15.154" 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: mon client directed command retry = 5 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: ms inject socket failures = 5000 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: osd heartbeat grace = 40 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: osd_max_pg_log_entries = 10 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: osd_min_pg_log_entries = 10 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: rgw override bucket index max shards = 1 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:35:19.122 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd mclock profile = high_recovery_ops 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug trim objects = true 2024-04-17T04:35:19.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2024-04-17T04:35:19.124 INFO:teuthology.orchestra.run.smithi088.stdout: bdev async discard = True 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: bdev enable discard = True 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore allocator = bitmap 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 1/20 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 1/20 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 4/10 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: osd fast shutdown = False 2024-04-17T04:35:19.125 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: osd shutdown pgref assert = True 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-17T04:35:19.126 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T04:35:19.127 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd crush smoke test = False 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = True 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T04:35:19.128 INFO:teuthology.orchestra.run.smithi088.stdout: debug rgw = 20 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw bucket counters cache = True 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw crypt require ssl = False 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw crypt s3 kms backend = testing 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw d3n l1 datacache size = 10737418240 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw d3n l1 local datacache enabled = True 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw max chunk size = 4194304 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw obj stripe size = 6291456 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw s3 auth use sts = True 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw sts key = abcdefghijklmnop 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw torrent flag = True 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: rgw user counters cache = True 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: setgroup = ceph 2024-04-17T04:35:19.129 INFO:teuthology.orchestra.run.smithi088.stdout: setuser = ceph 2024-04-17T04:35:19.130 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2024-04-17T04:35:19.130 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2024-04-17T04:35:19.130 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2024-04-17T04:35:19.130 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-04-17T04:35:19.130 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-04-17T04:35:19.130 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-04-17T04:35:19.131 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-04-17T04:35:19.132 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.88,[v2:172.21.15.88:3301,v1:172.21.15.88:6790],172.21.15.154" 2024-04-17T04:35:19.133 INFO:teuthology.orchestra.run.smithi154.stdout: mon client directed command retry = 5 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject socket failures = 5000 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd heartbeat grace = 40 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd_max_pg_log_entries = 10 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd_min_pg_log_entries = 10 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: rgw override bucket index max shards = 1 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-04-17T04:35:19.134 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug trim objects = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.135 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2024-04-17T04:35:19.136 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: osd fast shutdown = False 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: osd shutdown pgref assert = True 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-04-17T04:35:19.137 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-04-17T04:35:19.138 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.139 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd crush smoke test = False 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = True 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: debug rgw = 20 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw bucket counters cache = True 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw crypt require ssl = False 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw crypt s3 kms backend = testing 2024-04-17T04:35:19.140 INFO:teuthology.orchestra.run.smithi154.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw d3n l1 datacache size = 10737418240 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw d3n l1 local datacache enabled = True 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw max chunk size = 4194304 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw obj stripe size = 6291456 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw s3 auth use sts = True 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw sts key = abcdefghijklmnop 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw torrent flag = True 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: rgw user counters cache = True 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: setgroup = ceph 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout: setuser = ceph 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-04-17T04:35:19.141 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-04-17T04:35:19.143 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-17T04:35:19.143 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-17T04:35:19.231 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-17T04:35:19.231 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:19.231 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-17T04:35:19.236 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:19.236 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-17T04:35:19.280 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:35:19.280 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:19.280 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T04:35:19.280 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T04:35:19.346 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:19.346 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:19.393 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:35:19.393 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:19.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T04:35:19.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T04:35:19.416 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:19.416 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:19.464 INFO:tasks.ceph:Setting up mon nodes... 2024-04-17T04:35:19.464 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-17T04:35:19.464 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T04:35:19.501 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T04:35:19.504 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T04:35:19.548 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T04:35:19.563 INFO:tasks.ceph:Setting up mds nodes... 2024-04-17T04:35:19.563 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-17T04:35:19.563 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T04:35:19.597 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-17T04:35:19.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-17T04:35:19.643 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-17T04:35:19.654 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-17T04:35:19.655 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T04:35:19.655 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-17T04:35:19.666 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:19.666 INFO:tasks.ceph:role: osd.0 2024-04-17T04:35:19.666 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:35:19.666 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T04:35:19.729 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:19.729 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:19.729 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:19.729 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:19.729 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:19.730 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:19.730 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:19.730 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:19.730 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:19.730 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:20.191 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-17T04:35:20.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-17T04:35:20.194 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-17T04:35:20.350 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-17T04:35:20.361 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:20.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:20.362 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-17T04:35:20.421 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:20.421 INFO:tasks.ceph:role: osd.1 2024-04-17T04:35:20.421 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:35:20.421 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T04:35:20.485 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:20.485 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:20.485 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:20.485 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:20.485 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:20.485 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:20.486 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:20.486 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:20.486 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:20.486 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:20.920 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-17T04:35:20.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-17T04:35:20.924 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-17T04:35:21.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-17T04:35:21.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:21.025 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-17T04:35:21.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:21.078 INFO:tasks.ceph:role: osd.2 2024-04-17T04:35:21.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:35:21.079 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:21.139 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:21.591 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-17T04:35:21.594 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-17T04:35:21.594 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-17T04:35:21.693 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-17T04:35:21.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:21.705 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:21.706 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-17T04:35:21.759 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:21.759 INFO:tasks.ceph:role: osd.3 2024-04-17T04:35:21.759 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:35:21.760 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:21.822 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:21.823 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:21.823 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:21.823 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:22.321 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-17T04:35:22.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-17T04:35:22.324 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-17T04:35:22.478 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-17T04:35:22.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:22.490 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:22.490 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:22.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:22.605+0000 7f10067487c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-17T04:35:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:22.609+0000 7f10067487c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-17T04:35:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:22.609+0000 7f10067487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T04:35:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:22.609+0000 7f10067487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T04:35:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:22.609+0000 7f10067487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T04:35:22.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:22.617+0000 7f10067487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-17T04:35:25.604 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-17T04:35:25.618 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:25.697+0000 7f36dd12b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-17T04:35:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:25.697+0000 7f36dd12b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-17T04:35:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:25.697+0000 7f36dd12b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T04:35:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:25.697+0000 7f36dd12b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T04:35:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:25.697+0000 7f36dd12b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T04:35:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:25.697+0000 7f36dd12b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-17T04:35:28.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-17T04:35:28.734 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:28.813+0000 7fc5e42927c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-17T04:35:28.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:28.813+0000 7fc5e42927c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-17T04:35:28.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:28.813+0000 7fc5e42927c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T04:35:28.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:28.813+0000 7fc5e42927c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T04:35:28.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:28.813+0000 7fc5e42927c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T04:35:28.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:28.813+0000 7fc5e42927c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-17T04:35:31.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-17T04:35:31.846 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:31.925+0000 7f37183757c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-17T04:35:31.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:31.925+0000 7f37183757c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-17T04:35:31.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:31.925+0000 7f37183757c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T04:35:31.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:31.925+0000 7f37183757c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T04:35:31.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:31.925+0000 7f37183757c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T04:35:31.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:35:31.929+0000 7f37183757c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-17T04:35:34.900 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-17T04:35:34.954 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-17T04:35:34.969 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:34.969 INFO:tasks.ceph:role: osd.4 2024-04-17T04:35:34.969 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:35:34.969 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T04:35:35.030 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:35.030 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:35.030 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:35.031 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:35.510 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-17T04:35:35.512 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-17T04:35:35.512 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-17T04:35:35.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-17T04:35:35.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:35.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-17T04:35:35.735 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:35.735 INFO:tasks.ceph:role: osd.5 2024-04-17T04:35:35.735 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:35:35.735 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T04:35:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:35.799 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:35.800 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:36.239 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-17T04:35:36.241 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-17T04:35:36.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-17T04:35:36.327 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-17T04:35:36.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:36.337 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:36.337 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-17T04:35:36.393 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:36.393 INFO:tasks.ceph:role: osd.6 2024-04-17T04:35:36.393 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:35:36.393 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:36.457 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:36.914 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-17T04:35:36.916 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-17T04:35:36.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-17T04:35:37.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-17T04:35:37.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:37.054 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:37.054 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-17T04:35:37.107 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T04:35:37.107 INFO:tasks.ceph:role: osd.7 2024-04-17T04:35:37.107 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:35:37.107 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T04:35:37.171 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T04:35:37.171 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T04:35:37.171 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T04:35:37.171 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-17T04:35:37.171 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T04:35:37.172 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-04-17T04:35:37.172 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T04:35:37.172 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-17T04:35:37.172 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T04:35:37.172 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T04:35:37.662 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-04-17T04:35:37.666 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-04-17T04:35:37.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-17T04:35:37.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-17T04:35:37.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:35:37.830 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2024-04-17T04:35:37.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:37.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:37.951+0000 7fbc13aed7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-17T04:35:37.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:37.955+0000 7fbc13aed7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-17T04:35:37.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:37.955+0000 7fbc13aed7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T04:35:37.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:37.955+0000 7fbc13aed7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T04:35:37.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:37.955+0000 7fbc13aed7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T04:35:37.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:37.959+0000 7fbc13aed7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-17T04:35:40.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-17T04:35:40.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:41.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:41.027+0000 7f20263d97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-17T04:35:41.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:41.027+0000 7f20263d97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-17T04:35:41.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:41.027+0000 7f20263d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T04:35:41.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:41.027+0000 7f20263d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T04:35:41.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:41.027+0000 7f20263d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T04:35:41.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:41.031+0000 7f20263d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-17T04:35:44.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-17T04:35:44.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:44.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:44.163+0000 7fd47db227c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-17T04:35:44.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:44.163+0000 7fd47db227c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-17T04:35:44.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:44.163+0000 7fd47db227c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T04:35:44.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:44.163+0000 7fd47db227c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T04:35:44.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:44.163+0000 7fd47db227c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T04:35:44.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:44.167+0000 7fd47db227c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-17T04:35:47.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-17T04:35:47.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:47.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:47.271+0000 7f3c032127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-17T04:35:47.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:47.271+0000 7f3c032127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-17T04:35:47.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:47.271+0000 7f3c032127c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T04:35:47.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:47.271+0000 7f3c032127c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T04:35:47.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:47.271+0000 7f3c032127c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T04:35:47.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-17T04:35:47.271+0000 7f3c032127c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-17T04:35:50.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-17T04:35:50.292 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-17T04:35:50.293 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:50.293 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-17T04:35:50.307 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:50.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-17T04:35:50.361 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:50.361 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-17T04:35:50.416 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:50.416 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-17T04:35:50.473 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:50.473 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-17T04:35:50.527 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:50.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-17T04:35:50.541 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:50.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-17T04:35:50.595 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:50.595 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-17T04:35:50.651 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:50.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-17T04:35:50.707 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:50.707 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-17T04:35:50.762 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:50.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-17T04:35:50.768 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:50.768 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-17T04:35:50.811 INFO:tasks.ceph:Adding keys to all mons... 2024-04-17T04:35:50.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T04:35:50.814 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCHUR9mqEUbHhAAe75pp6gs6+cB4MLvc3euAQ== 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCKUR9mXcZSJBAAFDrErygaand5TOPIfhBY1A== 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCNUR9mCq/YKRAA0+xkFGmg6/a6CVrpYrOQBQ== 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCQUR9mtjvEMBAAv1wkkGFtxyz8bWgxXfvlgA== 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCTUR9mo/yHNxAAl5E9GavfP/b0vBfyQxORHw== 2024-04-17T04:35:50.857 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCHUR9mYAP/IBAAu4/HshJYulii60qH3yPt9A== 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCZUR9m2nvFOBAAFPA3uqxHYlaBQb0cdPiXqA== 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCdUR9mTAPfARAAFNbwezBp+MLg1NgYGC95zw== 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCgUR9mZw/4CRAAxixOLv1RxBNA9WzZ5Uouuw== 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCjUR9mfmJOEBAAsKWAuyLa3iFpGoWF7FG6Fg== 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCHUR9mCe3UIxAATsRBEhkMNxaGEegqiKbM9A== 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2024-04-17T04:35:50.858 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCHUR9m606pJhAAHpio2EWjtFQ/EtSO7THgrw== 2024-04-17T04:35:50.864 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCHUR9mqEUbHhAAe75pp6gs6+cB4MLvc3euAQ== 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCKUR9mXcZSJBAAFDrErygaand5TOPIfhBY1A== 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCNUR9mCq/YKRAA0+xkFGmg6/a6CVrpYrOQBQ== 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCQUR9mtjvEMBAAv1wkkGFtxyz8bWgxXfvlgA== 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCTUR9mo/yHNxAAl5E9GavfP/b0vBfyQxORHw== 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCHUR9mYAP/IBAAu4/HshJYulii60qH3yPt9A== 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-04-17T04:35:50.865 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCZUR9m2nvFOBAAFPA3uqxHYlaBQb0cdPiXqA== 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCdUR9mTAPfARAAFNbwezBp+MLg1NgYGC95zw== 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCgUR9mZw/4CRAAxixOLv1RxBNA9WzZ5Uouuw== 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCjUR9mfmJOEBAAsKWAuyLa3iFpGoWF7FG6Fg== 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCHUR9mCe3UIxAATsRBEhkMNxaGEegqiKbM9A== 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2024-04-17T04:35:50.866 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCHUR9m606pJhAAHpio2EWjtFQ/EtSO7THgrw== 2024-04-17T04:35:50.867 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T04:35:50.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T04:35:50.955 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:50.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.039 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.127 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.213 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.294 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T04:35:51.298 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T04:35:51.383 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.428 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.466 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.509 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.546 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.593 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T04:35:51.678 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T04:35:51.710 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T04:35:51.765 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T04:35:51.793 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T04:35:51.837 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-17T04:35:51.837 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-17T04:35:51.886 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T04:35:51.970 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-17T04:35:52.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-17T04:35:52.073 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T04:35:52.166 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-17T04:35:52.176 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-17T04:35:52.191 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T04:35:52.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-17T04:35:52.291 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:52.294 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T04:35:52.339 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-17T04:35:52.339 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-17T04:35:52.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-17T04:35:52.342 INFO:tasks.ceph.mon.a:Started 2024-04-17T04:35:52.342 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-17T04:35:52.342 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-17T04:35:52.344 INFO:tasks.ceph.mon.c:Started 2024-04-17T04:35:52.344 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-17T04:35:52.344 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-17T04:35:52.384 INFO:tasks.ceph.mon.b:Started 2024-04-17T04:35:52.384 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-17T04:35:52.384 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-17T04:35:52.384 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-17T04:35:52.389 INFO:tasks.ceph.mgr.y:Started 2024-04-17T04:35:52.389 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-17T04:35:52.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-17T04:35:52.391 INFO:tasks.ceph.mgr.x:Started 2024-04-17T04:35:52.391 INFO:tasks.ceph:Setting crush tunables to default 2024-04-17T04:35:52.391 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd crush tunables default 2024-04-17T04:35:57.918 INFO:teuthology.orchestra.run.smithi088.stderr:adjusted tunables profile to default 2024-04-17T04:35:57.938 INFO:tasks.ceph:check_enable_crimson: False 2024-04-17T04:35:57.938 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-17T04:35:57.938 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:57.938 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-17T04:35:57.949 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:57.949 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-17T04:35:58.002 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:58.002 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-17T04:35:58.037 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:58.037+0000 7f5d8429c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T04:35:58.047 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:35:58.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-17T04:35:58.099 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:58.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-17T04:35:58.115 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:58.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-17T04:35:58.171 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:58.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-17T04:35:58.230 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-17T04:35:58.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-17T04:35:58.270 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:58.271+0000 7ff457709180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T04:35:58.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 369d3f72-44a9-4b30-aa1c-c88765270bb9 0 2024-04-17T04:35:58.541 INFO:teuthology.orchestra.run.smithi154.stdout:0 2024-04-17T04:35:58.549 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:58.549+0000 7f5d8429c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T04:35:58.558 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1 1 2024-04-17T04:35:58.741 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:58.741+0000 7f5d8429c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T04:35:58.776 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-04-17T04:35:58.794 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 3d978434-aeba-4bdf-ab28-21dfd67bed1b 2 2024-04-17T04:35:58.807 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:58.811+0000 7ff457709180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T04:35:58.993 INFO:teuthology.orchestra.run.smithi154.stdout:2 2024-04-17T04:35:59.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 158b3428-03f3-4d64-a5e1-ec30159d91c8 3 2024-04-17T04:35:59.026 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.027+0000 7ff457709180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T04:35:59.235 INFO:teuthology.orchestra.run.smithi154.stdout:3 2024-04-17T04:35:59.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 62360a76-7bc7-4668-ba57-cbcf6cd589d1 4 2024-04-17T04:35:59.277 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.277+0000 7f5d8429c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T04:35:59.330 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.329+0000 7f5d8429c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T04:35:59.382 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.381+0000 7f5d8429c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T04:35:59.448 INFO:teuthology.orchestra.run.smithi154.stdout:4 2024-04-17T04:35:59.459 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.457+0000 7f5d8429c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T04:35:59.464 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new fd243a29-01d5-43c0-acaf-d36ab1e3d981 5 2024-04-17T04:35:59.510 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.509+0000 7f5d8429c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T04:35:59.586 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.585+0000 7f5d8429c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T04:35:59.610 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.611+0000 7ff457709180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T04:35:59.639 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.637+0000 7f5d8429c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T04:35:59.660 INFO:teuthology.orchestra.run.smithi154.stdout:5 2024-04-17T04:35:59.665 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.667+0000 7ff457709180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T04:35:59.677 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 7d374000-ae06-4891-98bf-073f3737dd03 6 2024-04-17T04:35:59.721 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.723+0000 7ff457709180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T04:35:59.787 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.785+0000 7f5d8429c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T04:35:59.802 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.803+0000 7ff457709180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T04:35:59.849 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:35:59.849+0000 7f5d8429c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T04:35:59.856 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.859+0000 7ff457709180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T04:35:59.875 INFO:teuthology.orchestra.run.smithi154.stdout:6 2024-04-17T04:35:59.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 01de9b40-4992-4f7f-9408-98cd9533e7e3 7 2024-04-17T04:35:59.936 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.939+0000 7ff457709180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T04:35:59.990 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:35:59.991+0000 7ff457709180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T04:36:00.138 INFO:teuthology.orchestra.run.smithi154.stdout:7 2024-04-17T04:36:00.144 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:00.147+0000 7ff457709180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T04:36:00.154 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-17T04:36:00.155 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-17T04:36:00.157 INFO:tasks.ceph.osd.0:Started 2024-04-17T04:36:00.157 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-17T04:36:00.157 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-17T04:36:00.160 INFO:tasks.ceph.osd.1:Started 2024-04-17T04:36:00.160 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-17T04:36:00.160 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-17T04:36:00.162 INFO:tasks.ceph.osd.2:Started 2024-04-17T04:36:00.162 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-17T04:36:00.162 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-17T04:36:00.165 INFO:tasks.ceph.osd.3:Started 2024-04-17T04:36:00.165 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-17T04:36:00.165 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-17T04:36:00.167 INFO:tasks.ceph.osd.4:Started 2024-04-17T04:36:00.167 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-17T04:36:00.167 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-17T04:36:00.169 INFO:tasks.ceph.osd.5:Started 2024-04-17T04:36:00.170 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-17T04:36:00.170 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-17T04:36:00.172 INFO:tasks.ceph.osd.6:Started 2024-04-17T04:36:00.172 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-17T04:36:00.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-17T04:36:00.174 INFO:tasks.ceph.osd.7:Started 2024-04-17T04:36:00.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T04:36:00.231 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:00.235+0000 7ff457709180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T04:36:00.409 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:00.409 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":10,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T04:36:00.137416+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T04:36:00.425 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-17T04:36:00.426 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-17T04:36:00.483 INFO:tasks.ceph.mgr.y.smithi088.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-17T04:36:00.484 INFO:tasks.ceph.mgr.y.smithi088.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-17T04:36:00.484 INFO:tasks.ceph.mgr.y.smithi088.stderr: from numpy import show_config as show_numpy_config 2024-04-17T04:36:00.583 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:00.581+0000 7f5d8429c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T04:36:00.766 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:00.765+0000 7f5d8429c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T04:36:00.780 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-17T04:36:00.783+0000 7f27e12a37c0 -1 Falling back to public interface 2024-04-17T04:36:00.791 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-17T04:36:00.795+0000 7f821b4b57c0 -1 Falling back to public interface 2024-04-17T04:36:00.791 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-17T04:36:00.795+0000 7fd7e86337c0 -1 Falling back to public interface 2024-04-17T04:36:00.792 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-17T04:36:00.795+0000 7fb49f4787c0 -1 Falling back to public interface 2024-04-17T04:36:00.822 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-17T04:36:00.821+0000 7f106d3277c0 -1 Falling back to public interface 2024-04-17T04:36:00.822 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-17T04:36:00.821+0000 7fe154c747c0 -1 Falling back to public interface 2024-04-17T04:36:00.833 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-17T04:36:00.833+0000 7f14107127c0 -1 Falling back to public interface 2024-04-17T04:36:00.834 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-17T04:36:00.833+0000 7f25a9df27c0 -1 Falling back to public interface 2024-04-17T04:36:00.835 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-17T04:36:00.833+0000 7f25a8db2640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6823/0: (98) Address already in use 2024-04-17T04:36:00.891 INFO:tasks.ceph.mgr.x.smithi154.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-17T04:36:00.891 INFO:tasks.ceph.mgr.x.smithi154.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-17T04:36:00.891 INFO:tasks.ceph.mgr.x.smithi154.stderr: from numpy import show_config as show_numpy_config 2024-04-17T04:36:00.933 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:00.933+0000 7f5d8429c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T04:36:00.951 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:00.955+0000 7ff457709180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T04:36:01.002 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:01.001+0000 7f5d8429c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T04:36:01.130 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:01.131+0000 7ff457709180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T04:36:01.143 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:01.141+0000 7f5d8429c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T04:36:01.229 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T04:36:01.302 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:01.303+0000 7ff457709180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T04:36:01.372 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:01.375+0000 7ff457709180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T04:36:01.407 INFO:teuthology.misc.health.smithi088.stdout: 2024-04-17T04:36:01.408 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":10,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T04:36:00.137416+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T04:36:01.423 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T04:36:01.518 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:01.519+0000 7ff457709180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T04:36:01.774 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:01.773+0000 7f5d8429c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T04:36:01.849 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:01.849+0000 7f5d8429c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T04:36:02.027 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:02.025+0000 7f5d8429c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T04:36:02.100 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:02.101+0000 7f5d8429c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T04:36:02.143 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:02.147+0000 7ff457709180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T04:36:02.217 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:02.219+0000 7ff457709180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T04:36:02.385 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:02.385+0000 7f5d8429c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T04:36:02.397 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:02.399+0000 7ff457709180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T04:36:02.446 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:02.445+0000 7f5d8429c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T04:36:02.471 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:02.475+0000 7ff457709180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T04:36:02.764 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:02.767+0000 7ff457709180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T04:36:02.826 INFO:tasks.ceph.mgr.x.smithi154.stderr:2024-04-17T04:36:02.827+0000 7ff457709180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T04:36:03.493 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:03.489+0000 7f5d2b6e9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-17T04:36:03.494 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-17T04:36:03.489+0000 7f5d2b6e9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-17T04:36:04.680 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-17T04:36:04.683+0000 7f27e12a37c0 -1 osd.7 0 log_to_monitors true 2024-04-17T04:36:04.681 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-17T04:36:04.683+0000 7fb49f4787c0 -1 osd.5 0 log_to_monitors true 2024-04-17T04:36:04.699 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-17T04:36:04.703+0000 7fd7e86337c0 -1 osd.4 0 log_to_monitors true 2024-04-17T04:36:04.752 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-17T04:36:04.755+0000 7f821b4b57c0 -1 osd.6 0 log_to_monitors true 2024-04-17T04:36:04.781 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-17T04:36:04.781+0000 7f25a9df27c0 -1 osd.3 0 log_to_monitors true 2024-04-17T04:36:04.805 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-17T04:36:04.805+0000 7fe154c747c0 -1 osd.2 0 log_to_monitors true 2024-04-17T04:36:04.805 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-17T04:36:04.805+0000 7f14107127c0 -1 osd.1 0 log_to_monitors true 2024-04-17T04:36:04.806 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-17T04:36:04.805+0000 7f106d3277c0 -1 osd.0 0 log_to_monitors true 2024-04-17T04:36:07.918 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-17T04:36:07.919+0000 7fb49c434640 -1 osd.5 0 waiting for initial osdmap 2024-04-17T04:36:07.935 INFO:tasks.ceph.osd.5.smithi154.stderr:2024-04-17T04:36:07.939+0000 7fb493245640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:07.937 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-17T04:36:07.939+0000 7fd7e3db9640 -1 osd.4 0 waiting for initial osdmap 2024-04-17T04:36:07.945 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-17T04:36:07.947+0000 7f8216c3b640 -1 osd.6 0 waiting for initial osdmap 2024-04-17T04:36:07.975 INFO:tasks.ceph.osd.4.smithi154.stderr:2024-04-17T04:36:07.979+0000 7fd7dc400640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:07.997 INFO:tasks.ceph.osd.6.smithi154.stderr:2024-04-17T04:36:07.999+0000 7f820f282640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:07.999 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-17T04:36:07.997+0000 7f25a5578640 -1 osd.3 0 waiting for initial osdmap 2024-04-17T04:36:08.004 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-17T04:36:08.005+0000 7f140d6ce640 -1 osd.1 0 waiting for initial osdmap 2024-04-17T04:36:08.005 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-17T04:36:08.005+0000 7f1068aad640 -1 osd.0 0 waiting for initial osdmap 2024-04-17T04:36:08.007 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-17T04:36:08.011+0000 7f27de25f640 -1 osd.7 0 waiting for initial osdmap 2024-04-17T04:36:08.015 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-17T04:36:08.013+0000 7fe1503fa640 -1 osd.2 0 waiting for initial osdmap 2024-04-17T04:36:08.033 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-17T04:36:08.033+0000 7f259dbbf640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:08.035 INFO:tasks.ceph.osd.7.smithi154.stderr:2024-04-17T04:36:08.039+0000 7f27d5070640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:08.049 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-17T04:36:08.049+0000 7fe148a41640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:08.049 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-17T04:36:08.049+0000 7f10610f4640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:08.053 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-17T04:36:08.053+0000 7f14044df640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T04:36:08.230 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T04:36:08.527 INFO:teuthology.misc.health.smithi088.stdout: 2024-04-17T04:36:08.527 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":12,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T04:36:05.828051+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T04:36:08.543 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T04:36:15.350 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T04:36:15.650 INFO:teuthology.misc.health.smithi088.stdout: 2024-04-17T04:36:15.651 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":17,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T04:36:13.076240+0000","last_up_change":"2024-04-17T04:36:08.952272+0000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T04:36:10.517211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6801","nonce":1670897847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6803","nonce":1670897847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6811","nonce":1670897847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6807","nonce":1670897847}]},"public_addr":"172.21.15.88:6801/1670897847","cluster_addr":"172.21.15.88:6803/1670897847","heartbeat_back_addr":"172.21.15.88:6811/1670897847","heartbeat_front_addr":"172.21.15.88:6807/1670897847","state":["exists","up"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6817","nonce":3314042889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6819","nonce":3314042889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6823","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6825","nonce":3314042889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6822","nonce":3314042889}]},"public_addr":"172.21.15.88:6817/3314042889","cluster_addr":"172.21.15.88:6819/3314042889","heartbeat_back_addr":"172.21.15.88:6825/3314042889","heartbeat_front_addr":"172.21.15.88:6822/3314042889","state":["exists","up"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6806","nonce":508820735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6810","nonce":508820735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6815","nonce":508820735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6813","nonce":508820735}]},"public_addr":"172.21.15.88:6806/508820735","cluster_addr":"172.21.15.88:6810/508820735","heartbeat_back_addr":"172.21.15.88:6815/508820735","heartbeat_front_addr":"172.21.15.88:6813/508820735","state":["exists","up"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6824","nonce":217804779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6827","nonce":217804779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6831","nonce":217804779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6829","nonce":217804779}]},"public_addr":"172.21.15.88:6824/217804779","cluster_addr":"172.21.15.88:6827/217804779","heartbeat_back_addr":"172.21.15.88:6831/217804779","heartbeat_front_addr":"172.21.15.88:6829/217804779","state":["exists","up"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6810","nonce":2413638230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6814","nonce":2413638230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6822","nonce":2413638230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6818","nonce":2413638230}]},"public_addr":"172.21.15.154:6810/2413638230","cluster_addr":"172.21.15.154:6814/2413638230","heartbeat_back_addr":"172.21.15.154:6822/2413638230","heartbeat_front_addr":"172.21.15.154:6818/2413638230","state":["exists","up"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6813","nonce":3628934341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6819","nonce":3628934341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6828","nonce":3628934341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6824","nonce":3628934341}]},"public_addr":"172.21.15.154:6813/3628934341","cluster_addr":"172.21.15.154:6819/3628934341","heartbeat_back_addr":"172.21.15.154:6828/3628934341","heartbeat_front_addr":"172.21.15.154:6824/3628934341","state":["exists","up"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6815","nonce":1439702329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6825","nonce":1439702329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6831","nonce":1439702329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6827","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6829","nonce":1439702329}]},"public_addr":"172.21.15.154:6815/1439702329","cluster_addr":"172.21.15.154:6825/1439702329","heartbeat_back_addr":"172.21.15.154:6831/1439702329","heartbeat_front_addr":"172.21.15.154:6829/1439702329","state":["exists","up"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6801","nonce":1245445760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6803","nonce":1245445760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6807","nonce":1245445760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6805","nonce":1245445760}]},"public_addr":"172.21.15.154:6801/1245445760","cluster_addr":"172.21.15.154:6803/1245445760","heartbeat_back_addr":"172.21.15.154:6807/1245445760","heartbeat_front_addr":"172.21.15.154:6805/1245445760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:08.009769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.775680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.793533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.757704+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.747333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.725796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.771745+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.730259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T04:36:15.669 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-17T04:36:15.669 INFO:tasks.ceph:Creating RBD pool 2024-04-17T04:36:15.669 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-17T04:36:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'rbd' created 2024-04-17T04:36:16.215 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-17T04:36:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-17T04:36:17.226 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-17T04:36:17.227 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T04:36:17.227 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-17T04:36:17.227 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-17T04:36:17.228 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-17T04:36:17.569 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:17.585 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2906002900},{"type":"v1","addr":"172.21.15.88:6833","nonce":2906002900}]},"active_addr":"172.21.15.88:6833/2906002900","active_change":"2024-04-17T04:36:02.471384+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":1840664217}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":1705517231}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":801961689}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":329803440}]}]} 2024-04-17T04:36:17.589 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-17T04:36:17.589 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-17T04:36:17.589 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T04:36:17.882 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:17.882 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T04:36:17.193860+0000","last_up_change":"2024-04-17T04:36:08.952272+0000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T04:36:10.517211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-17T04:36:15.959683+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6801","nonce":1670897847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6803","nonce":1670897847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6811","nonce":1670897847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6807","nonce":1670897847}]},"public_addr":"172.21.15.88:6801/1670897847","cluster_addr":"172.21.15.88:6803/1670897847","heartbeat_back_addr":"172.21.15.88:6811/1670897847","heartbeat_front_addr":"172.21.15.88:6807/1670897847","state":["exists","up"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6817","nonce":3314042889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6819","nonce":3314042889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6823","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6825","nonce":3314042889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6822","nonce":3314042889}]},"public_addr":"172.21.15.88:6817/3314042889","cluster_addr":"172.21.15.88:6819/3314042889","heartbeat_back_addr":"172.21.15.88:6825/3314042889","heartbeat_front_addr":"172.21.15.88:6822/3314042889","state":["exists","up"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6806","nonce":508820735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6810","nonce":508820735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6815","nonce":508820735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6813","nonce":508820735}]},"public_addr":"172.21.15.88:6806/508820735","cluster_addr":"172.21.15.88:6810/508820735","heartbeat_back_addr":"172.21.15.88:6815/508820735","heartbeat_front_addr":"172.21.15.88:6813/508820735","state":["exists","up"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6824","nonce":217804779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6827","nonce":217804779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6831","nonce":217804779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6829","nonce":217804779}]},"public_addr":"172.21.15.88:6824/217804779","cluster_addr":"172.21.15.88:6827/217804779","heartbeat_back_addr":"172.21.15.88:6831/217804779","heartbeat_front_addr":"172.21.15.88:6829/217804779","state":["exists","up"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6810","nonce":2413638230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6814","nonce":2413638230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6822","nonce":2413638230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6818","nonce":2413638230}]},"public_addr":"172.21.15.154:6810/2413638230","cluster_addr":"172.21.15.154:6814/2413638230","heartbeat_back_addr":"172.21.15.154:6822/2413638230","heartbeat_front_addr":"172.21.15.154:6818/2413638230","state":["exists","up"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6813","nonce":3628934341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6819","nonce":3628934341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6828","nonce":3628934341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6824","nonce":3628934341}]},"public_addr":"172.21.15.154:6813/3628934341","cluster_addr":"172.21.15.154:6819/3628934341","heartbeat_back_addr":"172.21.15.154:6828/3628934341","heartbeat_front_addr":"172.21.15.154:6824/3628934341","state":["exists","up"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6815","nonce":1439702329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6825","nonce":1439702329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6831","nonce":1439702329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6827","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6829","nonce":1439702329}]},"public_addr":"172.21.15.154:6815/1439702329","cluster_addr":"172.21.15.154:6825/1439702329","heartbeat_back_addr":"172.21.15.154:6831/1439702329","heartbeat_front_addr":"172.21.15.154:6829/1439702329","state":["exists","up"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6801","nonce":1245445760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6803","nonce":1245445760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6807","nonce":1245445760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6805","nonce":1245445760}]},"public_addr":"172.21.15.154:6801/1245445760","cluster_addr":"172.21.15.154:6803/1245445760","heartbeat_back_addr":"172.21.15.154:6807/1245445760","heartbeat_front_addr":"172.21.15.154:6805/1245445760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:08.009769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.775680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.793533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.757704+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.747333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.725796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.771745+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.730259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T04:36:17.899 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-17T04:36:17.900 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T04:36:18.184 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:18.184 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T04:36:17.193860+0000","last_up_change":"2024-04-17T04:36:08.952272+0000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T04:36:10.517211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-17T04:36:15.959683+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6801","nonce":1670897847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6803","nonce":1670897847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6811","nonce":1670897847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6807","nonce":1670897847}]},"public_addr":"172.21.15.88:6801/1670897847","cluster_addr":"172.21.15.88:6803/1670897847","heartbeat_back_addr":"172.21.15.88:6811/1670897847","heartbeat_front_addr":"172.21.15.88:6807/1670897847","state":["exists","up"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6817","nonce":3314042889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6819","nonce":3314042889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6823","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6825","nonce":3314042889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6822","nonce":3314042889}]},"public_addr":"172.21.15.88:6817/3314042889","cluster_addr":"172.21.15.88:6819/3314042889","heartbeat_back_addr":"172.21.15.88:6825/3314042889","heartbeat_front_addr":"172.21.15.88:6822/3314042889","state":["exists","up"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6806","nonce":508820735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6810","nonce":508820735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6815","nonce":508820735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6813","nonce":508820735}]},"public_addr":"172.21.15.88:6806/508820735","cluster_addr":"172.21.15.88:6810/508820735","heartbeat_back_addr":"172.21.15.88:6815/508820735","heartbeat_front_addr":"172.21.15.88:6813/508820735","state":["exists","up"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6824","nonce":217804779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6827","nonce":217804779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6831","nonce":217804779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6829","nonce":217804779}]},"public_addr":"172.21.15.88:6824/217804779","cluster_addr":"172.21.15.88:6827/217804779","heartbeat_back_addr":"172.21.15.88:6831/217804779","heartbeat_front_addr":"172.21.15.88:6829/217804779","state":["exists","up"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6810","nonce":2413638230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6814","nonce":2413638230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6822","nonce":2413638230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6818","nonce":2413638230}]},"public_addr":"172.21.15.154:6810/2413638230","cluster_addr":"172.21.15.154:6814/2413638230","heartbeat_back_addr":"172.21.15.154:6822/2413638230","heartbeat_front_addr":"172.21.15.154:6818/2413638230","state":["exists","up"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6813","nonce":3628934341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6819","nonce":3628934341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6828","nonce":3628934341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6824","nonce":3628934341}]},"public_addr":"172.21.15.154:6813/3628934341","cluster_addr":"172.21.15.154:6819/3628934341","heartbeat_back_addr":"172.21.15.154:6828/3628934341","heartbeat_front_addr":"172.21.15.154:6824/3628934341","state":["exists","up"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6815","nonce":1439702329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6825","nonce":1439702329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6831","nonce":1439702329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6827","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6829","nonce":1439702329}]},"public_addr":"172.21.15.154:6815/1439702329","cluster_addr":"172.21.15.154:6825/1439702329","heartbeat_back_addr":"172.21.15.154:6831/1439702329","heartbeat_front_addr":"172.21.15.154:6829/1439702329","state":["exists","up"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6801","nonce":1245445760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6803","nonce":1245445760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6807","nonce":1245445760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6805","nonce":1245445760}]},"public_addr":"172.21.15.154:6801/1245445760","cluster_addr":"172.21.15.154:6803/1245445760","heartbeat_back_addr":"172.21.15.154:6807/1245445760","heartbeat_front_addr":"172.21.15.154:6805/1245445760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:08.009769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.775680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.793533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.757704+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.747333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.725796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.771745+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.730259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T04:36:18.201 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-17T04:36:18.201 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-17T04:36:18.202 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-17T04:36:18.202 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-17T04:36:18.203 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-17T04:36:18.203 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-17T04:36:18.203 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-17T04:36:18.204 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-17T04:36:18.445 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:18.445 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T04:36:18.456 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:18.456 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T04:36:18.465 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:18.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T04:36:18.473 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:18.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T04:36:18.474 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:18.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T04:36:18.480 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-17T04:36:18.480 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T04:36:18.482 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-17T04:36:18.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T04:36:18.483 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:18.483 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T04:36:18.950 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:18.951 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:18.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-04-17T04:36:18.971 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:18.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-04-17T04:36:18.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-04-17T04:36:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:19.013 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:19.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-04-17T04:36:19.033 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:19.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-04-17T04:36:19.042 INFO:teuthology.orchestra.run.smithi088.stdout:55834574851 2024-04-17T04:36:19.043 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:19.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-04-17T04:36:19.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2024-04-17T04:36:19.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-17T04:36:19.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T04:36:19.990 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T04:36:20.001 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T04:36:20.018 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T04:36:20.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T04:36:20.056 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T04:36:20.064 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T04:36:20.065 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T04:36:20.477 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:20.488 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:20.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-04-17T04:36:20.501 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-04-17T04:36:20.510 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.549 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:20.565 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:20.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-17T04:36:20.567 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.579 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-17T04:36:20.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-04-17T04:36:20.591 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.591 INFO:teuthology.orchestra.run.smithi088.stdout:55834574852 2024-04-17T04:36:20.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-17T04:36:20.597 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.599 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-17T04:36:20.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2024-04-17T04:36:20.610 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.619 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-17T04:36:20.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-04-17T04:36:20.622 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-17T04:36:20.635 DEBUG:teuthology.parallel:result is None 2024-04-17T04:36:20.636 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T04:36:20.636 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T04:36:20.936 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:20.937 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T04:36:20.953 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-17T04:36:20.488551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218940,"kb_used_data":2748,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755780,"statfs":{"total":773094113280,"available":772869918720,"internally_reserved":0,"allocated":2813952,"data_stored":1716868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.293584"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.864314+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T04:36:18.864314+0000","last_peered":"2024-04-17T04:36:18.864314+0000","last_clean":"2024-04-17T04:36:18.864314+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T04:36:18.864314+0000","last_undegraded":"2024-04-17T04:36:18.864314+0000","last_fullsized":"2024-04-17T04:36:18.864314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:24.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.224760+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T04:36:18.224760+0000","last_peered":"2024-04-17T04:36:18.224760+0000","last_clean":"2024-04-17T04:36:18.224760+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T04:36:18.224760+0000","last_undegraded":"2024-04-17T04:36:18.224760+0000","last_fullsized":"2024-04-17T04:36:18.224760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:38.757105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.593575+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T04:36:18.593575+0000","last_peered":"2024-04-17T04:36:18.593575+0000","last_clean":"2024-04-17T04:36:18.593575+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T04:36:18.593575+0000","last_undegraded":"2024-04-17T04:36:18.593575+0000","last_fullsized":"2024-04-17T04:36:18.593575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:33.083623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.224651+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T04:36:18.224651+0000","last_peered":"2024-04-17T04:36:18.224651+0000","last_clean":"2024-04-17T04:36:18.224651+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T04:36:18.224651+0000","last_undegraded":"2024-04-17T04:36:18.224651+0000","last_fullsized":"2024-04-17T04:36:18.224651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:16.202650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:19.211563+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T04:36:19.211563+0000","last_peered":"2024-04-17T04:36:19.211563+0000","last_clean":"2024-04-17T04:36:19.211563+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T04:36:19.211563+0000","last_undegraded":"2024-04-17T04:36:19.211563+0000","last_fullsized":"2024-04-17T04:36:19.211563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:44:32.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.222304+0000","last_change":"2024-04-17T04:36:17.752374+0000","last_active":"2024-04-17T04:36:18.222304+0000","last_peered":"2024-04-17T04:36:18.222304+0000","last_clean":"2024-04-17T04:36:18.222304+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T04:36:18.222304+0000","last_undegraded":"2024-04-17T04:36:18.222304+0000","last_fullsized":"2024-04-17T04:36:18.222304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:41.044005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.593567+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T04:36:18.593567+0000","last_peered":"2024-04-17T04:36:18.593567+0000","last_clean":"2024-04-17T04:36:18.593567+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T04:36:18.593567+0000","last_undegraded":"2024-04-17T04:36:18.593567+0000","last_fullsized":"2024-04-17T04:36:18.593567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:06.659093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:19.211688+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T04:36:19.211688+0000","last_peered":"2024-04-17T04:36:19.211688+0000","last_clean":"2024-04-17T04:36:19.211688+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T04:36:19.211688+0000","last_undegraded":"2024-04-17T04:36:19.211688+0000","last_fullsized":"2024-04-17T04:36:19.211688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:13.926220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.593413+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T04:36:18.593413+0000","last_peered":"2024-04-17T04:36:18.593413+0000","last_clean":"2024-04-17T04:36:18.593413+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T04:36:18.593413+0000","last_undegraded":"2024-04-17T04:36:18.593413+0000","last_fullsized":"2024-04-17T04:36:18.593413+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:24:00.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":658103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":652355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T04:36:20.954 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T04:36:21.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:21.226 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T04:36:21.242 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-17T04:36:20.488551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218940,"kb_used_data":2748,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755780,"statfs":{"total":773094113280,"available":772869918720,"internally_reserved":0,"allocated":2813952,"data_stored":1716868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.293584"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.864314+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T04:36:18.864314+0000","last_peered":"2024-04-17T04:36:18.864314+0000","last_clean":"2024-04-17T04:36:18.864314+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T04:36:18.864314+0000","last_undegraded":"2024-04-17T04:36:18.864314+0000","last_fullsized":"2024-04-17T04:36:18.864314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:24.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.224760+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T04:36:18.224760+0000","last_peered":"2024-04-17T04:36:18.224760+0000","last_clean":"2024-04-17T04:36:18.224760+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T04:36:18.224760+0000","last_undegraded":"2024-04-17T04:36:18.224760+0000","last_fullsized":"2024-04-17T04:36:18.224760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:38.757105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.593575+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T04:36:18.593575+0000","last_peered":"2024-04-17T04:36:18.593575+0000","last_clean":"2024-04-17T04:36:18.593575+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T04:36:18.593575+0000","last_undegraded":"2024-04-17T04:36:18.593575+0000","last_fullsized":"2024-04-17T04:36:18.593575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:33.083623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.224651+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T04:36:18.224651+0000","last_peered":"2024-04-17T04:36:18.224651+0000","last_clean":"2024-04-17T04:36:18.224651+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T04:36:18.224651+0000","last_undegraded":"2024-04-17T04:36:18.224651+0000","last_fullsized":"2024-04-17T04:36:18.224651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:16.202650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:19.211563+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T04:36:19.211563+0000","last_peered":"2024-04-17T04:36:19.211563+0000","last_clean":"2024-04-17T04:36:19.211563+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T04:36:19.211563+0000","last_undegraded":"2024-04-17T04:36:19.211563+0000","last_fullsized":"2024-04-17T04:36:19.211563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:44:32.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.222304+0000","last_change":"2024-04-17T04:36:17.752374+0000","last_active":"2024-04-17T04:36:18.222304+0000","last_peered":"2024-04-17T04:36:18.222304+0000","last_clean":"2024-04-17T04:36:18.222304+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T04:36:18.222304+0000","last_undegraded":"2024-04-17T04:36:18.222304+0000","last_fullsized":"2024-04-17T04:36:18.222304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:41.044005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.593567+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T04:36:18.593567+0000","last_peered":"2024-04-17T04:36:18.593567+0000","last_clean":"2024-04-17T04:36:18.593567+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T04:36:18.593567+0000","last_undegraded":"2024-04-17T04:36:18.593567+0000","last_fullsized":"2024-04-17T04:36:18.593567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:06.659093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:19.211688+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T04:36:19.211688+0000","last_peered":"2024-04-17T04:36:19.211688+0000","last_clean":"2024-04-17T04:36:19.211688+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T04:36:19.211688+0000","last_undegraded":"2024-04-17T04:36:19.211688+0000","last_fullsized":"2024-04-17T04:36:19.211688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:13.926220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-17T04:36:18.593413+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T04:36:18.593413+0000","last_peered":"2024-04-17T04:36:18.593413+0000","last_clean":"2024-04-17T04:36:18.593413+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T04:36:18.593413+0000","last_undegraded":"2024-04-17T04:36:18.593413+0000","last_fullsized":"2024-04-17T04:36:18.593413+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:24:00.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":658103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":67735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":652355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T04:36:21.243 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T04:36:21.243 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T04:36:21.243 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-17T04:36:21.243 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T04:36:21.546 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:36:21.546 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T04:36:21.563 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-17T04:36:21.563 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-04-17T04:36:21.575 INFO:teuthology.run_tasks:Running task rgw... 2024-04-17T04:36:21.618 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-04-17T04:36:21.618 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-04-17T04:36:21.619 INFO:tasks.rgw:Creating data pools 2024-04-17T04:36:21.619 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-04-17T04:36:21.619 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2024-04-17T04:36:22.279 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-04-17T04:36:24.268 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'default.rgw.buckets.data' created 2024-04-17T04:36:24.305 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-04-17T04:36:25.290 INFO:teuthology.orchestra.run.smithi088.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-17T04:36:25.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-04-17T04:36:26.331 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'default.rgw.buckets.index' created 2024-04-17T04:36:26.364 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-04-17T04:36:27.350 INFO:teuthology.orchestra.run.smithi088.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-17T04:36:27.396 DEBUG:tasks.rgw:Pools created 2024-04-17T04:36:27.397 INFO:tasks.rgw:Configuring compression type = random 2024-04-17T04:36:27.397 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-04-17T04:36:27.397 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-04-17T04:36:27.397 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-04-17T04:36:27.452 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:27.452 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:29.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.436+0000 7f0b532cd640 0 -- 172.21.15.88:0/2866202710 >> [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] conn(0x7f0a28065d10 msgr2=0x7f0a280681d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:36:29.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.440+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.444+0000 7f0b522cb640 0 -- 172.21.15.88:0/4121477225 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5584965db010 msgr2=0x5584965dd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:36:29.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:29.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 realm 2024-04-17T04:36:29.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:29.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:36:29.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:29.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.648+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.652+0000 7f0b54fb1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:36:29.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.652+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.652+0000 7f0b54fb1980 20 rados_obj.operate() r=0 bl.length=993 2024-04-17T04:36:29.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.652+0000 7f0b54fb1980 20 searching for the correct realm 2024-04-17T04:36:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:36:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:36:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.684+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 20 rados_obj.operate() r=0 bl.length=436 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 20 zone default found 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 4 Realm: () 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 20 zonegroup default 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:29.688+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:31.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:31.396+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:31.396+0000 7f0b54fb1980 20 rados->read ofs=0 len=0 2024-04-17T04:36:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:31.400+0000 7f0b54fb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:31.400+0000 7f0b54fb1980 20 started sync module instance, tier type = 2024-04-17T04:36:31.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:31.400+0000 7f0b54fb1980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:36:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=0 2024-04-17T04:36:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=1 2024-04-17T04:36:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=2 2024-04-17T04:36:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=3 2024-04-17T04:36:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=4 2024-04-17T04:36:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=5 2024-04-17T04:36:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=6 2024-04-17T04:36:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 add_watcher() i=7 2024-04-17T04:36:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 2 all 8 watchers are set, enabling cache 2024-04-17T04:36:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-04-17T04:36:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-04-17T04:36:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.444+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-04-17T04:36:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-04-17T04:36:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-04-17T04:36:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-04-17T04:36:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-04-17T04:36:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-04-17T04:36:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-04-17T04:36:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-04-17T04:36:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-04-17T04:36:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-04-17T04:36:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-04-17T04:36:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-04-17T04:36:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-04-17T04:36:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-04-17T04:36:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-04-17T04:36:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-04-17T04:36:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.448+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-04-17T04:36:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-04-17T04:36:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-04-17T04:36:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-04-17T04:36:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-04-17T04:36:33.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-04-17T04:36:33.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-04-17T04:36:33.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-04-17T04:36:33.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-04-17T04:36:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-04-17T04:36:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.452+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-04-17T04:36:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-04-17T04:36:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-04-17T04:36:33.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-04-17T04:36:33.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-04-17T04:36:33.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-04-17T04:36:33.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-04-17T04:36:33.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-04-17T04:36:33.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-04-17T04:36:33.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-04-17T04:36:33.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-04-17T04:36:33.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-04-17T04:36:33.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-04-17T04:36:33.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-04-17T04:36:33.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.456+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-04-17T04:36:33.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-04-17T04:36:33.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-04-17T04:36:33.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-04-17T04:36:33.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-04-17T04:36:33.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-04-17T04:36:33.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-04-17T04:36:33.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-04-17T04:36:33.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-04-17T04:36:33.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-04-17T04:36:33.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-04-17T04:36:33.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-04-17T04:36:33.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-04-17T04:36:33.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-04-17T04:36:33.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.460+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-04-17T04:36:33.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-04-17T04:36:33.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-04-17T04:36:33.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-04-17T04:36:33.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-04-17T04:36:33.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-04-17T04:36:33.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-04-17T04:36:33.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-04-17T04:36:33.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-04-17T04:36:33.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-04-17T04:36:33.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-04-17T04:36:33.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-04-17T04:36:33.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.464+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-04-17T04:36:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-04-17T04:36:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-04-17T04:36:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-04-17T04:36:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-04-17T04:36:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-04-17T04:36:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-04-17T04:36:33.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-04-17T04:36:33.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.468+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-04-17T04:36:33.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-04-17T04:36:33.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-04-17T04:36:33.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-04-17T04:36:33.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-04-17T04:36:33.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-04-17T04:36:33.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-04-17T04:36:33.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-04-17T04:36:33.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-04-17T04:36:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-04-17T04:36:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-04-17T04:36:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-04-17T04:36:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-04-17T04:36:33.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-04-17T04:36:33.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-04-17T04:36:33.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-04-17T04:36:33.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.472+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-04-17T04:36:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-04-17T04:36:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-04-17T04:36:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-04-17T04:36:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-04-17T04:36:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-04-17T04:36:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-04-17T04:36:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-04-17T04:36:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-04-17T04:36:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-04-17T04:36:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-04-17T04:36:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-04-17T04:36:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-04-17T04:36:33.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-04-17T04:36:33.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-04-17T04:36:33.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-04-17T04:36:33.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.476+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-04-17T04:36:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-04-17T04:36:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-04-17T04:36:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-04-17T04:36:33.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-04-17T04:36:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-04-17T04:36:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-04-17T04:36:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-04-17T04:36:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-04-17T04:36:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-04-17T04:36:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-04-17T04:36:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-04-17T04:36:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-04-17T04:36:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-04-17T04:36:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.480+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-04-17T04:36:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-04-17T04:36:33.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-04-17T04:36:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-04-17T04:36:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-04-17T04:36:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-04-17T04:36:33.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-04-17T04:36:33.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-04-17T04:36:33.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-04-17T04:36:33.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-04-17T04:36:33.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-04-17T04:36:33.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-04-17T04:36:33.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-04-17T04:36:33.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-04-17T04:36:33.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-04-17T04:36:33.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-04-17T04:36:33.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.484+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-04-17T04:36:33.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-04-17T04:36:33.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-04-17T04:36:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-04-17T04:36:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-04-17T04:36:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-04-17T04:36:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-04-17T04:36:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-04-17T04:36:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-04-17T04:36:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.488+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-04-17T04:36:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-04-17T04:36:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-04-17T04:36:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-04-17T04:36:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-04-17T04:36:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-04-17T04:36:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-04-17T04:36:33.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-04-17T04:36:33.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-04-17T04:36:33.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-04-17T04:36:33.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-04-17T04:36:33.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.492+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-04-17T04:36:33.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-04-17T04:36:33.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-04-17T04:36:33.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-04-17T04:36:33.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-04-17T04:36:33.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-04-17T04:36:33.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-04-17T04:36:33.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-04-17T04:36:33.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-04-17T04:36:33.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-04-17T04:36:33.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-04-17T04:36:33.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-04-17T04:36:33.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-04-17T04:36:33.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-04-17T04:36:33.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-04-17T04:36:33.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-04-17T04:36:33.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.496+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-04-17T04:36:33.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-04-17T04:36:33.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-04-17T04:36:33.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-04-17T04:36:33.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-04-17T04:36:33.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-04-17T04:36:33.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-04-17T04:36:33.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-04-17T04:36:33.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.500+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-04-17T04:36:33.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-04-17T04:36:33.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-04-17T04:36:33.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-04-17T04:36:33.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-04-17T04:36:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-04-17T04:36:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-04-17T04:36:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-04-17T04:36:33.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-04-17T04:36:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-04-17T04:36:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-04-17T04:36:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-04-17T04:36:33.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-04-17T04:36:33.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-04-17T04:36:33.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.504+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-04-17T04:36:33.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-04-17T04:36:33.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-04-17T04:36:33.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-04-17T04:36:33.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-04-17T04:36:33.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-04-17T04:36:33.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-04-17T04:36:33.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-04-17T04:36:33.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-04-17T04:36:33.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-04-17T04:36:33.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.508+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-04-17T04:36:33.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-04-17T04:36:33.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-04-17T04:36:33.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-04-17T04:36:33.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-04-17T04:36:33.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-04-17T04:36:33.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-04-17T04:36:33.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-04-17T04:36:33.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-04-17T04:36:33.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-04-17T04:36:33.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-04-17T04:36:33.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-04-17T04:36:33.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-04-17T04:36:33.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-04-17T04:36:33.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.512+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-04-17T04:36:33.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-04-17T04:36:33.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-04-17T04:36:33.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-04-17T04:36:33.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-04-17T04:36:33.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-04-17T04:36:33.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-04-17T04:36:33.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-04-17T04:36:33.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-04-17T04:36:33.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-04-17T04:36:33.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-04-17T04:36:33.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-04-17T04:36:33.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-04-17T04:36:33.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-04-17T04:36:33.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.516+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.520+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-04-17T04:36:33.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.520+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-04-17T04:36:33.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.520+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.520+0000 7f0b54fb1980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-04-17T04:36:33.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.520+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-04-17T04:36:33.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.524+0000 7f0b54fb1980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-17T04:36:33.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.532+0000 7f0a057e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:36:33.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.532+0000 7f0b54fb1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:33.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:33.532+0000 7f0b54fb1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:36.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.396+0000 7f0b54fb1980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-04-17T04:36:36.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.396+0000 7f0b54fb1980 5 note: GC not initialized 2024-04-17T04:36:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.408+0000 7f096ffff640 20 reqs_thread_entry: start 2024-04-17T04:36:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.408+0000 7f0b54fb1980 20 init_complete bucket index max shards: 1 2024-04-17T04:36:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.408+0000 7f0b54fb1980 20 Filter name: none 2024-04-17T04:36:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.408+0000 7f096dffb640 20 reqs_thread_entry: start 2024-04-17T04:36:36.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.440+0000 7f0a057e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:36:36.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.448+0000 7f0b54fb1980 20 remove_watcher() i=0 2024-04-17T04:36:36.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.448+0000 7f0b54fb1980 2 removed watcher, disabling cache 2024-04-17T04:36:36.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.448+0000 7f0b54fb1980 20 remove_watcher() i=1 2024-04-17T04:36:36.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.452+0000 7f0b54fb1980 20 remove_watcher() i=2 2024-04-17T04:36:36.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.452+0000 7f0b54fb1980 20 remove_watcher() i=3 2024-04-17T04:36:36.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.456+0000 7f0b54fb1980 20 remove_watcher() i=4 2024-04-17T04:36:36.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.456+0000 7f0b54fb1980 20 remove_watcher() i=5 2024-04-17T04:36:36.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.460+0000 7f0b54fb1980 20 remove_watcher() i=6 2024-04-17T04:36:36.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.460+0000 7f0b54fb1980 20 remove_watcher() i=7 2024-04-17T04:36:36.475 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-04-17T04:36:36.475 DEBUG:tasks.util.rgw: json result: [] 2024-04-17T04:36:36.475 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-17T04:36:36.476 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-17T04:36:36.476 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-04-17T04:36:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.548+0000 7fb657c24980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.548+0000 7fb657c24980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.552+0000 7fb4937fe640 20 reqs_thread_entry: start 2024-04-17T04:36:36.573 INFO:teuthology.orchestra.run.smithi088.stdout:{"id":"52c3b34b-e732-40a8-b5aa-dfbef5c9e391","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-17T04:36:36.573 DEBUG:tasks.util.rgw: json result: {'id': '52c3b34b-e732-40a8-b5aa-dfbef5c9e391', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-17T04:36:36.573 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-04-17T04:36:36.573 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-04-17T04:36:36.573 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-04-17T04:36:36.650 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:36.650 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:36.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 realm 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.692+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.696+0000 7fc2f64eb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:36:36.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.696+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.696+0000 7fc2f64eb980 20 rados_obj.operate() r=0 bl.length=1009 2024-04-17T04:36:36.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.696+0000 7fc2f64eb980 20 searching for the correct realm 2024-04-17T04:36:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.704+0000 7fc2f4807640 0 -- 172.21.15.88:0/1223007843 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fc2dc0d2030 msgr2=0x7fc2dc0e6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:36:36.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.920+0000 7fc2f4006640 0 -- 172.21.15.88:0/1223007843 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5637ff61f8b0 msgr2=0x5637ff621cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:36:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:36:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados_obj.operate() r=0 bl.length=436 2024-04-17T04:36:36.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 zone default found 2024-04-17T04:36:36.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 4 Realm: () 2024-04-17T04:36:36.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:36:36.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:36:36.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:36:36.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 zonegroup default 2024-04-17T04:36:36.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.928+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 rados->read ofs=0 len=0 2024-04-17T04:36:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 started sync module instance, tier type = 2024-04-17T04:36:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.932+0000 7fc2f64eb980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=0 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=1 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=2 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=3 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=4 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=5 2024-04-17T04:36:36.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.964+0000 7fc2f64eb980 20 add_watcher() i=6 2024-04-17T04:36:36.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.968+0000 7fc2f64eb980 20 add_watcher() i=7 2024-04-17T04:36:36.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.968+0000 7fc2f64eb980 2 all 8 watchers are set, enabling cache 2024-04-17T04:36:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc1a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:36:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc2f64eb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc2f64eb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc2f64eb980 5 note: GC not initialized 2024-04-17T04:36:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc1a8fe1640 20 reqs_thread_entry: start 2024-04-17T04:36:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc2f64eb980 20 init_complete bucket index max shards: 1 2024-04-17T04:36:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc2f64eb980 20 Filter name: none 2024-04-17T04:36:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.972+0000 7fc1227fc640 20 reqs_thread_entry: start 2024-04-17T04:36:36.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:36.996+0000 7fc1a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:36:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.000+0000 7fc2f64eb980 20 remove_watcher() i=0 2024-04-17T04:36:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.000+0000 7fc2f64eb980 2 removed watcher, disabling cache 2024-04-17T04:36:37.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.004+0000 7fc2f64eb980 20 remove_watcher() i=1 2024-04-17T04:36:37.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.004+0000 7fc2f64eb980 20 remove_watcher() i=2 2024-04-17T04:36:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.008+0000 7fc2f64eb980 20 remove_watcher() i=3 2024-04-17T04:36:37.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.008+0000 7fc2f64eb980 20 remove_watcher() i=4 2024-04-17T04:36:37.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.012+0000 7fc2f64eb980 20 remove_watcher() i=5 2024-04-17T04:36:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.012+0000 7fc2f64eb980 20 remove_watcher() i=6 2024-04-17T04:36:37.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.016+0000 7fc2f64eb980 20 remove_watcher() i=7 2024-04-17T04:36:37.026 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-04-17T04:36:37.027 DEBUG:tasks.util.rgw: json result: [] 2024-04-17T04:36:37.027 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2024-04-17T04:36:37.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2024-04-17T04:36:37.027 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --placement-inline-data false 2024-04-17T04:36:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:37.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.140+0000 7efd67c06980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:37.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.140+0000 7efd67c06980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:37.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.144+0000 7efba37fe640 20 reqs_thread_entry: start 2024-04-17T04:36:37.165 INFO:teuthology.orchestra.run.smithi088.stdout:{"id":"52c3b34b-e732-40a8-b5aa-dfbef5c9e391","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-17T04:36:37.165 DEBUG:tasks.util.rgw: json result: {'id': '52c3b34b-e732-40a8-b5aa-dfbef5c9e391', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-17T04:36:37.166 INFO:tasks.rgw:Preparing directory for rgw datacache at /tmp/rgw_datacache 2024-04-17T04:36:37.166 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p /tmp/rgw_datacache 2024-04-17T04:36:37.214 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a+rwx /tmp/rgw_datacache 2024-04-17T04:36:37.269 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-04-17T04:36:37.269 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-04-17T04:36:37.269 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-04-17T04:36:37.351 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:37.351 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:37.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.396+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 realm 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:36:37.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 rados_obj.operate() r=0 bl.length=1009 2024-04-17T04:36:37.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.400+0000 7f9c338c4980 20 searching for the correct realm 2024-04-17T04:36:37.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.404+0000 7f9c313df640 0 -- 172.21.15.88:0/191965457 >> [v2:172.21.15.88:6804/508820735,v1:172.21.15.88:6806/508820735] conn(0x55fc7f531620 msgr2=0x7f9c2c010930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:36:37.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.412+0000 7f9c313df640 0 -- 172.21.15.88:0/191965457 >> [v2:172.21.15.88:6804/508820735,v1:172.21.15.88:6806/508820735] conn(0x7f9afc073640 msgr2=0x7f9afc075a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:36:37.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:36:37.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:36:37.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:36:37.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.420+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados_obj.operate() r=0 bl.length=436 2024-04-17T04:36:37.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 zone default found 2024-04-17T04:36:37.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 4 Realm: () 2024-04-17T04:36:37.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:36:37.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:36:37.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:36:37.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 zonegroup default 2024-04-17T04:36:37.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados->read ofs=0 len=0 2024-04-17T04:36:37.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:36:37.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 started sync module instance, tier type = 2024-04-17T04:36:37.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.424+0000 7f9c338c4980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=0 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=1 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=2 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=3 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=4 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=5 2024-04-17T04:36:37.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=6 2024-04-17T04:36:37.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 20 add_watcher() i=7 2024-04-17T04:36:37.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.444+0000 7f9c338c4980 2 all 8 watchers are set, enabling cache 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9ae77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9c338c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9c338c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9c338c4980 5 note: GC not initialized 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9ae67e4640 20 reqs_thread_entry: start 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9c338c4980 20 init_complete bucket index max shards: 1 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9c338c4980 20 Filter name: none 2024-04-17T04:36:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.448+0000 7f9a57fff640 20 reqs_thread_entry: start 2024-04-17T04:36:37.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.468+0000 7f9ae77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:36:37.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.472+0000 7f9c338c4980 20 remove_watcher() i=0 2024-04-17T04:36:37.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.472+0000 7f9c338c4980 2 removed watcher, disabling cache 2024-04-17T04:36:37.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.476+0000 7f9c338c4980 20 remove_watcher() i=1 2024-04-17T04:36:37.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.476+0000 7f9c338c4980 20 remove_watcher() i=2 2024-04-17T04:36:37.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.480+0000 7f9c338c4980 20 remove_watcher() i=3 2024-04-17T04:36:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.480+0000 7f9c338c4980 20 remove_watcher() i=4 2024-04-17T04:36:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.484+0000 7f9c338c4980 20 remove_watcher() i=5 2024-04-17T04:36:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.488+0000 7f9c338c4980 20 remove_watcher() i=6 2024-04-17T04:36:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.488+0000 7f9c338c4980 20 remove_watcher() i=7 2024-04-17T04:36:37.502 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-04-17T04:36:37.503 DEBUG:tasks.util.rgw: json result: [] 2024-04-17T04:36:37.503 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-04-17T04:36:37.503 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-04-17T04:36:37.503 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-04-17T04:36:37.503 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-04-17T04:36:37.586 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:37.586 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.616+0000 7f430e7f3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.616+0000 7f430e7f3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.620+0000 7f41527fc640 20 reqs_thread_entry: start 2024-04-17T04:36:37.641 INFO:teuthology.orchestra.run.smithi088.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-04-17T04:36:37.641 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-04-17T04:36:37.641 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-04-17T04:36:37.642 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-04-17T04:36:37.642 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-04-17T04:36:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:37.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.756+0000 7f0def435980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:37.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.756+0000 7f0def435980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:37.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.760+0000 7f0c32ffd640 20 reqs_thread_entry: start 2024-04-17T04:36:37.778 INFO:teuthology.orchestra.run.smithi088.stdout:{"id":"52c3b34b-e732-40a8-b5aa-dfbef5c9e391","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-17T04:36:37.778 DEBUG:tasks.util.rgw: json result: {'id': '52c3b34b-e732-40a8-b5aa-dfbef5c9e391', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-17T04:36:37.778 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-04-17T04:36:37.778 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-04-17T04:36:37.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-04-17T04:36:37.779 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-04-17T04:36:37.855 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:37.856 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:37.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.884+0000 7fdf11abc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:37.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.884+0000 7fdf11abc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:37.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:37.888+0000 7fdd75ffb640 20 reqs_thread_entry: start 2024-04-17T04:36:37.907 INFO:teuthology.orchestra.run.smithi088.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-04-17T04:36:37.907 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-04-17T04:36:37.907 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-04-17T04:36:37.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-04-17T04:36:37.907 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-04-17T04:36:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:36:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:36:38.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:38.016+0000 7f56dd58f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:36:38.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:38.016+0000 7f56dd58f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:36:38.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:36:38.020+0000 7f5541ffb640 20 reqs_thread_entry: start 2024-04-17T04:36:38.039 INFO:teuthology.orchestra.run.smithi088.stdout:{"id":"52c3b34b-e732-40a8-b5aa-dfbef5c9e391","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-17T04:36:38.039 DEBUG:tasks.util.rgw: json result: {'id': '52c3b34b-e732-40a8-b5aa-dfbef5c9e391', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-17T04:36:38.039 INFO:tasks.rgw:Starting rgw... 2024-04-17T04:36:38.039 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-04-17T04:36:38.039 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-17T04:36:38.039 DEBUG:teuthology.orchestra.run.smithi088:> sudo echo -n http://smithi088.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-17T04:36:38.091 INFO:teuthology.orchestra.run.smithi088.stdout:http://smithi088.front.sepia.ceph.com:80 2024-04-17T04:36:38.091 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-17T04:36:38.146 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-04-17T04:36:38.146 INFO:tasks.rgw.client.0:Restarting daemon 2024-04-17T04:36:38.146 DEBUG:teuthology.orchestra.run.smithi088:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-04-17T04:36:38.194 INFO:tasks.rgw.client.0:Started 2024-04-17T04:36:38.194 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:36:38.194 DEBUG:teuthology.orchestra.run.smithi088:> curl http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:36:38.285 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-17T04:36:38.285 INFO:teuthology.orchestra.run.smithi088.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-17T04:36:38.285 INFO:teuthology.orchestra.run.smithi088.stderr: Dload Upload Total Spent Left Speed 2024-04-17T04:36:38.286 INFO:teuthology.orchestra.run.smithi088.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-17T04:36:38.286 INFO:teuthology.orchestra.run.smithi088.stderr:curl: (7) Failed to connect to smithi088.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-04-17T04:36:39.287 DEBUG:teuthology.orchestra.run.smithi088:> curl http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:36:39.298 INFO:teuthology.orchestra.run.smithi088.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-17T04:36:39.298 INFO:teuthology.orchestra.run.smithi088.stderr: Dload Upload Total Spent Left Speed 2024-04-17T04:36:39.298 INFO:teuthology.orchestra.run.smithi088.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-17T04:36:39.298 INFO:teuthology.orchestra.run.smithi088.stderr:curl: (7) Failed to connect to smithi088.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-04-17T04:36:39.299 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-17T04:36:41.300 DEBUG:teuthology.orchestra.run.smithi088:> curl http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:36:41.311 INFO:teuthology.orchestra.run.smithi088.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-17T04:36:41.311 INFO:teuthology.orchestra.run.smithi088.stderr: Dload Upload Total Spent Left Speed 2024-04-17T04:36:41.311 INFO:teuthology.orchestra.run.smithi088.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-17T04:36:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:curl: (7) Failed to connect to smithi088.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-04-17T04:36:41.312 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-17T04:36:45.313 DEBUG:teuthology.orchestra.run.smithi088:> curl http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:36:45.323 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-17T04:36:45.323 INFO:teuthology.orchestra.run.smithi088.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-17T04:36:45.323 INFO:teuthology.orchestra.run.smithi088.stderr: Dload Upload Total Spent Left Speed 2024-04-17T04:36:45.323 INFO:teuthology.orchestra.run.smithi088.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-17T04:36:45.324 INFO:teuthology.orchestra.run.smithi088.stderr:curl: (7) Failed to connect to smithi088.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-04-17T04:36:53.325 DEBUG:teuthology.orchestra.run.smithi088:> curl http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:36:53.336 INFO:teuthology.orchestra.run.smithi088.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-17T04:36:53.336 INFO:teuthology.orchestra.run.smithi088.stderr: Dload Upload Total Spent Left Speed 2024-04-17T04:36:53.336 INFO:teuthology.orchestra.run.smithi088.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-17T04:36:53.337 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-17T04:36:53.338 INFO:teuthology.orchestra.run.smithi088.stderr:curl: (7) Failed to connect to smithi088.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-04-17T04:37:09.339 DEBUG:teuthology.orchestra.run.smithi088:> curl http://smithi088.front.sepia.ceph.com:80/ 2024-04-17T04:37:09.351 INFO:teuthology.orchestra.run.smithi088.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-17T04:37:09.351 INFO:teuthology.orchestra.run.smithi088.stderr: Dload Upload Total Spent Left Speed 2024-04-17T04:37:09.850 INFO:teuthology.orchestra.run.smithi088.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 374 0 --:--:-- --:--:-- --:--:-- 375 2024-04-17T04:37:09.851 INFO:teuthology.orchestra.run.smithi088.stdout:anonymous 2024-04-17T04:37:09.852 INFO:teuthology.run_tasks:Running task tox... 2024-04-17T04:37:09.864 INFO:tasks.tox:Deploying tox from pip... 2024-04-17T04:37:09.864 DEBUG:teuthology.orchestra.run.smithi088:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-04-17T04:37:12.744 DEBUG:teuthology.orchestra.run.smithi088:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-04-17T04:37:13.443 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting tox==3.15.0 2024-04-17T04:37:13.551 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-04-17T04:37:13.614 INFO:teuthology.orchestra.run.smithi088.stdout: ---------------------------------------- 82.1/82.1 KB 1.3 MB/s eta 0:00:00 2024-04-17T04:37:13.730 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting toml>=0.9.4 2024-04-17T04:37:13.749 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-04-17T04:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting pluggy<1,>=0.12.0 2024-04-17T04:37:13.893 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-04-17T04:37:14.055 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting packaging>=14 2024-04-17T04:37:14.073 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading packaging-24.0-py3-none-any.whl (53 kB) 2024-04-17T04:37:14.117 INFO:teuthology.orchestra.run.smithi088.stdout: ---------------------------------------- 53.5/53.5 KB 1.1 MB/s eta 0:00:00 2024-04-17T04:37:14.200 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting six<2,>=1.14.0 2024-04-17T04:37:14.219 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-17T04:37:14.596 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting filelock<4,>=3.0.0 2024-04-17T04:37:14.614 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading filelock-3.13.4-py3-none-any.whl (11 kB) 2024-04-17T04:37:14.736 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting py<2,>=1.4.17 2024-04-17T04:37:14.755 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-04-17T04:37:14.811 INFO:teuthology.orchestra.run.smithi088.stdout: ---------------------------------------- 98.7/98.7 KB 1.7 MB/s eta 0:00:00 2024-04-17T04:37:15.067 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-04-17T04:37:15.084 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading virtualenv-20.25.2-py3-none-any.whl (3.9 MB) 2024-04-17T04:37:15.564 INFO:teuthology.orchestra.run.smithi088.stdout: ---------------------------------------- 3.9/3.9 MB 8.2 MB/s eta 0:00:00 2024-04-17T04:37:15.754 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting platformdirs<5,>=3.9.1 2024-04-17T04:37:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading platformdirs-4.2.0-py3-none-any.whl (17 kB) 2024-04-17T04:37:15.876 INFO:teuthology.orchestra.run.smithi088.stdout:Collecting distlib<1,>=0.3.7 2024-04-17T04:37:15.896 INFO:teuthology.orchestra.run.smithi088.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-04-17T04:37:15.964 INFO:teuthology.orchestra.run.smithi088.stdout: --------------------------------------- 468.9/468.9 KB 7.2 MB/s eta 0:00:00 2024-04-17T04:37:16.126 INFO:teuthology.orchestra.run.smithi088.stdout:Installing collected packages: distlib, toml, six, py, pluggy, platformdirs, packaging, filelock, virtualenv, tox 2024-04-17T04:37:17.234 INFO:teuthology.orchestra.run.smithi088.stdout:Successfully installed distlib-0.3.8 filelock-3.13.4 packaging-24.0 platformdirs-4.2.0 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.25.2 2024-04-17T04:37:17.327 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T04:37:17.343 INFO:tasks.workunit:Pulling workunits from ref 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:37:17.343 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T04:37:17.343 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:37:17.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:37:17.349 INFO:teuthology.orchestra.run.smithi088.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T04:37:17.349 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:37:17.398 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-17T04:37:17.398 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-17T04:37:17.450 INFO:tasks.workunit:timeout=3h 2024-04-17T04:37:17.450 INFO:tasks.workunit:cleanup=True 2024-04-17T04:37:17.451 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:37:17.500 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T04:38:26.239 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '41c500dd60c93c8d83053b72cffc3dd1a952a336'. 2024-04-17T04:38:26.239 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.239 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T04:38:26.239 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T04:38:26.239 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:38:26.240 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 41c500dd60c Merge pull request #56902 from adk3798/wip-65415-squid 2024-04-17T04:38:26.242 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T04:38:26.292 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T04:38:26.294 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:38:26.294 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T04:38:27.060 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T04:38:27.140 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T04:38:27.191 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:38:27.193 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:38:27.194 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T04:38:27.242 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:38:27.246 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:38:27.246 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T04:38:27.295 INFO:tasks.workunit:Running workunits matching rgw/run-datacache.sh on client.0... 2024-04-17T04:38:27.296 INFO:tasks.workunit:Running workunit rgw/run-datacache.sh... 2024-04-17T04:38:27.296 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/run-datacache.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RGW_DATACACHE_PATH=/tmp/rgw_datacache adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-04-17T04:38:27.348 INFO:tasks.workunit.client.0.smithi088.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-04-17T04:38:27.348 INFO:tasks.workunit.client.0.smithi088.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:38:27.348 INFO:tasks.workunit.client.0.smithi088.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:38:30.293 INFO:tasks.workunit.client.0.smithi088.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-04-17T04:38:30.293 INFO:tasks.workunit.client.0.smithi088.stderr:++ deactivate nondestructive 2024-04-17T04:38:30.293 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:38:30.293 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:38:30.293 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:38:30.294 INFO:tasks.workunit.client.0.smithi088.stderr:++ hash -r 2024-04-17T04:38:30.294 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:38:30.294 INFO:tasks.workunit.client.0.smithi088.stderr:++ unset VIRTUAL_ENV 2024-04-17T04:38:30.294 INFO:tasks.workunit.client.0.smithi088.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-04-17T04:38:30.294 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-04-17T04:38:30.294 INFO:tasks.workunit.client.0.smithi088.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:38:30.295 INFO:tasks.workunit.client.0.smithi088.stderr:++ export VIRTUAL_ENV 2024-04-17T04:38:30.295 INFO:tasks.workunit.client.0.smithi088.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:38:30.295 INFO:tasks.workunit.client.0.smithi088.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:38:30.295 INFO:tasks.workunit.client.0.smithi088.stderr:++ export PATH 2024-04-17T04:38:30.295 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:38:30.295 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -z '' ']' 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ _OLD_VIRTUAL_PS1= 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ PS1='(rgw) ' 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ export PS1 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ export VIRTUAL_ENV_PROMPT 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:++ hash -r 2024-04-17T04:38:30.296 INFO:tasks.workunit.client.0.smithi088.stderr:+ pip install pip --upgrade 2024-04-17T04:38:30.676 INFO:tasks.workunit.client.0.smithi088.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-04-17T04:38:30.928 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting pip 2024-04-17T04:38:31.038 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-17T04:38:31.220 INFO:tasks.workunit.client.0.smithi088.stdout: ---------------------------------------- 2.1/2.1 MB 11.8 MB/s eta 0:00:00 2024-04-17T04:38:31.295 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: pip 2024-04-17T04:38:31.295 INFO:tasks.workunit.client.0.smithi088.stdout: Attempting uninstall: pip 2024-04-17T04:38:31.295 INFO:tasks.workunit.client.0.smithi088.stdout: Found existing installation: pip 22.0.2 2024-04-17T04:38:31.586 INFO:tasks.workunit.client.0.smithi088.stdout: Uninstalling pip-22.0.2: 2024-04-17T04:38:31.593 INFO:tasks.workunit.client.0.smithi088.stdout: Successfully uninstalled pip-22.0.2 2024-04-17T04:38:32.608 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed pip-24.0 2024-04-17T04:38:32.678 INFO:tasks.workunit.client.0.smithi088.stderr:+ pip install configobj 2024-04-17T04:38:33.232 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting configobj 2024-04-17T04:38:33.360 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading configobj-5.0.8-py2.py3-none-any.whl.metadata (3.4 kB) 2024-04-17T04:38:33.524 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting six (from configobj) 2024-04-17T04:38:33.546 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-17T04:38:33.620 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading configobj-5.0.8-py2.py3-none-any.whl (36 kB) 2024-04-17T04:38:33.715 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-17T04:38:33.837 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: six, configobj 2024-04-17T04:38:33.989 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed configobj-5.0.8 six-1.16.0 2024-04-17T04:38:34.426 INFO:tasks.workunit.client.0.smithi088.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_datacache.py 2024-04-17T04:38:34.464 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:datacache dir from config is: /tmp/rgw_datacache 2024-04-17T04:38:34.465 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(pwd) 2024-04-17T04:38:34.466 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:34.466 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b'/home/ubuntu/cephtest/mnt.0/client.0/tmp\n' 2024-04-17T04:38:34.466 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:pwd is: /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:38:34.466 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(sudo netstat -nltp | egrep "rados|valgr") 2024-04-17T04:38:34.512 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:34.512 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b'tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 37533/valgrind.bin \ntcp6 0 0 :::80 :::* LISTEN 37533/valgrind.bin \n' 2024-04-17T04:38:34.512 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:radosgw port: 80 2024-04-17T04:38:34.512 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:radosgw endpoint is: 127.0.0.1 2024-04-17T04:38:34.512 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat bs=1M count=7) 2024-04-17T04:38:34.538 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:34.538 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b'' 2024-04-17T04:38:34.538 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(radosgw-admin user create --uid rgw_datacache_user --display-name DatacacheUser --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn) 2024-04-17T04:38:35.016 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:35.016 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b'{\n "user_id": "rgw_datacache_user",\n "display_name": "DatacacheUser",\n "email": "",\n "suspended": 0,\n "max_buckets": 1000,\n "subusers": [],\n "keys": [\n {\n "user": "rgw_datacache_user",\n "access_key": "NX5QOQKC6BH2IDN8HC7A",\n "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn",\n "active": true,\n "create_date": "2024-04-17T04:38:34.899871Z"\n }\n ],\n "swift_keys": [],\n "caps": [],\n "op_mask": "read, write, delete",\n "default_placement": "",\n "default_storage_class": "",\n "placement_tags": [],\n "bucket_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "user_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "temp_url_keys": [],\n "type": "rgw",\n "mfa_ids": [],\n "account_id": "",\n "path": "/",\n "create_date": "2024-04-17T04:38:34.899843Z",\n "tags": [],\n "group_ids": []\n}\n\n' 2024-04-17T04:38:35.016 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:Creating s3cmd config... 2024-04-17T04:38:35.016 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:proto for s3cmd config is http 2024-04-17T04:38:35.016 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:s3cmd config written 2024-04-17T04:38:35.016 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 mb s3://datacachebucket) 2024-04-17T04:38:36.049 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:36.049 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b"Bucket 's3://datacachebucket/' created\n" 2024-04-17T04:38:36.049 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 put /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat s3://datacachebucket) 2024-04-17T04:38:38.454 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:38.454 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b"upload: '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat' -> 's3://datacachebucket/7M.dat' (7340032 bytes in 2.1 seconds, 3.34 MB/s) [1 of 1]\n" 2024-04-17T04:38:38.454 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 get s3://datacachebucket/7M.dat /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat --force) 2024-04-17T04:38:39.374 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:39.374 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b"download: 's3://datacachebucket/7M.dat' -> '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat' (7340032 bytes in 0.7 seconds, 9.84 MB/s)\n" 2024-04-17T04:38:39.374 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(radosgw-admin object stat --bucket=datacachebucket --object=7M.dat) 2024-04-17T04:38:39.969 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:39.969 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b'{\n "name": "7M.dat",\n "size": 7368844,\n "policy": {\n "acl": {\n "acl_user_map": [\n {\n "user": "rgw_datacache_user",\n "acl": 15\n }\n ],\n "acl_group_map": [],\n "grant_map": [\n {\n "id": "rgw_datacache_user",\n "grant": {\n "type": {\n "type": 0\n },\n "id": "rgw_datacache_user",\n "name": "DatacacheUser",\n "permission": {\n "flags": 15\n }\n }\n }\n ]\n },\n "owner": {\n "id": "rgw_datacache_user",\n "display_name": "DatacacheUser"\n }\n },\n "compression": {\n "compression_type": "lz4",\n "orig_size": 7340032,\n "blocks": [\n {\n "old_ofs": 0,\n "new_ofs": 0,\n "len": 4210766\n },\n {\n "old_ofs": 4194304,\n "new_ofs": 4210766,\n "len": 3158078\n }\n ]\n },\n "etag": "73d24262f1ade6a68bc4df903caeb1b0",\n "tag": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5038382721159750457",\n "manifest": {\n "objs": [],\n "obj_size": 7368844,\n "explicit_objs": false,\n "head_size": 0,\n "max_head_size": 0,\n "prefix": ".WLorTiBn_lkqwoBg1iCnyFbfmxP0lZA_",\n "rules": [\n {\n "key": 0,\n "val": {\n "start_part_num": 0,\n "start_ofs": 0,\n "part_size": 0,\n "stripe_max_size": 6291456,\n "override_prefix": ""\n }\n }\n ],\n "tail_instance": "",\n "tail_placement": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1",\n "bucket_id": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "placement_rule": "default-placement"\n },\n "tier_type": "",\n "begin_iter": {\n "part_ofs": 0,\n "stripe_ofs": 0,\n "ofs": 0,\n "stripe_size": 6291456,\n "cur_part_id": 0,\n "cur_stripe": 0,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1",\n "bucket_id": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".WLorTiBn_lkqwoBg1iCnyFbfmxP0lZA_0",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n },\n "end_iter": {\n "part_ofs": 0,\n "stripe_ofs": 6291456,\n "ofs": 7368844,\n "stripe_size": 1077388,\n "cur_part_id": 0,\n "cur_stripe": 1,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1",\n "bucket_id": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".WLorTiBn_lkqwoBg1iCnyFbfmxP0lZA_1",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n }\n },\n "pg_ver": 0,\n "source_zone": 0,\n "torrent": "",\n "attrs": {\n "user.rgw.content_type": "application/octet-stream",\n "user.rgw.storage_class": "STANDARD",\n "user.rgw.tail_tag": "52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5038382721159750457",\n "user.rgw.x-amz-content-sha256": "01783a235a7f4bf2f38bea94604ac42e962698fcaee5633546a313b6a428443d",\n "user.rgw.x-amz-date": "20240417T043836Z",\n "user.rgw.x-amz-meta-s3cmd-attrs": "atime:1713328714/ctime:1713328714/gid:1271/gname:ubuntu/md5:73d24262f1ade6a68bc4df903caeb1b0/mode:33204/mtime:1713328714/uid:1000/uname:ubuntu"\n }\n}\n' 2024-04-17T04:38:39.970 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:Cached object name is: .WLorTiBn_lkqwoBg1iCnyFbfmxP0lZA_ 2024-04-17T04:38:39.970 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:exec_cmd(find /tmp/rgw_datacache -type f | wc -l) 2024-04-17T04:38:39.974 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:command succeeded 2024-04-17T04:38:39.974 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:b'0\n' 2024-04-17T04:38:39.974 INFO:tasks.workunit.client.0.smithi088.stderr:DEBUG:root:Check cache dir content: 0 2024-04-17T04:38:39.974 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:NOTICE: datacache test object not found, inspect if datacache was bypassed or disabled during this check. 2024-04-17T04:38:39.974 INFO:tasks.workunit.client.0.smithi088.stderr:INFO:root:Completed Datacache tests 2024-04-17T04:38:39.980 INFO:tasks.workunit.client.0.smithi088.stderr:+ deactivate 2024-04-17T04:38:39.980 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-04-17T04:38:39.980 INFO:tasks.workunit.client.0.smithi088.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ export PATH 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset _OLD_VIRTUAL_PATH 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n '' ']' 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ hash -r 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n '' ']' 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset VIRTUAL_ENV 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' '!' '' = nondestructive ']' 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset -f deactivate 2024-04-17T04:38:39.981 INFO:tasks.workunit.client.0.smithi088.stderr:+ echo OK. 2024-04-17T04:38:39.982 INFO:tasks.workunit.client.0.smithi088.stdout:OK. 2024-04-17T04:38:39.982 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:38:39.983 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:38:40.043 INFO:tasks.workunit:Stopping ['rgw/run-datacache.sh'] on client.0... 2024-04-17T04:38:40.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T04:38:40.387 DEBUG:teuthology.parallel:result is None 2024-04-17T04:38:40.387 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:38:40.399 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:38:40.399 DEBUG:teuthology.orchestra.run.smithi088:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:38:40.452 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:38:40.453 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T04:38:40.462 INFO:tasks.workunit:Pulling workunits from ref 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:38:40.462 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T04:38:40.462 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:38:40.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:38:40.502 INFO:teuthology.orchestra.run.smithi088.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T04:38:40.502 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:38:40.550 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-17T04:38:40.550 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-17T04:38:40.598 INFO:tasks.workunit:timeout=3h 2024-04-17T04:38:40.598 INFO:tasks.workunit:cleanup=True 2024-04-17T04:38:40.599 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:38:40.648 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T04:39:46.280 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '41c500dd60c93c8d83053b72cffc3dd1a952a336'. 2024-04-17T04:39:46.280 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T04:39:46.281 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:39:46.282 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 41c500dd60c Merge pull request #56902 from adk3798/wip-65415-squid 2024-04-17T04:39:46.284 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T04:39:46.331 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T04:39:46.333 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:39:46.333 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T04:39:46.386 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T04:39:46.438 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T04:39:46.478 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:39:46.479 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:39:46.479 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T04:39:46.520 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:39:46.523 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:39:46.523 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T04:39:46.570 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-04-17T04:39:46.572 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-04-17T04:39:46.572 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-04-17T04:39:46.623 INFO:tasks.workunit.client.0.smithi088.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-04-17T04:39:46.624 INFO:tasks.workunit.client.0.smithi088.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:39:46.624 INFO:tasks.workunit.client.0.smithi088.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ deactivate nondestructive 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ hash -r 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:39:49.563 INFO:tasks.workunit.client.0.smithi088.stderr:++ unset VIRTUAL_ENV 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ export VIRTUAL_ENV 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ export PATH 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -z '' ']' 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ _OLD_VIRTUAL_PS1= 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ PS1='(rgw) ' 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ export PS1 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ export VIRTUAL_ENV_PROMPT 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:39:49.564 INFO:tasks.workunit.client.0.smithi088.stderr:++ hash -r 2024-04-17T04:39:49.565 INFO:tasks.workunit.client.0.smithi088.stderr:+ pip install pip --upgrade 2024-04-17T04:39:49.953 INFO:tasks.workunit.client.0.smithi088.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-04-17T04:39:50.183 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting pip 2024-04-17T04:39:50.189 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-17T04:39:50.228 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: pip 2024-04-17T04:39:50.228 INFO:tasks.workunit.client.0.smithi088.stdout: Attempting uninstall: pip 2024-04-17T04:39:50.228 INFO:tasks.workunit.client.0.smithi088.stdout: Found existing installation: pip 22.0.2 2024-04-17T04:39:50.499 INFO:tasks.workunit.client.0.smithi088.stdout: Uninstalling pip-22.0.2: 2024-04-17T04:39:50.505 INFO:tasks.workunit.client.0.smithi088.stdout: Successfully uninstalled pip-22.0.2 2024-04-17T04:39:51.525 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed pip-24.0 2024-04-17T04:39:51.594 INFO:tasks.workunit.client.0.smithi088.stderr:+ pip install boto3 2024-04-17T04:39:52.915 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting boto3 2024-04-17T04:39:53.037 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading boto3-1.34.85-py3-none-any.whl.metadata (6.6 kB) 2024-04-17T04:39:53.926 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting botocore<1.35.0,>=1.34.85 (from boto3) 2024-04-17T04:39:53.943 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading botocore-1.34.85-py3-none-any.whl.metadata (5.7 kB) 2024-04-17T04:39:54.112 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-04-17T04:39:54.132 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-04-17T04:39:54.300 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-04-17T04:39:54.320 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-04-17T04:39:54.504 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:39:54.524 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-04-17T04:39:54.750 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:39:54.768 INFO:tasks.workunit.client.0.smithi088.stdout: Downloading urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-04-17T04:39:55.135 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:39:55.136 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-17T04:39:55.164 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading boto3-1.34.85-py3-none-any.whl (139 kB) 2024-04-17T04:39:55.275 INFO:tasks.workunit.client.0.smithi088.stdout: ---------------------------------------- 139.3/139.3 kB 1.2 MB/s eta 0:00:00 2024-04-17T04:39:55.296 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading botocore-1.34.85-py3-none-any.whl (12.1 MB) 2024-04-17T04:39:55.625 INFO:tasks.workunit.client.0.smithi088.stdout: ---------------------------------------- 12.1/12.1 MB 37.8 MB/s eta 0:00:00 2024-04-17T04:39:55.645 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-04-17T04:39:55.737 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-04-17T04:39:55.804 INFO:tasks.workunit.client.0.smithi088.stdout: ---------------------------------------- 82.2/82.2 kB 1.2 MB/s eta 0:00:00 2024-04-17T04:39:55.825 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-17T04:39:55.888 INFO:tasks.workunit.client.0.smithi088.stdout: ---------------------------------------- 229.9/229.9 kB 3.8 MB/s eta 0:00:00 2024-04-17T04:39:55.908 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-04-17T04:39:55.964 INFO:tasks.workunit.client.0.smithi088.stdout: ---------------------------------------- 121.1/121.1 kB 2.2 MB/s eta 0:00:00 2024-04-17T04:39:55.967 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-17T04:39:56.115 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-04-17T04:39:57.347 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed boto3-1.34.85 botocore-1.34.85 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-04-17T04:39:57.514 INFO:tasks.workunit.client.0.smithi088.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-04-17T04:39:57.664 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-04-17T04:39:57.701 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:39:57.702 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:39:57.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.744+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:57.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.744+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.744+0000 7f6289d60980 20 realm 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.744+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:57.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:57.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:39:57.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:57.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:39:57.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.748+0000 7f6289d60980 20 searching for the correct realm 2024-04-17T04:39:57.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.752+0000 7f628707a640 0 -- 172.21.15.88:0/2929156454 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f6278049040 msgr2=0x7f6278088820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:39:57.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:57.968+0000 7f628787b640 0 -- 172.21.15.88:0/2929156454 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f627c0b83e0 msgr2=0x7f627c0b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:39:58.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:39:58.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:39:58.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.184+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:39:58.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 zone default found 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 4 Realm: () 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 zonegroup default 2024-04-17T04:39:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:58.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.188+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.192+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:58.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.192+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.192+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:58.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.192+0000 7f6289d60980 20 started sync module instance, tier type = 2024-04-17T04:39:58.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.192+0000 7f6289d60980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:39:58.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.216+0000 7f6289d60980 20 add_watcher() i=0 2024-04-17T04:39:58.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.216+0000 7f6289d60980 20 add_watcher() i=1 2024-04-17T04:39:58.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.216+0000 7f6289d60980 20 add_watcher() i=2 2024-04-17T04:39:58.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.216+0000 7f6289d60980 20 add_watcher() i=3 2024-04-17T04:39:58.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.216+0000 7f6289d60980 20 add_watcher() i=4 2024-04-17T04:39:58.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.216+0000 7f6289d60980 20 add_watcher() i=5 2024-04-17T04:39:58.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.220+0000 7f6289d60980 20 add_watcher() i=6 2024-04-17T04:39:58.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.224+0000 7f6289d60980 20 add_watcher() i=7 2024-04-17T04:39:58.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.224+0000 7f6289d60980 2 all 8 watchers are set, enabling cache 2024-04-17T04:39:58.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f613dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:39:58.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:39:58.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:39:58.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 5 note: GC not initialized 2024-04-17T04:39:58.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f613cfe1640 20 reqs_thread_entry: start 2024-04-17T04:39:58.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 20 init_complete bucket index max shards: 1 2024-04-17T04:39:58.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 20 Filter name: none 2024-04-17T04:39:58.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f60b67fc640 20 reqs_thread_entry: start 2024-04-17T04:39:58.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-04-17T04:39:58.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.228+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 20 rados->read ofs=0 len=0 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-04-17T04:39:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-04-17T04:39:58.230 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-04-17T04:39:58.230 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.232+0000 7f6289d60980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-04-17T04:39:58.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.236+0000 7f6289d60980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-04-17T04:39:58.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.236+0000 7f6289d60980 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-04-17T04:39:58.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.236+0000 7f6289d60980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-04-17T04:39:58.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.236+0000 7f6166ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 158913789953 cookie 94212884483024 notifier 4565 bl.length()=593 2024-04-17T04:39:58.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.236+0000 7f6166ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-04-17T04:39:58.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.236+0000 7f6166ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-04-17T04:39:58.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.240+0000 7f6289d60980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-04-17T04:39:58.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.240+0000 7f6289d60980 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-04-17T04:39:58.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.240+0000 7f6289d60980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-04-17T04:39:58.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.244+0000 7f6166ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 158913789953 cookie 94212884476896 notifier 4565 bl.length()=186 2024-04-17T04:39:58.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.244+0000 7f6166ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-04-17T04:39:58.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.244+0000 7f6166ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-04-17T04:39:58.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.248+0000 7f613dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:39:58.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.256+0000 7f6289d60980 20 remove_watcher() i=0 2024-04-17T04:39:58.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.256+0000 7f6289d60980 2 removed watcher, disabling cache 2024-04-17T04:39:58.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.256+0000 7f6289d60980 20 remove_watcher() i=1 2024-04-17T04:39:58.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.260+0000 7f6289d60980 20 remove_watcher() i=2 2024-04-17T04:39:58.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.260+0000 7f6289d60980 20 remove_watcher() i=3 2024-04-17T04:39:58.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.264+0000 7f6289d60980 20 remove_watcher() i=4 2024-04-17T04:39:58.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.264+0000 7f6289d60980 20 remove_watcher() i=5 2024-04-17T04:39:58.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.268+0000 7f6289d60980 20 remove_watcher() i=6 2024-04-17T04:39:58.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:39:58.268+0000 7f6289d60980 20 remove_watcher() i=7 2024-04-17T04:40:00.497 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: recalculated bucket check stats are correct 2024-04-17T04:40:00.497 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:00.497 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-04-17T04:40:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:00.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.576+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 realm 2024-04-17T04:40:00.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:00.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:00.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:00.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.580+0000 7f1ae716c980 20 searching for the correct realm 2024-04-17T04:40:00.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.588+0000 7f1ae5488640 0 -- 172.21.15.88:0/2967867562 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55ca16599530 msgr2=0x55ca1659b9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:00.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.596+0000 7f1ae4c87640 0 -- 172.21.15.88:0/2967867562 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f1ad8024770 msgr2=0x7f1ad8022800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:00.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 zone default found 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 4 Realm: () 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 zonegroup default 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 started sync module instance, tier type = 2024-04-17T04:40:00.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.604+0000 7f1ae716c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:00.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.620+0000 7f1ae716c980 20 add_watcher() i=0 2024-04-17T04:40:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.620+0000 7f1ae716c980 20 add_watcher() i=1 2024-04-17T04:40:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.620+0000 7f1ae716c980 20 add_watcher() i=2 2024-04-17T04:40:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.620+0000 7f1ae716c980 20 add_watcher() i=3 2024-04-17T04:40:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.620+0000 7f1ae716c980 20 add_watcher() i=4 2024-04-17T04:40:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.620+0000 7f1ae716c980 20 add_watcher() i=5 2024-04-17T04:40:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.624+0000 7f1ae716c980 20 add_watcher() i=6 2024-04-17T04:40:00.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.624+0000 7f1ae716c980 20 add_watcher() i=7 2024-04-17T04:40:00.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.624+0000 7f1ae716c980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f199afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 5 note: GC not initialized 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1999fe3640 20 reqs_thread_entry: start 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 Filter name: none 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f190b7fe640 20 reqs_thread_entry: start 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:00.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:00.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:00.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:00.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:00.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:00.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:00.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:40:00.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.628+0000 7f1ae716c980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:40:00.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.632+0000 7f199afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:00.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.640+0000 7f1ae716c980 20 remove_watcher() i=0 2024-04-17T04:40:00.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.640+0000 7f1ae716c980 2 removed watcher, disabling cache 2024-04-17T04:40:00.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.640+0000 7f1ae716c980 20 remove_watcher() i=1 2024-04-17T04:40:00.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.644+0000 7f1ae716c980 20 remove_watcher() i=2 2024-04-17T04:40:00.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.644+0000 7f1ae716c980 20 remove_watcher() i=3 2024-04-17T04:40:00.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.648+0000 7f1ae716c980 20 remove_watcher() i=4 2024-04-17T04:40:00.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.648+0000 7f1ae716c980 20 remove_watcher() i=5 2024-04-17T04:40:00.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.648+0000 7f1ae716c980 20 remove_watcher() i=6 2024-04-17T04:40:00.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.652+0000 7f1ae716c980 20 remove_watcher() i=7 2024-04-17T04:40:00.664 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-04-17T04:40:00.690 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:00.690 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:00.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.736+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 realm 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:00.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:00.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.740+0000 7f36e0586980 20 searching for the correct realm 2024-04-17T04:40:00.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.748+0000 7f36de8a2640 0 -- 172.21.15.88:0/1706220281 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f36d40c0040 msgr2=0x7f36d40e4410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:00.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.752+0000 7f36de0a1640 0 -- 172.21.15.88:0/1706220281 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f36d40c7900 msgr2=0x7f36d40b5410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:00.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:00.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.968+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:00.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 zone default found 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 4 Realm: () 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 zonegroup default 2024-04-17T04:40:00.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:00.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:00.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 started sync module instance, tier type = 2024-04-17T04:40:00.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:00.972+0000 7f36e0586980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:00.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=0 2024-04-17T04:40:00.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=1 2024-04-17T04:40:00.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=2 2024-04-17T04:40:00.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=3 2024-04-17T04:40:00.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=4 2024-04-17T04:40:00.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=5 2024-04-17T04:40:01.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.000+0000 7f36e0586980 20 add_watcher() i=6 2024-04-17T04:40:01.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.004+0000 7f36e0586980 20 add_watcher() i=7 2024-04-17T04:40:01.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.004+0000 7f36e0586980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:01.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f3594fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:01.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f36e0586980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:01.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f36e0586980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:01.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f36e0586980 5 note: GC not initialized 2024-04-17T04:40:01.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f350f7fe640 20 reqs_thread_entry: start 2024-04-17T04:40:01.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f36e0586980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:01.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f36e0586980 20 Filter name: none 2024-04-17T04:40:01.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f350d7fa640 20 reqs_thread_entry: start 2024-04-17T04:40:01.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.008+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:01.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:01.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:01.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:01.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:01.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.012+0000 7f36e0586980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:01.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.016+0000 7f3594fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:01.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.020+0000 7f36e0586980 20 remove_watcher() i=0 2024-04-17T04:40:01.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.020+0000 7f36e0586980 2 removed watcher, disabling cache 2024-04-17T04:40:01.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.024+0000 7f36e0586980 20 remove_watcher() i=1 2024-04-17T04:40:01.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.024+0000 7f36e0586980 20 remove_watcher() i=2 2024-04-17T04:40:01.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.028+0000 7f36e0586980 20 remove_watcher() i=3 2024-04-17T04:40:01.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.028+0000 7f36e0586980 20 remove_watcher() i=4 2024-04-17T04:40:01.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.032+0000 7f36e0586980 20 remove_watcher() i=5 2024-04-17T04:40:01.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.032+0000 7f36e0586980 20 remove_watcher() i=6 2024-04-17T04:40:01.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.036+0000 7f36e0586980 20 remove_watcher() i=7 2024-04-17T04:40:01.050 INFO:tasks.workunit.client.0.smithi088.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 74, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-04-17T04:40:01.050 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check unlinked does not report normal entries 2024-04-17T04:40:01.050 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:01.051 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-04-17T04:40:01.086 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:01.086 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:01.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 realm 2024-04-17T04:40:01.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:01.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.132+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.136+0000 7ffbe3a28980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:01.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.136+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.136+0000 7ffbe3a28980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:01.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.136+0000 7ffbe3a28980 20 searching for the correct realm 2024-04-17T04:40:01.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.140+0000 7ffbe0d42640 0 -- 172.21.15.88:0/1353103100 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7ffbd407b040 msgr2=0x7ffbd4087ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:01.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.148+0000 7ffbe0d42640 0 -- 172.21.15.88:0/1353103100 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7ffaac0738e0 msgr2=0x7ffaac075d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:01.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:01.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 zone default found 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 4 Realm: () 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 zonegroup default 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:01.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 started sync module instance, tier type = 2024-04-17T04:40:01.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.156+0000 7ffbe3a28980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:01.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=0 2024-04-17T04:40:01.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=1 2024-04-17T04:40:01.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=2 2024-04-17T04:40:01.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=3 2024-04-17T04:40:01.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=4 2024-04-17T04:40:01.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=5 2024-04-17T04:40:01.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=6 2024-04-17T04:40:01.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 20 add_watcher() i=7 2024-04-17T04:40:01.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.176+0000 7ffbe3a28980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:01.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffa977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:01.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:01.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:01.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 5 note: GC not initialized 2024-04-17T04:40:01.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffa967e4640 20 reqs_thread_entry: start 2024-04-17T04:40:01.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:01.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 Filter name: none 2024-04-17T04:40:01.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffa07fff640 20 reqs_thread_entry: start 2024-04-17T04:40:01.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:01.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 rados->read ofs=0 len=0 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:01.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:01.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:01.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:01.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 1 NOTICE: finished shard 0 (0 entries found) 2024-04-17T04:40:01.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.180+0000 7ffbe3a28980 1 NOTICE: finished all shards (0 entries found) 2024-04-17T04:40:01.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.184+0000 7ffa977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:01.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.188+0000 7ffbe3a28980 20 remove_watcher() i=0 2024-04-17T04:40:01.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.188+0000 7ffbe3a28980 2 removed watcher, disabling cache 2024-04-17T04:40:01.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.192+0000 7ffbe3a28980 20 remove_watcher() i=1 2024-04-17T04:40:01.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.192+0000 7ffbe3a28980 20 remove_watcher() i=2 2024-04-17T04:40:01.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.196+0000 7ffbe3a28980 20 remove_watcher() i=3 2024-04-17T04:40:01.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.196+0000 7ffbe3a28980 20 remove_watcher() i=4 2024-04-17T04:40:01.198 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.200+0000 7ffbe3a28980 20 remove_watcher() i=5 2024-04-17T04:40:01.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.200+0000 7ffbe3a28980 20 remove_watcher() i=6 2024-04-17T04:40:01.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:01.204+0000 7ffbe3a28980 20 remove_watcher() i=7 2024-04-17T04:40:01.216 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-04-17T04:40:01.652 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-04-17T04:40:03.270 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:03.270 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-17T04:40:03.306 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:03.306 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:03.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 realm 2024-04-17T04:40:03.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:03.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.351+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.355+0000 7f69f6c22980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:03.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.355+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.355+0000 7f69f6c22980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:03.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.355+0000 7f69f6c22980 20 searching for the correct realm 2024-04-17T04:40:03.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.359+0000 7f69f3f3c640 0 -- 172.21.15.88:0/2319785459 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x55b971c0f3c0 msgr2=0x55b971c117b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:03.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.367+0000 7f69f473d640 0 -- 172.21.15.88:0/2319785459 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f69e810ac20 msgr2=0x7f69e812b0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:03.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 zone default found 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 4 Realm: () 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 zonegroup default 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.375+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.379+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.379+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.379+0000 7f69f6c22980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.379+0000 7f69f6c22980 20 started sync module instance, tier type = 2024-04-17T04:40:03.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.379+0000 7f69f6c22980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:03.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=0 2024-04-17T04:40:03.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=1 2024-04-17T04:40:03.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=2 2024-04-17T04:40:03.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=3 2024-04-17T04:40:03.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=4 2024-04-17T04:40:03.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=5 2024-04-17T04:40:03.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.399+0000 7f69f6c22980 20 add_watcher() i=6 2024-04-17T04:40:03.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.403+0000 7f69f6c22980 20 add_watcher() i=7 2024-04-17T04:40:03.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.403+0000 7f69f6c22980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:03.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f68aafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:03.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 5 note: GC not initialized 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f68a9fdb640 20 reqs_thread_entry: start 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 Filter name: none 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f681b7fe640 20 reqs_thread_entry: start 2024-04-17T04:40:03.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 INFO: main: starting shard=0 2024-04-17T04:40:03.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.407+0000 7f69f6c22980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:03.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.411+0000 7f69f6c22980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-17T04:40:03.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.411+0000 7f69f6c22980 20 INFO: main: done 2024-04-17T04:40:03.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.411+0000 7f68aafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:03.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.415+0000 7f69f6c22980 20 remove_watcher() i=0 2024-04-17T04:40:03.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.415+0000 7f69f6c22980 2 removed watcher, disabling cache 2024-04-17T04:40:03.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.419+0000 7f69f6c22980 20 remove_watcher() i=1 2024-04-17T04:40:03.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.419+0000 7f69f6c22980 20 remove_watcher() i=2 2024-04-17T04:40:03.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.423+0000 7f69f6c22980 20 remove_watcher() i=3 2024-04-17T04:40:03.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.423+0000 7f69f6c22980 20 remove_watcher() i=4 2024-04-17T04:40:03.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.427+0000 7f69f6c22980 20 remove_watcher() i=5 2024-04-17T04:40:03.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.427+0000 7f69f6c22980 20 remove_watcher() i=6 2024-04-17T04:40:03.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.427+0000 7f69f6c22980 20 remove_watcher() i=7 2024-04-17T04:40:03.621 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:03.622 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-17T04:40:03.657 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:03.657 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:03.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 realm 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.703+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.707+0000 7fa8f20e5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:03.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.707+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.707+0000 7fa8f20e5980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:03.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.707+0000 7fa8f20e5980 20 searching for the correct realm 2024-04-17T04:40:03.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.711+0000 7fa8f0401640 0 -- 172.21.15.88:0/3239893514 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55d9ddc093a0 msgr2=0x55d9ddc0b810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:03.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.719+0000 7fa8efc00640 0 -- 172.21.15.88:0/3239893514 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fa8d8057c20 msgr2=0x7fa8d8055650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:03.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 zone default found 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 4 Realm: () 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 zonegroup default 2024-04-17T04:40:03.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 started sync module instance, tier type = 2024-04-17T04:40:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.727+0000 7fa8f20e5980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:03.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=0 2024-04-17T04:40:03.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=1 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=2 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=3 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=4 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=5 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.747+0000 7fa8f20e5980 20 add_watcher() i=6 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 20 add_watcher() i=7 2024-04-17T04:40:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:03.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa7a5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:03.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:03.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:03.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 5 note: GC not initialized 2024-04-17T04:40:03.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa7a4fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:03.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:03.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 20 Filter name: none 2024-04-17T04:40:03.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa71e7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:03.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.751+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:03.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 rados->read ofs=0 len=0 2024-04-17T04:40:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 INFO: main: starting shard=0 2024-04-17T04:40:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:03.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-17T04:40:03.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa8f20e5980 20 INFO: main: done 2024-04-17T04:40:03.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.755+0000 7fa7a5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:03.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.763+0000 7fa8f20e5980 20 remove_watcher() i=0 2024-04-17T04:40:03.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.763+0000 7fa8f20e5980 2 removed watcher, disabling cache 2024-04-17T04:40:03.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.763+0000 7fa8f20e5980 20 remove_watcher() i=1 2024-04-17T04:40:03.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.767+0000 7fa8f20e5980 20 remove_watcher() i=2 2024-04-17T04:40:03.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.767+0000 7fa8f20e5980 20 remove_watcher() i=3 2024-04-17T04:40:03.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.771+0000 7fa8f20e5980 20 remove_watcher() i=4 2024-04-17T04:40:03.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.771+0000 7fa8f20e5980 20 remove_watcher() i=5 2024-04-17T04:40:03.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.771+0000 7fa8f20e5980 20 remove_watcher() i=6 2024-04-17T04:40:03.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:03.775+0000 7fa8f20e5980 20 remove_watcher() i=7 2024-04-17T04:40:03.969 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:03.970 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-04-17T04:40:04.005 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:04.005 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:04.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 realm 2024-04-17T04:40:04.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:04.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.051+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.055+0000 7f1b01841980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:04.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.055+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.055+0000 7f1b01841980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:04.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.055+0000 7f1b01841980 20 searching for the correct realm 2024-04-17T04:40:04.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.063+0000 7f1afeb5b640 0 -- 172.21.15.88:0/2279898636 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f1af0085600 msgr2=0x7f1af0084860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:04.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.067+0000 7f1affb5d640 0 -- 172.21.15.88:0/2279898636 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f1af4008040 msgr2=0x7f1af407c6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:04.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:04.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:04.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 zone default found 2024-04-17T04:40:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 4 Realm: () 2024-04-17T04:40:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:04.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 zonegroup default 2024-04-17T04:40:04.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.075+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.079+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.079+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.079+0000 7f1b01841980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.079+0000 7f1b01841980 20 started sync module instance, tier type = 2024-04-17T04:40:04.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.079+0000 7f1b01841980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.099+0000 7f1b01841980 20 add_watcher() i=0 2024-04-17T04:40:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.099+0000 7f1b01841980 20 add_watcher() i=1 2024-04-17T04:40:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.099+0000 7f1b01841980 20 add_watcher() i=2 2024-04-17T04:40:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.099+0000 7f1b01841980 20 add_watcher() i=3 2024-04-17T04:40:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.099+0000 7f1b01841980 20 add_watcher() i=4 2024-04-17T04:40:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.099+0000 7f1b01841980 20 add_watcher() i=5 2024-04-17T04:40:04.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f1b01841980 20 add_watcher() i=6 2024-04-17T04:40:04.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f1b01841980 20 add_watcher() i=7 2024-04-17T04:40:04.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f1b01841980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:04.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f19b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:04.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f1b01841980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f1b01841980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f1b01841980 5 note: GC not initialized 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.103+0000 7f19b4fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 Filter name: none 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f192e7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:04.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:04.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:04.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:04.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 INFO: main: starting shard=0 2024-04-17T04:40:04.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:04.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:40:04.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f1b01841980 20 INFO: main: done 2024-04-17T04:40:04.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.107+0000 7f19b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:04.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.115+0000 7f1b01841980 20 remove_watcher() i=0 2024-04-17T04:40:04.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.115+0000 7f1b01841980 2 removed watcher, disabling cache 2024-04-17T04:40:04.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.115+0000 7f1b01841980 20 remove_watcher() i=1 2024-04-17T04:40:04.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.119+0000 7f1b01841980 20 remove_watcher() i=2 2024-04-17T04:40:04.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.119+0000 7f1b01841980 20 remove_watcher() i=3 2024-04-17T04:40:04.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.123+0000 7f1b01841980 20 remove_watcher() i=4 2024-04-17T04:40:04.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.123+0000 7f1b01841980 20 remove_watcher() i=5 2024-04-17T04:40:04.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.127+0000 7f1b01841980 20 remove_watcher() i=6 2024-04-17T04:40:04.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.127+0000 7f1b01841980 20 remove_watcher() i=7 2024-04-17T04:40:04.281 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:04.282 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-04-17T04:40:04.316 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:04.316 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 realm 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.363+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.367+0000 7f355343b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:04.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.367+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.367+0000 7f355343b980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:04.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.367+0000 7f355343b980 20 searching for the correct realm 2024-04-17T04:40:04.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.371+0000 7f3551757640 0 -- 172.21.15.88:0/2587661256 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f341c073860 msgr2=0x7f341c075c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:04.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.583+0000 7f3550f56640 0 -- 172.21.15.88:0/2587661256 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f35440093d0 msgr2=0x7f3544001000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:04.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.799+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:04.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 zone default found 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 4 Realm: () 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:04.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 zonegroup default 2024-04-17T04:40:04.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.803+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.807+0000 7f355343b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:04.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.807+0000 7f355343b980 20 started sync module instance, tier type = 2024-04-17T04:40:04.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.807+0000 7f355343b980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:04.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.831+0000 7f355343b980 20 add_watcher() i=0 2024-04-17T04:40:04.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.831+0000 7f355343b980 20 add_watcher() i=1 2024-04-17T04:40:04.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.831+0000 7f355343b980 20 add_watcher() i=2 2024-04-17T04:40:04.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.831+0000 7f355343b980 20 add_watcher() i=3 2024-04-17T04:40:04.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.831+0000 7f355343b980 20 add_watcher() i=4 2024-04-17T04:40:04.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.831+0000 7f355343b980 20 add_watcher() i=5 2024-04-17T04:40:04.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.835+0000 7f355343b980 20 add_watcher() i=6 2024-04-17T04:40:04.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.835+0000 7f355343b980 20 add_watcher() i=7 2024-04-17T04:40:04.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.835+0000 7f355343b980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:04.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f3406fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:04.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:04.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:04.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 5 note: GC not initialized 2024-04-17T04:40:04.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f3405fe3640 20 reqs_thread_entry: start 2024-04-17T04:40:04.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:04.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 20 Filter name: none 2024-04-17T04:40:04.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f33777fe640 20 reqs_thread_entry: start 2024-04-17T04:40:04.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.839+0000 7f355343b980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:04.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:04.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:04.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:04.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:04.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 INFO: main: starting shard=0 2024-04-17T04:40:04.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:04.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:40:04.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f355343b980 20 INFO: main: done 2024-04-17T04:40:04.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.843+0000 7f3406fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:04.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.851+0000 7f355343b980 20 remove_watcher() i=0 2024-04-17T04:40:04.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.851+0000 7f355343b980 2 removed watcher, disabling cache 2024-04-17T04:40:04.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.851+0000 7f355343b980 20 remove_watcher() i=1 2024-04-17T04:40:04.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.855+0000 7f355343b980 20 remove_watcher() i=2 2024-04-17T04:40:04.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.859+0000 7f355343b980 20 remove_watcher() i=3 2024-04-17T04:40:04.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.859+0000 7f355343b980 20 remove_watcher() i=4 2024-04-17T04:40:04.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.859+0000 7f355343b980 20 remove_watcher() i=5 2024-04-17T04:40:04.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.863+0000 7f355343b980 20 remove_watcher() i=6 2024-04-17T04:40:04.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:04.863+0000 7f355343b980 20 remove_watcher() i=7 2024-04-17T04:40:04.929 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-04-17T04:40:05.370 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-04-17T04:40:05.793 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check unlinked finds unlistable entries 2024-04-17T04:40:05.793 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:05.793 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-04-17T04:40:05.826 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:05.826 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:05.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.867+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.867+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.867+0000 7f0e116d2980 20 realm 2024-04-17T04:40:05.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.867+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:05.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:05.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:05.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.871+0000 7f0e116d2980 20 searching for the correct realm 2024-04-17T04:40:05.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.879+0000 7f0e0e9ec640 0 -- 172.21.15.88:0/36042537 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f0e00008050 msgr2=0x7f0e000548d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:05.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.883+0000 7f0e0f9ee640 0 -- 172.21.15.88:0/36042537 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f0e0405d9d0 msgr2=0x7f0e04062a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:05.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 zone default found 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 4 Realm: () 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 zonegroup default 2024-04-17T04:40:05.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.891+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.895+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.895+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.895+0000 7f0e116d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:05.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.895+0000 7f0e116d2980 20 started sync module instance, tier type = 2024-04-17T04:40:05.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.895+0000 7f0e116d2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:05.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.911+0000 7f0e116d2980 20 add_watcher() i=0 2024-04-17T04:40:05.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.911+0000 7f0e116d2980 20 add_watcher() i=1 2024-04-17T04:40:05.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.911+0000 7f0e116d2980 20 add_watcher() i=2 2024-04-17T04:40:05.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.911+0000 7f0e116d2980 20 add_watcher() i=3 2024-04-17T04:40:05.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.911+0000 7f0e116d2980 20 add_watcher() i=4 2024-04-17T04:40:05.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.911+0000 7f0e116d2980 20 add_watcher() i=5 2024-04-17T04:40:05.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.915+0000 7f0e116d2980 20 add_watcher() i=6 2024-04-17T04:40:05.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.915+0000 7f0e116d2980 20 add_watcher() i=7 2024-04-17T04:40:05.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.915+0000 7f0e116d2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:05.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0cc5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 5 note: GC not initialized 2024-04-17T04:40:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0cc4fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 Filter name: none 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0c3e7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 rados->read ofs=0 len=0 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:05.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.919+0000 7f0e116d2980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:05.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.923+0000 7f0e116d2980 1 NOTICE: finished shard 0 (4 entries found) 2024-04-17T04:40:05.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.923+0000 7f0e116d2980 1 NOTICE: finished all shards (4 entries found) 2024-04-17T04:40:05.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.923+0000 7f0cc5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:05.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.927+0000 7f0e116d2980 20 remove_watcher() i=0 2024-04-17T04:40:05.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.927+0000 7f0e116d2980 2 removed watcher, disabling cache 2024-04-17T04:40:05.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.931+0000 7f0e116d2980 20 remove_watcher() i=1 2024-04-17T04:40:05.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.931+0000 7f0e116d2980 20 remove_watcher() i=2 2024-04-17T04:40:05.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.935+0000 7f0e116d2980 20 remove_watcher() i=3 2024-04-17T04:40:05.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.935+0000 7f0e116d2980 20 remove_watcher() i=4 2024-04-17T04:40:05.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.935+0000 7f0e116d2980 20 remove_watcher() i=5 2024-04-17T04:40:05.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.939+0000 7f0e116d2980 20 remove_watcher() i=6 2024-04-17T04:40:05.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:05.939+0000 7f0e116d2980 20 remove_watcher() i=7 2024-04-17T04:40:05.949 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: unlinked entries are not listable 2024-04-17T04:40:05.949 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:06.063 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-04-17T04:40:06.063 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:06.285 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-04-17T04:40:06.286 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:06.286 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-17T04:40:06.326 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:06.326 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:06.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 20 realm 2024-04-17T04:40:06.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:06.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.371+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.375+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.375+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.375+0000 7f1175593980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:06.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.375+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.375+0000 7f1175593980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:06.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.375+0000 7f1175593980 20 searching for the correct realm 2024-04-17T04:40:06.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.383+0000 7f11738af640 0 -- 172.21.15.88:0/1270315342 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f115c063410 msgr2=0x7f115c059590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:06.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.599+0000 7f11730ae640 0 -- 172.21.15.88:0/1270315342 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x557186e50cf0 msgr2=0x557186e53120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:06.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 zone default found 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 4 Realm: () 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 zonegroup default 2024-04-17T04:40:06.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.607+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.611+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.611+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.611+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.611+0000 7f1175593980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:06.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.611+0000 7f1175593980 20 started sync module instance, tier type = 2024-04-17T04:40:06.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.611+0000 7f1175593980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:06.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=0 2024-04-17T04:40:06.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=1 2024-04-17T04:40:06.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=2 2024-04-17T04:40:06.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=3 2024-04-17T04:40:06.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=4 2024-04-17T04:40:06.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=5 2024-04-17T04:40:06.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.631+0000 7f1175593980 20 add_watcher() i=6 2024-04-17T04:40:06.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.635+0000 7f1175593980 20 add_watcher() i=7 2024-04-17T04:40:06.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.635+0000 7f1175593980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:06.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1029fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:06.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:06.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:06.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 5 note: GC not initialized 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1028fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 20 Filter name: none 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f0fa27fc640 20 reqs_thread_entry: start 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.639+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:06.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:06.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:06.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:06.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 20 rados->read ofs=0 len=0 2024-04-17T04:40:06.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:06.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:06.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:06.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:06.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:06.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:06.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.643+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:c state=0x557186e884f8 s->prefetch_data=0 2024-04-17T04:40:06.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.647+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to 22fzg1ogghu44ymeh9pour1rbr8lsbjv 2024-04-17T04:40:06.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.647+0000 7f1175593980 20 get_obj_state_impl: setting s->olh_tag to hc1shpegqsm08pmis10oq4d922n4m6jz 2024-04-17T04:40:06.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.647+0000 7f1175593980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=hc1shpegqsm08pmis10oq4d922n4m6jz 2024-04-17T04:40:06.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.647+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 20 olh_log_entry: epoch=3 op=3 key=c[Q1W6sTMwRSc1oAMbNivOgwoEovF8kRI] 2024-04-17T04:40:06.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:Q1W6sTMwRSc1oAMbNivOgwoEovF8kRI_c state=0x557186eb6908 s->prefetch_data=0 2024-04-17T04:40:06.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 10 manifest: total_size = 9 2024-04-17T04:40:06.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17417976924950659453 2024-04-17T04:40:06.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:Q1W6sTMwRSc1oAMbNivOgwoEovF8kRI_c state=0x557186eb6908 s->prefetch_data=0 2024-04-17T04:40:06.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.651+0000 7f1175593980 20 prepare_atomic_modification: state is not atomic. state=0x557186eb6908 2024-04-17T04:40:06.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.655+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.659+0000 7f1175593980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:06.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.659+0000 7f1175593980 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-04-17T04:40:06.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.659+0000 7f1175593980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:06.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.659+0000 7f1175593980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:Q1W6sTMwRSc1oAMbNivOgwoEovF8kRI_.64jHjbZMk0kmCx1BAAjHiWwY4AJmzKc_0 2024-04-17T04:40:06.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.659+0000 7f1056ffd640 20 handle_completion(): completion ok for obj=c[Q1W6sTMwRSc1oAMbNivOgwoEovF8kRI] 2024-04-17T04:40:06.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.663+0000 7f1175593980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:06.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.667+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.667+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.667+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.667+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.667+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:d state=0x557186e89528 s->prefetch_data=0 2024-04-17T04:40:06.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.671+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to cz5v1k5x65sr7q91evkt4azru2vt83fi 2024-04-17T04:40:06.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.671+0000 7f1175593980 20 get_obj_state_impl: setting s->olh_tag to 70xzqg1c21ce4xxv1qpcpib002j7i9my 2024-04-17T04:40:06.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.671+0000 7f1175593980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=70xzqg1c21ce4xxv1qpcpib002j7i9my 2024-04-17T04:40:06.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.671+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 olh_log_entry: epoch=3 op=3 key=d[fc9frzke3de49KNllXcZSvNMYFSbfNv] 2024-04-17T04:40:06.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:fc9frzke3de49KNllXcZSvNMYFSbfNv_d state=0x557186eb8928 s->prefetch_data=0 2024-04-17T04:40:06.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 10 manifest: total_size = 12 2024-04-17T04:40:06.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8106822682460837162 2024-04-17T04:40:06.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:fc9frzke3de49KNllXcZSvNMYFSbfNv_d state=0x557186eb8928 s->prefetch_data=0 2024-04-17T04:40:06.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 prepare_atomic_modification: state is not atomic. state=0x557186eb8928 2024-04-17T04:40:06.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.675+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.683+0000 7f1175593980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:06.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.683+0000 7f1175593980 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-04-17T04:40:06.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.683+0000 7f1175593980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:06.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.683+0000 7f1175593980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:fc9frzke3de49KNllXcZSvNMYFSbfNv_.7KIPLf0qByPYtAZuqCN_sdqM6CGsMkv_0 2024-04-17T04:40:06.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.687+0000 7f10567fc640 20 handle_completion(): completion ok for obj=d[fc9frzke3de49KNllXcZSvNMYFSbfNv] 2024-04-17T04:40:06.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.687+0000 7f1175593980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:06.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.691+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.691+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.691+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.691+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.695+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:e state=0x557186e89528 s->prefetch_data=0 2024-04-17T04:40:06.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.695+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to g24aul3kf5k9aij1eeoyo4sr4oubghyy 2024-04-17T04:40:06.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.695+0000 7f1175593980 20 get_obj_state_impl: setting s->olh_tag to dkj619o1mmq9e7w57tqyeucc4vn6b3hf 2024-04-17T04:40:06.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.695+0000 7f1175593980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=dkj619o1mmq9e7w57tqyeucc4vn6b3hf 2024-04-17T04:40:06.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.699+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.699+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.699+0000 7f1175593980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-04-17T04:40:06.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.699+0000 7f1175593980 20 olh_log_entry: epoch=2 op=3 key=e[c6dYycai73GPuOfquwiold9huR7dtO7] 2024-04-17T04:40:06.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.699+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:c6dYycai73GPuOfquwiold9huR7dtO7_e state=0x557186eb8928 s->prefetch_data=0 2024-04-17T04:40:06.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.703+0000 7f1175593980 10 manifest: total_size = 11 2024-04-17T04:40:06.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.703+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14610719020357558967 2024-04-17T04:40:06.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.703+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:c6dYycai73GPuOfquwiold9huR7dtO7_e state=0x557186eb8928 s->prefetch_data=0 2024-04-17T04:40:06.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.703+0000 7f1175593980 20 prepare_atomic_modification: state is not atomic. state=0x557186eb8928 2024-04-17T04:40:06.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.703+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.707+0000 7f1175593980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:06.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.707+0000 7f1175593980 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-04-17T04:40:06.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.707+0000 7f1175593980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:06.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.707+0000 7f1175593980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:c6dYycai73GPuOfquwiold9huR7dtO7_.2Lb4RGtAdQz0sXncqP0uMEjYMwf169a_0 2024-04-17T04:40:06.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.707+0000 7f1056ffd640 20 handle_completion(): completion ok for obj=e[c6dYycai73GPuOfquwiold9huR7dtO7] 2024-04-17T04:40:06.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.711+0000 7f1175593980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:06.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.715+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:e state=0x557186e89528 s->prefetch_data=0 2024-04-17T04:40:06.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.715+0000 7f1175593980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=dkj619o1mmq9e7w57tqyeucc4vn6b3hf 2024-04-17T04:40:06.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.723+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.727+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.727+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.727+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:f state=0x557186e89528 s->prefetch_data=0 2024-04-17T04:40:06.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.727+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to aqv2obvux0p73tt89syoybru0qsp3662 2024-04-17T04:40:06.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.727+0000 7f1175593980 20 get_obj_state_impl: setting s->olh_tag to 7kfemmk5jc3spte98gkc17fuy2e3620a 2024-04-17T04:40:06.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.727+0000 7f1175593980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7kfemmk5jc3spte98gkc17fuy2e3620a 2024-04-17T04:40:06.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.731+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-04-17T04:40:06.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 olh_log_entry: epoch=2 op=3 key=f[9hpnhSSYND-ExiGHtgjCWtk.KBNgFvd] 2024-04-17T04:40:06.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:9hpnhSSYND-ExiGHtgjCWtk.KBNgFvd_f state=0x557186eb8928 s->prefetch_data=0 2024-04-17T04:40:06.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 10 manifest: total_size = 12 2024-04-17T04:40:06.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18126687312819002628 2024-04-17T04:40:06.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:_:9hpnhSSYND-ExiGHtgjCWtk.KBNgFvd_f state=0x557186eb8928 s->prefetch_data=0 2024-04-17T04:40:06.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 prepare_atomic_modification: state is not atomic. state=0x557186eb8928 2024-04-17T04:40:06.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.735+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.743+0000 7f1175593980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:06.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.743+0000 7f1175593980 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-04-17T04:40:06.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.743+0000 7f1175593980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:06.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.743+0000 7f1175593980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:9hpnhSSYND-ExiGHtgjCWtk.KBNgFvd_.s8HKcm-BVmQFtBU3qQc1UpbQiVhq_KV_0 2024-04-17T04:40:06.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.743+0000 7f10567fc640 20 handle_completion(): completion ok for obj=f[9hpnhSSYND-ExiGHtgjCWtk.KBNgFvd] 2024-04-17T04:40:06.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.743+0000 7f1175593980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:06.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.747+0000 7f1175593980 20 get_obj_state: octx=0x557186bdd160 obj=check-bucket:f state=0x557186e89528 s->prefetch_data=0 2024-04-17T04:40:06.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.747+0000 7f1175593980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7kfemmk5jc3spte98gkc17fuy2e3620a 2024-04-17T04:40:06.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.759+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.763+0000 7f1175593980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:06.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.763+0000 7f1175593980 1 NOTICE: finished shard 0 (4 entries removed) 2024-04-17T04:40:06.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.763+0000 7f1175593980 1 NOTICE: finished all shards (4 entries removed) 2024-04-17T04:40:06.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.763+0000 7f1029fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:06.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.771+0000 7f1175593980 20 remove_watcher() i=0 2024-04-17T04:40:06.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.771+0000 7f1175593980 2 removed watcher, disabling cache 2024-04-17T04:40:06.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.771+0000 7f1175593980 20 remove_watcher() i=1 2024-04-17T04:40:06.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.775+0000 7f1175593980 20 remove_watcher() i=2 2024-04-17T04:40:06.777 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.779+0000 7f1175593980 20 remove_watcher() i=3 2024-04-17T04:40:06.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.779+0000 7f1175593980 20 remove_watcher() i=4 2024-04-17T04:40:06.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.783+0000 7f1175593980 20 remove_watcher() i=5 2024-04-17T04:40:06.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.787+0000 7f1175593980 20 remove_watcher() i=6 2024-04-17T04:40:06.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:06.791+0000 7f1175593980 20 remove_watcher() i=7 2024-04-17T04:40:07.009 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check unlinked does not affect normal entries 2024-04-17T04:40:07.010 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:07.383 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-04-17T04:40:07.383 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:07.384 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-04-17T04:40:07.421 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:07.421 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:07.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 realm 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.467+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.471+0000 7f8471ca0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:07.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.471+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.471+0000 7f8471ca0980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:07.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.471+0000 7f8471ca0980 20 searching for the correct realm 2024-04-17T04:40:07.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.475+0000 7f846ffbc640 0 -- 172.21.15.88:0/3266326194 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f846403b040 msgr2=0x7f846403b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:07.481 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.483+0000 7f846f7bb640 0 -- 172.21.15.88:0/3266326194 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f8464008050 msgr2=0x7f8464049400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:07.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.699+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:07.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:07.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 zone default found 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 4 Realm: () 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 zonegroup default 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.703+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.707+0000 7f8471ca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:07.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.707+0000 7f8471ca0980 20 started sync module instance, tier type = 2024-04-17T04:40:07.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.707+0000 7f8471ca0980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:07.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=0 2024-04-17T04:40:07.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=1 2024-04-17T04:40:07.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=2 2024-04-17T04:40:07.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=3 2024-04-17T04:40:07.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=4 2024-04-17T04:40:07.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=5 2024-04-17T04:40:07.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.731+0000 7f8471ca0980 20 add_watcher() i=6 2024-04-17T04:40:07.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.735+0000 7f8471ca0980 20 add_watcher() i=7 2024-04-17T04:40:07.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.735+0000 7f8471ca0980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:07.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f83267e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:07.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f8471ca0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:07.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f8471ca0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:07.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f8471ca0980 5 note: GC not initialized 2024-04-17T04:40:07.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f83257e2640 20 reqs_thread_entry: start 2024-04-17T04:40:07.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f8471ca0980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:07.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.739+0000 7f8471ca0980 20 Filter name: none 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f831e7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:07.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:07.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:07.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:07.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 1 NOTICE: finished shard 0 (0 entries found) 2024-04-17T04:40:07.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.743+0000 7f8471ca0980 1 NOTICE: finished all shards (0 entries found) 2024-04-17T04:40:07.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.747+0000 7f83267e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:07.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.751+0000 7f8471ca0980 20 remove_watcher() i=0 2024-04-17T04:40:07.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.751+0000 7f8471ca0980 2 removed watcher, disabling cache 2024-04-17T04:40:07.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.755+0000 7f8471ca0980 20 remove_watcher() i=1 2024-04-17T04:40:07.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.755+0000 7f8471ca0980 20 remove_watcher() i=2 2024-04-17T04:40:07.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.759+0000 7f8471ca0980 20 remove_watcher() i=3 2024-04-17T04:40:07.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.759+0000 7f8471ca0980 20 remove_watcher() i=4 2024-04-17T04:40:07.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.763+0000 7f8471ca0980 20 remove_watcher() i=5 2024-04-17T04:40:07.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.763+0000 7f8471ca0980 20 remove_watcher() i=6 2024-04-17T04:40:07.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:07.767+0000 7f8471ca0980 20 remove_watcher() i=7 2024-04-17T04:40:08.245 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-17T04:40:08.279 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:08.279 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:08.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 20 realm 2024-04-17T04:40:08.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:08.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.323+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.327+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.327+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.327+0000 7fbd23f49980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:08.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.327+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.327+0000 7fbd23f49980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:08.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.327+0000 7fbd23f49980 20 searching for the correct realm 2024-04-17T04:40:08.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.335+0000 7fbd22265640 0 -- 172.21.15.88:0/857963184 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55b35596b910 msgr2=0x55b35596dd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:08.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.339+0000 7fbd21263640 0 -- 172.21.15.88:0/857963184 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbd14054a20 msgr2=0x7fbd1405be50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:08.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:08.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:08.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:08.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:08.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:08.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:08.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:08.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 zone default found 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 4 Realm: () 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 zonegroup default 2024-04-17T04:40:08.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.347+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.351+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.351+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.351+0000 7fbd23f49980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.351+0000 7fbd23f49980 20 started sync module instance, tier type = 2024-04-17T04:40:08.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.351+0000 7fbd23f49980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:08.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=0 2024-04-17T04:40:08.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=1 2024-04-17T04:40:08.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=2 2024-04-17T04:40:08.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=3 2024-04-17T04:40:08.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=4 2024-04-17T04:40:08.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=5 2024-04-17T04:40:08.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.371+0000 7fbd23f49980 20 add_watcher() i=6 2024-04-17T04:40:08.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.375+0000 7fbd23f49980 20 add_watcher() i=7 2024-04-17T04:40:08.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.375+0000 7fbd23f49980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:08.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.375+0000 7fbbd37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.375+0000 7fbd23f49980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.375+0000 7fbd23f49980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.375+0000 7fbd23f49980 5 note: GC not initialized 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbbd27fc640 20 reqs_thread_entry: start 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 Filter name: none 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbb43fff640 20 reqs_thread_entry: start 2024-04-17T04:40:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:08.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:08.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 INFO: main: starting shard=0 2024-04-17T04:40:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-04-17T04:40:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbd23f49980 20 INFO: main: done 2024-04-17T04:40:08.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.379+0000 7fbbd37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:08.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.387+0000 7fbd23f49980 20 remove_watcher() i=0 2024-04-17T04:40:08.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.387+0000 7fbd23f49980 2 removed watcher, disabling cache 2024-04-17T04:40:08.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.387+0000 7fbd23f49980 20 remove_watcher() i=1 2024-04-17T04:40:08.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.391+0000 7fbd23f49980 20 remove_watcher() i=2 2024-04-17T04:40:08.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.391+0000 7fbd23f49980 20 remove_watcher() i=3 2024-04-17T04:40:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.395+0000 7fbd23f49980 20 remove_watcher() i=4 2024-04-17T04:40:08.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.395+0000 7fbd23f49980 20 remove_watcher() i=5 2024-04-17T04:40:08.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.399+0000 7fbd23f49980 20 remove_watcher() i=6 2024-04-17T04:40:08.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.399+0000 7fbd23f49980 20 remove_watcher() i=7 2024-04-17T04:40:08.412 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-17T04:40:08.441 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:08.441 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:08.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.483+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 realm 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:08.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:08.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.487+0000 7f230f7a4980 20 searching for the correct realm 2024-04-17T04:40:08.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.495+0000 7f230cabe640 0 -- 172.21.15.88:0/3785522989 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5586007bb300 msgr2=0x5586007bd6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:08.500 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.499+0000 7f230cabe640 0 -- 172.21.15.88:0/3785522989 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f2304008070 msgr2=0x7f230405e8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.507+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:08.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 zone default found 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 4 Realm: () 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 zonegroup default 2024-04-17T04:40:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 started sync module instance, tier type = 2024-04-17T04:40:08.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.511+0000 7f230f7a4980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=0 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=1 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=2 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=3 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=4 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=5 2024-04-17T04:40:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=6 2024-04-17T04:40:08.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 20 add_watcher() i=7 2024-04-17T04:40:08.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.535+0000 7f230f7a4980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:08.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f21c37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:08.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:08.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:08.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 5 note: GC not initialized 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f21c27e4640 20 reqs_thread_entry: start 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 Filter name: none 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f2133fff640 20 reqs_thread_entry: start 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:08.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:08.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:08.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 INFO: main: starting shard=0 2024-04-17T04:40:08.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.539+0000 7f230f7a4980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:08.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.543+0000 7f230f7a4980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-04-17T04:40:08.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.543+0000 7f230f7a4980 20 INFO: main: done 2024-04-17T04:40:08.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.543+0000 7f21c37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:08.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.547+0000 7f230f7a4980 20 remove_watcher() i=0 2024-04-17T04:40:08.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.547+0000 7f230f7a4980 2 removed watcher, disabling cache 2024-04-17T04:40:08.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.551+0000 7f230f7a4980 20 remove_watcher() i=1 2024-04-17T04:40:08.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.551+0000 7f230f7a4980 20 remove_watcher() i=2 2024-04-17T04:40:08.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.555+0000 7f230f7a4980 20 remove_watcher() i=3 2024-04-17T04:40:08.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.555+0000 7f230f7a4980 20 remove_watcher() i=4 2024-04-17T04:40:08.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.559+0000 7f230f7a4980 20 remove_watcher() i=5 2024-04-17T04:40:08.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.563+0000 7f230f7a4980 20 remove_watcher() i=6 2024-04-17T04:40:08.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.563+0000 7f230f7a4980 20 remove_watcher() i=7 2024-04-17T04:40:08.575 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check olh finds leftover OLH entries 2024-04-17T04:40:08.575 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:08.575 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-04-17T04:40:08.603 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:08.603 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:08.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 realm 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:08.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.647+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.651+0000 7f24f4bf1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:08.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.651+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:08.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.651+0000 7f24f4bf1980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:08.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.651+0000 7f24f4bf1980 20 searching for the correct realm 2024-04-17T04:40:08.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.655+0000 7f24f1f0b640 0 -- 172.21.15.88:0/1464821339 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f24e4008050 msgr2=0x7f24e400bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:08.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:08.871+0000 7f24f270c640 0 -- 172.21.15.88:0/1464821339 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f24e80ba930 msgr2=0x7f24e80e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:09.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.087+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 zone default found 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 4 Realm: () 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 zonegroup default 2024-04-17T04:40:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 started sync module instance, tier type = 2024-04-17T04:40:09.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.091+0000 7f24f4bf1980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:09.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=0 2024-04-17T04:40:09.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=1 2024-04-17T04:40:09.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=2 2024-04-17T04:40:09.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=3 2024-04-17T04:40:09.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=4 2024-04-17T04:40:09.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=5 2024-04-17T04:40:09.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.115+0000 7f24f4bf1980 20 add_watcher() i=6 2024-04-17T04:40:09.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.119+0000 7f24f4bf1980 20 add_watcher() i=7 2024-04-17T04:40:09.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.119+0000 7f24f4bf1980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:09.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.123+0000 7f23a8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.123+0000 7f24f4bf1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:09.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.123+0000 7f24f4bf1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:09.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.123+0000 7f24f4bf1980 5 note: GC not initialized 2024-04-17T04:40:09.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.123+0000 7f23237fe640 20 reqs_thread_entry: start 2024-04-17T04:40:09.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.123+0000 7f24f4bf1980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:09.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 20 Filter name: none 2024-04-17T04:40:09.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f23217fa640 20 reqs_thread_entry: start 2024-04-17T04:40:09.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:09.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:09.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:09.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:09.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:09.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:09.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 1 NOTICE: finished shard 0 (2 entries found) 2024-04-17T04:40:09.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.127+0000 7f24f4bf1980 1 NOTICE: finished all shards (2 entries found) 2024-04-17T04:40:09.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.131+0000 7f23a8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.135+0000 7f24f4bf1980 20 remove_watcher() i=0 2024-04-17T04:40:09.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.135+0000 7f24f4bf1980 2 removed watcher, disabling cache 2024-04-17T04:40:09.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.139+0000 7f24f4bf1980 20 remove_watcher() i=1 2024-04-17T04:40:09.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.139+0000 7f24f4bf1980 20 remove_watcher() i=2 2024-04-17T04:40:09.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.143+0000 7f24f4bf1980 20 remove_watcher() i=3 2024-04-17T04:40:09.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.143+0000 7f24f4bf1980 20 remove_watcher() i=4 2024-04-17T04:40:09.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.147+0000 7f24f4bf1980 20 remove_watcher() i=5 2024-04-17T04:40:09.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.147+0000 7f24f4bf1980 20 remove_watcher() i=6 2024-04-17T04:40:09.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.151+0000 7f24f4bf1980 20 remove_watcher() i=7 2024-04-17T04:40:09.165 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-04-17T04:40:09.165 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:09.166 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-17T04:40:09.201 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:09.201 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:09.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.243+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 realm 2024-04-17T04:40:09.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:09.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:09.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.247+0000 7f1691cb4980 20 searching for the correct realm 2024-04-17T04:40:09.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.255+0000 7f168efce640 0 -- 172.21.15.88:0/64587643 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x556bb8e96550 msgr2=0x556bb8e989c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:40:09.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.259+0000 7f168f7cf640 0 -- 172.21.15.88:0/64587643 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f168408e0b0 msgr2=0x7f168406e830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:09.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:09.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:09.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:09.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:09.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 zone default found 2024-04-17T04:40:09.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 4 Realm: () 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 zonegroup default 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.271+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.275+0000 7f1691cb4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.275+0000 7f1691cb4980 20 started sync module instance, tier type = 2024-04-17T04:40:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.275+0000 7f1691cb4980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:09.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=0 2024-04-17T04:40:09.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=1 2024-04-17T04:40:09.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=2 2024-04-17T04:40:09.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=3 2024-04-17T04:40:09.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=4 2024-04-17T04:40:09.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=5 2024-04-17T04:40:09.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=6 2024-04-17T04:40:09.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 20 add_watcher() i=7 2024-04-17T04:40:09.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.291+0000 7f1691cb4980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:09.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1545fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:09.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:09.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 5 note: GC not initialized 2024-04-17T04:40:09.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1544fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 Filter name: none 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f14be7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:09.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:09.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:09.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:09.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:09.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:09.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:09.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:09.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.295+0000 7f1691cb4980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 get_obj_state: octx=0x556bb8c28150 obj=check-bucket:c state=0x556bb8ed11f8 s->prefetch_data=0 2024-04-17T04:40:09.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 get_obj_state: setting s->obj_tag to 22fzg1ogghu44ymeh9pour1rbr8lsbjv 2024-04-17T04:40:09.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 get_obj_state_impl: setting s->olh_tag to hc1shpegqsm08pmis10oq4d922n4m6jz 2024-04-17T04:40:09.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-04-17T04:40:09.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 olh_log_entry: epoch=4 op=3 key=c[rsbd0kyFhgP9cZGVW7JcdIfH3Zk0-7J] 2024-04-17T04:40:09.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.299+0000 7f1691cb4980 20 get_obj_state: octx=0x556bb8efd9b0 obj=check-bucket:_:rsbd0kyFhgP9cZGVW7JcdIfH3Zk0-7J_c state=0x556bb8ecfac8 s->prefetch_data=0 2024-04-17T04:40:09.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.303+0000 7f1691cb4980 20 get_obj_state: octx=0x556bb8efd9b0 obj=check-bucket:c state=0x556bb8eceab8 s->prefetch_data=0 2024-04-17T04:40:09.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.303+0000 7f1691cb4980 20 get_obj_state: setting s->obj_tag to 22fzg1ogghu44ymeh9pour1rbr8lsbjv 2024-04-17T04:40:09.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.303+0000 7f1691cb4980 20 get_obj_state_impl: setting s->olh_tag to hc1shpegqsm08pmis10oq4d922n4m6jz 2024-04-17T04:40:09.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.303+0000 7f1691cb4980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=hc1shpegqsm08pmis10oq4d922n4m6jz 2024-04-17T04:40:09.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.311+0000 7f1691cb4980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 get_obj_state: octx=0x556bb8ed0d10 obj=check-bucket:d state=0x556bb8ed11f8 s->prefetch_data=0 2024-04-17T04:40:09.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 get_obj_state: setting s->obj_tag to cz5v1k5x65sr7q91evkt4azru2vt83fi 2024-04-17T04:40:09.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 get_obj_state_impl: setting s->olh_tag to 70xzqg1c21ce4xxv1qpcpib002j7i9my 2024-04-17T04:40:09.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-04-17T04:40:09.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 olh_log_entry: epoch=4 op=3 key=d[xgWyrXbfOpBQPJnJ1G64Ic.ySEcmGdD] 2024-04-17T04:40:09.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.315+0000 7f1691cb4980 20 get_obj_state: octx=0x556bb8efd9b0 obj=check-bucket:_:xgWyrXbfOpBQPJnJ1G64Ic.ySEcmGdD_d state=0x556bb8f00148 s->prefetch_data=0 2024-04-17T04:40:09.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.319+0000 7f1691cb4980 20 get_obj_state: octx=0x556bb8efd9b0 obj=check-bucket:d state=0x556bb8ecdaa8 s->prefetch_data=0 2024-04-17T04:40:09.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.319+0000 7f1691cb4980 20 get_obj_state: setting s->obj_tag to cz5v1k5x65sr7q91evkt4azru2vt83fi 2024-04-17T04:40:09.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.319+0000 7f1691cb4980 20 get_obj_state_impl: setting s->olh_tag to 70xzqg1c21ce4xxv1qpcpib002j7i9my 2024-04-17T04:40:09.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.319+0000 7f1691cb4980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=70xzqg1c21ce4xxv1qpcpib002j7i9my 2024-04-17T04:40:09.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.323+0000 7f1691cb4980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.323+0000 7f1691cb4980 1 NOTICE: finished shard 0 (2 entries removed) 2024-04-17T04:40:09.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.323+0000 7f1691cb4980 1 NOTICE: finished all shards (2 entries removed) 2024-04-17T04:40:09.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.327+0000 7f1545fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.331+0000 7f1691cb4980 20 remove_watcher() i=0 2024-04-17T04:40:09.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.331+0000 7f1691cb4980 2 removed watcher, disabling cache 2024-04-17T04:40:09.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.331+0000 7f1691cb4980 20 remove_watcher() i=1 2024-04-17T04:40:09.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.335+0000 7f1691cb4980 20 remove_watcher() i=2 2024-04-17T04:40:09.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.339+0000 7f1691cb4980 20 remove_watcher() i=3 2024-04-17T04:40:09.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.339+0000 7f1691cb4980 20 remove_watcher() i=4 2024-04-17T04:40:09.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.339+0000 7f1691cb4980 20 remove_watcher() i=5 2024-04-17T04:40:09.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.343+0000 7f1691cb4980 20 remove_watcher() i=6 2024-04-17T04:40:09.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.343+0000 7f1691cb4980 20 remove_watcher() i=7 2024-04-17T04:40:09.359 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-17T04:40:09.388 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:09.388 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:09.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 20 realm 2024-04-17T04:40:09.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:09.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.431+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.435+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.435+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.435+0000 7f42ecef0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.435+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.435+0000 7f42ecef0980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:09.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.435+0000 7f42ecef0980 20 searching for the correct realm 2024-04-17T04:40:09.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.439+0000 7f42ea20a640 0 -- 172.21.15.88:0/18732261 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x55a8f46f0750 msgr2=0x55a8f46f2b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.447+0000 7f42eaa0b640 0 -- 172.21.15.88:0/18732261 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f42d406bbf0 msgr2=0x7f42d4056000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:09.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 zone default found 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 4 Realm: () 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 zonegroup default 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.455+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.459+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.459+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.459+0000 7f42ecef0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.459+0000 7f42ecef0980 20 started sync module instance, tier type = 2024-04-17T04:40:09.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.459+0000 7f42ecef0980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:09.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=0 2024-04-17T04:40:09.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=1 2024-04-17T04:40:09.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=2 2024-04-17T04:40:09.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=3 2024-04-17T04:40:09.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=4 2024-04-17T04:40:09.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=5 2024-04-17T04:40:09.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=6 2024-04-17T04:40:09.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 20 add_watcher() i=7 2024-04-17T04:40:09.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.475+0000 7f42ecef0980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:09.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f41a17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 5 note: GC not initialized 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f411bfff640 20 reqs_thread_entry: start 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 Filter name: none 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f4119ffb640 20 reqs_thread_entry: start 2024-04-17T04:40:09.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:09.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 INFO: main: starting shard=0 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:40:09.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.479+0000 7f42ecef0980 20 INFO: main: done 2024-04-17T04:40:09.481 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.483+0000 7f41a17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.487+0000 7f42ecef0980 20 remove_watcher() i=0 2024-04-17T04:40:09.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.487+0000 7f42ecef0980 2 removed watcher, disabling cache 2024-04-17T04:40:09.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.487+0000 7f42ecef0980 20 remove_watcher() i=1 2024-04-17T04:40:09.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.491+0000 7f42ecef0980 20 remove_watcher() i=2 2024-04-17T04:40:09.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.491+0000 7f42ecef0980 20 remove_watcher() i=3 2024-04-17T04:40:09.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.495+0000 7f42ecef0980 20 remove_watcher() i=4 2024-04-17T04:40:09.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.495+0000 7f42ecef0980 20 remove_watcher() i=5 2024-04-17T04:40:09.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.499+0000 7f42ecef0980 20 remove_watcher() i=6 2024-04-17T04:40:09.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.499+0000 7f42ecef0980 20 remove_watcher() i=7 2024-04-17T04:40:09.512 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-17T04:40:09.540 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:09.540 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:09.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.583+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 realm 2024-04-17T04:40:09.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:09.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:09.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.587+0000 7fcbc76c7980 20 searching for the correct realm 2024-04-17T04:40:09.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.595+0000 7fcbc49e1640 0 -- 172.21.15.88:0/445876963 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fcbb8033040 msgr2=0x7fcbb8033930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:40:09.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.603+0000 7fcbc49e1640 0 -- 172.21.15.88:0/445876963 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fca900738f0 msgr2=0x7fca90075d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 zone default found 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 4 Realm: () 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 zonegroup default 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 started sync module instance, tier type = 2024-04-17T04:40:09.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.611+0000 7fcbc76c7980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=0 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=1 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=2 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=3 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=4 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=5 2024-04-17T04:40:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=6 2024-04-17T04:40:09.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 20 add_watcher() i=7 2024-04-17T04:40:09.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.631+0000 7fcbc76c7980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:09.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fca7bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fcbc76c7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:09.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fcbc76c7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:09.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fcbc76c7980 5 note: GC not initialized 2024-04-17T04:40:09.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fca7afe5640 20 reqs_thread_entry: start 2024-04-17T04:40:09.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fcbc76c7980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:09.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fcbc76c7980 20 Filter name: none 2024-04-17T04:40:09.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fca78fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:09.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.635+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:09.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:09.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:09.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:09.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 INFO: main: starting shard=0 2024-04-17T04:40:09.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:40:09.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fcbc76c7980 20 INFO: main: done 2024-04-17T04:40:09.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.639+0000 7fca7bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.647+0000 7fcbc76c7980 20 remove_watcher() i=0 2024-04-17T04:40:09.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.647+0000 7fcbc76c7980 2 removed watcher, disabling cache 2024-04-17T04:40:09.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.647+0000 7fcbc76c7980 20 remove_watcher() i=1 2024-04-17T04:40:09.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.651+0000 7fcbc76c7980 20 remove_watcher() i=2 2024-04-17T04:40:09.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.651+0000 7fcbc76c7980 20 remove_watcher() i=3 2024-04-17T04:40:09.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.655+0000 7fcbc76c7980 20 remove_watcher() i=4 2024-04-17T04:40:09.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.655+0000 7fcbc76c7980 20 remove_watcher() i=5 2024-04-17T04:40:09.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.655+0000 7fcbc76c7980 20 remove_watcher() i=6 2024-04-17T04:40:09.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.659+0000 7fcbc76c7980 20 remove_watcher() i=7 2024-04-17T04:40:09.670 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-04-17T04:40:09.671 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:09.671 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-04-17T04:40:09.701 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:09.701 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:09.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 realm 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.743+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.747+0000 7ff3f9b3e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.747+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.747+0000 7ff3f9b3e980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:09.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.747+0000 7ff3f9b3e980 20 searching for the correct realm 2024-04-17T04:40:09.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.755+0000 7ff3f6e58640 0 -- 172.21.15.88:0/2335055890 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7ff3e807a040 msgr2=0x7ff3e8055a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.759+0000 7ff3f6e58640 0 -- 172.21.15.88:0/2335055890 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7ff2c00738e0 msgr2=0x7ff2c0075d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:09.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 zone default found 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 4 Realm: () 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 zonegroup default 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:09.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 started sync module instance, tier type = 2024-04-17T04:40:09.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.767+0000 7ff3f9b3e980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=0 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=1 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=2 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=3 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=4 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=5 2024-04-17T04:40:09.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=6 2024-04-17T04:40:09.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 20 add_watcher() i=7 2024-04-17T04:40:09.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.783+0000 7ff3f9b3e980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:09.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff2adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 5 note: GC not initialized 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff2acfe1640 20 reqs_thread_entry: start 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 Filter name: none 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff2267fc640 20 reqs_thread_entry: start 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:09.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:09.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:09.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:09.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:09.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 rados->read ofs=0 len=0 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.787+0000 7ff3f9b3e980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.791+0000 7ff3f9b3e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.791+0000 7ff3f9b3e980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.791+0000 7ff3f9b3e980 1 NOTICE: finished shard 0 (0 entries found) 2024-04-17T04:40:09.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.791+0000 7ff3f9b3e980 1 NOTICE: finished all shards (0 entries found) 2024-04-17T04:40:09.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.791+0000 7ff2adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:09.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.795+0000 7ff3f9b3e980 20 remove_watcher() i=0 2024-04-17T04:40:09.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.795+0000 7ff3f9b3e980 2 removed watcher, disabling cache 2024-04-17T04:40:09.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.799+0000 7ff3f9b3e980 20 remove_watcher() i=1 2024-04-17T04:40:09.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.799+0000 7ff3f9b3e980 20 remove_watcher() i=2 2024-04-17T04:40:09.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.803+0000 7ff3f9b3e980 20 remove_watcher() i=3 2024-04-17T04:40:09.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.803+0000 7ff3f9b3e980 20 remove_watcher() i=4 2024-04-17T04:40:09.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.803+0000 7ff3f9b3e980 20 remove_watcher() i=5 2024-04-17T04:40:09.805 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.807+0000 7ff3f9b3e980 20 remove_watcher() i=6 2024-04-17T04:40:09.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:09.807+0000 7ff3f9b3e980 20 remove_watcher() i=7 2024-04-17T04:40:09.817 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-04-17T04:40:09.817 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:09.936 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-04-17T04:40:09.936 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:40:11.832 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-04-17T04:40:12.257 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-04-17T04:40:13.829 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:13.830 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-17T04:40:13.864 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:13.864 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:13.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.907+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 realm 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:13.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:13.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.911+0000 7f1874c2c980 20 searching for the correct realm 2024-04-17T04:40:13.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.919+0000 7f1871f46640 0 -- 172.21.15.88:0/907586757 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f185c0c6040 msgr2=0x7f185c0f2400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:13.925 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.927+0000 7f1871f46640 0 -- 172.21.15.88:0/907586757 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f173c073910 msgr2=0x7f173c075d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:13.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:13.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:13.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:13.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.931+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:13.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 zone default found 2024-04-17T04:40:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 4 Realm: () 2024-04-17T04:40:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 zonegroup default 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:13.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 started sync module instance, tier type = 2024-04-17T04:40:13.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.935+0000 7f1874c2c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:13.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=0 2024-04-17T04:40:13.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=1 2024-04-17T04:40:13.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=2 2024-04-17T04:40:13.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=3 2024-04-17T04:40:13.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=4 2024-04-17T04:40:13.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=5 2024-04-17T04:40:13.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=6 2024-04-17T04:40:13.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 20 add_watcher() i=7 2024-04-17T04:40:13.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.951+0000 7f1874c2c980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:13.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1728fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 5 note: GC not initialized 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f16a37fe640 20 reqs_thread_entry: start 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 20 Filter name: none 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f16a17fa640 20 reqs_thread_entry: start 2024-04-17T04:40:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.955+0000 7f1874c2c980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:13.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 rados->read ofs=0 len=0 2024-04-17T04:40:13.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:13.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:13.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:13.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 INFO: main: starting shard=0 2024-04-17T04:40:13.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:13.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-17T04:40:13.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1874c2c980 20 INFO: main: done 2024-04-17T04:40:13.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.959+0000 7f1728fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:13.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.963+0000 7f1874c2c980 20 remove_watcher() i=0 2024-04-17T04:40:13.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.963+0000 7f1874c2c980 2 removed watcher, disabling cache 2024-04-17T04:40:13.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.967+0000 7f1874c2c980 20 remove_watcher() i=1 2024-04-17T04:40:13.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.967+0000 7f1874c2c980 20 remove_watcher() i=2 2024-04-17T04:40:13.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.971+0000 7f1874c2c980 20 remove_watcher() i=3 2024-04-17T04:40:13.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.971+0000 7f1874c2c980 20 remove_watcher() i=4 2024-04-17T04:40:13.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.975+0000 7f1874c2c980 20 remove_watcher() i=5 2024-04-17T04:40:13.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.975+0000 7f1874c2c980 20 remove_watcher() i=6 2024-04-17T04:40:13.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:13.975+0000 7f1874c2c980 20 remove_watcher() i=7 2024-04-17T04:40:14.241 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:14.242 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-17T04:40:14.276 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:14.276 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:14.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 realm 2024-04-17T04:40:14.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:14.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:14.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.323+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.327+0000 7f21f8826980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:14.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.327+0000 7f21f8826980 20 searching for the correct realm 2024-04-17T04:40:14.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.331+0000 7f21f6b42640 0 -- 172.21.15.88:0/3141305206 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f21ec022260 msgr2=0x7f21ec08ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:14.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.547+0000 7f21f6341640 0 -- 172.21.15.88:0/3141305206 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5582e99e5050 msgr2=0x5582e99e7510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:14.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:14.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:14.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:14.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:14.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:14.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:14.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:14.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.555+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 zone default found 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 4 Realm: () 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 zonegroup default 2024-04-17T04:40:14.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 started sync module instance, tier type = 2024-04-17T04:40:14.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.559+0000 7f21f8826980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:14.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=0 2024-04-17T04:40:14.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=1 2024-04-17T04:40:14.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=2 2024-04-17T04:40:14.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=3 2024-04-17T04:40:14.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=4 2024-04-17T04:40:14.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=5 2024-04-17T04:40:14.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=6 2024-04-17T04:40:14.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 20 add_watcher() i=7 2024-04-17T04:40:14.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.587+0000 7f21f8826980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:14.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.591+0000 7f20acfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:14.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:14.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:14.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 5 note: GC not initialized 2024-04-17T04:40:14.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f20277fe640 20 reqs_thread_entry: start 2024-04-17T04:40:14.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:14.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 Filter name: none 2024-04-17T04:40:14.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f20257fa640 20 reqs_thread_entry: start 2024-04-17T04:40:14.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:14.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:14.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:14.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:14.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 INFO: main: starting shard=0 2024-04-17T04:40:14.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:14.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.595+0000 7f21f8826980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-17T04:40:14.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.599+0000 7f21f8826980 20 INFO: main: done 2024-04-17T04:40:14.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.599+0000 7f20acfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:14.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.603+0000 7f21f8826980 20 remove_watcher() i=0 2024-04-17T04:40:14.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.603+0000 7f21f8826980 2 removed watcher, disabling cache 2024-04-17T04:40:14.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.607+0000 7f21f8826980 20 remove_watcher() i=1 2024-04-17T04:40:14.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.607+0000 7f21f8826980 20 remove_watcher() i=2 2024-04-17T04:40:14.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.611+0000 7f21f8826980 20 remove_watcher() i=3 2024-04-17T04:40:14.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.615+0000 7f21f8826980 20 remove_watcher() i=4 2024-04-17T04:40:14.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.615+0000 7f21f8826980 20 remove_watcher() i=5 2024-04-17T04:40:14.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.615+0000 7f21f8826980 20 remove_watcher() i=6 2024-04-17T04:40:14.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.619+0000 7f21f8826980 20 remove_watcher() i=7 2024-04-17T04:40:14.825 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:14.826 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-04-17T04:40:14.861 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:14.861 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:14.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.907+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 realm 2024-04-17T04:40:14.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:14.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:14.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:14.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.911+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:14.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.915+0000 7f862dc6b980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:14.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.915+0000 7f862dc6b980 20 searching for the correct realm 2024-04-17T04:40:14.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:14.919+0000 7f862bf87640 0 -- 172.21.15.88:0/2411678163 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f86200e3400 msgr2=0x7f86200e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:40:15.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.135+0000 7f862b786640 0 -- 172.21.15.88:0/2411678163 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x561b4903fd50 msgr2=0x561b49042140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:15.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:15.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:15.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 zone default found 2024-04-17T04:40:15.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 4 Realm: () 2024-04-17T04:40:15.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:15.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:15.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:15.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 zonegroup default 2024-04-17T04:40:15.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.143+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.147+0000 7f862dc6b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.147+0000 7f862dc6b980 20 started sync module instance, tier type = 2024-04-17T04:40:15.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.147+0000 7f862dc6b980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=0 2024-04-17T04:40:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=1 2024-04-17T04:40:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=2 2024-04-17T04:40:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=3 2024-04-17T04:40:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=4 2024-04-17T04:40:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=5 2024-04-17T04:40:15.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.167+0000 7f862dc6b980 20 add_watcher() i=6 2024-04-17T04:40:15.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.171+0000 7f862dc6b980 20 add_watcher() i=7 2024-04-17T04:40:15.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.171+0000 7f862dc6b980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:15.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.175+0000 7f84e1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:15.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.175+0000 7f862dc6b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:15.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.175+0000 7f862dc6b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:15.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.175+0000 7f862dc6b980 5 note: GC not initialized 2024-04-17T04:40:15.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.175+0000 7f84e0fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:15.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.175+0000 7f862dc6b980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:15.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 Filter name: none 2024-04-17T04:40:15.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f845a7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:15.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:15.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:15.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:15.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:15.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 INFO: main: starting shard=0 2024-04-17T04:40:15.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:15.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:40:15.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.179+0000 7f862dc6b980 20 INFO: main: done 2024-04-17T04:40:15.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.183+0000 7f84e1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:15.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.187+0000 7f862dc6b980 20 remove_watcher() i=0 2024-04-17T04:40:15.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.187+0000 7f862dc6b980 2 removed watcher, disabling cache 2024-04-17T04:40:15.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.191+0000 7f862dc6b980 20 remove_watcher() i=1 2024-04-17T04:40:15.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.191+0000 7f862dc6b980 20 remove_watcher() i=2 2024-04-17T04:40:15.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.195+0000 7f862dc6b980 20 remove_watcher() i=3 2024-04-17T04:40:15.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.195+0000 7f862dc6b980 20 remove_watcher() i=4 2024-04-17T04:40:15.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.199+0000 7f862dc6b980 20 remove_watcher() i=5 2024-04-17T04:40:15.198 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.199+0000 7f862dc6b980 20 remove_watcher() i=6 2024-04-17T04:40:15.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.199+0000 7f862dc6b980 20 remove_watcher() i=7 2024-04-17T04:40:15.434 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:40:15.434 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-04-17T04:40:15.470 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:15.470 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:15.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 realm 2024-04-17T04:40:15.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:15.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.515+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.519+0000 7f40ec367980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:15.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.519+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.519+0000 7f40ec367980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:15.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.519+0000 7f40ec367980 20 searching for the correct realm 2024-04-17T04:40:15.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.527+0000 7f40e9681640 0 -- 172.21.15.88:0/1141532215 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f40dc034060 msgr2=0x7f40dc076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:15.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.531+0000 7f40e9681640 0 -- 172.21.15.88:0/1141532215 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f3fb40738e0 msgr2=0x7f3fb4075d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:15.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:15.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:15.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:15.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:15.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:15.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:15.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:15.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:15.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 zone default found 2024-04-17T04:40:15.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 4 Realm: () 2024-04-17T04:40:15.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:15.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:15.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:15.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 zonegroup default 2024-04-17T04:40:15.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.539+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.543+0000 7f40ec367980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:15.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.543+0000 7f40ec367980 20 started sync module instance, tier type = 2024-04-17T04:40:15.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.543+0000 7f40ec367980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:15.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=0 2024-04-17T04:40:15.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=1 2024-04-17T04:40:15.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=2 2024-04-17T04:40:15.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=3 2024-04-17T04:40:15.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=4 2024-04-17T04:40:15.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=5 2024-04-17T04:40:15.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=6 2024-04-17T04:40:15.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 20 add_watcher() i=7 2024-04-17T04:40:15.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f40ec367980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:15.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.559+0000 7f3f9b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 5 note: GC not initialized 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f3f9a7fc640 20 reqs_thread_entry: start 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 Filter name: none 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f3f0bfff640 20 reqs_thread_entry: start 2024-04-17T04:40:15.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:15.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 rados->read ofs=0 len=0 2024-04-17T04:40:15.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:15.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:15.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-17T04:40:15.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 INFO: main: starting shard=0 2024-04-17T04:40:15.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:15.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:40:15.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f40ec367980 20 INFO: main: done 2024-04-17T04:40:15.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.563+0000 7f3f9b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:15.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.571+0000 7f40ec367980 20 remove_watcher() i=0 2024-04-17T04:40:15.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.571+0000 7f40ec367980 2 removed watcher, disabling cache 2024-04-17T04:40:15.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.571+0000 7f40ec367980 20 remove_watcher() i=1 2024-04-17T04:40:15.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.575+0000 7f40ec367980 20 remove_watcher() i=2 2024-04-17T04:40:15.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.575+0000 7f40ec367980 20 remove_watcher() i=3 2024-04-17T04:40:15.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.579+0000 7f40ec367980 20 remove_watcher() i=4 2024-04-17T04:40:15.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.579+0000 7f40ec367980 20 remove_watcher() i=5 2024-04-17T04:40:15.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.583+0000 7f40ec367980 20 remove_watcher() i=6 2024-04-17T04:40:15.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:15.583+0000 7f40ec367980 20 remove_watcher() i=7 2024-04-17T04:40:15.621 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-04-17T04:40:16.033 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-04-17T04:40:16.447 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-04-17T04:40:16.477 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:16.477 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:16.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.519+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 realm 2024-04-17T04:40:16.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:16.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:16.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:16.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.523+0000 7ff7bc763980 20 searching for the correct realm 2024-04-17T04:40:16.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.531+0000 7ff7baa7f640 0 -- 172.21.15.88:0/2203817102 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7ff7b005d5c0 msgr2=0x7ff7b005be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:16.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.747+0000 7ff7ba27e640 0 -- 172.21.15.88:0/2203817102 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x556a56c8c2d0 msgr2=0x556a56c8e6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 zone default found 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 4 Realm: () 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 zonegroup default 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.755+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 started sync module instance, tier type = 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.759+0000 7ff7bc763980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=0 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=1 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=2 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=3 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=4 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=5 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.783+0000 7ff7bc763980 20 add_watcher() i=6 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.787+0000 7ff7bc763980 20 add_watcher() i=7 2024-04-17T04:40:16.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.787+0000 7ff7bc763980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff670fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 5 note: GC not initialized 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff5eb7fe640 20 reqs_thread_entry: start 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 Filter name: none 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff5e97fa640 20 reqs_thread_entry: start 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:16.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.791+0000 7ff7bc763980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.795+0000 7ff7bc763980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.799+0000 7ff670fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.807+0000 7ff7bc763980 20 remove_watcher() i=0 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.807+0000 7ff7bc763980 2 removed watcher, disabling cache 2024-04-17T04:40:16.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.811+0000 7ff7bc763980 20 remove_watcher() i=1 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.811+0000 7ff7bc763980 20 remove_watcher() i=2 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.815+0000 7ff7bc763980 20 remove_watcher() i=3 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.815+0000 7ff7bc763980 20 remove_watcher() i=4 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.819+0000 7ff7bc763980 20 remove_watcher() i=5 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.819+0000 7ff7bc763980 20 remove_watcher() i=6 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.819+0000 7ff7bc763980 20 remove_watcher() i=7 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:16.906 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:16.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 realm 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:16.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.915+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.919+0000 7f2c840c1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:16.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.919+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:16.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.919+0000 7f2c840c1980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:16.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.919+0000 7f2c840c1980 20 searching for the correct realm 2024-04-17T04:40:16.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.923+0000 7f2c813db640 0 -- 172.21.15.88:0/3358755395 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f2c74019070 msgr2=0x7f2c740960d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:16.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:16.931+0000 7f2c81bdc640 0 -- 172.21.15.88:0/3358755395 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f2c78027040 msgr2=0x7f2c78019920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:17.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.147+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:17.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:17.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 zone default found 2024-04-17T04:40:17.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 4 Realm: () 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 zonegroup default 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.151+0000 7f2c840c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.155+0000 7f2c840c1980 20 started sync module instance, tier type = 2024-04-17T04:40:17.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.155+0000 7f2c840c1980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=0 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=1 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=2 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=3 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=4 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=5 2024-04-17T04:40:17.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.175+0000 7f2c840c1980 20 add_watcher() i=6 2024-04-17T04:40:17.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.179+0000 7f2c840c1980 20 add_watcher() i=7 2024-04-17T04:40:17.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.179+0000 7f2c840c1980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:17.182 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2b337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:17.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:17.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:17.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 5 note: GC not initialized 2024-04-17T04:40:17.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2b327fc640 20 reqs_thread_entry: start 2024-04-17T04:40:17.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:17.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 20 Filter name: none 2024-04-17T04:40:17.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2aa3fff640 20 reqs_thread_entry: start 2024-04-17T04:40:17.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-17T04:40:17.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.183+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:17.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-17T04:40:17.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-17T04:40:17.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 : miss 2024-04-17T04:40:17.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 info.flags=0x17 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 to cache LRU end 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2 2024-04-17T04:40:17.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.187+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:c state=0x557b67838128 s->prefetch_data=0 2024-04-17T04:40:17.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.191+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to svay1kcyhpjzloj6w801p9bzq2ciklor 2024-04-17T04:40:17.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.191+0000 7f2c840c1980 20 get_obj_state_impl: setting s->olh_tag to djc5hmjup2j61113x8r2u1fvaj66w73i 2024-04-17T04:40:17.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.191+0000 7f2c840c1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=djc5hmjup2j61113x8r2u1fvaj66w73i 2024-04-17T04:40:17.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.191+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 olh_log_entry: epoch=3 op=3 key=c[c-8ViyPrAzD88exNm8Ix6647LIYBU7P] 2024-04-17T04:40:17.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:c-8ViyPrAzD88exNm8Ix6647LIYBU7P_c state=0x557b678660c8 s->prefetch_data=0 2024-04-17T04:40:17.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 10 manifest: total_size = 12 2024-04-17T04:40:17.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15488047808737594323 2024-04-17T04:40:17.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:c-8ViyPrAzD88exNm8Ix6647LIYBU7P_c state=0x557b678660c8 s->prefetch_data=0 2024-04-17T04:40:17.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 prepare_atomic_modification: state is not atomic. state=0x557b678660c8 2024-04-17T04:40:17.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.195+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.199+0000 7f2c840c1980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:17.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.199+0000 7f2c840c1980 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-04-17T04:40:17.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.199+0000 7f2c840c1980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:17.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.199+0000 7f2c840c1980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:c-8ViyPrAzD88exNm8Ix6647LIYBU7P_.O3mS0rck_zRCfuwVMyIskAU9tS-YVql_0 2024-04-17T04:40:17.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.203+0000 7f2b657fa640 20 handle_completion(): completion ok for obj=c[c-8ViyPrAzD88exNm8Ix6647LIYBU7P] 2024-04-17T04:40:17.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.203+0000 7f2c840c1980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:17.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.207+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.207+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.211+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.211+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.215+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:d state=0x557b678640a8 s->prefetch_data=0 2024-04-17T04:40:17.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.215+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to prhodod6hcqd0zwye8r2lib97gs6pz4m 2024-04-17T04:40:17.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.215+0000 7f2c840c1980 20 get_obj_state_impl: setting s->olh_tag to guatcz1qo0lvtfevps6l0y1kxbzmmlm0 2024-04-17T04:40:17.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.215+0000 7f2c840c1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=guatcz1qo0lvtfevps6l0y1kxbzmmlm0 2024-04-17T04:40:17.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.215+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 olh_log_entry: epoch=3 op=3 key=d[P0.miErz6OZSsA1qBZQ21NREGFVX7sv] 2024-04-17T04:40:17.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:P0.miErz6OZSsA1qBZQ21NREGFVX7sv_d state=0x557b67833578 s->prefetch_data=0 2024-04-17T04:40:17.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 10 manifest: total_size = 22 2024-04-17T04:40:17.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15946577799751828884 2024-04-17T04:40:17.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:P0.miErz6OZSsA1qBZQ21NREGFVX7sv_d state=0x557b67833578 s->prefetch_data=0 2024-04-17T04:40:17.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 prepare_atomic_modification: state is not atomic. state=0x557b67833578 2024-04-17T04:40:17.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.219+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.227+0000 7f2c840c1980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:17.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.227+0000 7f2c840c1980 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-17T04:40:17.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.227+0000 7f2c840c1980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:17.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.227+0000 7f2c840c1980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:P0.miErz6OZSsA1qBZQ21NREGFVX7sv_.2Uui1Ky38vYvsoSpHXz8qPiiCdi51t-_0 2024-04-17T04:40:17.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.227+0000 7f2b64ff9640 20 handle_completion(): completion ok for obj=d[P0.miErz6OZSsA1qBZQ21NREGFVX7sv] 2024-04-17T04:40:17.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.231+0000 7f2c840c1980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:17.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.231+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.231+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.235+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.235+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.235+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:e state=0x557b678640a8 s->prefetch_data=0 2024-04-17T04:40:17.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.235+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to rmjiig6uap8gs29lijw5oeyvqbbo7k1g 2024-04-17T04:40:17.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.235+0000 7f2c840c1980 20 get_obj_state_impl: setting s->olh_tag to 7bjuvgh9f08f05jm92x4rbel6cixirz2 2024-04-17T04:40:17.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.235+0000 7f2c840c1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7bjuvgh9f08f05jm92x4rbel6cixirz2 2024-04-17T04:40:17.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.239+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-04-17T04:40:17.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 olh_log_entry: epoch=2 op=3 key=e[DJJY1r3RYS6MO2GAK5ojhQW6e.85AKR] 2024-04-17T04:40:17.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:DJJY1r3RYS6MO2GAK5ojhQW6e.85AKR_e state=0x557b67835598 s->prefetch_data=0 2024-04-17T04:40:17.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 10 manifest: total_size = 12 2024-04-17T04:40:17.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4983881766724816005 2024-04-17T04:40:17.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:DJJY1r3RYS6MO2GAK5ojhQW6e.85AKR_e state=0x557b67835598 s->prefetch_data=0 2024-04-17T04:40:17.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 prepare_atomic_modification: state is not atomic. state=0x557b67835598 2024-04-17T04:40:17.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.243+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.251+0000 7f2c840c1980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:17.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.251+0000 7f2c840c1980 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-04-17T04:40:17.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.251+0000 7f2c840c1980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:17.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.251+0000 7f2c840c1980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:DJJY1r3RYS6MO2GAK5ojhQW6e.85AKR_.r8UTfm-KBkFh9xjc-7aA_x7FD4NCTMj_0 2024-04-17T04:40:17.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.251+0000 7f2b657fa640 20 handle_completion(): completion ok for obj=e[DJJY1r3RYS6MO2GAK5ojhQW6e.85AKR] 2024-04-17T04:40:17.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.259+0000 7f2c840c1980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:17.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.263+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:e state=0x557b678640a8 s->prefetch_data=0 2024-04-17T04:40:17.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.263+0000 7f2c840c1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7bjuvgh9f08f05jm92x4rbel6cixirz2 2024-04-17T04:40:17.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.271+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.275+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.275+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.275+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:f state=0x557b678640a8 s->prefetch_data=0 2024-04-17T04:40:17.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.275+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to 2vwytygp6go9b0do7sbbuxi9xb0be955 2024-04-17T04:40:17.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.275+0000 7f2c840c1980 20 get_obj_state_impl: setting s->olh_tag to h97nzd11f0urjms3texup1awdcanwcs0 2024-04-17T04:40:17.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.275+0000 7f2c840c1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=h97nzd11f0urjms3texup1awdcanwcs0 2024-04-17T04:40:17.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.279+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-04-17T04:40:17.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 olh_log_entry: epoch=2 op=3 key=f[uDR0o7CqK3U6B2xHJjWgQf5IsmCiAmE] 2024-04-17T04:40:17.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:uDR0o7CqK3U6B2xHJjWgQf5IsmCiAmE_f state=0x557b67835598 s->prefetch_data=0 2024-04-17T04:40:17.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 10 manifest: total_size = 22 2024-04-17T04:40:17.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8417747584224782681 2024-04-17T04:40:17.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:_:uDR0o7CqK3U6B2xHJjWgQf5IsmCiAmE_f state=0x557b67835598 s->prefetch_data=0 2024-04-17T04:40:17.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 prepare_atomic_modification: state is not atomic. state=0x557b67835598 2024-04-17T04:40:17.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.283+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.291+0000 7f2c840c1980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:40:17.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.291+0000 7f2c840c1980 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-17T04:40:17.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.291+0000 7f2c840c1980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:40:17.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.291+0000 7f2c840c1980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2__shadow:uDR0o7CqK3U6B2xHJjWgQf5IsmCiAmE_.pXfbV2nQOf8U-CYUhYDGXeySg9hBZHV_0 2024-04-17T04:40:17.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.291+0000 7f2b64ff9640 20 handle_completion(): completion ok for obj=f[uDR0o7CqK3U6B2xHJjWgQf5IsmCiAmE] 2024-04-17T04:40:17.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.295+0000 7f2c840c1980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:40:17.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.299+0000 7f2c840c1980 20 get_obj_state: octx=0x557b6758c160 obj=check-bucket:f state=0x557b678640a8 s->prefetch_data=0 2024-04-17T04:40:17.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.299+0000 7f2c840c1980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=h97nzd11f0urjms3texup1awdcanwcs0 2024-04-17T04:40:17.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.303+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.307+0000 7f2c840c1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2.0 2024-04-17T04:40:17.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.307+0000 7f2c840c1980 1 NOTICE: finished shard 0 (4 entries removed) 2024-04-17T04:40:17.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.307+0000 7f2c840c1980 1 NOTICE: finished all shards (4 entries removed) 2024-04-17T04:40:17.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.307+0000 7f2b337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:17.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.315+0000 7f2c840c1980 20 remove_watcher() i=0 2024-04-17T04:40:17.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.315+0000 7f2c840c1980 2 removed watcher, disabling cache 2024-04-17T04:40:17.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.315+0000 7f2c840c1980 20 remove_watcher() i=1 2024-04-17T04:40:17.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.319+0000 7f2c840c1980 20 remove_watcher() i=2 2024-04-17T04:40:17.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.319+0000 7f2c840c1980 20 remove_watcher() i=3 2024-04-17T04:40:17.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.323+0000 7f2c840c1980 20 remove_watcher() i=4 2024-04-17T04:40:17.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.323+0000 7f2c840c1980 20 remove_watcher() i=5 2024-04-17T04:40:17.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.327+0000 7f2c840c1980 20 remove_watcher() i=6 2024-04-17T04:40:17.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.327+0000 7f2c840c1980 20 remove_watcher() i=7 2024-04-17T04:40:17.865 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-04-17T04:40:17.896 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:40:17.897 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:40:17.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 realm 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:17.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.943+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.947+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:17.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.947+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:17.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.947+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:40:17.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.947+0000 7fab45659980 20 searching for the correct realm 2024-04-17T04:40:17.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:17.951+0000 7fab43975640 0 -- 172.21.15.88:0/1103512075 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5609dc7cfd80 msgr2=0x5609dc7d2170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:18.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.163+0000 7fab43174640 0 -- 172.21.15.88:0/1103512075 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fab38084400 msgr2=0x7fab38063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:40:18.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:40:18.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:40:18.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:40:18.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:40:18.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:40:18.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:40:18.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:18.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.579+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:40:18.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 zone default found 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 4 Realm: () 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 zonegroup default 2024-04-17T04:40:18.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:18.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:18.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:40:18.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 started sync module instance, tier type = 2024-04-17T04:40:18.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.583+0000 7fab45659980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=0 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=1 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=2 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=3 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=4 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=5 2024-04-17T04:40:18.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=6 2024-04-17T04:40:18.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 20 add_watcher() i=7 2024-04-17T04:40:18.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.611+0000 7fab45659980 2 all 8 watchers are set, enabling cache 2024-04-17T04:40:18.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.615+0000 7fa9f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:18.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.615+0000 7fab45659980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:40:18.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.615+0000 7fab45659980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:40:18.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 5 note: GC not initialized 2024-04-17T04:40:18.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fa9f8fe1640 20 reqs_thread_entry: start 2024-04-17T04:40:18.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 init_complete bucket index max shards: 1 2024-04-17T04:40:18.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 Filter name: none 2024-04-17T04:40:18.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fa9727fc640 20 reqs_thread_entry: start 2024-04-17T04:40:18.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:18.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:18.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:18.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=190 2024-04-17T04:40:18.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.619+0000 7fab45659980 20 rados->read ofs=0 len=0 2024-04-17T04:40:18.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.623+0000 7fab45659980 20 rados_obj.operate() r=0 bl.length=404 2024-04-17T04:40:18.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.623+0000 7fab45659980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:40:18.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.623+0000 7fa9f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:40:18.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.631+0000 7fab45659980 20 remove_watcher() i=0 2024-04-17T04:40:18.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.631+0000 7fab45659980 2 removed watcher, disabling cache 2024-04-17T04:40:18.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.631+0000 7fab45659980 20 remove_watcher() i=1 2024-04-17T04:40:18.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.635+0000 7fab45659980 20 remove_watcher() i=2 2024-04-17T04:40:18.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.639+0000 7fab45659980 20 remove_watcher() i=3 2024-04-17T04:40:18.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.639+0000 7fab45659980 20 remove_watcher() i=4 2024-04-17T04:40:18.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.643+0000 7fab45659980 20 remove_watcher() i=5 2024-04-17T04:40:18.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.643+0000 7fab45659980 20 remove_watcher() i=6 2024-04-17T04:40:18.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:40:18.647+0000 7fab45659980 20 remove_watcher() i=7 2024-04-17T04:40:18.661 INFO:tasks.workunit.client.0.smithi088.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-04-17T04:40:18.661 INFO:tasks.workunit.client.0.smithi088.stderr:Deleting bucket check-bucket 2024-04-17T04:40:19.019 INFO:tasks.workunit.client.0.smithi088.stderr:Completed bucket check tests 2024-04-17T04:40:19.020 INFO:tasks.workunit.client.0.smithi088.stdout:connected to http://localhost:80 2024-04-17T04:40:19.110 INFO:tasks.workunit.client.0.smithi088.stderr:+ deactivate 2024-04-17T04:40:19.110 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-04-17T04:40:19.110 INFO:tasks.workunit.client.0.smithi088.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:40:19.110 INFO:tasks.workunit.client.0.smithi088.stderr:+ export PATH 2024-04-17T04:40:19.110 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset _OLD_VIRTUAL_PATH 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n '' ']' 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ hash -r 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n '' ']' 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset VIRTUAL_ENV 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' '!' '' = nondestructive ']' 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset -f deactivate 2024-04-17T04:40:19.111 INFO:tasks.workunit.client.0.smithi088.stdout:OK. 2024-04-17T04:40:19.112 INFO:tasks.workunit.client.0.smithi088.stderr:+ echo OK. 2024-04-17T04:40:19.112 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:40:19.113 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:40:19.125 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-04-17T04:40:19.126 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T04:40:19.560 DEBUG:teuthology.parallel:result is None 2024-04-17T04:40:19.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:40:19.573 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:40:19.573 DEBUG:teuthology.orchestra.run.smithi088:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:40:19.621 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:40:19.621 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T04:40:19.631 INFO:tasks.workunit:Pulling workunits from ref 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:40:19.631 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T04:40:19.632 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:40:19.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:40:19.669 INFO:teuthology.orchestra.run.smithi088.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T04:40:19.670 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:40:19.717 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-17T04:40:19.717 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-17T04:40:19.765 INFO:tasks.workunit:timeout=3h 2024-04-17T04:40:19.765 INFO:tasks.workunit:cleanup=True 2024-04-17T04:40:19.765 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:40:19.815 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T04:41:32.676 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '41c500dd60c93c8d83053b72cffc3dd1a952a336'. 2024-04-17T04:41:32.676 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.676 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T04:41:32.676 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T04:41:32.676 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T04:41:32.676 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.677 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-04-17T04:41:32.678 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.678 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T04:41:32.678 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:41:32.678 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 41c500dd60c Merge pull request #56902 from adk3798/wip-65415-squid 2024-04-17T04:41:32.680 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T04:41:32.729 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T04:41:32.733 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:41:32.733 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T04:41:32.785 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T04:41:32.841 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T04:41:32.882 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:41:32.883 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:41:32.883 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T04:41:32.924 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:41:32.927 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:41:32.927 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T04:41:32.979 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-04-17T04:41:32.980 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-04-17T04:41:32.980 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-04-17T04:41:33.041 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:41:33.041 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-17T04:41:33.041 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:41:33.042 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 11 tests from ClsLock 2024-04-17T04:41:33.042 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-04-17T04:41:34.874 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:34.875 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: exclusive 2024-04-17T04:41:34.875 INFO:tasks.workunit.client.0.smithi088.stdout: tag: 2024-04-17T04:41:34.875 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:34.875 INFO:tasks.workunit.client.0.smithi088.stdout: client.5154 expiration=2024-04-17T04:43:34.872224+0000 addr=v1:172.21.15.88:0/192728290 cookie= 2024-04-17T04:41:34.881 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:34.881 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: shared 2024-04-17T04:41:34.881 INFO:tasks.workunit.client.0.smithi088.stdout: tag: 2024-04-17T04:41:34.881 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:34.881 INFO:tasks.workunit.client.0.smithi088.stdout: client.5154 expiration=2024-04-17T04:43:34.883148+0000 addr=v1:172.21.15.88:0/192728290 cookie= 2024-04-17T04:41:34.881 INFO:tasks.workunit.client.0.smithi088.stdout: client.5163 expiration=0.000000 addr=v1:172.21.15.88:0/3148228866 cookie= 2024-04-17T04:41:34.883 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:34.884 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: shared 2024-04-17T04:41:34.884 INFO:tasks.workunit.client.0.smithi088.stdout: tag: 2024-04-17T04:41:34.884 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:34.884 INFO:tasks.workunit.client.0.smithi088.stdout: client.5163 expiration=0.000000 addr=v1:172.21.15.88:0/3148228866 cookie= 2024-04-17T04:41:35.833 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestMultiLocking (2792 ms) 2024-04-17T04:41:35.834 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestMeta 2024-04-17T04:41:37.939 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:37.939 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: shared 2024-04-17T04:41:37.940 INFO:tasks.workunit.client.0.smithi088.stdout: tag: 2024-04-17T04:41:37.940 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:37.940 INFO:tasks.workunit.client.0.smithi088.stdout: client.5178 expiration=0.000000 addr=v1:172.21.15.88:0/1997814192 cookie= 2024-04-17T04:41:37.944 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:37.944 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: exclusive 2024-04-17T04:41:37.944 INFO:tasks.workunit.client.0.smithi088.stdout: tag: new_tag 2024-04-17T04:41:37.944 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:37.944 INFO:tasks.workunit.client.0.smithi088.stdout: client.5172 expiration=0.000000 addr=v1:172.21.15.88:0/485514041 cookie= 2024-04-17T04:41:38.897 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestMeta (3063 ms) 2024-04-17T04:41:38.897 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestCookie 2024-04-17T04:41:40.917 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:40.917 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: exclusive 2024-04-17T04:41:40.918 INFO:tasks.workunit.client.0.smithi088.stdout: tag: 2024-04-17T04:41:40.918 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:40.924 INFO:tasks.workunit.client.0.smithi088.stdout:lock: mylock 2024-04-17T04:41:40.924 INFO:tasks.workunit.client.0.smithi088.stdout: lock_type: shared 2024-04-17T04:41:40.924 INFO:tasks.workunit.client.0.smithi088.stdout: tag: 2024-04-17T04:41:40.924 INFO:tasks.workunit.client.0.smithi088.stdout: lockers: 2024-04-17T04:41:40.924 INFO:tasks.workunit.client.0.smithi088.stdout: client.5184 expiration=0.000000 addr=v1:172.21.15.88:0/3705352906 cookie= 2024-04-17T04:41:40.924 INFO:tasks.workunit.client.0.smithi088.stdout: client.5184 expiration=0.000000 addr=v1:172.21.15.88:0/3705352906 cookie=new cookie 2024-04-17T04:41:41.906 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestCookie (3010 ms) 2024-04-17T04:41:41.906 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-04-17T04:41:44.952 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestMultipleLocks (3046 ms) 2024-04-17T04:41:44.952 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestLockDuration 2024-04-17T04:41:52.058 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestLockDuration (7106 ms) 2024-04-17T04:41:52.058 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-04-17T04:41:55.097 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestAssertLocked (3038 ms) 2024-04-17T04:41:55.097 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestSetCookie 2024-04-17T04:41:58.107 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestSetCookie (3011 ms) 2024-04-17T04:41:58.107 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestRenew 2024-04-17T04:42:18.755 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestRenew (20648 ms) 2024-04-17T04:42:18.755 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-04-17T04:42:24.881 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6126 ms) 2024-04-17T04:42:24.882 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-04-17T04:42:31.960 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7079 ms) 2024-04-17T04:42:31.960 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-04-17T04:42:39.744 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7784 ms) 2024-04-17T04:42:39.744 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 11 tests from ClsLock (66703 ms total) 2024-04-17T04:42:39.744 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:42:39.744 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:42:39.744 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 11 tests from 1 test suite ran. (66703 ms total) 2024-04-17T04:42:39.744 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 11 tests. 2024-04-17T04:42:39.748 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:42:39.748 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:42:39.809 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-04-17T04:42:39.810 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-04-17T04:42:39.811 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-04-17T04:42:39.874 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:42:39.874 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-17T04:42:39.875 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:42:39.875 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 4 tests from cls_log 2024-04-17T04:42:39.875 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-04-17T04:42:42.760 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_log.test_log_add_same_time (2885 ms) 2024-04-17T04:42:42.760 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-04-17T04:42:45.831 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_log.test_log_add_different_time (3072 ms) 2024-04-17T04:42:45.832 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_log.trim_by_time 2024-04-17T04:42:48.832 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_log.trim_by_time (3000 ms) 2024-04-17T04:42:48.832 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_log.trim_by_marker 2024-04-17T04:42:51.894 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_log.trim_by_marker (3062 ms) 2024-04-17T04:42:51.894 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 4 tests from cls_log (12019 ms total) 2024-04-17T04:42:51.894 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:42:51.894 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:42:51.894 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 4 tests from 1 test suite ran. (12019 ms total) 2024-04-17T04:42:51.894 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 4 tests. 2024-04-17T04:42:51.899 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:42:51.899 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:42:51.957 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-04-17T04:42:51.958 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-04-17T04:42:51.958 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-04-17T04:42:52.025 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:42:52.025 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 10 tests from 1 test suite. 2024-04-17T04:42:52.025 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:42:52.025 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 10 tests from cls_refcount 2024-04-17T04:42:52.025 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_implicit 2024-04-17T04:42:54.940 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_implicit (2915 ms) 2024-04-17T04:42:54.940 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-04-17T04:42:57.957 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3018 ms) 2024-04-17T04:42:57.958 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_put_snap 2024-04-17T04:43:06.791 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_put_snap (8826 ms) 2024-04-17T04:43:06.791 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_explicit 2024-04-17T04:43:09.809 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_explicit (3026 ms) 2024-04-17T04:43:09.809 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.set 2024-04-17T04:43:12.861 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.set (3052 ms) 2024-04-17T04:43:12.862 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-04-17T04:43:19.965 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_implicit_ec (7103 ms) 2024-04-17T04:43:19.965 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-04-17T04:43:26.991 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7025 ms) 2024-04-17T04:43:26.991 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-04-17T04:43:39.140 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_put_snap_ec (12150 ms) 2024-04-17T04:43:39.140 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-04-17T04:43:46.189 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.test_explicit_ec (7050 ms) 2024-04-17T04:43:46.190 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_refcount.set_ec 2024-04-17T04:43:53.253 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_refcount.set_ec (7064 ms) 2024-04-17T04:43:53.253 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 10 tests from cls_refcount (61229 ms total) 2024-04-17T04:43:53.253 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:43:53.253 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:43:53.253 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 10 tests from 1 test suite ran. (61229 ms total) 2024-04-17T04:43:53.253 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 10 tests. 2024-04-17T04:43:53.257 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:43:53.257 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:43:53.316 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-04-17T04:43:53.317 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-04-17T04:43:53.317 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-04-17T04:43:53.382 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:43:53.382 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 16 tests from 1 test suite. 2024-04-17T04:43:53.382 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:43:53.382 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 16 tests from cls_rgw 2024-04-17T04:43:55.283 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_basic 2024-04-17T04:43:55.363 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_basic (79 ms) 2024-04-17T04:43:55.363 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-04-17T04:43:55.430 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (67 ms) 2024-04-17T04:43:55.431 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_remove_object 2024-04-17T04:43:55.547 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_remove_object (117 ms) 2024-04-17T04:43:55.547 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_suggest 2024-04-17T04:43:58.052 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_suggest (2505 ms) 2024-04-17T04:43:58.052 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-04-17T04:43:58.069 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_suggest_complete (17 ms) 2024-04-17T04:43:58.069 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_list 2024-04-17T04:43:58.124 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_list (55 ms) 2024-04-17T04:43:58.124 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-04-17T04:44:39.407 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_list_delimited (41283 ms) 2024-04-17T04:44:39.408 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.bi_list 2024-04-17T04:44:39.573 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.bi_list (167 ms) 2024-04-17T04:44:39.573 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.gc_set 2024-04-17T04:44:39.599 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.gc_set (25 ms) 2024-04-17T04:44:39.599 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.gc_list 2024-04-17T04:44:39.625 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.gc_list (27 ms) 2024-04-17T04:44:39.625 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.gc_defer 2024-04-17T04:44:46.800 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.gc_defer (7174 ms) 2024-04-17T04:44:46.800 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.usage_basic 2024-04-17T04:44:47.000 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.usage_basic (201 ms) 2024-04-17T04:44:47.001 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-04-17T04:44:47.003 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.usage_clear_no_obj (3 ms) 2024-04-17T04:44:47.003 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.usage_clear 2024-04-17T04:44:47.301 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.usage_clear (297 ms) 2024-04-17T04:44:47.301 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-04-17T04:44:47.627 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.bi_log_trim (327 ms) 2024-04-17T04:44:47.627 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-04-17T04:44:47.661 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw.index_racing_removes (33 ms) 2024-04-17T04:44:47.804 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 16 tests from cls_rgw (52378 ms total) 2024-04-17T04:44:47.804 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:44:47.805 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:44:47.805 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 16 tests from 1 test suite ran. (54423 ms total) 2024-04-17T04:44:47.805 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 16 tests. 2024-04-17T04:44:47.816 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:44:47.816 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:44:47.873 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-04-17T04:44:47.874 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-04-17T04:44:47.874 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-04-17T04:44:47.937 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:44:47.937 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-17T04:44:47.937 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:44:47.937 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 9 tests from cls_rgw_gc 2024-04-17T04:44:47.937 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.init 2024-04-17T04:44:49.823 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.init (1886 ms) 2024-04-17T04:44:49.823 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-04-17T04:44:49.865 INFO:tasks.workunit.client.0.smithi088.stderr:[ ] list info tag = chain-0 2024-04-17T04:44:49.865 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (42 ms) 2024-04-17T04:44:49.865 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-04-17T04:44:49.876 INFO:tasks.workunit.client.0.smithi088.stderr:[ ] next_marker is: = 0/4263 2024-04-17T04:44:49.877 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (12 ms) 2024-04-17T04:44:49.877 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-04-17T04:44:49.891 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (14 ms) 2024-04-17T04:44:49.891 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-04-17T04:44:49.906 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (15 ms) 2024-04-17T04:44:49.906 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-04-17T04:44:49.975 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (68 ms) 2024-04-17T04:44:49.975 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-04-17T04:44:49.992 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (18 ms) 2024-04-17T04:44:49.993 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-04-17T04:44:50.011 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (19 ms) 2024-04-17T04:44:50.011 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_gc.finalize 2024-04-17T04:44:50.820 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_gc.finalize (809 ms) 2024-04-17T04:44:50.820 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 9 tests from cls_rgw_gc (2884 ms total) 2024-04-17T04:44:50.820 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:44:50.820 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:44:50.821 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 9 tests from 1 test suite ran. (2884 ms total) 2024-04-17T04:44:50.821 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 9 tests. 2024-04-17T04:44:50.826 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:44:50.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:44:50.883 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-04-17T04:44:50.885 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-04-17T04:44:50.885 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-04-17T04:44:50.959 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:44:50.959 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:44:50.960 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 1 test from 1 test suite. 2024-04-17T04:44:50.960 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:44:52.851 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 1 test from cls_rgw_stats 2024-04-17T04:44:52.851 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] cls_rgw_stats.simulate 2024-04-17T04:44:52.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~QXWWASCQDT86.0 size=1048576 2024-04-17T04:44:52.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~QXWWASCQDT86.1 size=298649 2024-04-17T04:44:52.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.900+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=QXWWASCQDT86 tag=2O0VWSSBQSN0 type=write size=2395801 2024-04-17T04:44:52.900 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~BSIG92IKINGQ.0 size=1048576 2024-04-17T04:44:52.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~BSIG92IKINGQ.1 size=181968 2024-04-17T04:44:52.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.908+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=BSIG92IKINGQ tag=CSODA7IHZ72L type=write size=2279120 2024-04-17T04:44:52.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.908+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=J6VVFZIUR2JQ type=del size=2045131 2024-04-17T04:44:52.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.912+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=G8RTO2JSAJQ5 type=write size=375219 2024-04-17T04:44:52.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~0DKLQMVGR3BZ.0 size=92781 2024-04-17T04:44:52.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.916+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=0DKLQMVGR3BZ tag=4N2M2E2E7CM8 type=write size=1141357 2024-04-17T04:44:52.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.916+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=DJUOAFGHE3BH type=del size=870386 2024-04-17T04:44:52.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.920+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=3ED0AX33JTT2 type=del size=2102878 2024-04-17T04:44:52.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.924+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=PDJS32EDEDUI type=del size=487211 2024-04-17T04:44:52.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~BPIKAPC0JZ2W.0 size=44274 2024-04-17T04:44:52.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.928+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=BPIKAPC0JZ2W tag=0HVWIG3N3R9Z type=write size=1092850 2024-04-17T04:44:52.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.928+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=F6IRVJR60GV1 type=write size=926304 2024-04-17T04:44:52.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.932+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=E0LU38J41B2R type=del size=3143878 2024-04-17T04:44:52.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.932+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=8DZ12UV78XXU type=del size=1854896 2024-04-17T04:44:52.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~3CWQONI8KXCZ.0 size=850494 2024-04-17T04:44:52.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.940+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=3CWQONI8KXCZ tag=E6YYCEB1JKCZ type=write size=1899070 2024-04-17T04:44:52.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.940+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=6G6EGTUE0IPU type=del size=469665 2024-04-17T04:44:52.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.944+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=C10PF3HX0W02 type=del size=2344349 2024-04-17T04:44:52.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.944+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=06F877BJSRT6 type=del size=1737365 2024-04-17T04:44:52.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~QXWWASCQDT86.0 size=1048576 2024-04-17T04:44:52.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~QXWWASCQDT86.1 size=298649 2024-04-17T04:44:52.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.952+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=QXWWASCQDT86 tag=2O0VWSSBQSN0 type=write size=2395801 2024-04-17T04:44:52.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.956+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~DS9IIG03H1HS.0 size=1048576 2024-04-17T04:44:52.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~DS9IIG03H1HS.1 size=183651 2024-04-17T04:44:52.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.960+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=DS9IIG03H1HS tag=MYCOO4IN1D5Y type=write size=2280803 2024-04-17T04:44:52.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.960+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~BSIG92IKINGQ.0 size=1048576 2024-04-17T04:44:52.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.960+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1JOWGWFR7R6P.2~BSIG92IKINGQ.1 size=181968 2024-04-17T04:44:52.961 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.964+0000 7fd8ef2218c0 1 < canceled multipart upload key=1JOWGWFR7R6P upload=BSIG92IKINGQ tag=CSODA7IHZ72L type=write size=2279120 2024-04-17T04:44:52.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.968+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=4ZY6HSNFRYWI type=write size=474161 2024-04-17T04:44:52.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.972+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=J6VVFZIUR2JQ type=del 2024-04-17T04:44:52.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.976+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~H28JE765C41Q.0 size=864805 2024-04-17T04:44:52.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.976+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=H28JE765C41Q tag=R4EBOYACJ80J type=write size=1913381 2024-04-17T04:44:52.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.984+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=G8RTO2JSAJQ5 type=write size=375219 2024-04-17T04:44:52.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.984+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=MEJXCCKUE3VR type=del size=2783565 2024-04-17T04:44:52.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.988+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~0DKLQMVGR3BZ.0 size=92781 2024-04-17T04:44:52.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.992+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=0DKLQMVGR3BZ tag=4N2M2E2E7CM8 type=write size=1141357 2024-04-17T04:44:52.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:52.996+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=2O92W702OG4C type=del size=3345682 2024-04-17T04:44:52.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.000+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=DJUOAFGHE3BH type=del 2024-04-17T04:44:52.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.000+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=RSKZGNK9DDOC type=del size=621411 2024-04-17T04:44:53.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.004+0000 7fd8ef2218c0 1 < suggested update operation key=3OFQH4KVYN18 tag=3ED0AX33JTT2 type=del size=2102878 2024-04-17T04:44:53.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.008+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=D0BDHXRKDEQ0 type=del size=598203 2024-04-17T04:44:53.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.012+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=PDJS32EDEDUI type=del 2024-04-17T04:44:53.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.012+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=UMS0UH8MMX62 type=del size=606125 2024-04-17T04:44:53.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.016+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2S1IT3TO1BPU.2~BPIKAPC0JZ2W.0 size=44274 2024-04-17T04:44:53.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.020+0000 7fd8ef2218c0 1 < canceled multipart upload key=2S1IT3TO1BPU upload=BPIKAPC0JZ2W tag=0HVWIG3N3R9Z type=write size=1092850 2024-04-17T04:44:53.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.020+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=LEVKHEF2LPEM type=write size=1036350 2024-04-17T04:44:53.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.024+0000 7fd8ef2218c0 1 < completed write operation key=B63EZS0LAC30 tag=F6IRVJR60GV1 type=write size=926304 2024-04-17T04:44:53.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~U1GHPQV9VZ4H.0 size=482510 2024-04-17T04:44:53.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.028+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=U1GHPQV9VZ4H tag=Z7UZKMFJJPUX type=write size=1531086 2024-04-17T04:44:53.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.036+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=E0LU38J41B2R type=del 2024-04-17T04:44:53.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.040+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~9A7101CT632C.0 size=1048576 2024-04-17T04:44:53.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.040+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~9A7101CT632C.1 size=1048576 2024-04-17T04:44:53.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.044+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~9A7101CT632C.2 size=765608 2024-04-17T04:44:53.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.044+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=9A7101CT632C tag=U0DQU2WZT94W type=write size=3911336 2024-04-17T04:44:53.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.044+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=8DZ12UV78XXU type=del 2024-04-17T04:44:53.044 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.048+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=EFL05O57UB1N type=del size=796924 2024-04-17T04:44:53.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.052+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~3CWQONI8KXCZ.0 size=850494 2024-04-17T04:44:53.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.056+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=3CWQONI8KXCZ tag=E6YYCEB1JKCZ type=write size=1899070 2024-04-17T04:44:53.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.056+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=XHYRQRP9QF39 type=write size=686768 2024-04-17T04:44:53.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.060+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=6G6EGTUE0IPU type=del 2024-04-17T04:44:53.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.064+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=AI142HK5IHVR type=del size=2332018 2024-04-17T04:44:53.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.064+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=C10PF3HX0W02 type=del 2024-04-17T04:44:53.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.068+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=IOWD01F78SRO type=del size=3286356 2024-04-17T04:44:53.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.072+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=06F877BJSRT6 type=del size=1737365 2024-04-17T04:44:53.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~FHT58ZIR4EDV.0 size=802571 2024-04-17T04:44:53.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.076+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=FHT58ZIR4EDV tag=03QFXRAU397S type=write size=1851147 2024-04-17T04:44:53.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~DS9IIG03H1HS.0 size=1048576 2024-04-17T04:44:53.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~DS9IIG03H1HS.1 size=183651 2024-04-17T04:44:53.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.084+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=DS9IIG03H1HS tag=MYCOO4IN1D5Y type=write size=2280803 2024-04-17T04:44:53.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.088+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=IP2QT09PFQTP type=del size=3706299 2024-04-17T04:44:53.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.092+0000 7fd8ef2218c0 1 < completed write operation key=6219DPY72LWC tag=4ZY6HSNFRYWI type=write size=474161 2024-04-17T04:44:53.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.092+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=5FP07LPRQTV5 type=del size=432937 2024-04-17T04:44:53.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.096+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~H28JE765C41Q.0 size=864805 2024-04-17T04:44:53.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.100+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=H28JE765C41Q tag=R4EBOYACJ80J type=write size=1913381 2024-04-17T04:44:53.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.104+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=3COUVN17WXO0 type=del size=2570806 2024-04-17T04:44:53.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.104+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=MEJXCCKUE3VR type=del 2024-04-17T04:44:53.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~KFPUO5HR2G98.0 size=1048576 2024-04-17T04:44:53.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~KFPUO5HR2G98.1 size=1048576 2024-04-17T04:44:53.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~KFPUO5HR2G98.2 size=343546 2024-04-17T04:44:53.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=KFPUO5HR2G98 tag=MGJ0PX8QTCUU type=write size=3489274 2024-04-17T04:44:53.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.116+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=2O92W702OG4C type=del 2024-04-17T04:44:53.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.120+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=BADR79VYC8ZB type=del size=3128988 2024-04-17T04:44:53.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.124+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=RSKZGNK9DDOC type=del 2024-04-17T04:44:53.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.128+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~753FPH3SCQ9Q.0 size=1048576 2024-04-17T04:44:53.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.128+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~753FPH3SCQ9Q.1 size=1048576 2024-04-17T04:44:53.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~753FPH3SCQ9Q.2 size=611823 2024-04-17T04:44:53.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.132+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=753FPH3SCQ9Q tag=EV8BMK8VGX4S type=write size=3757551 2024-04-17T04:44:53.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.132+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=D0BDHXRKDEQ0 type=del 2024-04-17T04:44:53.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.140+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~6G2I6VUZDRE8.0 size=1048576 2024-04-17T04:44:53.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.140+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~6G2I6VUZDRE8.1 size=483957 2024-04-17T04:44:53.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.140+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=6G2I6VUZDRE8 tag=SKPWZ8ANNFJP type=write size=2581109 2024-04-17T04:44:53.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.144+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=UMS0UH8MMX62 type=del 2024-04-17T04:44:53.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.148+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=GFSYCZ5TBT80 type=write size=689828 2024-04-17T04:44:53.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.152+0000 7fd8ef2218c0 1 < completed write operation key=509DZ9DBJWC5 tag=LEVKHEF2LPEM type=write size=1036350 2024-04-17T04:44:53.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.156+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=US3TX04SHQZ6 type=del size=3971100 2024-04-17T04:44:53.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~U1GHPQV9VZ4H.0 size=482510 2024-04-17T04:44:53.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.168+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=U1GHPQV9VZ4H tag=Z7UZKMFJJPUX type=write size=1531086 2024-04-17T04:44:53.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.172+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~Q2EE4JG0Y0RO.0 size=17951 2024-04-17T04:44:53.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.172+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=Q2EE4JG0Y0RO tag=9O8H3YS6SQ4Y type=write size=1066527 2024-04-17T04:44:53.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~9A7101CT632C.0 size=1048576 2024-04-17T04:44:53.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~9A7101CT632C.1 size=1048576 2024-04-17T04:44:53.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.180+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~9A7101CT632C.2 size=765608 2024-04-17T04:44:53.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.184+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=9A7101CT632C tag=U0DQU2WZT94W type=write size=3911336 2024-04-17T04:44:53.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~NUZOQA5VIZRH.0 size=134387 2024-04-17T04:44:53.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.192+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=NUZOQA5VIZRH tag=AFC8LHIXJGL9 type=write size=1182963 2024-04-17T04:44:53.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.196+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=EFL05O57UB1N type=del 2024-04-17T04:44:53.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.196+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=7AMWBK02I77H type=del size=1693706 2024-04-17T04:44:53.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.200+0000 7fd8ef2218c0 1 < completed write operation key=1JOWGWFR7R6P tag=XHYRQRP9QF39 type=write size=686768 2024-04-17T04:44:53.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.204+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~8P1LAKA6GFMY.0 size=1048576 2024-04-17T04:44:53.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~8P1LAKA6GFMY.1 size=1048576 2024-04-17T04:44:53.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.212+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~8P1LAKA6GFMY.2 size=116397 2024-04-17T04:44:53.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.212+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=8P1LAKA6GFMY tag=4U32W25QMH0Q type=write size=3262125 2024-04-17T04:44:53.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.216+0000 7fd8ef2218c0 1 < suggested update operation key=R3C8PAVPVGBY tag=AI142HK5IHVR type=del size=2332018 2024-04-17T04:44:53.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.220+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=Z0TVE1Y8ORNJ type=del size=3139437 2024-04-17T04:44:53.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.224+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=IOWD01F78SRO type=del 2024-04-17T04:44:53.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.224+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=6IODBNQ25Y08 type=write size=486354 2024-04-17T04:44:53.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.228+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~FHT58ZIR4EDV.0 size=802571 2024-04-17T04:44:53.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.232+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=FHT58ZIR4EDV tag=03QFXRAU397S type=write size=1851147 2024-04-17T04:44:53.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.236+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~0PX59503938L.0 size=1048576 2024-04-17T04:44:53.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.240+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~0PX59503938L.1 size=1048576 2024-04-17T04:44:53.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~0PX59503938L.2 size=124300 2024-04-17T04:44:53.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.244+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=0PX59503938L tag=HRPHEFZ4JSHN type=write size=3270028 2024-04-17T04:44:53.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.244+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=IP2QT09PFQTP type=del 2024-04-17T04:44:53.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~8PUXH9V8WZRG.0 size=743635 2024-04-17T04:44:53.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.252+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=8PUXH9V8WZRG tag=KKXTVQJL10PL type=write size=1792211 2024-04-17T04:44:53.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.252+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=5FP07LPRQTV5 type=del 2024-04-17T04:44:53.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.256+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=OKNOJEINAUYQ type=del size=4029592 2024-04-17T04:44:53.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.260+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=3COUVN17WXO0 type=del 2024-04-17T04:44:53.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.264+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=S0OJZ8VVG7Y5 type=del size=1061538 2024-04-17T04:44:53.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~KFPUO5HR2G98.0 size=1048576 2024-04-17T04:44:53.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~KFPUO5HR2G98.1 size=1048576 2024-04-17T04:44:53.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.272+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~KFPUO5HR2G98.2 size=343546 2024-04-17T04:44:53.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.276+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=KFPUO5HR2G98 tag=MGJ0PX8QTCUU type=write size=3489274 2024-04-17T04:44:53.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.280+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=2SB3WSV1MGTA type=write size=1022961 2024-04-17T04:44:53.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.288+0000 7fd8ef2218c0 1 < suggested update operation key=B63EZS0LAC30 tag=BADR79VYC8ZB type=del size=3128988 2024-04-17T04:44:53.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.292+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=TRAWYGU3NZIF type=del size=905204 2024-04-17T04:44:53.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~753FPH3SCQ9Q.0 size=1048576 2024-04-17T04:44:53.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~753FPH3SCQ9Q.1 size=1048576 2024-04-17T04:44:53.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.300+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~753FPH3SCQ9Q.2 size=611823 2024-04-17T04:44:53.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.304+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=753FPH3SCQ9Q tag=EV8BMK8VGX4S type=write size=3757551 2024-04-17T04:44:53.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.308+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=AX9TLKIQR1EN type=del size=3766482 2024-04-17T04:44:53.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~6G2I6VUZDRE8.0 size=1048576 2024-04-17T04:44:53.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~6G2I6VUZDRE8.1 size=483957 2024-04-17T04:44:53.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.320+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=6G2I6VUZDRE8 tag=SKPWZ8ANNFJP type=write size=2581109 2024-04-17T04:44:53.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.324+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=G97IMYA3872K type=write size=267519 2024-04-17T04:44:53.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.328+0000 7fd8ef2218c0 1 < completed write operation key=5FR1NEWE0LD1 tag=GFSYCZ5TBT80 type=write size=689828 2024-04-17T04:44:53.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.332+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=X6LHYK4TJ48L type=write size=361837 2024-04-17T04:44:53.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.332+0000 7fd8ef2218c0 1 < canceled operation key=AE4CYL334ONO tag=US3TX04SHQZ6 type=del size=3971100 2024-04-17T04:44:53.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~4E2RRDGO71HY.0 size=1048576 2024-04-17T04:44:53.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~4E2RRDGO71HY.1 size=418141 2024-04-17T04:44:53.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.344+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=4E2RRDGO71HY tag=49ICSGBOLWTR type=write size=2515293 2024-04-17T04:44:53.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.344+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~Q2EE4JG0Y0RO.0 size=17951 2024-04-17T04:44:53.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.352+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=Q2EE4JG0Y0RO tag=9O8H3YS6SQ4Y type=write size=1066527 2024-04-17T04:44:53.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.352+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=GYJICFLLYGPF type=write size=45941 2024-04-17T04:44:53.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.356+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~NUZOQA5VIZRH.0 size=134387 2024-04-17T04:44:53.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.360+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=NUZOQA5VIZRH tag=AFC8LHIXJGL9 type=write size=1182963 2024-04-17T04:44:53.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.364+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=PEI59FZXBMP1 type=del size=31538 2024-04-17T04:44:53.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.372+0000 7fd8ef2218c0 1 < suggested update operation key=GX9X0D45GC2I tag=7AMWBK02I77H type=del size=1693706 2024-04-17T04:44:53.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.376+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=AGK9NGGH2W51 type=del size=4010524 2024-04-17T04:44:53.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~8P1LAKA6GFMY.0 size=1048576 2024-04-17T04:44:53.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~8P1LAKA6GFMY.1 size=1048576 2024-04-17T04:44:53.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.384+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~8P1LAKA6GFMY.2 size=116397 2024-04-17T04:44:53.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.388+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=8P1LAKA6GFMY tag=4U32W25QMH0Q type=write size=3262125 2024-04-17T04:44:53.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~DB1S6BCLJRJW.0 size=1048576 2024-04-17T04:44:53.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~DB1S6BCLJRJW.1 size=1048576 2024-04-17T04:44:53.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.400+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~DB1S6BCLJRJW.2 size=700232 2024-04-17T04:44:53.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.400+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=DB1S6BCLJRJW tag=BCM033TI4O1V type=write size=3845960 2024-04-17T04:44:53.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.404+0000 7fd8ef2218c0 1 < canceled operation key=5FR1NEWE0LD1 tag=Z0TVE1Y8ORNJ type=del size=3139437 2024-04-17T04:44:53.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.404+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=4PSFWYRW72FC type=del size=3567555 2024-04-17T04:44:53.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.408+0000 7fd8ef2218c0 1 < completed write operation key=AE4CYL334ONO tag=6IODBNQ25Y08 type=write size=486354 2024-04-17T04:44:53.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.412+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=XJ0YD1E8ZAMC type=del size=998732 2024-04-17T04:44:53.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.416+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~0PX59503938L.0 size=1048576 2024-04-17T04:44:53.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~0PX59503938L.1 size=1048576 2024-04-17T04:44:53.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~0PX59503938L.2 size=124300 2024-04-17T04:44:53.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.428+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=0PX59503938L tag=HRPHEFZ4JSHN type=write size=3270028 2024-04-17T04:44:53.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.428+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=0OC2WSKLFWW7 type=del size=662176 2024-04-17T04:44:53.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.432+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~8PUXH9V8WZRG.0 size=743635 2024-04-17T04:44:53.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.436+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=8PUXH9V8WZRG tag=KKXTVQJL10PL type=write size=1792211 2024-04-17T04:44:53.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~5VAH9S64N83O.0 size=1048576 2024-04-17T04:44:53.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~5VAH9S64N83O.1 size=1048576 2024-04-17T04:44:53.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~5VAH9S64N83O.2 size=527717 2024-04-17T04:44:53.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.448+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=5VAH9S64N83O tag=4CWBOGY529AY type=write size=3673445 2024-04-17T04:44:53.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.456+0000 7fd8ef2218c0 1 < suggested update operation key=UNOVYXDS5JEB tag=OKNOJEINAUYQ type=del size=4029592 2024-04-17T04:44:53.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.460+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=VDT0NZTBVWIJ type=write size=835280 2024-04-17T04:44:53.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.464+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=S0OJZ8VVG7Y5 type=del 2024-04-17T04:44:53.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.464+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=XIVDU0D0PTCI type=del size=2546977 2024-04-17T04:44:53.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.468+0000 7fd8ef2218c0 1 < canceled operation key=5FR1NEWE0LD1 tag=2SB3WSV1MGTA type=write size=1022961 2024-04-17T04:44:53.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.476+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~RM8S0RW73HTH.0 size=255214 2024-04-17T04:44:53.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.476+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=RM8S0RW73HTH tag=FZ9C3C8LUPKN type=write size=1303790 2024-04-17T04:44:53.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.480+0000 7fd8ef2218c0 1 < suggested remove operation key=5K50O96P7OGD tag=TRAWYGU3NZIF type=del 2024-04-17T04:44:53.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.484+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=P2K0NHK5SNB2 type=write size=284916 2024-04-17T04:44:53.483 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.488+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=AX9TLKIQR1EN type=del 2024-04-17T04:44:53.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.492+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=ISQUKW7CRSRI type=write size=155258 2024-04-17T04:44:53.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.496+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=G97IMYA3872K type=write size=267519 2024-04-17T04:44:53.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~XUUZCFM21FNY.0 size=1048576 2024-04-17T04:44:53.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.504+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~XUUZCFM21FNY.1 size=1013959 2024-04-17T04:44:53.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.504+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=XUUZCFM21FNY tag=069UE476I9OV type=write size=3111111 2024-04-17T04:44:53.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.508+0000 7fd8ef2218c0 1 < suggested update operation key=BCG05NIQFA53 tag=X6LHYK4TJ48L type=write size=361837 2024-04-17T04:44:53.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.512+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=3K68MIU7L6FK type=write size=940152 2024-04-17T04:44:53.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~4E2RRDGO71HY.0 size=1048576 2024-04-17T04:44:53.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~4E2RRDGO71HY.1 size=418141 2024-04-17T04:44:53.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.524+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=4E2RRDGO71HY tag=49ICSGBOLWTR type=write size=2515293 2024-04-17T04:44:53.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.524+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=L4XL35TWQ2Y4 type=del size=1235735 2024-04-17T04:44:53.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.528+0000 7fd8ef2218c0 1 < canceled operation key=M3U8L7M8JKG9 tag=GYJICFLLYGPF type=write size=45941 2024-04-17T04:44:53.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.532+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=T8HS03NYBH6E type=del size=2295749 2024-04-17T04:44:53.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.536+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=PEI59FZXBMP1 type=del 2024-04-17T04:44:53.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.540+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=BDXL6RHEH376 type=del size=2583491 2024-04-17T04:44:53.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.540+0000 7fd8ef2218c0 1 < canceled operation key=2S1IT3TO1BPU tag=AGK9NGGH2W51 type=del size=4010524 2024-04-17T04:44:53.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GPOZ75MGUFR8.0 size=1048576 2024-04-17T04:44:53.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GPOZ75MGUFR8.1 size=1048576 2024-04-17T04:44:53.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.552+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GPOZ75MGUFR8.2 size=933533 2024-04-17T04:44:53.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.552+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=GPOZ75MGUFR8 tag=3EYGMAGB6RGR type=write size=4079261 2024-04-17T04:44:53.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.552+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~DB1S6BCLJRJW.0 size=1048576 2024-04-17T04:44:53.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.556+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~DB1S6BCLJRJW.1 size=1048576 2024-04-17T04:44:53.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.556+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~DB1S6BCLJRJW.2 size=700232 2024-04-17T04:44:53.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.564+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=DB1S6BCLJRJW tag=BCM033TI4O1V type=write size=3845960 2024-04-17T04:44:53.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.564+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=0E24NEY2BZB5 type=del size=2124462 2024-04-17T04:44:53.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.568+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=4PSFWYRW72FC type=del 2024-04-17T04:44:53.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~C3PK5XH7HD27.0 size=490761 2024-04-17T04:44:53.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.576+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=C3PK5XH7HD27 tag=WWP8081MW17V type=write size=1539337 2024-04-17T04:44:53.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.580+0000 7fd8ef2218c0 1 < suggested remove operation key=1ZTJP8DRITEK tag=XJ0YD1E8ZAMC type=del 2024-04-17T04:44:53.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.584+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=3RC7RMBSOQXE type=del size=1392216 2024-04-17T04:44:53.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.588+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=0OC2WSKLFWW7 type=del 2024-04-17T04:44:53.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~28L8S6KROUFX.0 size=1048576 2024-04-17T04:44:53.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~28L8S6KROUFX.1 size=34867 2024-04-17T04:44:53.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=28L8S6KROUFX tag=3GHWLKBJYWUX type=write size=2132019 2024-04-17T04:44:53.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~5VAH9S64N83O.0 size=1048576 2024-04-17T04:44:53.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.604+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~5VAH9S64N83O.1 size=1048576 2024-04-17T04:44:53.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.604+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~5VAH9S64N83O.2 size=527717 2024-04-17T04:44:53.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.608+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=5VAH9S64N83O tag=4CWBOGY529AY type=write size=3673445 2024-04-17T04:44:53.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~OHXF75USGHIJ.0 size=1048576 2024-04-17T04:44:53.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~OHXF75USGHIJ.1 size=216398 2024-04-17T04:44:53.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.616+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=OHXF75USGHIJ tag=9Y1Q23Y7M6S5 type=write size=2313550 2024-04-17T04:44:53.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.620+0000 7fd8ef2218c0 -1 < failed to complete operation key=5K50O96P7OGD tag=VDT0NZTBVWIJ type=write size=835280: (22) Invalid argument 2024-04-17T04:44:53.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~2RUER60DUXC9.0 size=1048576 2024-04-17T04:44:53.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~2RUER60DUXC9.1 size=1048576 2024-04-17T04:44:53.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.632+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~2RUER60DUXC9.2 size=327117 2024-04-17T04:44:53.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.632+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=2RUER60DUXC9 tag=OZ796745H0HY type=write size=3472845 2024-04-17T04:44:53.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.636+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=XIVDU0D0PTCI type=del 2024-04-17T04:44:53.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.640+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~ZHRBXABPJEKJ.0 size=1048576 2024-04-17T04:44:53.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~ZHRBXABPJEKJ.1 size=202672 2024-04-17T04:44:53.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.644+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=ZHRBXABPJEKJ tag=QVEBU1X4KNES type=write size=2299824 2024-04-17T04:44:53.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.648+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~RM8S0RW73HTH.0 size=255214 2024-04-17T04:44:53.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.652+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=RM8S0RW73HTH tag=FZ9C3C8LUPKN type=write size=1303790 2024-04-17T04:44:53.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~FIH9PZ025GXS.0 size=1048576 2024-04-17T04:44:53.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.660+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~FIH9PZ025GXS.1 size=1048576 2024-04-17T04:44:53.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.664+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~FIH9PZ025GXS.2 size=498494 2024-04-17T04:44:53.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.664+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=FIH9PZ025GXS tag=WVCIHJNS2YJZ type=write size=3644222 2024-04-17T04:44:53.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.668+0000 7fd8ef2218c0 1 < suggested remove operation key=N11Y8FPN9XES tag=P2K0NHK5SNB2 type=write 2024-04-17T04:44:53.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.676+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~BVCVP1QOBPN2.0 size=1048576 2024-04-17T04:44:53.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~BVCVP1QOBPN2.1 size=478473 2024-04-17T04:44:53.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=BVCVP1QOBPN2 tag=FA28FOHOAVF2 type=write size=2575625 2024-04-17T04:44:53.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.684+0000 7fd8ef2218c0 1 < completed write operation key=UYCKMXE6LBUH tag=ISQUKW7CRSRI type=write size=155258 2024-04-17T04:44:53.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.684+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=6DZQAXXJV58F type=write size=905715 2024-04-17T04:44:53.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~XUUZCFM21FNY.0 size=1048576 2024-04-17T04:44:53.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.692+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~XUUZCFM21FNY.1 size=1013959 2024-04-17T04:44:53.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.696+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=XUUZCFM21FNY tag=069UE476I9OV type=write size=3111111 2024-04-17T04:44:53.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.700+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=6QTFFXHQWWOI type=del size=2116455 2024-04-17T04:44:53.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.704+0000 7fd8ef2218c0 1 < suggested remove operation key=BCG05NIQFA53 tag=3K68MIU7L6FK type=write 2024-04-17T04:44:53.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.708+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=TTPGXG9AAJNC type=del size=528541 2024-04-17T04:44:53.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.712+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=L4XL35TWQ2Y4 type=del 2024-04-17T04:44:53.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.716+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=403FG7V3MXDF type=write size=19566 2024-04-17T04:44:53.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.720+0000 7fd8ef2218c0 1 < suggested remove operation key=VUYM3JC57QRP tag=T8HS03NYBH6E type=del 2024-04-17T04:44:53.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.720+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=APJLXWMI6CER type=del size=3585179 2024-04-17T04:44:53.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.728+0000 7fd8ef2218c0 1 < suggested update operation key=B63EZS0LAC30 tag=BDXL6RHEH376 type=del size=2583491 2024-04-17T04:44:53.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.732+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~BENPN2OTD59M.0 size=1048576 2024-04-17T04:44:53.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~BENPN2OTD59M.1 size=1048576 2024-04-17T04:44:53.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~BENPN2OTD59M.2 size=492391 2024-04-17T04:44:53.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.736+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=BENPN2OTD59M tag=U4X459JVGNF0 type=write size=3638119 2024-04-17T04:44:53.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GPOZ75MGUFR8.0 size=1048576 2024-04-17T04:44:53.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GPOZ75MGUFR8.1 size=1048576 2024-04-17T04:44:53.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GPOZ75MGUFR8.2 size=933533 2024-04-17T04:44:53.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.752+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=GPOZ75MGUFR8 tag=3EYGMAGB6RGR type=write size=4079261 2024-04-17T04:44:53.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~2CTZOION3UKB.0 size=1048576 2024-04-17T04:44:53.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~2CTZOION3UKB.1 size=9363 2024-04-17T04:44:53.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.756+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=2CTZOION3UKB tag=NPWTSNYH1KC4 type=write size=2106515 2024-04-17T04:44:53.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.760+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=0E24NEY2BZB5 type=del 2024-04-17T04:44:53.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.764+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=GENOAB6HDHEH type=del size=1986415 2024-04-17T04:44:53.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.764+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~C3PK5XH7HD27.0 size=490761 2024-04-17T04:44:53.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.772+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=C3PK5XH7HD27 tag=WWP8081MW17V type=write size=1539337 2024-04-17T04:44:53.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.772+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=AH0OPIIOR2S3 type=del size=4137375 2024-04-17T04:44:53.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.776+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=3RC7RMBSOQXE type=del 2024-04-17T04:44:53.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.780+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~7IS6XT5U8Q5I.0 size=1048576 2024-04-17T04:44:53.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~7IS6XT5U8Q5I.1 size=1048576 2024-04-17T04:44:53.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~7IS6XT5U8Q5I.2 size=823521 2024-04-17T04:44:53.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.784+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=7IS6XT5U8Q5I tag=8X9WOEVUVS05 type=write size=3969249 2024-04-17T04:44:53.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~28L8S6KROUFX.0 size=1048576 2024-04-17T04:44:53.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~28L8S6KROUFX.1 size=34867 2024-04-17T04:44:53.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.792+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=28L8S6KROUFX tag=3GHWLKBJYWUX type=write size=2132019 2024-04-17T04:44:53.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.796+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=RQS168FKE6C7 type=write size=974346 2024-04-17T04:44:53.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~OHXF75USGHIJ.0 size=1048576 2024-04-17T04:44:53.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~OHXF75USGHIJ.1 size=216398 2024-04-17T04:44:53.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.804+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=OHXF75USGHIJ tag=9Y1Q23Y7M6S5 type=write size=2313550 2024-04-17T04:44:53.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.808+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=XJLBOLQZ3IHZ type=del size=1893792 2024-04-17T04:44:53.805 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~2RUER60DUXC9.0 size=1048576 2024-04-17T04:44:53.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~2RUER60DUXC9.1 size=1048576 2024-04-17T04:44:53.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~2RUER60DUXC9.2 size=327117 2024-04-17T04:44:53.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.816+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=2RUER60DUXC9 tag=OZ796745H0HY type=write size=3472845 2024-04-17T04:44:53.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.820+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=769DUV2KGMRG type=del size=52876 2024-04-17T04:44:53.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~ZHRBXABPJEKJ.0 size=1048576 2024-04-17T04:44:53.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~ZHRBXABPJEKJ.1 size=202672 2024-04-17T04:44:53.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.828+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=ZHRBXABPJEKJ tag=QVEBU1X4KNES type=write size=2299824 2024-04-17T04:44:53.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~R4N12IT94BS7.0 size=761439 2024-04-17T04:44:53.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=R4N12IT94BS7 tag=BB48K5P7FQ62 type=write size=1810015 2024-04-17T04:44:53.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.836+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~FIH9PZ025GXS.0 size=1048576 2024-04-17T04:44:53.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.836+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1JOWGWFR7R6P.2~FIH9PZ025GXS.1 size=1048576 2024-04-17T04:44:53.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.840+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~FIH9PZ025GXS.2 size=498494 2024-04-17T04:44:53.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.844+0000 7fd8ef2218c0 1 < canceled multipart upload key=1JOWGWFR7R6P upload=FIH9PZ025GXS tag=WVCIHJNS2YJZ type=write size=3644222 2024-04-17T04:44:53.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.848+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=2TXU93I2O140 type=del size=483370 2024-04-17T04:44:53.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.852+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~BVCVP1QOBPN2.0 size=1048576 2024-04-17T04:44:53.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.856+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~BVCVP1QOBPN2.1 size=478473 2024-04-17T04:44:53.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.860+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=BVCVP1QOBPN2 tag=FA28FOHOAVF2 type=write size=2575625 2024-04-17T04:44:53.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~MDO14S5S9329.0 size=1048576 2024-04-17T04:44:53.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~MDO14S5S9329.1 size=1048576 2024-04-17T04:44:53.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~MDO14S5S9329.2 size=28426 2024-04-17T04:44:53.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.872+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=MDO14S5S9329 tag=EH7XPX26ILAD type=write size=3174154 2024-04-17T04:44:53.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.888+0000 7fd8ef2218c0 1 < completed write operation key=82D6YWKJ1TFQ tag=6DZQAXXJV58F type=write size=905715 2024-04-17T04:44:53.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.892+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=LQX80KTQ7YRD type=del size=3438487 2024-04-17T04:44:53.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.896+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=6QTFFXHQWWOI type=del 2024-04-17T04:44:53.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.896+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=YMGEMJ261Y5B type=del size=728995 2024-04-17T04:44:53.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.900+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=TTPGXG9AAJNC type=del 2024-04-17T04:44:53.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.900+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=SZ2A4OEJCVE0 type=del size=3111300 2024-04-17T04:44:53.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.904+0000 7fd8ef2218c0 1 < completed write operation key=G03VLS09F3BT tag=403FG7V3MXDF type=write size=19566 2024-04-17T04:44:53.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~E1098O8IRURQ.0 size=1048576 2024-04-17T04:44:53.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~E1098O8IRURQ.1 size=1048576 2024-04-17T04:44:53.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.912+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~E1098O8IRURQ.2 size=646986 2024-04-17T04:44:53.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.912+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=E1098O8IRURQ tag=DDFL6N6OL2Q4 type=write size=3792714 2024-04-17T04:44:53.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.916+0000 7fd8ef2218c0 1 < suggested update operation key=82D6YWKJ1TFQ tag=APJLXWMI6CER type=del size=3585179 2024-04-17T04:44:53.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.916+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=3TGX0HN0QJV1 type=write size=875499 2024-04-17T04:44:53.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.920+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~BENPN2OTD59M.0 size=1048576 2024-04-17T04:44:53.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.920+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~BENPN2OTD59M.1 size=1048576 2024-04-17T04:44:53.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.924+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~BENPN2OTD59M.2 size=492391 2024-04-17T04:44:53.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.928+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=BENPN2OTD59M tag=U4X459JVGNF0 type=write size=3638119 2024-04-17T04:44:53.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~2KD00GF0C2OY.0 size=172601 2024-04-17T04:44:53.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.932+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=2KD00GF0C2OY tag=DZ8HA4J18NCY type=write size=1221177 2024-04-17T04:44:53.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~2CTZOION3UKB.0 size=1048576 2024-04-17T04:44:53.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~2CTZOION3UKB.1 size=9363 2024-04-17T04:44:53.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.936+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=2CTZOION3UKB tag=NPWTSNYH1KC4 type=write size=2106515 2024-04-17T04:44:53.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.940+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=W403O15V7YPD type=del size=3967561 2024-04-17T04:44:53.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.944+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=GENOAB6HDHEH type=del 2024-04-17T04:44:53.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.944+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=1C3XILJ1E3AN type=write size=709695 2024-04-17T04:44:53.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.948+0000 7fd8ef2218c0 1 < suggested remove operation key=N11Y8FPN9XES tag=AH0OPIIOR2S3 type=del 2024-04-17T04:44:53.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.952+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=R8GF4CWAC3YA type=del size=1620249 2024-04-17T04:44:53.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~7IS6XT5U8Q5I.0 size=1048576 2024-04-17T04:44:53.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.956+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~7IS6XT5U8Q5I.1 size=1048576 2024-04-17T04:44:53.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.956+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~7IS6XT5U8Q5I.2 size=823521 2024-04-17T04:44:53.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.960+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=7IS6XT5U8Q5I tag=8X9WOEVUVS05 type=write size=3969249 2024-04-17T04:44:53.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.964+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=BCC0L7YT96D3 type=write size=1007778 2024-04-17T04:44:53.961 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.964+0000 7fd8ef2218c0 1 < completed write operation key=6219DPY72LWC tag=RQS168FKE6C7 type=write size=974346 2024-04-17T04:44:53.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~IGQF88SBRCLC.0 size=1048576 2024-04-17T04:44:53.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~IGQF88SBRCLC.1 size=1048576 2024-04-17T04:44:53.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~IGQF88SBRCLC.2 size=839288 2024-04-17T04:44:53.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.972+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=IGQF88SBRCLC tag=W1XB2QSSRED8 type=write size=3985016 2024-04-17T04:44:53.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.976+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=N11Y8FPN9XES tag=XJLBOLQZ3IHZ type=del size=1893792: (2) No such file or directory 2024-04-17T04:44:53.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.976+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=W5CAD14ZZ29H type=del size=2675084 2024-04-17T04:44:53.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.980+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=769DUV2KGMRG type=del 2024-04-17T04:44:53.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~ET90SPLV3SGR.0 size=1048576 2024-04-17T04:44:53.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~ET90SPLV3SGR.1 size=463419 2024-04-17T04:44:53.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.984+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=ET90SPLV3SGR tag=MRPQ1NF8X37G type=write size=2560571 2024-04-17T04:44:53.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.984+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~R4N12IT94BS7.0 size=761439 2024-04-17T04:44:53.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.992+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=R4N12IT94BS7 tag=BB48K5P7FQ62 type=write size=1810015 2024-04-17T04:44:53.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.992+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=TWAL2LR67XG0 type=write size=250281 2024-04-17T04:44:53.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.996+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=2TXU93I2O140 type=del 2024-04-17T04:44:53.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:53.996+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=30DBJWTP0UXD type=del size=2619876 2024-04-17T04:44:53.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.000+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~MDO14S5S9329.0 size=1048576 2024-04-17T04:44:53.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.000+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~MDO14S5S9329.1 size=1048576 2024-04-17T04:44:53.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.004+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~MDO14S5S9329.2 size=28426 2024-04-17T04:44:54.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.008+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=MDO14S5S9329 tag=EH7XPX26ILAD type=write size=3174154 2024-04-17T04:44:54.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~ZHBNJRH52FEJ.0 size=1048576 2024-04-17T04:44:54.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~ZHBNJRH52FEJ.1 size=1048576 2024-04-17T04:44:54.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~ZHBNJRH52FEJ.2 size=892086 2024-04-17T04:44:54.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.012+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=ZHBNJRH52FEJ tag=QLK909QO0GBZ type=write size=4037814 2024-04-17T04:44:54.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.016+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=LQX80KTQ7YRD type=del 2024-04-17T04:44:54.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.016+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=88QGEPPVYAO4 type=del size=2237294 2024-04-17T04:44:54.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.020+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=YMGEMJ261Y5B type=del 2024-04-17T04:44:54.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.024+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=Y8393EW8L0MT type=del size=1581021 2024-04-17T04:44:54.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.024+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=SZ2A4OEJCVE0 type=del 2024-04-17T04:44:54.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~P7NLHEDCASN2.0 size=1048576 2024-04-17T04:44:54.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~P7NLHEDCASN2.1 size=1048576 2024-04-17T04:44:54.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.032+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~P7NLHEDCASN2.2 size=44401 2024-04-17T04:44:54.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.032+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=P7NLHEDCASN2 tag=1RAU2CX2AZP0 type=write size=3190129 2024-04-17T04:44:54.029 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.032+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~E1098O8IRURQ.0 size=1048576 2024-04-17T04:44:54.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~E1098O8IRURQ.1 size=1048576 2024-04-17T04:44:54.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~E1098O8IRURQ.2 size=646986 2024-04-17T04:44:54.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.040+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=E1098O8IRURQ tag=DDFL6N6OL2Q4 type=write size=3792714 2024-04-17T04:44:54.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.044+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=0MMQXX0F19ZB type=del size=3278629 2024-04-17T04:44:54.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.048+0000 7fd8ef2218c0 1 < suggested update operation key=1JOWGWFR7R6P tag=3TGX0HN0QJV1 type=write size=875499 2024-04-17T04:44:54.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.052+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~22QYS2UAXG1K.0 size=487762 2024-04-17T04:44:54.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.052+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=22QYS2UAXG1K tag=CMS4Q03AKXKP type=write size=1536338 2024-04-17T04:44:54.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.052+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~2KD00GF0C2OY.0 size=172601 2024-04-17T04:44:54.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.056+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=2KD00GF0C2OY tag=DZ8HA4J18NCY type=write size=1221177 2024-04-17T04:44:54.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.060+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=KJXP9FSZCCGJ type=del size=237006 2024-04-17T04:44:54.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.064+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=W403O15V7YPD type=del 2024-04-17T04:44:54.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.064+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=X16B1QFQ9JDT type=del size=3361914 2024-04-17T04:44:54.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.068+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=1C3XILJ1E3AN type=write 2024-04-17T04:44:54.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.072+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=7T672JNO426D type=del size=3560553 2024-04-17T04:44:54.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.076+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=R8GF4CWAC3YA type=del 2024-04-17T04:44:54.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.076+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=V2ME740KJI0X type=write size=493580 2024-04-17T04:44:54.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.080+0000 7fd8ef2218c0 1 < completed write operation key=82D6YWKJ1TFQ tag=BCC0L7YT96D3 type=write size=1007778 2024-04-17T04:44:54.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.080+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=D2F2NAOF4JJX type=del size=60053 2024-04-17T04:44:54.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~IGQF88SBRCLC.0 size=1048576 2024-04-17T04:44:54.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.084+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~IGQF88SBRCLC.1 size=1048576 2024-04-17T04:44:54.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~IGQF88SBRCLC.2 size=839288 2024-04-17T04:44:54.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.088+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=IGQF88SBRCLC tag=W1XB2QSSRED8 type=write size=3985016 2024-04-17T04:44:54.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~PQEFZ8U104BR.0 size=1048576 2024-04-17T04:44:54.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.096+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~PQEFZ8U104BR.1 size=1048576 2024-04-17T04:44:54.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.096+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~PQEFZ8U104BR.2 size=537699 2024-04-17T04:44:54.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.096+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=PQEFZ8U104BR tag=WSQ6EXF10DHJ type=write size=3683427 2024-04-17T04:44:54.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.100+0000 7fd8ef2218c0 1 < canceled operation key=MS3T476JOEON tag=W5CAD14ZZ29H type=del size=2675084 2024-04-17T04:44:54.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~EKWY9QN92935.0 size=255097 2024-04-17T04:44:54.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.104+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=EKWY9QN92935 tag=U5VNJJ8YRF68 type=write size=1303673 2024-04-17T04:44:54.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.104+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~ET90SPLV3SGR.0 size=1048576 2024-04-17T04:44:54.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~ET90SPLV3SGR.1 size=463419 2024-04-17T04:44:54.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.112+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=ET90SPLV3SGR tag=MRPQ1NF8X37G type=write size=2560571 2024-04-17T04:44:54.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~07IPN8ALJ6R9.0 size=1048576 2024-04-17T04:44:54.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.120+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~07IPN8ALJ6R9.1 size=330495 2024-04-17T04:44:54.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.120+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=07IPN8ALJ6R9 tag=GOTXHBMV563T type=write size=2427647 2024-04-17T04:44:54.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.120+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=TWAL2LR67XG0 type=write size=250281 2024-04-17T04:44:54.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~IEUXIY6RCXLN.0 size=1048576 2024-04-17T04:44:54.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.128+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~IEUXIY6RCXLN.1 size=376539 2024-04-17T04:44:54.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.128+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=IEUXIY6RCXLN tag=GGVFW9R8D5X4 type=write size=2473691 2024-04-17T04:44:54.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.128+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=30DBJWTP0UXD type=del 2024-04-17T04:44:54.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.132+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=OMC0WY2ULZCL type=write size=1012217 2024-04-17T04:44:54.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~ZHBNJRH52FEJ.0 size=1048576 2024-04-17T04:44:54.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~ZHBNJRH52FEJ.1 size=1048576 2024-04-17T04:44:54.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.136+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5ZULD0PHLS4V.2~ZHBNJRH52FEJ.2 size=892086 2024-04-17T04:44:54.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.140+0000 7fd8ef2218c0 1 < canceled multipart upload key=5ZULD0PHLS4V upload=ZHBNJRH52FEJ tag=QLK909QO0GBZ type=write size=4037814 2024-04-17T04:44:54.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.144+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=0WF5LIZM5KVH type=del size=369140 2024-04-17T04:44:54.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.148+0000 7fd8ef2218c0 1 < suggested remove operation key=5ZULD0PHLS4V tag=88QGEPPVYAO4 type=del 2024-04-17T04:44:54.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.152+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=VXKT29LYMJ1A type=write size=326392 2024-04-17T04:44:54.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.152+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=Y8393EW8L0MT type=del 2024-04-17T04:44:54.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.156+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~0U18UMAW9B5D.0 size=758770 2024-04-17T04:44:54.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.156+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=0U18UMAW9B5D tag=7XP25LGX70VG type=write size=1807346 2024-04-17T04:44:54.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.156+0000 7fd8ef2218c0 1 < canceled part key=_multipart_EWAW6EBW6W6U.2~P7NLHEDCASN2.0 size=1048576 2024-04-17T04:44:54.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~P7NLHEDCASN2.1 size=1048576 2024-04-17T04:44:54.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~P7NLHEDCASN2.2 size=44401 2024-04-17T04:44:54.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.164+0000 7fd8ef2218c0 1 < canceled multipart upload key=EWAW6EBW6W6U upload=P7NLHEDCASN2 tag=1RAU2CX2AZP0 type=write size=3190129 2024-04-17T04:44:54.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.168+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=U3TCQ4XCI36I type=del size=717560 2024-04-17T04:44:54.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.168+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=0MMQXX0F19ZB type=del 2024-04-17T04:44:54.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.172+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=YVI8UD13GGQ4 type=del size=2221369 2024-04-17T04:44:54.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~22QYS2UAXG1K.0 size=487762 2024-04-17T04:44:54.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.180+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=22QYS2UAXG1K tag=CMS4Q03AKXKP type=write size=1536338 2024-04-17T04:44:54.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.180+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=2ZPMUOMBATNY type=del size=1495754 2024-04-17T04:44:54.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.184+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=KJXP9FSZCCGJ type=del 2024-04-17T04:44:54.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~0SWE3YN4QP34.0 size=1048576 2024-04-17T04:44:54.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~0SWE3YN4QP34.1 size=564872 2024-04-17T04:44:54.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.192+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=0SWE3YN4QP34 tag=SKWPCR2T9OWV type=write size=2662024 2024-04-17T04:44:54.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.196+0000 7fd8ef2218c0 1 < suggested update operation key=82D6YWKJ1TFQ tag=X16B1QFQ9JDT type=del size=3361914 2024-04-17T04:44:54.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.196+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=QYAAC4FJ5IP8 type=del size=1857644 2024-04-17T04:44:54.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.204+0000 7fd8ef2218c0 1 < suggested remove operation key=3OFQH4KVYN18 tag=7T672JNO426D type=del 2024-04-17T04:44:54.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.212+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~6GDCXFF1QZJ7.0 size=1048576 2024-04-17T04:44:54.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~6GDCXFF1QZJ7.1 size=606732 2024-04-17T04:44:54.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.216+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=6GDCXFF1QZJ7 tag=U37JWVEKMM2K type=write size=2703884 2024-04-17T04:44:54.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.216+0000 7fd8ef2218c0 1 < completed write operation key=GX9X0D45GC2I tag=V2ME740KJI0X type=write size=493580 2024-04-17T04:44:54.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.220+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=DL4OL12WLDVE type=del size=2870504 2024-04-17T04:44:54.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.220+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=D2F2NAOF4JJX type=del 2024-04-17T04:44:54.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.228+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~PZDLGWTODDPJ.0 size=846628 2024-04-17T04:44:54.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.228+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=PZDLGWTODDPJ tag=M19G8NIFDS3H type=write size=1895204 2024-04-17T04:44:54.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~PQEFZ8U104BR.0 size=1048576 2024-04-17T04:44:54.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~PQEFZ8U104BR.1 size=1048576 2024-04-17T04:44:54.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~PQEFZ8U104BR.2 size=537699 2024-04-17T04:44:54.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.236+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=PQEFZ8U104BR tag=WSQ6EXF10DHJ type=write size=3683427 2024-04-17T04:44:54.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~V3QIQ59AA60G.0 size=1048576 2024-04-17T04:44:54.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~V3QIQ59AA60G.1 size=1048576 2024-04-17T04:44:54.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.248+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~V3QIQ59AA60G.2 size=97958 2024-04-17T04:44:54.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.248+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=V3QIQ59AA60G tag=M585ER53N5FZ type=write size=3243686 2024-04-17T04:44:54.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.248+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~EKWY9QN92935.0 size=255097 2024-04-17T04:44:54.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.252+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=EKWY9QN92935 tag=U5VNJJ8YRF68 type=write size=1303673 2024-04-17T04:44:54.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.256+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=VYV2Y5MN14E5 type=write size=538022 2024-04-17T04:44:54.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.260+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~07IPN8ALJ6R9.0 size=1048576 2024-04-17T04:44:54.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.260+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~07IPN8ALJ6R9.1 size=330495 2024-04-17T04:44:54.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.264+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=07IPN8ALJ6R9 tag=GOTXHBMV563T type=write size=2427647 2024-04-17T04:44:54.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.268+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=X57U427JRBFG type=del size=1229704 2024-04-17T04:44:54.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.272+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~IEUXIY6RCXLN.0 size=1048576 2024-04-17T04:44:54.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.272+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~IEUXIY6RCXLN.1 size=376539 2024-04-17T04:44:54.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.280+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=IEUXIY6RCXLN tag=GGVFW9R8D5X4 type=write size=2473691 2024-04-17T04:44:54.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.280+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=H30EPTE53L7N type=del size=842588 2024-04-17T04:44:54.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.284+0000 7fd8ef2218c0 1 < completed write operation key=MS3T476JOEON tag=OMC0WY2ULZCL type=write size=1012217 2024-04-17T04:44:54.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.288+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=QLIUDCX6MGUR type=del size=17875 2024-04-17T04:44:54.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.288+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=0WF5LIZM5KVH type=del 2024-04-17T04:44:54.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.292+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~Q8AVZG1IPVOA.0 size=1048576 2024-04-17T04:44:54.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.296+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~Q8AVZG1IPVOA.1 size=317170 2024-04-17T04:44:54.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.296+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=Q8AVZG1IPVOA tag=928LDPFMT2B3 type=write size=2414322 2024-04-17T04:44:54.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.300+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=VXKT29LYMJ1A type=write size=326392 2024-04-17T04:44:54.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.300+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=KRWNE1NZF7CY type=del size=1100741 2024-04-17T04:44:54.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~0U18UMAW9B5D.0 size=758770 2024-04-17T04:44:54.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.308+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=0U18UMAW9B5D tag=7XP25LGX70VG type=write size=1807346 2024-04-17T04:44:54.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.312+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=QKKG8UB05O81 type=write size=743075 2024-04-17T04:44:54.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.316+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=U3TCQ4XCI36I type=del 2024-04-17T04:44:54.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.316+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=LNBX7FEX84NK type=del size=738630 2024-04-17T04:44:54.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.320+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=YVI8UD13GGQ4 type=del 2024-04-17T04:44:54.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.324+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=ZBY261274O1Y type=del size=3105781 2024-04-17T04:44:54.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.324+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=2ZPMUOMBATNY type=del 2024-04-17T04:44:54.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.328+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=TUM8HOXIQT6I type=del size=451733 2024-04-17T04:44:54.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.332+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~0SWE3YN4QP34.0 size=1048576 2024-04-17T04:44:54.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.332+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~0SWE3YN4QP34.1 size=564872 2024-04-17T04:44:54.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.336+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=0SWE3YN4QP34 tag=SKWPCR2T9OWV type=write size=2662024 2024-04-17T04:44:54.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~JFQF99I8WVPW.0 size=1048576 2024-04-17T04:44:54.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~JFQF99I8WVPW.1 size=414652 2024-04-17T04:44:54.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.344+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=JFQF99I8WVPW tag=86IOY1A99UAI type=write size=2511804 2024-04-17T04:44:54.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.348+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=QYAAC4FJ5IP8 type=del 2024-04-17T04:44:54.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.348+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=MW7IW2M52JAV type=write size=1039205 2024-04-17T04:44:54.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.352+0000 7fd8ef2218c0 1 < canceled part key=_multipart_R3C8PAVPVGBY.2~6GDCXFF1QZJ7.0 size=1048576 2024-04-17T04:44:54.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~6GDCXFF1QZJ7.1 size=606732 2024-04-17T04:44:54.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.356+0000 7fd8ef2218c0 1 < canceled multipart upload key=R3C8PAVPVGBY upload=6GDCXFF1QZJ7 tag=U37JWVEKMM2K type=write size=2703884 2024-04-17T04:44:54.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~SOW430RMUZND.0 size=1027417 2024-04-17T04:44:54.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.360+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=SOW430RMUZND tag=DCG8WM6H2WR1 type=write size=2075993 2024-04-17T04:44:54.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.364+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=DL4OL12WLDVE type=del 2024-04-17T04:44:54.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.368+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=IH074WO4STU1 type=del size=525459 2024-04-17T04:44:54.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.368+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~PZDLGWTODDPJ.0 size=846628 2024-04-17T04:44:54.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.376+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=PZDLGWTODDPJ tag=M19G8NIFDS3H type=write size=1895204 2024-04-17T04:44:54.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.376+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=XO08WHF3K3PG type=del size=2733541 2024-04-17T04:44:54.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~V3QIQ59AA60G.0 size=1048576 2024-04-17T04:44:54.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~V3QIQ59AA60G.1 size=1048576 2024-04-17T04:44:54.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.380+0000 7fd8ef2218c0 1 < canceled part key=_multipart_VUYM3JC57QRP.2~V3QIQ59AA60G.2 size=97958 2024-04-17T04:44:54.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.388+0000 7fd8ef2218c0 1 < canceled multipart upload key=VUYM3JC57QRP upload=V3QIQ59AA60G tag=M585ER53N5FZ type=write size=3243686 2024-04-17T04:44:54.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~52552VJ32KYL.0 size=1048576 2024-04-17T04:44:54.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~52552VJ32KYL.1 size=1048576 2024-04-17T04:44:54.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~52552VJ32KYL.2 size=455300 2024-04-17T04:44:54.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.396+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=52552VJ32KYL tag=A9V0NPA7RNIB type=write size=3601028 2024-04-17T04:44:54.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.400+0000 7fd8ef2218c0 1 < completed write operation key=1JOWGWFR7R6P tag=VYV2Y5MN14E5 type=write size=538022 2024-04-17T04:44:54.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.404+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=DUZNZS0P4E5A type=del size=1351937 2024-04-17T04:44:54.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.404+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=X57U427JRBFG type=del 2024-04-17T04:44:54.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.412+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~WLERDZWZBP0L.0 size=632003 2024-04-17T04:44:54.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.412+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=WLERDZWZBP0L tag=JPENHW1G9AFJ type=write size=1680579 2024-04-17T04:44:54.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.416+0000 7fd8ef2218c0 1 < canceled operation key=5FR1NEWE0LD1 tag=H30EPTE53L7N type=del size=842588 2024-04-17T04:44:54.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.420+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=FDXMXGWR0TJM type=del size=2717368 2024-04-17T04:44:54.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.424+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=QLIUDCX6MGUR type=del 2024-04-17T04:44:54.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.428+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=JCX3UNUUBVIY type=write size=200800 2024-04-17T04:44:54.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.428+0000 7fd8ef2218c0 1 < canceled part key=_multipart_WCIJICR0FT2C.2~Q8AVZG1IPVOA.0 size=1048576 2024-04-17T04:44:54.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.428+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~Q8AVZG1IPVOA.1 size=317170 2024-04-17T04:44:54.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.432+0000 7fd8ef2218c0 1 < canceled multipart upload key=WCIJICR0FT2C upload=Q8AVZG1IPVOA tag=928LDPFMT2B3 type=write size=2414322 2024-04-17T04:44:54.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~1WFYMF5AA1I1.0 size=1048576 2024-04-17T04:44:54.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~1WFYMF5AA1I1.1 size=1048576 2024-04-17T04:44:54.438 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~1WFYMF5AA1I1.2 size=618008 2024-04-17T04:44:54.438 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.444+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=1WFYMF5AA1I1 tag=J1KWHFBFRG46 type=write size=3763736 2024-04-17T04:44:54.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.448+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=KRWNE1NZF7CY type=del 2024-04-17T04:44:54.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.452+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=FRLLU1I2FWOH type=del size=2319528 2024-04-17T04:44:54.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.452+0000 7fd8ef2218c0 1 < completed write operation key=5EFSN2FPUQB6 tag=QKKG8UB05O81 type=write size=743075 2024-04-17T04:44:54.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.456+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=D9J6VUL3H44X type=del size=2342324 2024-04-17T04:44:54.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.460+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=LNBX7FEX84NK type=del 2024-04-17T04:44:54.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.464+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=R14A8EB9VSBF type=write size=736119 2024-04-17T04:44:54.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.468+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=ZBY261274O1Y type=del 2024-04-17T04:44:54.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.472+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=KXF3S3K61C9W type=del size=1879518 2024-04-17T04:44:54.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.472+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=TUM8HOXIQT6I type=del 2024-04-17T04:44:54.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.476+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=6AX2LU9ZEDF6 type=del size=1628315 2024-04-17T04:44:54.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.476+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~JFQF99I8WVPW.0 size=1048576 2024-04-17T04:44:54.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.480+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~JFQF99I8WVPW.1 size=414652 2024-04-17T04:44:54.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.480+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=JFQF99I8WVPW tag=86IOY1A99UAI type=write size=2511804 2024-04-17T04:44:54.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.484+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=GD11QJ70D3OI type=del size=4024801 2024-04-17T04:44:54.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.492+0000 7fd8ef2218c0 1 < suggested remove operation key=6WSZ5UL0V1Y1 tag=MW7IW2M52JAV type=write 2024-04-17T04:44:54.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.492+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=KL1X1RCPESDR type=del size=1796116 2024-04-17T04:44:54.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.496+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~SOW430RMUZND.0 size=1027417 2024-04-17T04:44:54.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.500+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=SOW430RMUZND tag=DCG8WM6H2WR1 type=write size=2075993 2024-04-17T04:44:54.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~IO4VPAT3WNVR.0 size=1048576 2024-04-17T04:44:54.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~IO4VPAT3WNVR.1 size=765295 2024-04-17T04:44:54.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.508+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=IO4VPAT3WNVR tag=I50EHGNZMR0A type=write size=2862447 2024-04-17T04:44:54.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.512+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=IH074WO4STU1 type=del 2024-04-17T04:44:54.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.516+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=ZE0K8UBR5EKQ type=del size=1737577 2024-04-17T04:44:54.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.520+0000 7fd8ef2218c0 -1 < failed to complete operation key=6WSZ5UL0V1Y1 tag=XO08WHF3K3PG type=del size=2733541: (22) Invalid argument 2024-04-17T04:44:54.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.524+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~XPGVRR3QV7G2.0 size=1048576 2024-04-17T04:44:54.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.528+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~XPGVRR3QV7G2.1 size=168847 2024-04-17T04:44:54.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.528+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=XPGVRR3QV7G2 tag=SEH2IGX4QBPT type=write size=2265999 2024-04-17T04:44:54.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~52552VJ32KYL.0 size=1048576 2024-04-17T04:44:54.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.532+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~52552VJ32KYL.1 size=1048576 2024-04-17T04:44:54.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.536+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~52552VJ32KYL.2 size=455300 2024-04-17T04:44:54.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.540+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=52552VJ32KYL tag=A9V0NPA7RNIB type=write size=3601028 2024-04-17T04:44:54.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.544+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=47D6ZHNJWLEJ type=del size=746745 2024-04-17T04:44:54.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.548+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=DUZNZS0P4E5A type=del 2024-04-17T04:44:54.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.552+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~WT9TZYYZY9I4.0 size=1048576 2024-04-17T04:44:54.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.556+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~WT9TZYYZY9I4.1 size=875410 2024-04-17T04:44:54.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.556+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=WT9TZYYZY9I4 tag=B3OLP49PI2AK type=write size=2972562 2024-04-17T04:44:54.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.556+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~WLERDZWZBP0L.0 size=632003 2024-04-17T04:44:54.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.560+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=WLERDZWZBP0L tag=JPENHW1G9AFJ type=write size=1680579 2024-04-17T04:44:54.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.564+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=WQ1DEK19R6MM type=del size=1683304 2024-04-17T04:44:54.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.568+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=FDXMXGWR0TJM type=del 2024-04-17T04:44:54.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ER22HGQ3KKQW.0 size=1048576 2024-04-17T04:44:54.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ER22HGQ3KKQW.1 size=1048576 2024-04-17T04:44:54.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ER22HGQ3KKQW.2 size=645303 2024-04-17T04:44:54.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.580+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=ER22HGQ3KKQW tag=2WGIQQ618S5X type=write size=3791031 2024-04-17T04:44:54.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.584+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=JCX3UNUUBVIY type=write size=200800 2024-04-17T04:44:54.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.584+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=HH1YOJ2KP0F3 type=write size=490634 2024-04-17T04:44:54.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.588+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~1WFYMF5AA1I1.0 size=1048576 2024-04-17T04:44:54.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.592+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~1WFYMF5AA1I1.1 size=1048576 2024-04-17T04:44:54.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.592+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~1WFYMF5AA1I1.2 size=618008 2024-04-17T04:44:54.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.596+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=1WFYMF5AA1I1 tag=J1KWHFBFRG46 type=write size=3763736 2024-04-17T04:44:54.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.600+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=02TMU8JQNZIL type=del size=292712 2024-04-17T04:44:54.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.604+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=FRLLU1I2FWOH type=del 2024-04-17T04:44:54.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.608+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~FRR5AUYET1AX.0 size=1048576 2024-04-17T04:44:54.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~FRR5AUYET1AX.1 size=1048576 2024-04-17T04:44:54.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~FRR5AUYET1AX.2 size=469570 2024-04-17T04:44:54.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.612+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=FRR5AUYET1AX tag=WO8TITC8L8EK type=write size=3615298 2024-04-17T04:44:54.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.616+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=D9J6VUL3H44X type=del 2024-04-17T04:44:54.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.620+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=OOTIZV3R9P4X type=del size=916998 2024-04-17T04:44:54.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.624+0000 7fd8ef2218c0 1 < completed write operation key=2IY8DB37FI9H tag=R14A8EB9VSBF type=write size=736119 2024-04-17T04:44:54.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GXQBG6GI47I1.0 size=1048576 2024-04-17T04:44:54.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GXQBG6GI47I1.1 size=1011965 2024-04-17T04:44:54.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.628+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=GXQBG6GI47I1 tag=IPKGFQ0FVEI0 type=write size=3109117 2024-04-17T04:44:54.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.636+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=KXF3S3K61C9W type=del 2024-04-17T04:44:54.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.640+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=WJC7D0VXPKYG type=del size=3394413 2024-04-17T04:44:54.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.644+0000 7fd8ef2218c0 1 < suggested update operation key=WCIJICR0FT2C tag=6AX2LU9ZEDF6 type=del size=1628315 2024-04-17T04:44:54.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.648+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~LYNKVVNUGTGA.0 size=789754 2024-04-17T04:44:54.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.648+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=LYNKVVNUGTGA tag=2YLF6PPUBMII type=write size=1838330 2024-04-17T04:44:54.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.652+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=GD11QJ70D3OI type=del 2024-04-17T04:44:54.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~72442UPYNMYN.0 size=978879 2024-04-17T04:44:54.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.656+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=72442UPYNMYN tag=AWTM5VVQA3BS type=write size=2027455 2024-04-17T04:44:54.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.660+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=KL1X1RCPESDR type=del 2024-04-17T04:44:54.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.664+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=5X4M20D677DC type=del size=2658073 2024-04-17T04:44:54.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~IO4VPAT3WNVR.0 size=1048576 2024-04-17T04:44:54.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~IO4VPAT3WNVR.1 size=765295 2024-04-17T04:44:54.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.672+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=IO4VPAT3WNVR tag=I50EHGNZMR0A type=write size=2862447 2024-04-17T04:44:54.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.676+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=DV8QJ07NE333 type=del size=3948974 2024-04-17T04:44:54.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.680+0000 7fd8ef2218c0 1 < canceled operation key=1ZTJP8DRITEK tag=ZE0K8UBR5EKQ type=del size=1737577 2024-04-17T04:44:54.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.684+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=0CHVGXVKWGI8 type=del size=1064973 2024-04-17T04:44:54.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~XPGVRR3QV7G2.0 size=1048576 2024-04-17T04:44:54.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~XPGVRR3QV7G2.1 size=168847 2024-04-17T04:44:54.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.696+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=XPGVRR3QV7G2 tag=SEH2IGX4QBPT type=write size=2265999 2024-04-17T04:44:54.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~8VFVYPI1AXSD.0 size=1048576 2024-04-17T04:44:54.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~8VFVYPI1AXSD.1 size=949168 2024-04-17T04:44:54.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.704+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=8VFVYPI1AXSD tag=A6DAM73LIUKX type=write size=3046320 2024-04-17T04:44:54.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.708+0000 7fd8ef2218c0 1 < canceled operation key=2S1IT3TO1BPU tag=47D6ZHNJWLEJ type=del size=746745 2024-04-17T04:44:54.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~JS4DTVZ8GCRH.0 size=1048576 2024-04-17T04:44:54.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~JS4DTVZ8GCRH.1 size=1048576 2024-04-17T04:44:54.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~JS4DTVZ8GCRH.2 size=338125 2024-04-17T04:44:54.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.720+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=JS4DTVZ8GCRH tag=S9WN2BAARNLB type=write size=3483853 2024-04-17T04:44:54.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.724+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~WT9TZYYZY9I4.0 size=1048576 2024-04-17T04:44:54.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.724+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~WT9TZYYZY9I4.1 size=875410 2024-04-17T04:44:54.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.728+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=WT9TZYYZY9I4 tag=B3OLP49PI2AK type=write size=2972562 2024-04-17T04:44:54.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~MZ045FQI3D3Q.0 size=949871 2024-04-17T04:44:54.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.736+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=MZ045FQI3D3Q tag=GTMMY3YW0XHQ type=write size=1998447 2024-04-17T04:44:54.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.740+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=WQ1DEK19R6MM type=del 2024-04-17T04:44:54.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.744+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=6M1SXKTPSE50 type=del size=2044517 2024-04-17T04:44:54.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~ER22HGQ3KKQW.0 size=1048576 2024-04-17T04:44:54.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~ER22HGQ3KKQW.1 size=1048576 2024-04-17T04:44:54.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.756+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~ER22HGQ3KKQW.2 size=645303 2024-04-17T04:44:54.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.764+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=ER22HGQ3KKQW tag=2WGIQQ618S5X type=write size=3791031 2024-04-17T04:44:54.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.768+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=GJP0XES96SJD type=del size=1158186 2024-04-17T04:44:54.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.772+0000 7fd8ef2218c0 1 < completed write operation key=AE4CYL334ONO tag=HH1YOJ2KP0F3 type=write size=490634 2024-04-17T04:44:54.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.772+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=60IM0F8XIRBP type=del size=3188062 2024-04-17T04:44:54.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.776+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=02TMU8JQNZIL type=del 2024-04-17T04:44:54.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~0G6WJKIUQ6QL.0 size=1048576 2024-04-17T04:44:54.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~0G6WJKIUQ6QL.1 size=1019509 2024-04-17T04:44:54.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.784+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=0G6WJKIUQ6QL tag=R0I87NXL36XZ type=write size=3116661 2024-04-17T04:44:54.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~FRR5AUYET1AX.0 size=1048576 2024-04-17T04:44:54.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.792+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~FRR5AUYET1AX.1 size=1048576 2024-04-17T04:44:54.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.792+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~FRR5AUYET1AX.2 size=469570 2024-04-17T04:44:54.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.800+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=FRR5AUYET1AX tag=WO8TITC8L8EK type=write size=3615298 2024-04-17T04:44:54.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.800+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=12RIQ81C06CE type=del size=3474317 2024-04-17T04:44:54.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.804+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=OOTIZV3R9P4X type=del 2024-04-17T04:44:54.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.808+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=BN2LW052A0AX type=del size=741670 2024-04-17T04:44:54.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GXQBG6GI47I1.0 size=1048576 2024-04-17T04:44:54.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GXQBG6GI47I1.1 size=1011965 2024-04-17T04:44:54.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.820+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=GXQBG6GI47I1 tag=IPKGFQ0FVEI0 type=write size=3109117 2024-04-17T04:44:54.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.824+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=7D63FX5A0Y9J type=del size=1027821 2024-04-17T04:44:54.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.828+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=WJC7D0VXPKYG type=del 2024-04-17T04:44:54.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.828+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=I4S7BFLPF579 type=del size=4035043 2024-04-17T04:44:54.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.832+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~LYNKVVNUGTGA.0 size=789754 2024-04-17T04:44:54.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.836+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=LYNKVVNUGTGA tag=2YLF6PPUBMII type=write size=1838330 2024-04-17T04:44:54.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.840+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=P4XZZ9T0TGSA type=del size=2176049 2024-04-17T04:44:54.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.844+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~72442UPYNMYN.0 size=978879 2024-04-17T04:44:54.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.848+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=72442UPYNMYN tag=AWTM5VVQA3BS type=write size=2027455 2024-04-17T04:44:54.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.852+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~ONH580DQQZ2N.0 size=1048576 2024-04-17T04:44:54.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.856+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~ONH580DQQZ2N.1 size=921422 2024-04-17T04:44:54.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.856+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=ONH580DQQZ2N tag=T200JJZZR8M0 type=write size=3018574 2024-04-17T04:44:54.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.864+0000 7fd8ef2218c0 1 < suggested update operation key=82D6YWKJ1TFQ tag=5X4M20D677DC type=del size=2658073 2024-04-17T04:44:54.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.868+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=S9Q8I8K7LZVU type=del size=2534892 2024-04-17T04:44:54.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.872+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=DV8QJ07NE333 type=del 2024-04-17T04:44:54.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.872+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=D89EG1OBFOOO type=del size=3413161 2024-04-17T04:44:54.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.880+0000 7fd8ef2218c0 1 < suggested update operation key=JVWRJBMFIRLL tag=0CHVGXVKWGI8 type=del size=1064973 2024-04-17T04:44:54.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.884+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=TS0K5U1GSJD8 type=del size=3865825 2024-04-17T04:44:54.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~8VFVYPI1AXSD.0 size=1048576 2024-04-17T04:44:54.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~8VFVYPI1AXSD.1 size=949168 2024-04-17T04:44:54.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.896+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=8VFVYPI1AXSD tag=A6DAM73LIUKX type=write size=3046320 2024-04-17T04:44:54.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.900+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=T3H0N813Z3YY type=del size=1343256 2024-04-17T04:44:54.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.900+0000 7fd8ef2218c0 1 < canceled part key=_multipart_N11Y8FPN9XES.2~JS4DTVZ8GCRH.0 size=1048576 2024-04-17T04:44:54.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~JS4DTVZ8GCRH.1 size=1048576 2024-04-17T04:44:54.900 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~JS4DTVZ8GCRH.2 size=338125 2024-04-17T04:44:54.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.908+0000 7fd8ef2218c0 1 < canceled multipart upload key=N11Y8FPN9XES upload=JS4DTVZ8GCRH tag=S9WN2BAARNLB type=write size=3483853 2024-04-17T04:44:54.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.912+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=K4WAZKEWCMCP type=del size=1990183 2024-04-17T04:44:54.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.912+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~MZ045FQI3D3Q.0 size=949871 2024-04-17T04:44:54.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.920+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=MZ045FQI3D3Q tag=GTMMY3YW0XHQ type=write size=1998447 2024-04-17T04:44:54.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.920+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=VFQ77UON2FAI type=del size=2122529 2024-04-17T04:44:54.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.924+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=6M1SXKTPSE50 type=del 2024-04-17T04:44:54.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~F2CF8SY138F6.0 size=1048576 2024-04-17T04:44:54.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~F2CF8SY138F6.1 size=1048576 2024-04-17T04:44:54.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~F2CF8SY138F6.2 size=960102 2024-04-17T04:44:54.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.932+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=F2CF8SY138F6 tag=ZKLGJOKXT2JY type=write size=4105830 2024-04-17T04:44:54.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.936+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=GJP0XES96SJD type=del 2024-04-17T04:44:54.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.936+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=XAFHAPB5GMJZ type=del size=3383105 2024-04-17T04:44:54.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.940+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=60IM0F8XIRBP type=del 2024-04-17T04:44:54.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.944+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=T6O7TR4GPGHR type=del size=3081633 2024-04-17T04:44:54.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.944+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~0G6WJKIUQ6QL.0 size=1048576 2024-04-17T04:44:54.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~0G6WJKIUQ6QL.1 size=1019509 2024-04-17T04:44:54.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.952+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=0G6WJKIUQ6QL tag=R0I87NXL36XZ type=write size=3116661 2024-04-17T04:44:54.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.956+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=6WDLXFSWEEDX type=del size=226131 2024-04-17T04:44:54.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.956+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=12RIQ81C06CE type=del 2024-04-17T04:44:54.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~LTAN422MQU5L.0 size=1048576 2024-04-17T04:44:54.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.964+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~LTAN422MQU5L.1 size=1048576 2024-04-17T04:44:54.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.964+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~LTAN422MQU5L.2 size=907980 2024-04-17T04:44:54.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.964+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=LTAN422MQU5L tag=FKNNICMTDS8E type=write size=4053708 2024-04-17T04:44:54.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.968+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=BN2LW052A0AX type=del 2024-04-17T04:44:54.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.968+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=18WAK84PKGI6 type=write size=606628 2024-04-17T04:44:54.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.972+0000 7fd8ef2218c0 1 < canceled operation key=7BV3V2XMLSPH tag=7D63FX5A0Y9J type=del size=1027821 2024-04-17T04:44:54.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.976+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=NAJMK7YJIOC6 type=del size=1011425 2024-04-17T04:44:54.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.976+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=I4S7BFLPF579 type=del 2024-04-17T04:44:54.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.980+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=V768OTH0L36M type=del size=3994678 2024-04-17T04:44:54.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.980+0000 7fd8ef2218c0 -1 < failed to complete operation key=JVWRJBMFIRLL tag=P4XZZ9T0TGSA type=del size=2176049: (22) Invalid argument 2024-04-17T04:44:54.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~7QUEQFZN7VDS.0 size=1048576 2024-04-17T04:44:54.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~7QUEQFZN7VDS.1 size=397260 2024-04-17T04:44:54.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.988+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=7QUEQFZN7VDS tag=K0K452CFC6IG type=write size=2494412 2024-04-17T04:44:54.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.988+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~ONH580DQQZ2N.0 size=1048576 2024-04-17T04:44:54.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~ONH580DQQZ2N.1 size=921422 2024-04-17T04:44:54.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.996+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=ONH580DQQZ2N tag=T200JJZZR8M0 type=write size=3018574 2024-04-17T04:44:54.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:54.996+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=RD0A4MG9U962 type=del size=24188 2024-04-17T04:44:54.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.000+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=S9Q8I8K7LZVU type=del 2024-04-17T04:44:54.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~BP4MT5PD9ZSV.0 size=1048576 2024-04-17T04:44:55.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~BP4MT5PD9ZSV.1 size=1048576 2024-04-17T04:44:55.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~BP4MT5PD9ZSV.2 size=259335 2024-04-17T04:44:55.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=BP4MT5PD9ZSV tag=AJ7P2L1H4ILT type=write size=3405063 2024-04-17T04:44:55.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.008+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=D89EG1OBFOOO type=del 2024-04-17T04:44:55.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.012+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=CYPG2ZT344YJ type=write size=809090 2024-04-17T04:44:55.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.012+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=TS0K5U1GSJD8 type=del 2024-04-17T04:44:55.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.016+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=8UT0MUUMCLTH type=del size=203088 2024-04-17T04:44:55.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.020+0000 7fd8ef2218c0 1 < suggested remove operation key=B63EZS0LAC30 tag=T3H0N813Z3YY type=del 2024-04-17T04:44:55.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.024+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=K3BTHJ03V4V1 type=del size=3345955 2024-04-17T04:44:55.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.024+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=K4WAZKEWCMCP type=del 2024-04-17T04:44:55.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.028+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=23UQ3VRJ08C0 type=write size=910906 2024-04-17T04:44:55.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.028+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=VFQ77UON2FAI type=del 2024-04-17T04:44:55.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.032+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=JUPWW82QSWH8 type=del size=2176625 2024-04-17T04:44:55.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~F2CF8SY138F6.0 size=1048576 2024-04-17T04:44:55.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~F2CF8SY138F6.1 size=1048576 2024-04-17T04:44:55.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~F2CF8SY138F6.2 size=960102 2024-04-17T04:44:55.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.040+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=F2CF8SY138F6 tag=ZKLGJOKXT2JY type=write size=4105830 2024-04-17T04:44:55.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.044+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=0UDTJG3F1C3M type=del size=348907 2024-04-17T04:44:55.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.044+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=XAFHAPB5GMJZ type=del 2024-04-17T04:44:55.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.048+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=DIJVM72IZZVG type=del size=3581821 2024-04-17T04:44:55.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.052+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=T6O7TR4GPGHR type=del 2024-04-17T04:44:55.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.052+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~W6W5DJEX68F8.0 size=1048576 2024-04-17T04:44:55.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.056+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~W6W5DJEX68F8.1 size=774138 2024-04-17T04:44:55.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.056+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=W6W5DJEX68F8 tag=1KF8P0VP2RLH type=write size=2871290 2024-04-17T04:44:55.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.056+0000 7fd8ef2218c0 1 < canceled operation key=2IY8DB37FI9H tag=6WDLXFSWEEDX type=del size=226131 2024-04-17T04:44:55.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.060+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=7KKW3Q097W65 type=del size=1729095 2024-04-17T04:44:55.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~LTAN422MQU5L.0 size=1048576 2024-04-17T04:44:55.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~LTAN422MQU5L.1 size=1048576 2024-04-17T04:44:55.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~LTAN422MQU5L.2 size=907980 2024-04-17T04:44:55.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.068+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=LTAN422MQU5L tag=FKNNICMTDS8E type=write size=4053708 2024-04-17T04:44:55.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.072+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=AOG0IMX9X3PB type=del size=2959449 2024-04-17T04:44:55.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.076+0000 7fd8ef2218c0 1 < completed write operation key=5ZULD0PHLS4V tag=18WAK84PKGI6 type=write size=606628 2024-04-17T04:44:55.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.080+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~SESPFKUFZ13D.0 size=1048576 2024-04-17T04:44:55.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.080+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~SESPFKUFZ13D.1 size=252078 2024-04-17T04:44:55.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.080+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=SESPFKUFZ13D tag=V60GXYRMJS7T type=write size=2349230 2024-04-17T04:44:55.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.084+0000 7fd8ef2218c0 1 < suggested remove operation key=MS3T476JOEON tag=NAJMK7YJIOC6 type=del 2024-04-17T04:44:55.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.088+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=JL3ONWNZG0K0 type=write size=136737 2024-04-17T04:44:55.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.088+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=V768OTH0L36M type=del 2024-04-17T04:44:55.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~VFQLNX88N54E.0 size=1048576 2024-04-17T04:44:55.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.096+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~VFQLNX88N54E.1 size=605936 2024-04-17T04:44:55.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.096+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=VFQLNX88N54E tag=9YZLEZMKHPS9 type=write size=2703088 2024-04-17T04:44:55.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.096+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~7QUEQFZN7VDS.0 size=1048576 2024-04-17T04:44:55.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.100+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~7QUEQFZN7VDS.1 size=397260 2024-04-17T04:44:55.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.104+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=7QUEQFZN7VDS tag=K0K452CFC6IG type=write size=2494412 2024-04-17T04:44:55.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.108+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~A6L4WAN8MSIR.0 size=926574 2024-04-17T04:44:55.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.108+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=A6L4WAN8MSIR tag=2SCHVSIAHC8V type=write size=1975150 2024-04-17T04:44:55.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.112+0000 7fd8ef2218c0 1 < suggested update operation key=UEWLJ3E9ODUL tag=RD0A4MG9U962 type=del size=24188 2024-04-17T04:44:55.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~FO8FQ8FZTA5I.0 size=1048576 2024-04-17T04:44:55.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~FO8FQ8FZTA5I.1 size=953201 2024-04-17T04:44:55.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=FO8FQ8FZTA5I tag=UAGDAI4IIPF8 type=write size=3050353 2024-04-17T04:44:55.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~BP4MT5PD9ZSV.0 size=1048576 2024-04-17T04:44:55.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~BP4MT5PD9ZSV.1 size=1048576 2024-04-17T04:44:55.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.124+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~BP4MT5PD9ZSV.2 size=259335 2024-04-17T04:44:55.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.124+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=BP4MT5PD9ZSV tag=AJ7P2L1H4ILT type=write size=3405063 2024-04-17T04:44:55.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.128+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=NI994F3PTA2G type=del size=765462 2024-04-17T04:44:55.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.132+0000 7fd8ef2218c0 1 < suggested remove operation key=3OFQH4KVYN18 tag=CYPG2ZT344YJ type=write 2024-04-17T04:44:55.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.136+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=53CX0D54APL1 type=del size=943225 2024-04-17T04:44:55.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.140+0000 7fd8ef2218c0 1 < suggested remove operation key=JVWRJBMFIRLL tag=8UT0MUUMCLTH type=del 2024-04-17T04:44:55.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~3LMWF4ZO04BY.0 size=65548 2024-04-17T04:44:55.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.144+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=3LMWF4ZO04BY tag=1K1K84EFB43P type=write size=1114124 2024-04-17T04:44:55.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.148+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=K3BTHJ03V4V1 type=del 2024-04-17T04:44:55.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.148+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=RB6W2UCW6Q21 type=del size=2332488 2024-04-17T04:44:55.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.152+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=23UQ3VRJ08C0 type=write 2024-04-17T04:44:55.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.156+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~3CES30Q4EVLH.0 size=1048576 2024-04-17T04:44:55.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.160+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~3CES30Q4EVLH.1 size=1048576 2024-04-17T04:44:55.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.160+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~3CES30Q4EVLH.2 size=754937 2024-04-17T04:44:55.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.160+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=3CES30Q4EVLH tag=MT8RXPXJFETL type=write size=3900665 2024-04-17T04:44:55.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.164+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=JUPWW82QSWH8 type=del 2024-04-17T04:44:55.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.168+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~8IXVG1RHWKFD.0 size=1048576 2024-04-17T04:44:55.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.172+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~8IXVG1RHWKFD.1 size=646415 2024-04-17T04:44:55.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.172+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=8IXVG1RHWKFD tag=6IKN89PP0EY5 type=write size=2743567 2024-04-17T04:44:55.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.176+0000 7fd8ef2218c0 1 < suggested remove operation key=5ZULD0PHLS4V tag=0UDTJG3F1C3M type=del 2024-04-17T04:44:55.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.180+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=SYY5W3KYO5YV type=del size=2927089 2024-04-17T04:44:55.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.180+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=MS3T476JOEON tag=DIJVM72IZZVG type=del size=3581821: (2) No such file or directory 2024-04-17T04:44:55.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.184+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=146GF8IGNIOX type=del size=632573 2024-04-17T04:44:55.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.184+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~W6W5DJEX68F8.0 size=1048576 2024-04-17T04:44:55.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.188+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~W6W5DJEX68F8.1 size=774138 2024-04-17T04:44:55.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.192+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=W6W5DJEX68F8 tag=1KF8P0VP2RLH type=write size=2871290 2024-04-17T04:44:55.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.196+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=BW6JL1AWO06Z type=del size=278148 2024-04-17T04:44:55.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.196+0000 7fd8ef2218c0 -1 < failed to complete operation key=M3U8L7M8JKG9 tag=7KKW3Q097W65 type=del size=1729095: (22) Invalid argument 2024-04-17T04:44:55.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.200+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=P4L7EI8I6XCL type=del size=186096 2024-04-17T04:44:55.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.204+0000 7fd8ef2218c0 1 < suggested remove operation key=GX9X0D45GC2I tag=AOG0IMX9X3PB type=del 2024-04-17T04:44:55.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.208+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=FUXW0VMERHVG type=del size=2848795 2024-04-17T04:44:55.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.208+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~SESPFKUFZ13D.0 size=1048576 2024-04-17T04:44:55.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.212+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~SESPFKUFZ13D.1 size=252078 2024-04-17T04:44:55.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.216+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=SESPFKUFZ13D tag=V60GXYRMJS7T type=write size=2349230 2024-04-17T04:44:55.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~I0PU5XYK7U6A.0 size=469337 2024-04-17T04:44:55.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.220+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=I0PU5XYK7U6A tag=QR6MPMAWNHW4 type=write size=1517913 2024-04-17T04:44:55.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.220+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=JL3ONWNZG0K0 type=write size=136737 2024-04-17T04:44:55.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.224+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=YH0SVI7M1XPU type=del size=4151400 2024-04-17T04:44:55.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.224+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~VFQLNX88N54E.0 size=1048576 2024-04-17T04:44:55.223 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~VFQLNX88N54E.1 size=605936 2024-04-17T04:44:55.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.232+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=VFQLNX88N54E tag=9YZLEZMKHPS9 type=write size=2703088 2024-04-17T04:44:55.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.236+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=ZWW1AHTEJ2AK type=write size=747554 2024-04-17T04:44:55.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.240+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~A6L4WAN8MSIR.0 size=926574 2024-04-17T04:44:55.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.244+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=A6L4WAN8MSIR tag=2SCHVSIAHC8V type=write size=1975150 2024-04-17T04:44:55.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.244+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=RVZVVKR9UP94 type=del size=3963991 2024-04-17T04:44:55.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.248+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~FO8FQ8FZTA5I.0 size=1048576 2024-04-17T04:44:55.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.248+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~FO8FQ8FZTA5I.1 size=953201 2024-04-17T04:44:55.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.252+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=FO8FQ8FZTA5I tag=UAGDAI4IIPF8 type=write size=3050353 2024-04-17T04:44:55.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.256+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~TU23JGTUHMRM.0 size=241517 2024-04-17T04:44:55.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.256+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=TU23JGTUHMRM tag=1P37YHYT267E type=write size=1290093 2024-04-17T04:44:55.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.256+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=NI994F3PTA2G type=del 2024-04-17T04:44:55.256 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.260+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=C039VJF0DHLM type=del size=3398721 2024-04-17T04:44:55.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.264+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=53CX0D54APL1 type=del 2024-04-17T04:44:55.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.264+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=XS7R3JKCXDO2 type=del size=3744751 2024-04-17T04:44:55.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~3LMWF4ZO04BY.0 size=65548 2024-04-17T04:44:55.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.276+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=3LMWF4ZO04BY tag=1K1K84EFB43P type=write size=1114124 2024-04-17T04:44:55.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~10R84ZUI3OO2.0 size=293812 2024-04-17T04:44:55.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.280+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=10R84ZUI3OO2 tag=O8HM63RLCUW9 type=write size=1342388 2024-04-17T04:44:55.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.284+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=RB6W2UCW6Q21 type=del 2024-04-17T04:44:55.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.288+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=MOPTJ3S0KQLU type=del size=403093 2024-04-17T04:44:55.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.288+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~3CES30Q4EVLH.0 size=1048576 2024-04-17T04:44:55.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.288+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~3CES30Q4EVLH.1 size=1048576 2024-04-17T04:44:55.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.292+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~3CES30Q4EVLH.2 size=754937 2024-04-17T04:44:55.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.296+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=3CES30Q4EVLH tag=MT8RXPXJFETL type=write size=3900665 2024-04-17T04:44:55.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.300+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=Q73BIXA9T4A4 type=del size=469665 2024-04-17T04:44:55.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~8IXVG1RHWKFD.0 size=1048576 2024-04-17T04:44:55.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.308+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~8IXVG1RHWKFD.1 size=646415 2024-04-17T04:44:55.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.312+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=8IXVG1RHWKFD tag=6IKN89PP0EY5 type=write size=2743567 2024-04-17T04:44:55.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.316+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=L31D0318FJBZ type=del size=1159546 2024-04-17T04:44:55.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.316+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=SYY5W3KYO5YV type=del 2024-04-17T04:44:55.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.320+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=84TR1UQL3USZ type=write size=125805 2024-04-17T04:44:55.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.328+0000 7fd8ef2218c0 1 < suggested update operation key=M3U8L7M8JKG9 tag=146GF8IGNIOX type=del size=632573 2024-04-17T04:44:55.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.328+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=BTBLHV5C8USV type=del size=398045 2024-04-17T04:44:55.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.332+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=BW6JL1AWO06Z type=del 2024-04-17T04:44:55.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.336+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=KQ2ZZENYZV15 type=del size=3353685 2024-04-17T04:44:55.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.340+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=P4L7EI8I6XCL type=del 2024-04-17T04:44:55.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.340+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=JPFTL9O96QDC type=del size=2863147 2024-04-17T04:44:55.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.344+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=FUXW0VMERHVG type=del 2024-04-17T04:44:55.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.348+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=EYITYYDNFC0V type=del size=1020884 2024-04-17T04:44:55.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~I0PU5XYK7U6A.0 size=469337 2024-04-17T04:44:55.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.356+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=I0PU5XYK7U6A tag=QR6MPMAWNHW4 type=write size=1517913 2024-04-17T04:44:55.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.360+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=0C7NU8RD8A1N type=del size=1937792 2024-04-17T04:44:55.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.364+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=YH0SVI7M1XPU type=del 2024-04-17T04:44:55.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.368+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~1UHZOH1UEDYA.0 size=759817 2024-04-17T04:44:55.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.368+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=1UHZOH1UEDYA tag=DG4CFM00P9BC type=write size=1808393 2024-04-17T04:44:55.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.376+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=ZWW1AHTEJ2AK type=write 2024-04-17T04:44:55.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.380+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~CR6M6Q12Z13Y.0 size=1048576 2024-04-17T04:44:55.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~CR6M6Q12Z13Y.1 size=1048576 2024-04-17T04:44:55.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~CR6M6Q12Z13Y.2 size=483488 2024-04-17T04:44:55.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.384+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=CR6M6Q12Z13Y tag=P7GT8GOMREPM type=write size=3629216 2024-04-17T04:44:55.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.392+0000 7fd8ef2218c0 1 < suggested remove operation key=B63EZS0LAC30 tag=RVZVVKR9UP94 type=del 2024-04-17T04:44:55.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.400+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~N437JDPDRTP4.0 size=164452 2024-04-17T04:44:55.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.400+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=N437JDPDRTP4 tag=L1LZTPU682TZ type=write size=1213028 2024-04-17T04:44:55.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.400+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~TU23JGTUHMRM.0 size=241517 2024-04-17T04:44:55.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.408+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=TU23JGTUHMRM tag=1P37YHYT267E type=write size=1290093 2024-04-17T04:44:55.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.408+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=E79JJZ8GA4EJ type=write size=177746 2024-04-17T04:44:55.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.412+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=C039VJF0DHLM type=del 2024-04-17T04:44:55.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.416+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=39CXSDL70DQJ type=write size=578515 2024-04-17T04:44:55.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.420+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=XS7R3JKCXDO2 type=del 2024-04-17T04:44:55.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.424+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~0V2YGPF665HJ.0 size=318128 2024-04-17T04:44:55.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.424+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=0V2YGPF665HJ tag=QVBWJVRGOLLS type=write size=1366704 2024-04-17T04:44:55.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.428+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~10R84ZUI3OO2.0 size=293812 2024-04-17T04:44:55.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.436+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=10R84ZUI3OO2 tag=O8HM63RLCUW9 type=write size=1342388 2024-04-17T04:44:55.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~PKRT3OWDBYGW.0 size=1048576 2024-04-17T04:44:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~PKRT3OWDBYGW.1 size=470523 2024-04-17T04:44:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.444+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=PKRT3OWDBYGW tag=RW7Q4GSKCSQG type=write size=2567675 2024-04-17T04:44:55.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.448+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=MOPTJ3S0KQLU type=del 2024-04-17T04:44:55.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.452+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=N0UMQ35SPE1K type=del size=3293382 2024-04-17T04:44:55.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.456+0000 7fd8ef2218c0 1 < suggested update operation key=UEWLJ3E9ODUL tag=Q73BIXA9T4A4 type=del size=469665 2024-04-17T04:44:55.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.460+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=35XPK3VD9OGM type=del size=3799445 2024-04-17T04:44:55.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.464+0000 7fd8ef2218c0 1 < canceled operation key=5K50O96P7OGD tag=L31D0318FJBZ type=del size=1159546 2024-04-17T04:44:55.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.468+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~H4IZ9LEYC1KN.0 size=869413 2024-04-17T04:44:55.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.468+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=H4IZ9LEYC1KN tag=P3CWRF2QFLU9 type=write size=1917989 2024-04-17T04:44:55.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.472+0000 7fd8ef2218c0 1 < completed write operation key=AE4CYL334ONO tag=84TR1UQL3USZ type=write size=125805 2024-04-17T04:44:55.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.480+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~NKMG2AD4OCW7.0 size=152476 2024-04-17T04:44:55.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.480+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=NKMG2AD4OCW7 tag=03DZUVD24A4X type=write size=1201052 2024-04-17T04:44:55.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.484+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=BTBLHV5C8USV type=del 2024-04-17T04:44:55.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~10PC54M4WORO.0 size=1048576 2024-04-17T04:44:55.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.492+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~10PC54M4WORO.1 size=158389 2024-04-17T04:44:55.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.492+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=10PC54M4WORO tag=SHDKX6L4JDHI type=write size=2255541 2024-04-17T04:44:55.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.496+0000 7fd8ef2218c0 -1 < failed to complete operation key=2IY8DB37FI9H tag=KQ2ZZENYZV15 type=del size=3353685: (22) Invalid argument 2024-04-17T04:44:55.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~XV4QRSUUJN0E.0 size=361437 2024-04-17T04:44:55.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.500+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=XV4QRSUUJN0E tag=6YYRM5POSMEW type=write size=1410013 2024-04-17T04:44:55.500 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.504+0000 7fd8ef2218c0 1 < canceled operation key=R3C8PAVPVGBY tag=JPFTL9O96QDC type=del size=2863147 2024-04-17T04:44:55.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.508+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=Y6CO9EQYNMGN type=del size=1496032 2024-04-17T04:44:55.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.512+0000 7fd8ef2218c0 1 < canceled operation key=509DZ9DBJWC5 tag=EYITYYDNFC0V type=del size=1020884 2024-04-17T04:44:55.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.516+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~I2542I4FE9TS.0 size=1048576 2024-04-17T04:44:55.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.520+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~I2542I4FE9TS.1 size=799772 2024-04-17T04:44:55.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.520+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=I2542I4FE9TS tag=L2OI8HW7ETEY type=write size=2896924 2024-04-17T04:44:55.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.524+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=0C7NU8RD8A1N type=del 2024-04-17T04:44:55.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.528+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=ZHQ7L8HLBILT type=write size=35790 2024-04-17T04:44:55.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~1UHZOH1UEDYA.0 size=759817 2024-04-17T04:44:55.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.540+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=1UHZOH1UEDYA tag=DG4CFM00P9BC type=write size=1808393 2024-04-17T04:44:55.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.544+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=MNRVBNJ4FSH8 type=write size=147595 2024-04-17T04:44:55.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.548+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~CR6M6Q12Z13Y.0 size=1048576 2024-04-17T04:44:55.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.548+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~CR6M6Q12Z13Y.1 size=1048576 2024-04-17T04:44:55.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.548+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~CR6M6Q12Z13Y.2 size=483488 2024-04-17T04:44:55.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.556+0000 7fd8ef2218c0 1 < canceled multipart upload key=6WSZ5UL0V1Y1 upload=CR6M6Q12Z13Y tag=P7GT8GOMREPM type=write size=3629216 2024-04-17T04:44:55.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~DV1V2GYLMXQ8.0 size=1048576 2024-04-17T04:44:55.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~DV1V2GYLMXQ8.1 size=882984 2024-04-17T04:44:55.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.564+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=DV1V2GYLMXQ8 tag=WGMS3P6Y3IWD type=write size=2980136 2024-04-17T04:44:55.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.568+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~N437JDPDRTP4.0 size=164452 2024-04-17T04:44:55.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.576+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=N437JDPDRTP4 tag=L1LZTPU682TZ type=write size=1213028 2024-04-17T04:44:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.580+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=K73T72ATTVYN type=write size=149643 2024-04-17T04:44:55.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.580+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=E79JJZ8GA4EJ type=write size=177746 2024-04-17T04:44:55.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.588+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~AUMT45EXHZHU.0 size=1048576 2024-04-17T04:44:55.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~AUMT45EXHZHU.1 size=461255 2024-04-17T04:44:55.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.592+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=AUMT45EXHZHU tag=TZIZJ8E4TNUL type=write size=2558407 2024-04-17T04:44:55.589 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.592+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=39CXSDL70DQJ type=write size=578515 2024-04-17T04:44:55.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.596+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=2ZNI5J3OVAW7 type=del size=3274 2024-04-17T04:44:55.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.596+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~0V2YGPF665HJ.0 size=318128 2024-04-17T04:44:55.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.604+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=0V2YGPF665HJ tag=QVBWJVRGOLLS type=write size=1366704 2024-04-17T04:44:55.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.604+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=UOGN3ZU7C13N type=del size=445807 2024-04-17T04:44:55.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~PKRT3OWDBYGW.0 size=1048576 2024-04-17T04:44:55.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.608+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~PKRT3OWDBYGW.1 size=470523 2024-04-17T04:44:55.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.616+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=PKRT3OWDBYGW tag=RW7Q4GSKCSQG type=write size=2567675 2024-04-17T04:44:55.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.620+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~5949UEJMZZW9.0 size=256295 2024-04-17T04:44:55.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.620+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=5949UEJMZZW9 tag=A74MX2WD0LWI type=write size=1304871 2024-04-17T04:44:55.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.624+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=N0UMQ35SPE1K type=del 2024-04-17T04:44:55.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.628+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=668VDFO35E6W type=del size=2131159 2024-04-17T04:44:55.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.632+0000 7fd8ef2218c0 1 < canceled operation key=UYCKMXE6LBUH tag=35XPK3VD9OGM type=del size=3799445 2024-04-17T04:44:55.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.636+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=Q1SE5C789IGM type=write size=21378 2024-04-17T04:44:55.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.640+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~H4IZ9LEYC1KN.0 size=869413 2024-04-17T04:44:55.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.648+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=H4IZ9LEYC1KN tag=P3CWRF2QFLU9 type=write size=1917989 2024-04-17T04:44:55.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~J1E3DK7XAT1C.0 size=1048576 2024-04-17T04:44:55.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~J1E3DK7XAT1C.1 size=1048576 2024-04-17T04:44:55.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.660+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~J1E3DK7XAT1C.2 size=502596 2024-04-17T04:44:55.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.660+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=J1E3DK7XAT1C tag=BD9G67XFEHUL type=write size=3648324 2024-04-17T04:44:55.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.664+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~NKMG2AD4OCW7.0 size=152476 2024-04-17T04:44:55.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.672+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=NKMG2AD4OCW7 tag=03DZUVD24A4X type=write size=1201052 2024-04-17T04:44:55.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.676+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~6ZIJODNFIFQ3.0 size=1048576 2024-04-17T04:44:55.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~6ZIJODNFIFQ3.1 size=381386 2024-04-17T04:44:55.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=6ZIJODNFIFQ3 tag=KS6FQ2YC72IT type=write size=2478538 2024-04-17T04:44:55.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.684+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~10PC54M4WORO.0 size=1048576 2024-04-17T04:44:55.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~10PC54M4WORO.1 size=158389 2024-04-17T04:44:55.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.692+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=10PC54M4WORO tag=SHDKX6L4JDHI type=write size=2255541 2024-04-17T04:44:55.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~NE1WO109FSOE.0 size=1048576 2024-04-17T04:44:55.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~NE1WO109FSOE.1 size=1048576 2024-04-17T04:44:55.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.708+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~NE1WO109FSOE.2 size=307160 2024-04-17T04:44:55.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.708+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=NE1WO109FSOE tag=X28MEIT3Y7OH type=write size=3452888 2024-04-17T04:44:55.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.708+0000 7fd8ef2218c0 1 < canceled part key=_multipart_A5ZUB3YK06J3.2~XV4QRSUUJN0E.0 size=361437 2024-04-17T04:44:55.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.716+0000 7fd8ef2218c0 1 < canceled multipart upload key=A5ZUB3YK06J3 upload=XV4QRSUUJN0E tag=6YYRM5POSMEW type=write size=1410013 2024-04-17T04:44:55.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~S08MJ2HAM8NT.0 size=836974 2024-04-17T04:44:55.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.720+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=S08MJ2HAM8NT tag=XLGB5I8DYRYW type=write size=1885550 2024-04-17T04:44:55.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.724+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=Y6CO9EQYNMGN type=del 2024-04-17T04:44:55.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.728+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~582UUDOZNZAT.0 size=780600 2024-04-17T04:44:55.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.728+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=582UUDOZNZAT tag=BOX6HDZOBVL1 type=write size=1829176 2024-04-17T04:44:55.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.732+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~I2542I4FE9TS.0 size=1048576 2024-04-17T04:44:55.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.732+0000 7fd8ef2218c0 1 < canceled part key=_multipart_OZ1IST8DOGXW.2~I2542I4FE9TS.1 size=799772 2024-04-17T04:44:55.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.736+0000 7fd8ef2218c0 1 < canceled multipart upload key=OZ1IST8DOGXW upload=I2542I4FE9TS tag=L2OI8HW7ETEY type=write size=2896924 2024-04-17T04:44:55.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~LNF29TCYFVJL.0 size=749157 2024-04-17T04:44:55.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.740+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=LNF29TCYFVJL tag=8LB1FWRN50TU type=write size=1797733 2024-04-17T04:44:55.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.744+0000 7fd8ef2218c0 1 < completed write operation key=VUYM3JC57QRP tag=ZHQ7L8HLBILT type=write size=35790 2024-04-17T04:44:55.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.748+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=P5RK65NBB4XF type=del size=125660 2024-04-17T04:44:55.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.752+0000 7fd8ef2218c0 1 < completed write operation key=GX9X0D45GC2I tag=MNRVBNJ4FSH8 type=write size=147595 2024-04-17T04:44:55.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~SALJI2PA090B.0 size=625925 2024-04-17T04:44:55.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.756+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=SALJI2PA090B tag=XAV6Z8H028QM type=write size=1674501 2024-04-17T04:44:55.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.760+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~DV1V2GYLMXQ8.0 size=1048576 2024-04-17T04:44:55.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.764+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~DV1V2GYLMXQ8.1 size=882984 2024-04-17T04:44:55.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.768+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=DV1V2GYLMXQ8 tag=WGMS3P6Y3IWD type=write size=2980136 2024-04-17T04:44:55.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.776+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~HV2EOU9U6J3O.0 size=1048576 2024-04-17T04:44:55.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.776+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~HV2EOU9U6J3O.1 size=335373 2024-04-17T04:44:55.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.776+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=HV2EOU9U6J3O tag=SIUV158QE9AE type=write size=2432525 2024-04-17T04:44:55.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.780+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=K73T72ATTVYN type=write size=149643 2024-04-17T04:44:55.779 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.784+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=39M01TMBTHF7 type=del size=3214352 2024-04-17T04:44:55.779 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.784+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~AUMT45EXHZHU.0 size=1048576 2024-04-17T04:44:55.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~AUMT45EXHZHU.1 size=461255 2024-04-17T04:44:55.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.792+0000 7fd8ef2218c0 1 < canceled multipart upload key=6219DPY72LWC upload=AUMT45EXHZHU tag=TZIZJ8E4TNUL type=write size=2558407 2024-04-17T04:44:55.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~2PMRBG12VYCP.0 size=1048576 2024-04-17T04:44:55.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~2PMRBG12VYCP.1 size=1048576 2024-04-17T04:44:55.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.804+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~2PMRBG12VYCP.2 size=966074 2024-04-17T04:44:55.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.804+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=2PMRBG12VYCP tag=LTJZON5EZI9W type=write size=4111802 2024-04-17T04:44:55.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.812+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=2ZNI5J3OVAW7 type=del 2024-04-17T04:44:55.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.816+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=3NOYUN6Q7L0B type=del size=1034978 2024-04-17T04:44:55.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.820+0000 7fd8ef2218c0 1 < suggested update operation key=6219DPY72LWC tag=UOGN3ZU7C13N type=del size=445807 2024-04-17T04:44:55.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.828+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~IQFEJ1LUWJ7A.0 size=326464 2024-04-17T04:44:55.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.828+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=IQFEJ1LUWJ7A tag=9QXHLT33FQ60 type=write size=1375040 2024-04-17T04:44:55.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.828+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~5949UEJMZZW9.0 size=256295 2024-04-17T04:44:55.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.832+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=5949UEJMZZW9 tag=A74MX2WD0LWI type=write size=1304871 2024-04-17T04:44:55.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~JMXBTU1PKJLQ.0 size=1048576 2024-04-17T04:44:55.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~JMXBTU1PKJLQ.1 size=16054 2024-04-17T04:44:55.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.840+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=JMXBTU1PKJLQ tag=Y000CYGLMDTF type=write size=2113206 2024-04-17T04:44:55.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.848+0000 7fd8ef2218c0 1 < suggested remove operation key=G03VLS09F3BT tag=668VDFO35E6W type=del 2024-04-17T04:44:55.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.852+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~HLE2XOEXGAW4.0 size=920706 2024-04-17T04:44:55.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.852+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=HLE2XOEXGAW4 tag=LEDIE5Z0ICL6 type=write size=1969282 2024-04-17T04:44:55.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.856+0000 7fd8ef2218c0 1 < completed write operation key=AE4CYL334ONO tag=Q1SE5C789IGM type=write size=21378 2024-04-17T04:44:55.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~HURKGMED9X1G.0 size=334103 2024-04-17T04:44:55.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.860+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=HURKGMED9X1G tag=BN5H7GAVSL3E type=write size=1382679 2024-04-17T04:44:55.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.864+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~J1E3DK7XAT1C.0 size=1048576 2024-04-17T04:44:55.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.864+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~J1E3DK7XAT1C.1 size=1048576 2024-04-17T04:44:55.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.868+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~J1E3DK7XAT1C.2 size=502596 2024-04-17T04:44:55.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.872+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=J1E3DK7XAT1C tag=BD9G67XFEHUL type=write size=3648324 2024-04-17T04:44:55.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.876+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~VJ4F0EN3QBC0.0 size=884008 2024-04-17T04:44:55.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.876+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=VJ4F0EN3QBC0 tag=1MO3AI74DGX0 type=write size=1932584 2024-04-17T04:44:55.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.880+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~6ZIJODNFIFQ3.0 size=1048576 2024-04-17T04:44:55.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.884+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~6ZIJODNFIFQ3.1 size=381386 2024-04-17T04:44:55.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.888+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=6ZIJODNFIFQ3 tag=KS6FQ2YC72IT type=write size=2478538 2024-04-17T04:44:55.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.892+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~6WNEEJ7L7VMT.0 size=1048576 2024-04-17T04:44:55.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.892+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~6WNEEJ7L7VMT.1 size=297549 2024-04-17T04:44:55.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.892+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=6WNEEJ7L7VMT tag=WHSPLB4MRI2E type=write size=2394701 2024-04-17T04:44:55.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.896+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~NE1WO109FSOE.0 size=1048576 2024-04-17T04:44:55.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.896+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~NE1WO109FSOE.1 size=1048576 2024-04-17T04:44:55.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~NE1WO109FSOE.2 size=307160 2024-04-17T04:44:55.900 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.904+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=NE1WO109FSOE tag=X28MEIT3Y7OH type=write size=3452888 2024-04-17T04:44:55.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.908+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=RW7VD88FTLE5 type=del size=1270657 2024-04-17T04:44:55.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.912+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~S08MJ2HAM8NT.0 size=836974 2024-04-17T04:44:55.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.916+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=S08MJ2HAM8NT tag=XLGB5I8DYRYW type=write size=1885550 2024-04-17T04:44:55.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~W7KTYUDXVTS3.0 size=1048576 2024-04-17T04:44:55.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~W7KTYUDXVTS3.1 size=1048576 2024-04-17T04:44:55.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~W7KTYUDXVTS3.2 size=556466 2024-04-17T04:44:55.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.924+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=W7KTYUDXVTS3 tag=OT8LO1I23CUV type=write size=3702194 2024-04-17T04:44:55.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.924+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~582UUDOZNZAT.0 size=780600 2024-04-17T04:44:55.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.928+0000 7fd8ef2218c0 1 < canceled multipart upload key=6219DPY72LWC upload=582UUDOZNZAT tag=BOX6HDZOBVL1 type=write size=1829176 2024-04-17T04:44:55.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.932+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=RQQ382F1X4J6 type=del size=2308083 2024-04-17T04:44:55.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~LNF29TCYFVJL.0 size=749157 2024-04-17T04:44:55.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.940+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=LNF29TCYFVJL tag=8LB1FWRN50TU type=write size=1797733 2024-04-17T04:44:55.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.940+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=3JTJM8OQLFCJ type=del size=402922 2024-04-17T04:44:55.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.944+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=P5RK65NBB4XF type=del 2024-04-17T04:44:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.948+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~LNEV0HZ11BJN.0 size=1048576 2024-04-17T04:44:55.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~LNEV0HZ11BJN.1 size=1048576 2024-04-17T04:44:55.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~LNEV0HZ11BJN.2 size=565189 2024-04-17T04:44:55.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.952+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=LNEV0HZ11BJN tag=YHH0TF58JMI3 type=write size=3710917 2024-04-17T04:44:55.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.956+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~SALJI2PA090B.0 size=625925 2024-04-17T04:44:55.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.960+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=SALJI2PA090B tag=XAV6Z8H028QM type=write size=1674501 2024-04-17T04:44:55.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.964+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~LD54N3Y3YV3Q.0 size=1048576 2024-04-17T04:44:55.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~LD54N3Y3YV3Q.1 size=1048576 2024-04-17T04:44:55.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~LD54N3Y3YV3Q.2 size=868681 2024-04-17T04:44:55.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.968+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=LD54N3Y3YV3Q tag=TR3FMQMNDJUB type=write size=4014409 2024-04-17T04:44:55.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~HV2EOU9U6J3O.0 size=1048576 2024-04-17T04:44:55.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~HV2EOU9U6J3O.1 size=335373 2024-04-17T04:44:55.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.976+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=HV2EOU9U6J3O tag=SIUV158QE9AE type=write size=2432525 2024-04-17T04:44:55.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~N1XPENT5LV6B.0 size=1048576 2024-04-17T04:44:55.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~N1XPENT5LV6B.1 size=1048576 2024-04-17T04:44:55.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~N1XPENT5LV6B.2 size=429880 2024-04-17T04:44:55.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.984+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=N1XPENT5LV6B tag=68B672EHIBY2 type=write size=3575608 2024-04-17T04:44:55.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.988+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=39M01TMBTHF7 type=del 2024-04-17T04:44:55.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.988+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=UFVDG2QW8HL3 type=del size=1814120 2024-04-17T04:44:55.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~2PMRBG12VYCP.0 size=1048576 2024-04-17T04:44:55.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~2PMRBG12VYCP.1 size=1048576 2024-04-17T04:44:55.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:55.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~2PMRBG12VYCP.2 size=966074 2024-04-17T04:44:55.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.000+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=2PMRBG12VYCP tag=LTJZON5EZI9W type=write size=4111802 2024-04-17T04:44:56.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~1KRLCKSAP3Z4.0 size=1048576 2024-04-17T04:44:56.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~1KRLCKSAP3Z4.1 size=1048576 2024-04-17T04:44:56.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~1KRLCKSAP3Z4.2 size=217037 2024-04-17T04:44:56.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=1KRLCKSAP3Z4 tag=C481ZTVB89WX type=write size=3362765 2024-04-17T04:44:56.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.012+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=3NOYUN6Q7L0B type=del 2024-04-17T04:44:56.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.012+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=4IYVK10YO75Z type=del size=1355012 2024-04-17T04:44:56.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.016+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~IQFEJ1LUWJ7A.0 size=326464 2024-04-17T04:44:56.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.016+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=IQFEJ1LUWJ7A tag=9QXHLT33FQ60 type=write size=1375040 2024-04-17T04:44:56.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~27O5K2ZHOA9Z.0 size=1048576 2024-04-17T04:44:56.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~27O5K2ZHOA9Z.1 size=1048576 2024-04-17T04:44:56.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~27O5K2ZHOA9Z.2 size=355730 2024-04-17T04:44:56.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.028+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=27O5K2ZHOA9Z tag=L9HS61K9PDY4 type=write size=3501458 2024-04-17T04:44:56.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~JMXBTU1PKJLQ.0 size=1048576 2024-04-17T04:44:56.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.032+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~JMXBTU1PKJLQ.1 size=16054 2024-04-17T04:44:56.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.036+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=JMXBTU1PKJLQ tag=Y000CYGLMDTF type=write size=2113206 2024-04-17T04:44:56.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.036+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=OVNH9AF1G6WZ type=write size=773494 2024-04-17T04:44:56.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~HLE2XOEXGAW4.0 size=920706 2024-04-17T04:44:56.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.044+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=HLE2XOEXGAW4 tag=LEDIE5Z0ICL6 type=write size=1969282 2024-04-17T04:44:56.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.048+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=9H3BPZAB9RTH type=del size=185178 2024-04-17T04:44:56.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.048+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~HURKGMED9X1G.0 size=334103 2024-04-17T04:44:56.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.052+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=HURKGMED9X1G tag=BN5H7GAVSL3E type=write size=1382679 2024-04-17T04:44:56.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.056+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=7SUNTGVLINWU type=del size=2747727 2024-04-17T04:44:56.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~VJ4F0EN3QBC0.0 size=884008 2024-04-17T04:44:56.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.064+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=VJ4F0EN3QBC0 tag=1MO3AI74DGX0 type=write size=1932584 2024-04-17T04:44:56.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.068+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=458K9MGN0SKQ type=del size=901210 2024-04-17T04:44:56.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~6WNEEJ7L7VMT.0 size=1048576 2024-04-17T04:44:56.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~6WNEEJ7L7VMT.1 size=297549 2024-04-17T04:44:56.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.076+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=6WNEEJ7L7VMT tag=WHSPLB4MRI2E type=write size=2394701 2024-04-17T04:44:56.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.080+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=X7JHHIRDKHI9 type=del size=699931 2024-04-17T04:44:56.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.088+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=RW7VD88FTLE5 type=del 2024-04-17T04:44:56.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~LV8CHM7P8OV1.0 size=1048576 2024-04-17T04:44:56.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.096+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~LV8CHM7P8OV1.1 size=637357 2024-04-17T04:44:56.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.096+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=LV8CHM7P8OV1 tag=DMQHRPFTMVEQ type=write size=2734509 2024-04-17T04:44:56.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.096+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~W7KTYUDXVTS3.0 size=1048576 2024-04-17T04:44:56.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.100+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~W7KTYUDXVTS3.1 size=1048576 2024-04-17T04:44:56.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.100+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~W7KTYUDXVTS3.2 size=556466 2024-04-17T04:44:56.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.104+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=W7KTYUDXVTS3 tag=OT8LO1I23CUV type=write size=3702194 2024-04-17T04:44:56.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.104+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=ISG1XVRKNC93 type=del size=1452117 2024-04-17T04:44:56.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.108+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=RQQ382F1X4J6 type=del 2024-04-17T04:44:56.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~AL65R6BJ5H44.0 size=1048576 2024-04-17T04:44:56.111 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~AL65R6BJ5H44.1 size=387277 2024-04-17T04:44:56.111 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=AL65R6BJ5H44 tag=YN1NDDTIDWYI type=write size=2484429 2024-04-17T04:44:56.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.120+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=3JTJM8OQLFCJ type=del 2024-04-17T04:44:56.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~CLK8GPC3F7R8.0 size=1048576 2024-04-17T04:44:56.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~CLK8GPC3F7R8.1 size=1048576 2024-04-17T04:44:56.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.128+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~CLK8GPC3F7R8.2 size=697128 2024-04-17T04:44:56.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.128+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=CLK8GPC3F7R8 tag=BY8NNSTZOWS3 type=write size=3842856 2024-04-17T04:44:56.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~LNEV0HZ11BJN.0 size=1048576 2024-04-17T04:44:56.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~LNEV0HZ11BJN.1 size=1048576 2024-04-17T04:44:56.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~LNEV0HZ11BJN.2 size=565189 2024-04-17T04:44:56.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.140+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=LNEV0HZ11BJN tag=YHH0TF58JMI3 type=write size=3710917 2024-04-17T04:44:56.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.140+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=C05XDYVLHYDV type=del size=3102054 2024-04-17T04:44:56.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.144+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~LD54N3Y3YV3Q.0 size=1048576 2024-04-17T04:44:56.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.144+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~LD54N3Y3YV3Q.1 size=1048576 2024-04-17T04:44:56.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~LD54N3Y3YV3Q.2 size=868681 2024-04-17T04:44:56.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.152+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=LD54N3Y3YV3Q tag=TR3FMQMNDJUB type=write size=4014409 2024-04-17T04:44:56.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.156+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=VG8FBIV8I0K8 type=del size=2796949 2024-04-17T04:44:56.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.156+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~N1XPENT5LV6B.0 size=1048576 2024-04-17T04:44:56.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.156+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~N1XPENT5LV6B.1 size=1048576 2024-04-17T04:44:56.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~N1XPENT5LV6B.2 size=429880 2024-04-17T04:44:56.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.164+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=N1XPENT5LV6B tag=68B672EHIBY2 type=write size=3575608 2024-04-17T04:44:56.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.164+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=VSNA7GFHS3HF type=write size=182642 2024-04-17T04:44:56.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.168+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=UFVDG2QW8HL3 type=del 2024-04-17T04:44:56.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.172+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=B1AASV4BR6WK type=del size=2649294 2024-04-17T04:44:56.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.172+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~1KRLCKSAP3Z4.0 size=1048576 2024-04-17T04:44:56.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~1KRLCKSAP3Z4.1 size=1048576 2024-04-17T04:44:56.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~1KRLCKSAP3Z4.2 size=217037 2024-04-17T04:44:56.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.184+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=1KRLCKSAP3Z4 tag=C481ZTVB89WX type=write size=3362765 2024-04-17T04:44:56.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.188+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=0N06PW5DL518 type=del size=3820944 2024-04-17T04:44:56.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.188+0000 7fd8ef2218c0 1 < canceled operation key=AE4CYL334ONO tag=4IYVK10YO75Z type=del size=1355012 2024-04-17T04:44:56.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.192+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=WS254DHB7938 type=del size=3194259 2024-04-17T04:44:56.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~27O5K2ZHOA9Z.0 size=1048576 2024-04-17T04:44:56.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~27O5K2ZHOA9Z.1 size=1048576 2024-04-17T04:44:56.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~27O5K2ZHOA9Z.2 size=355730 2024-04-17T04:44:56.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.204+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=27O5K2ZHOA9Z tag=L9HS61K9PDY4 type=write size=3501458 2024-04-17T04:44:56.203 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.208+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=A8KOD8HOYF0F type=del size=1750330 2024-04-17T04:44:56.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.212+0000 7fd8ef2218c0 1 < canceled operation key=OZ1IST8DOGXW tag=OVNH9AF1G6WZ type=write size=773494 2024-04-17T04:44:56.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~LZCZVAGCA85T.0 size=1048576 2024-04-17T04:44:56.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~LZCZVAGCA85T.1 size=1048576 2024-04-17T04:44:56.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~LZCZVAGCA85T.2 size=508695 2024-04-17T04:44:56.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.220+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=LZCZVAGCA85T tag=MVTZJRE9G3XR type=write size=3654423 2024-04-17T04:44:56.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.224+0000 7fd8ef2218c0 1 < canceled operation key=EWAW6EBW6W6U tag=9H3BPZAB9RTH type=del size=185178 2024-04-17T04:44:56.223 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.228+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=IZS2U4WEMHLS type=del size=274222 2024-04-17T04:44:56.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.232+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=7SUNTGVLINWU type=del 2024-04-17T04:44:56.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.232+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=NGXUOG0R3GAF type=del size=1639456 2024-04-17T04:44:56.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.236+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=458K9MGN0SKQ type=del 2024-04-17T04:44:56.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~LVNG4GJ66KVA.0 size=1048576 2024-04-17T04:44:56.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~LVNG4GJ66KVA.1 size=1048576 2024-04-17T04:44:56.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.248+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~LVNG4GJ66KVA.2 size=707792 2024-04-17T04:44:56.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.248+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=LVNG4GJ66KVA tag=M4LYBICBJY58 type=write size=3853520 2024-04-17T04:44:56.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.256+0000 7fd8ef2218c0 1 < suggested remove operation key=6219DPY72LWC tag=X7JHHIRDKHI9 type=del 2024-04-17T04:44:56.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.260+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~VHJHZQQPKPAY.0 size=186087 2024-04-17T04:44:56.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.260+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=VHJHZQQPKPAY tag=A0MQ83XC4DGN type=write size=1234663 2024-04-17T04:44:56.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~LV8CHM7P8OV1.0 size=1048576 2024-04-17T04:44:56.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~LV8CHM7P8OV1.1 size=637357 2024-04-17T04:44:56.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.276+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=LV8CHM7P8OV1 tag=DMQHRPFTMVEQ type=write size=2734509 2024-04-17T04:44:56.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~CXAIUT7BKFO4.0 size=1048576 2024-04-17T04:44:56.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~CXAIUT7BKFO4.1 size=1048576 2024-04-17T04:44:56.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~CXAIUT7BKFO4.2 size=884567 2024-04-17T04:44:56.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.288+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=CXAIUT7BKFO4 tag=E0TGZA4UQ99Y type=write size=4030295 2024-04-17T04:44:56.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.292+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=ISG1XVRKNC93 type=del 2024-04-17T04:44:56.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.300+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~LCS2TNSNQLHH.0 size=580260 2024-04-17T04:44:56.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.300+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=LCS2TNSNQLHH tag=B2XAAUK84MPR type=write size=1628836 2024-04-17T04:44:56.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~AL65R6BJ5H44.0 size=1048576 2024-04-17T04:44:56.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.304+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2S1IT3TO1BPU.2~AL65R6BJ5H44.1 size=387277 2024-04-17T04:44:56.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.308+0000 7fd8ef2218c0 1 < canceled multipart upload key=2S1IT3TO1BPU upload=AL65R6BJ5H44 tag=YN1NDDTIDWYI type=write size=2484429 2024-04-17T04:44:56.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.312+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=MNS4LM3Q03BU type=del size=2613065 2024-04-17T04:44:56.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~CLK8GPC3F7R8.0 size=1048576 2024-04-17T04:44:56.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~CLK8GPC3F7R8.1 size=1048576 2024-04-17T04:44:56.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.320+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~CLK8GPC3F7R8.2 size=697128 2024-04-17T04:44:56.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.324+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=CLK8GPC3F7R8 tag=BY8NNSTZOWS3 type=write size=3842856 2024-04-17T04:44:56.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~SRNSKLBK0MO0.0 size=1048576 2024-04-17T04:44:56.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~SRNSKLBK0MO0.1 size=1048576 2024-04-17T04:44:56.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~SRNSKLBK0MO0.2 size=468514 2024-04-17T04:44:56.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.332+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=SRNSKLBK0MO0 tag=3BSLBE2Q3HM9 type=write size=3614242 2024-04-17T04:44:56.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.336+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=C05XDYVLHYDV type=del 2024-04-17T04:44:56.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.340+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=ARUN21OCBUYY type=write size=281860 2024-04-17T04:44:56.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.344+0000 7fd8ef2218c0 1 < suggested remove operation key=5FR1NEWE0LD1 tag=VG8FBIV8I0K8 type=del 2024-04-17T04:44:56.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.348+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=SBSVVUG9OJ5X type=del size=3717072 2024-04-17T04:44:56.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.352+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=VSNA7GFHS3HF type=write size=182642 2024-04-17T04:44:56.355 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~C13M8AL8D7E1.0 size=1048576 2024-04-17T04:44:56.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~C13M8AL8D7E1.1 size=3648 2024-04-17T04:44:56.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.360+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=C13M8AL8D7E1 tag=FM9Q7VTTL15P type=write size=2100800 2024-04-17T04:44:56.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.364+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=B1AASV4BR6WK type=del 2024-04-17T04:44:56.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.368+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=G64KZ1U2ZHP5 type=del size=391631 2024-04-17T04:44:56.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.372+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=0N06PW5DL518 type=del 2024-04-17T04:44:56.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.376+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=RRNMSRHU3DQ4 type=del size=621841 2024-04-17T04:44:56.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.376+0000 7fd8ef2218c0 -1 < failed to complete operation key=6219DPY72LWC tag=WS254DHB7938 type=del size=3194259: (22) Invalid argument 2024-04-17T04:44:56.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.380+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=E2P3DJW2RLR7 type=del size=1153753 2024-04-17T04:44:56.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.380+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=2IY8DB37FI9H tag=A8KOD8HOYF0F type=del size=1750330: (2) No such file or directory 2024-04-17T04:44:56.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.384+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=B2BZ6FWK8830 type=del size=1922371 2024-04-17T04:44:56.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.384+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~LZCZVAGCA85T.0 size=1048576 2024-04-17T04:44:56.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.384+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~LZCZVAGCA85T.1 size=1048576 2024-04-17T04:44:56.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~LZCZVAGCA85T.2 size=508695 2024-04-17T04:44:56.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.392+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=LZCZVAGCA85T tag=MVTZJRE9G3XR type=write size=3654423 2024-04-17T04:44:56.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.396+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=G84X0G4XN8GI type=del size=1050640 2024-04-17T04:44:56.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.400+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=IZS2U4WEMHLS type=del 2024-04-17T04:44:56.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.400+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=261RPFOMGE79 type=del size=802074 2024-04-17T04:44:56.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.404+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=NGXUOG0R3GAF type=del 2024-04-17T04:44:56.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.412+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~01PGHGXKT08H.0 size=1048576 2024-04-17T04:44:56.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.412+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~01PGHGXKT08H.1 size=906242 2024-04-17T04:44:56.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.412+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=01PGHGXKT08H tag=FD6CVNJPBMGE type=write size=3003394 2024-04-17T04:44:56.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.416+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~LVNG4GJ66KVA.0 size=1048576 2024-04-17T04:44:56.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~LVNG4GJ66KVA.1 size=1048576 2024-04-17T04:44:56.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.424+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~LVNG4GJ66KVA.2 size=707792 2024-04-17T04:44:56.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.428+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=LVNG4GJ66KVA tag=M4LYBICBJY58 type=write size=3853520 2024-04-17T04:44:56.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~2UFAWGF55HQT.0 size=16096 2024-04-17T04:44:56.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.436+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=2UFAWGF55HQT tag=M2XM2I8K66QV type=write size=1064672 2024-04-17T04:44:56.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~VHJHZQQPKPAY.0 size=186087 2024-04-17T04:44:56.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.448+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=VHJHZQQPKPAY tag=A0MQ83XC4DGN type=write size=1234663 2024-04-17T04:44:56.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.452+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=9QCBYJCVIXW2 type=del size=2587939 2024-04-17T04:44:56.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~CXAIUT7BKFO4.0 size=1048576 2024-04-17T04:44:56.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.460+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~CXAIUT7BKFO4.1 size=1048576 2024-04-17T04:44:56.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.460+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~CXAIUT7BKFO4.2 size=884567 2024-04-17T04:44:56.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.468+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=CXAIUT7BKFO4 tag=E0TGZA4UQ99Y type=write size=4030295 2024-04-17T04:44:56.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.472+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~PW2SVJYJX6N6.0 size=1048576 2024-04-17T04:44:56.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.476+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~PW2SVJYJX6N6.1 size=556247 2024-04-17T04:44:56.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.476+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=PW2SVJYJX6N6 tag=Y1UE353QGYKU type=write size=2653399 2024-04-17T04:44:56.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.480+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~LCS2TNSNQLHH.0 size=580260 2024-04-17T04:44:56.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.488+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=LCS2TNSNQLHH tag=B2XAAUK84MPR type=write size=1628836 2024-04-17T04:44:56.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.488+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=2EMSMEPJQ5JW type=del size=794252 2024-04-17T04:44:56.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.496+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=MNS4LM3Q03BU type=del 2024-04-17T04:44:56.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.500+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=D6B31MA00G3B type=write size=639232 2024-04-17T04:44:56.500 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~SRNSKLBK0MO0.0 size=1048576 2024-04-17T04:44:56.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.508+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~SRNSKLBK0MO0.1 size=1048576 2024-04-17T04:44:56.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.508+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~SRNSKLBK0MO0.2 size=468514 2024-04-17T04:44:56.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.512+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=SRNSKLBK0MO0 tag=3BSLBE2Q3HM9 type=write size=3614242 2024-04-17T04:44:56.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.516+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=AOBEXPA6FFIA type=del size=1258106 2024-04-17T04:44:56.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.520+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=ARUN21OCBUYY type=write size=281860 2024-04-17T04:44:56.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.524+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~UO4M1OWQLL4H.0 size=1048576 2024-04-17T04:44:56.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.528+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~UO4M1OWQLL4H.1 size=1048576 2024-04-17T04:44:56.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~UO4M1OWQLL4H.2 size=852542 2024-04-17T04:44:56.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.532+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=UO4M1OWQLL4H tag=MI9ORFKI7B29 type=write size=3998270 2024-04-17T04:44:56.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.536+0000 7fd8ef2218c0 1 < suggested update operation key=UNOVYXDS5JEB tag=SBSVVUG9OJ5X type=del size=3717072 2024-04-17T04:44:56.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.544+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~LH39PEV1I0OG.0 size=1048576 2024-04-17T04:44:56.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~LH39PEV1I0OG.1 size=877177 2024-04-17T04:44:56.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.548+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=LH39PEV1I0OG tag=G70XLONB805X type=write size=2974329 2024-04-17T04:44:56.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~C13M8AL8D7E1.0 size=1048576 2024-04-17T04:44:56.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~C13M8AL8D7E1.1 size=3648 2024-04-17T04:44:56.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.560+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=C13M8AL8D7E1 tag=FM9Q7VTTL15P type=write size=2100800 2024-04-17T04:44:56.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.564+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=ZRVPW502LBC4 type=del size=448440 2024-04-17T04:44:56.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.568+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=G64KZ1U2ZHP5 type=del 2024-04-17T04:44:56.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.572+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=K1X61KS0JKEI type=del size=3485752 2024-04-17T04:44:56.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.572+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=RRNMSRHU3DQ4 type=del 2024-04-17T04:44:56.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.576+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=D7BYO5WA1TNK type=del size=1246207 2024-04-17T04:44:56.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.580+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=E2P3DJW2RLR7 type=del 2024-04-17T04:44:56.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.584+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=TJAVVR3G3WGJ type=del size=2738795 2024-04-17T04:44:56.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.592+0000 7fd8ef2218c0 1 < suggested update operation key=2IY8DB37FI9H tag=B2BZ6FWK8830 type=del size=1922371 2024-04-17T04:44:56.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.596+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=650DD0H2496H type=del size=2137168 2024-04-17T04:44:56.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.600+0000 7fd8ef2218c0 1 < suggested update operation key=UYCKMXE6LBUH tag=G84X0G4XN8GI type=del size=1050640 2024-04-17T04:44:56.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.604+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=B5N8KLGINOMH type=del size=454888 2024-04-17T04:44:56.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.612+0000 7fd8ef2218c0 1 < suggested remove operation key=VUYM3JC57QRP tag=261RPFOMGE79 type=del 2024-04-17T04:44:56.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~N9E4DP6WOE3H.0 size=840477 2024-04-17T04:44:56.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.616+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=N9E4DP6WOE3H tag=77GXAISJQKRT type=write size=1889053 2024-04-17T04:44:56.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~01PGHGXKT08H.0 size=1048576 2024-04-17T04:44:56.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.624+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~01PGHGXKT08H.1 size=906242 2024-04-17T04:44:56.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.628+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=01PGHGXKT08H tag=FD6CVNJPBMGE type=write size=3003394 2024-04-17T04:44:56.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.632+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=T3NE5T0S3WJU type=write size=20017 2024-04-17T04:44:56.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.632+0000 7fd8ef2218c0 1 < canceled part key=_multipart_OZ1IST8DOGXW.2~2UFAWGF55HQT.0 size=16096 2024-04-17T04:44:56.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.640+0000 7fd8ef2218c0 1 < canceled multipart upload key=OZ1IST8DOGXW upload=2UFAWGF55HQT tag=M2XM2I8K66QV type=write size=1064672 2024-04-17T04:44:56.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.644+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=0W2FTDRKV6FX type=del size=2062700 2024-04-17T04:44:56.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.648+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=9QCBYJCVIXW2 type=del 2024-04-17T04:44:56.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.648+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=LW258XZ5NHAT type=del size=1795063 2024-04-17T04:44:56.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.652+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~PW2SVJYJX6N6.0 size=1048576 2024-04-17T04:44:56.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.652+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~PW2SVJYJX6N6.1 size=556247 2024-04-17T04:44:56.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.660+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=PW2SVJYJX6N6 tag=Y1UE353QGYKU type=write size=2653399 2024-04-17T04:44:56.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.668+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~SX4Y5LRZTV97.0 size=1048576 2024-04-17T04:44:56.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.668+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~SX4Y5LRZTV97.1 size=452286 2024-04-17T04:44:56.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.668+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=SX4Y5LRZTV97 tag=183PJ0J93T50 type=write size=2549438 2024-04-17T04:44:56.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.672+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=2EMSMEPJQ5JW type=del 2024-04-17T04:44:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~2RAH6V6Y08WN.0 size=439117 2024-04-17T04:44:56.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=2RAH6V6Y08WN tag=QSO3BXDZJ7WK type=write size=1487693 2024-04-17T04:44:56.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.684+0000 7fd8ef2218c0 1 < completed write operation key=G03VLS09F3BT tag=D6B31MA00G3B type=write size=639232 2024-04-17T04:44:56.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.688+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=6K78YNW35QWL type=write size=74050 2024-04-17T04:44:56.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.696+0000 7fd8ef2218c0 1 < suggested update operation key=OZ1IST8DOGXW tag=AOBEXPA6FFIA type=del size=1258106 2024-04-17T04:44:56.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.700+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=BOS12HK5MXVH type=del size=811341 2024-04-17T04:44:56.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.704+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~UO4M1OWQLL4H.0 size=1048576 2024-04-17T04:44:56.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.704+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~UO4M1OWQLL4H.1 size=1048576 2024-04-17T04:44:56.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~UO4M1OWQLL4H.2 size=852542 2024-04-17T04:44:56.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.716+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=UO4M1OWQLL4H tag=MI9ORFKI7B29 type=write size=3998270 2024-04-17T04:44:56.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.716+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=X969DXRPAKP1 type=write size=151295 2024-04-17T04:44:56.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.720+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~LH39PEV1I0OG.0 size=1048576 2024-04-17T04:44:56.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.720+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~LH39PEV1I0OG.1 size=877177 2024-04-17T04:44:56.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.728+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=LH39PEV1I0OG tag=G70XLONB805X type=write size=2974329 2024-04-17T04:44:56.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.732+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=H7BOJJQ2K22C type=del size=3864829 2024-04-17T04:44:56.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.736+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=ZRVPW502LBC4 type=del 2024-04-17T04:44:56.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~S4P5KIHZCFXE.0 size=1048576 2024-04-17T04:44:56.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~S4P5KIHZCFXE.1 size=968047 2024-04-17T04:44:56.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=S4P5KIHZCFXE tag=RDL47RPOV4JS type=write size=3065199 2024-04-17T04:44:56.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.748+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=K1X61KS0JKEI type=del 2024-04-17T04:44:56.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~J6HHVQLP1UYW.0 size=1048576 2024-04-17T04:44:56.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~J6HHVQLP1UYW.1 size=1048576 2024-04-17T04:44:56.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.760+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~J6HHVQLP1UYW.2 size=289933 2024-04-17T04:44:56.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.760+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=J6HHVQLP1UYW tag=HPTV53I89SSZ type=write size=3435661 2024-04-17T04:44:56.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.768+0000 7fd8ef2218c0 1 < suggested update operation key=5EFSN2FPUQB6 tag=D7BYO5WA1TNK type=del size=1246207 2024-04-17T04:44:56.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.768+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=NWPJRYBLSC90 type=del size=4108128 2024-04-17T04:44:56.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.772+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=TJAVVR3G3WGJ type=del 2024-04-17T04:44:56.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.780+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~IKR3Z4W972QT.0 size=882868 2024-04-17T04:44:56.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.780+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=IKR3Z4W972QT tag=B4FTM3C9AWAE type=write size=1931444 2024-04-17T04:44:56.779 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.784+0000 7fd8ef2218c0 -1 < failed to complete operation key=OZ1IST8DOGXW tag=650DD0H2496H type=del size=2137168: (22) Invalid argument 2024-04-17T04:44:56.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~QPZLZSFDC8OJ.0 size=648211 2024-04-17T04:44:56.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.788+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=QPZLZSFDC8OJ tag=B31CF3MYPOSR type=write size=1696787 2024-04-17T04:44:56.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.792+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=B5N8KLGINOMH type=del 2024-04-17T04:44:56.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.796+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=A2ZPYR82JPJF type=del size=2475632 2024-04-17T04:44:56.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~N9E4DP6WOE3H.0 size=840477 2024-04-17T04:44:56.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.808+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=N9E4DP6WOE3H tag=77GXAISJQKRT type=write size=1889053 2024-04-17T04:44:56.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~PYMQNADUH3EW.0 size=1048576 2024-04-17T04:44:56.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~PYMQNADUH3EW.1 size=1048576 2024-04-17T04:44:56.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.820+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~PYMQNADUH3EW.2 size=320452 2024-04-17T04:44:56.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.820+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=PYMQNADUH3EW tag=H2QZGW3B0CU3 type=write size=3466180 2024-04-17T04:44:56.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.828+0000 7fd8ef2218c0 1 < suggested remove operation key=7BV3V2XMLSPH tag=T3NE5T0S3WJU type=write 2024-04-17T04:44:56.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.832+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=QVDVMRES9G71 type=del size=4057497 2024-04-17T04:44:56.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.836+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=0W2FTDRKV6FX type=del size=2062700 2024-04-17T04:44:56.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.840+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=91QZS6Q2THS0 type=del size=2476128 2024-04-17T04:44:56.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.844+0000 7fd8ef2218c0 -1 < failed to complete operation key=7BV3V2XMLSPH tag=LW258XZ5NHAT type=del size=1795063: (22) Invalid argument 2024-04-17T04:44:56.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.848+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=5PX1QYHEW0C8 type=del size=2439456 2024-04-17T04:44:56.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.848+0000 7fd8ef2218c0 1 < canceled part key=_multipart_A5ZUB3YK06J3.2~SX4Y5LRZTV97.0 size=1048576 2024-04-17T04:44:56.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~SX4Y5LRZTV97.1 size=452286 2024-04-17T04:44:56.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.856+0000 7fd8ef2218c0 1 < canceled multipart upload key=A5ZUB3YK06J3 upload=SX4Y5LRZTV97 tag=183PJ0J93T50 type=write size=2549438 2024-04-17T04:44:56.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~772I1X0UBXHI.0 size=1048576 2024-04-17T04:44:56.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~772I1X0UBXHI.1 size=1048576 2024-04-17T04:44:56.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~772I1X0UBXHI.2 size=655802 2024-04-17T04:44:56.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.868+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=772I1X0UBXHI tag=3IJ2ZO7PJ1DT type=write size=3801530 2024-04-17T04:44:56.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.872+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~2RAH6V6Y08WN.0 size=439117 2024-04-17T04:44:56.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.876+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=2RAH6V6Y08WN tag=QSO3BXDZJ7WK type=write size=1487693 2024-04-17T04:44:56.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.884+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~H4KEPCWRPPMG.0 size=399150 2024-04-17T04:44:56.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.884+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=H4KEPCWRPPMG tag=UNC5Q3ZBBX6A type=write size=1447726 2024-04-17T04:44:56.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.888+0000 7fd8ef2218c0 1 < completed write operation key=A5ZUB3YK06J3 tag=6K78YNW35QWL type=write size=74050 2024-04-17T04:44:56.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.892+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~50PSSYYPQA61.0 size=1048576 2024-04-17T04:44:56.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~50PSSYYPQA61.1 size=1048576 2024-04-17T04:44:56.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~50PSSYYPQA61.2 size=762205 2024-04-17T04:44:56.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.900+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=50PSSYYPQA61 tag=UBVPHAD060DN type=write size=3907933 2024-04-17T04:44:56.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.904+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=BOS12HK5MXVH type=del 2024-04-17T04:44:56.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.908+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=QKH0ARP1ZUNJ type=del size=4148304 2024-04-17T04:44:56.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.916+0000 7fd8ef2218c0 1 < suggested remove operation key=5ZULD0PHLS4V tag=X969DXRPAKP1 type=write 2024-04-17T04:44:56.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.916+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=Z5O7CY8WOB32 type=del size=148741 2024-04-17T04:44:56.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.920+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=H7BOJJQ2K22C type=del 2024-04-17T04:44:56.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.924+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=Q06RBAGGYIQV type=del size=869451 2024-04-17T04:44:56.923 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.928+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~S4P5KIHZCFXE.0 size=1048576 2024-04-17T04:44:56.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~S4P5KIHZCFXE.1 size=968047 2024-04-17T04:44:56.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.936+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=S4P5KIHZCFXE tag=RDL47RPOV4JS type=write size=3065199 2024-04-17T04:44:56.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.940+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=P0Z4IFO2TMOY type=del size=3353437 2024-04-17T04:44:56.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.944+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~J6HHVQLP1UYW.0 size=1048576 2024-04-17T04:44:56.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~J6HHVQLP1UYW.1 size=1048576 2024-04-17T04:44:56.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~J6HHVQLP1UYW.2 size=289933 2024-04-17T04:44:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.956+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=J6HHVQLP1UYW tag=HPTV53I89SSZ type=write size=3435661 2024-04-17T04:44:56.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.956+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=9952YBP3AK6A type=del size=739364 2024-04-17T04:44:56.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.960+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=NWPJRYBLSC90 type=del 2024-04-17T04:44:56.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~PAO9C5QXQJ1W.0 size=806959 2024-04-17T04:44:56.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.968+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=PAO9C5QXQJ1W tag=TP7BL1ON34MU type=write size=1855535 2024-04-17T04:44:56.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~IKR3Z4W972QT.0 size=882868 2024-04-17T04:44:56.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.976+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=IKR3Z4W972QT tag=B4FTM3C9AWAE type=write size=1931444 2024-04-17T04:44:56.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.980+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=VNJSRK1DNERO type=del size=3053355 2024-04-17T04:44:56.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.984+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~QPZLZSFDC8OJ.0 size=648211 2024-04-17T04:44:56.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.988+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=QPZLZSFDC8OJ tag=B31CF3MYPOSR type=write size=1696787 2024-04-17T04:44:56.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.996+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~U3T58X2JPJGC.0 size=875127 2024-04-17T04:44:56.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:56.996+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=U3T58X2JPJGC tag=8REK3YR74PZO type=write size=1923703 2024-04-17T04:44:56.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.000+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=A2ZPYR82JPJF type=del 2024-04-17T04:44:56.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.000+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=BEXIW9LMDBHM type=write size=794624 2024-04-17T04:44:57.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.004+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~PYMQNADUH3EW.0 size=1048576 2024-04-17T04:44:57.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~PYMQNADUH3EW.1 size=1048576 2024-04-17T04:44:57.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.012+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UYCKMXE6LBUH.2~PYMQNADUH3EW.2 size=320452 2024-04-17T04:44:57.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.016+0000 7fd8ef2218c0 1 < canceled multipart upload key=UYCKMXE6LBUH upload=PYMQNADUH3EW tag=H2QZGW3B0CU3 type=write size=3466180 2024-04-17T04:44:57.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.020+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~1ML129V0OKVU.0 size=427012 2024-04-17T04:44:57.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.020+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=1ML129V0OKVU tag=AL9C28M8GE8S type=write size=1475588 2024-04-17T04:44:57.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.028+0000 7fd8ef2218c0 1 < suggested remove operation key=UNOVYXDS5JEB tag=QVDVMRES9G71 type=del 2024-04-17T04:44:57.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.032+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=BZGDTNUXDAYG type=del size=1432891 2024-04-17T04:44:57.029 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.032+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=91QZS6Q2THS0 type=del 2024-04-17T04:44:57.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.036+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=WZQ30EDKL3M8 type=del size=1619235 2024-04-17T04:44:57.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.040+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=5PX1QYHEW0C8 type=del 2024-04-17T04:44:57.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.040+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=T272QL3MXF0Z type=del size=374244 2024-04-17T04:44:57.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.044+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~772I1X0UBXHI.0 size=1048576 2024-04-17T04:44:57.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.048+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~772I1X0UBXHI.1 size=1048576 2024-04-17T04:44:57.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.048+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~772I1X0UBXHI.2 size=655802 2024-04-17T04:44:57.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.056+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=772I1X0UBXHI tag=3IJ2ZO7PJ1DT type=write size=3801530 2024-04-17T04:44:57.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.060+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=TCMLFBPVUWMQ type=del size=2932206 2024-04-17T04:44:57.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~H4KEPCWRPPMG.0 size=399150 2024-04-17T04:44:57.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.068+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=H4KEPCWRPPMG tag=UNC5Q3ZBBX6A type=write size=1447726 2024-04-17T04:44:57.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.072+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=JI4CF3ML54KB type=del size=2421914 2024-04-17T04:44:57.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.076+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~50PSSYYPQA61.0 size=1048576 2024-04-17T04:44:57.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.076+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~50PSSYYPQA61.1 size=1048576 2024-04-17T04:44:57.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~50PSSYYPQA61.2 size=762205 2024-04-17T04:44:57.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.088+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=50PSSYYPQA61 tag=UBVPHAD060DN type=write size=3907933 2024-04-17T04:44:57.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.092+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=WOUP43UDV7X9 type=del size=3505004 2024-04-17T04:44:57.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.096+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=QKH0ARP1ZUNJ type=del 2024-04-17T04:44:57.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.096+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=W4W0E7UFOJM1 type=del size=164599 2024-04-17T04:44:57.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.104+0000 7fd8ef2218c0 1 < suggested remove operation key=6WSZ5UL0V1Y1 tag=Z5O7CY8WOB32 type=del 2024-04-17T04:44:57.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.108+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=0YM47DGOASUV type=del size=3385952 2024-04-17T04:44:57.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.112+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=Q06RBAGGYIQV type=del 2024-04-17T04:44:57.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.120+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~CAK9Q2YGB6AG.0 size=1048576 2024-04-17T04:44:57.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.120+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~CAK9Q2YGB6AG.1 size=45426 2024-04-17T04:44:57.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.120+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=CAK9Q2YGB6AG tag=H99923VZZSCG type=write size=2142578 2024-04-17T04:44:57.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.124+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=P0Z4IFO2TMOY type=del 2024-04-17T04:44:57.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~IKSGXA85IDTD.0 size=1048576 2024-04-17T04:44:57.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~IKSGXA85IDTD.1 size=112510 2024-04-17T04:44:57.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.132+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=IKSGXA85IDTD tag=ZEXP90TY0M2E type=write size=2209662 2024-04-17T04:44:57.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.136+0000 7fd8ef2218c0 1 < canceled operation key=1JOWGWFR7R6P tag=9952YBP3AK6A type=del size=739364 2024-04-17T04:44:57.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.140+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=R9G6R2QN08CX type=write size=290270 2024-04-17T04:44:57.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.144+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~PAO9C5QXQJ1W.0 size=806959 2024-04-17T04:44:57.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.148+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=PAO9C5QXQJ1W tag=TP7BL1ON34MU type=write size=1855535 2024-04-17T04:44:57.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.156+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~A3G1OQ25XR92.0 size=151438 2024-04-17T04:44:57.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.156+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=A3G1OQ25XR92 tag=TGZ9HQPEOGH3 type=write size=1200014 2024-04-17T04:44:57.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.160+0000 7fd8ef2218c0 1 < suggested update operation key=7BV3V2XMLSPH tag=VNJSRK1DNERO type=del size=3053355 2024-04-17T04:44:57.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.164+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=5X493ECR7P4L type=del size=2473200 2024-04-17T04:44:57.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.164+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1JOWGWFR7R6P.2~U3T58X2JPJGC.0 size=875127 2024-04-17T04:44:57.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.168+0000 7fd8ef2218c0 1 < canceled multipart upload key=1JOWGWFR7R6P upload=U3T58X2JPJGC tag=8REK3YR74PZO type=write size=1923703 2024-04-17T04:44:57.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.176+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~RNVGNG4N3OEN.0 size=1048576 2024-04-17T04:44:57.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.180+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~RNVGNG4N3OEN.1 size=230811 2024-04-17T04:44:57.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.180+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=RNVGNG4N3OEN tag=XZ6LBECJKKCM type=write size=2327963 2024-04-17T04:44:57.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.184+0000 7fd8ef2218c0 1 < suggested update operation key=UEWLJ3E9ODUL tag=BEXIW9LMDBHM type=write size=794624 2024-04-17T04:44:57.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~6K6WJ1TQNP8I.0 size=1048576 2024-04-17T04:44:57.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~6K6WJ1TQNP8I.1 size=880969 2024-04-17T04:44:57.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.192+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=6K6WJ1TQNP8I tag=2CHYG66CZ2CL type=write size=2978121 2024-04-17T04:44:57.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~1ML129V0OKVU.0 size=427012 2024-04-17T04:44:57.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.200+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=1ML129V0OKVU tag=AL9C28M8GE8S type=write size=1475588 2024-04-17T04:44:57.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.204+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=AOVKIPY4EWGG type=del size=3139843 2024-04-17T04:44:57.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.208+0000 7fd8ef2218c0 -1 < failed to complete operation key=7BV3V2XMLSPH tag=BZGDTNUXDAYG type=del size=1432891: (22) Invalid argument 2024-04-17T04:44:57.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.212+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=9BUWB9TCLNHX type=del size=4084912 2024-04-17T04:44:57.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.216+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=WZQ30EDKL3M8 type=del 2024-04-17T04:44:57.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.220+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=3NZL40DQDY8Q type=write size=678018 2024-04-17T04:44:57.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.224+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=T272QL3MXF0Z type=del 2024-04-17T04:44:57.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.224+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=D0CQ7DCUNDGH type=del size=1871508 2024-04-17T04:44:57.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.228+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=TCMLFBPVUWMQ type=del 2024-04-17T04:44:57.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.232+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=XYPZ3ZFXGDQ7 type=write size=92526 2024-04-17T04:44:57.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.236+0000 7fd8ef2218c0 1 < canceled operation key=EWAW6EBW6W6U tag=JI4CF3ML54KB type=del size=2421914 2024-04-17T04:44:57.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.240+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~CDXLBPBD2S9L.0 size=1048576 2024-04-17T04:44:57.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~CDXLBPBD2S9L.1 size=10750 2024-04-17T04:44:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.244+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=CDXLBPBD2S9L tag=S2XDS54RU1TE type=write size=2107902 2024-04-17T04:44:57.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.248+0000 7fd8ef2218c0 1 < canceled operation key=1JOWGWFR7R6P tag=WOUP43UDV7X9 type=del size=3505004 2024-04-17T04:44:57.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.252+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=O97O4SKH0T6T type=del size=3798512 2024-04-17T04:44:57.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.256+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=W4W0E7UFOJM1 type=del 2024-04-17T04:44:57.256 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.260+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=O8XXSXNKDJGV type=write size=216915 2024-04-17T04:44:57.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.264+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=0YM47DGOASUV type=del 2024-04-17T04:44:57.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.268+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=BXSIBK95ZF2Z type=del size=1212619 2024-04-17T04:44:57.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~CAK9Q2YGB6AG.0 size=1048576 2024-04-17T04:44:57.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.272+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~CAK9Q2YGB6AG.1 size=45426 2024-04-17T04:44:57.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.280+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=CAK9Q2YGB6AG tag=H99923VZZSCG type=write size=2142578 2024-04-17T04:44:57.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.280+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=TBJ1Z4ZTVNL8 type=del size=1924941 2024-04-17T04:44:57.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.284+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~IKSGXA85IDTD.0 size=1048576 2024-04-17T04:44:57.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.284+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~IKSGXA85IDTD.1 size=112510 2024-04-17T04:44:57.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.288+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=IKSGXA85IDTD tag=ZEXP90TY0M2E type=write size=2209662 2024-04-17T04:44:57.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.296+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~STU3FOTW6SOU.0 size=1048576 2024-04-17T04:44:57.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.296+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~STU3FOTW6SOU.1 size=1048576 2024-04-17T04:44:57.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.300+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~STU3FOTW6SOU.2 size=629547 2024-04-17T04:44:57.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.300+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=STU3FOTW6SOU tag=4IVOZ7FILNT4 type=write size=3775275 2024-04-17T04:44:57.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.304+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=R9G6R2QN08CX type=write size=290270 2024-04-17T04:44:57.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.304+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=CN1TPJ9QFD7U type=del size=1143269 2024-04-17T04:44:57.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.308+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~A3G1OQ25XR92.0 size=151438 2024-04-17T04:44:57.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.316+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=A3G1OQ25XR92 tag=TGZ9HQPEOGH3 type=write size=1200014 2024-04-17T04:44:57.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.320+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~N8YRCJW9BTUP.0 size=1048576 2024-04-17T04:44:57.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~N8YRCJW9BTUP.1 size=1048576 2024-04-17T04:44:57.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~N8YRCJW9BTUP.2 size=330457 2024-04-17T04:44:57.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.324+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=N8YRCJW9BTUP tag=OJRX1H2RJKGN type=write size=3476185 2024-04-17T04:44:57.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.332+0000 7fd8ef2218c0 1 < suggested remove operation key=R3C8PAVPVGBY tag=5X493ECR7P4L type=del 2024-04-17T04:44:57.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.336+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~Q9C3HPAZS3YB.0 size=1048576 2024-04-17T04:44:57.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~Q9C3HPAZS3YB.1 size=1048576 2024-04-17T04:44:57.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~Q9C3HPAZS3YB.2 size=7051 2024-04-17T04:44:57.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.344+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=Q9C3HPAZS3YB tag=CR6ZZ2U526NO type=write size=3152779 2024-04-17T04:44:57.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.344+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~RNVGNG4N3OEN.0 size=1048576 2024-04-17T04:44:57.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.348+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~RNVGNG4N3OEN.1 size=230811 2024-04-17T04:44:57.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.356+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=RNVGNG4N3OEN tag=XZ6LBECJKKCM type=write size=2327963 2024-04-17T04:44:57.355 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~1WIFUQF0SW2Q.0 size=330152 2024-04-17T04:44:57.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.360+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=1WIFUQF0SW2Q tag=I7WX9YYWY2PL type=write size=1378728 2024-04-17T04:44:57.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.364+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~6K6WJ1TQNP8I.0 size=1048576 2024-04-17T04:44:57.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.368+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~6K6WJ1TQNP8I.1 size=880969 2024-04-17T04:44:57.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.372+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=6K6WJ1TQNP8I tag=2CHYG66CZ2CL type=write size=2978121 2024-04-17T04:44:57.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.376+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~GLME22WLYHOB.0 size=1048576 2024-04-17T04:44:57.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.380+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~GLME22WLYHOB.1 size=125307 2024-04-17T04:44:57.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.380+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=GLME22WLYHOB tag=W9F838NHHPX7 type=write size=2222459 2024-04-17T04:44:57.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.380+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=R3C8PAVPVGBY tag=AOVKIPY4EWGG type=del size=3139843: (2) No such file or directory 2024-04-17T04:44:57.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4LHO5ZGGNKNE.0 size=1048576 2024-04-17T04:44:57.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4LHO5ZGGNKNE.1 size=1048576 2024-04-17T04:44:57.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4LHO5ZGGNKNE.2 size=1018142 2024-04-17T04:44:57.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.388+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=4LHO5ZGGNKNE tag=S1I3MOTLIKJT type=write size=4163870 2024-04-17T04:44:57.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.392+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=9BUWB9TCLNHX type=del 2024-04-17T04:44:57.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.396+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=OQAYZI62AD47 type=del size=3025228 2024-04-17T04:44:57.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.396+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=3NZL40DQDY8Q type=write size=678018 2024-04-17T04:44:57.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.400+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=UAKP8TO0B4HK type=write size=253256 2024-04-17T04:44:57.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.400+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=D0CQ7DCUNDGH type=del 2024-04-17T04:44:57.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~QSL5HRGG53I5.0 size=1048576 2024-04-17T04:44:57.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~QSL5HRGG53I5.1 size=929883 2024-04-17T04:44:57.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.408+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=QSL5HRGG53I5 tag=0QK6BOVTVLI1 type=write size=3027035 2024-04-17T04:44:57.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.412+0000 7fd8ef2218c0 1 < completed write operation key=2IY8DB37FI9H tag=XYPZ3ZFXGDQ7 type=write size=92526 2024-04-17T04:44:57.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.412+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=V5JBKDB3141J type=write size=906327 2024-04-17T04:44:57.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.416+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~CDXLBPBD2S9L.0 size=1048576 2024-04-17T04:44:57.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.416+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~CDXLBPBD2S9L.1 size=10750 2024-04-17T04:44:57.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.420+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=CDXLBPBD2S9L tag=S2XDS54RU1TE type=write size=2107902 2024-04-17T04:44:57.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.424+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~5RCNQXHMLFJW.0 size=1048576 2024-04-17T04:44:57.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.428+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~5RCNQXHMLFJW.1 size=1048576 2024-04-17T04:44:57.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.428+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~5RCNQXHMLFJW.2 size=463455 2024-04-17T04:44:57.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.428+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=5RCNQXHMLFJW tag=T9A85178MAC7 type=write size=3609183 2024-04-17T04:44:57.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.432+0000 7fd8ef2218c0 1 < suggested update operation key=509DZ9DBJWC5 tag=O97O4SKH0T6T type=del size=3798512 2024-04-17T04:44:57.431 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.436+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=8311ISMAD8KR type=del size=163074 2024-04-17T04:44:57.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.440+0000 7fd8ef2218c0 1 < suggested update operation key=5K50O96P7OGD tag=O8XXSXNKDJGV type=write size=216915 2024-04-17T04:44:57.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~TEUSZDAYTOY2.0 size=1048576 2024-04-17T04:44:57.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~TEUSZDAYTOY2.1 size=1048576 2024-04-17T04:44:57.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~TEUSZDAYTOY2.2 size=736369 2024-04-17T04:44:57.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.448+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=TEUSZDAYTOY2 tag=7GLPWMUJIHYA type=write size=3882097 2024-04-17T04:44:57.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.452+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=BXSIBK95ZF2Z type=del 2024-04-17T04:44:57.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.452+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=78V4MKA3T90Z type=del size=3402042 2024-04-17T04:44:57.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.456+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=TBJ1Z4ZTVNL8 type=del 2024-04-17T04:44:57.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.460+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=DY2GOFYIDSXZ type=del size=1478254 2024-04-17T04:44:57.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.464+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~STU3FOTW6SOU.0 size=1048576 2024-04-17T04:44:57.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.464+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~STU3FOTW6SOU.1 size=1048576 2024-04-17T04:44:57.464 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.468+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~STU3FOTW6SOU.2 size=629547 2024-04-17T04:44:57.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.472+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=STU3FOTW6SOU tag=4IVOZ7FILNT4 type=write size=3775275 2024-04-17T04:44:57.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.476+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=M40K5905AHPW type=del size=1029368 2024-04-17T04:44:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.476+0000 7fd8ef2218c0 -1 < failed to complete operation key=R3C8PAVPVGBY tag=CN1TPJ9QFD7U type=del size=1143269: (22) Invalid argument 2024-04-17T04:44:57.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.480+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~965ZKFWR1D8R.0 size=1048576 2024-04-17T04:44:57.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.484+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~965ZKFWR1D8R.1 size=346383 2024-04-17T04:44:57.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.484+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=965ZKFWR1D8R tag=HAWUXCPCC3SI type=write size=2443535 2024-04-17T04:44:57.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.488+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~N8YRCJW9BTUP.0 size=1048576 2024-04-17T04:44:57.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.488+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~N8YRCJW9BTUP.1 size=1048576 2024-04-17T04:44:57.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.492+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~N8YRCJW9BTUP.2 size=330457 2024-04-17T04:44:57.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.496+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=N8YRCJW9BTUP tag=OJRX1H2RJKGN type=write size=3476185 2024-04-17T04:44:57.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.500+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=B9RFKXM8LE06 type=del size=3623289 2024-04-17T04:44:57.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~Q9C3HPAZS3YB.0 size=1048576 2024-04-17T04:44:57.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~Q9C3HPAZS3YB.1 size=1048576 2024-04-17T04:44:57.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~Q9C3HPAZS3YB.2 size=7051 2024-04-17T04:44:57.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.508+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=Q9C3HPAZS3YB tag=CR6ZZ2U526NO type=write size=3152779 2024-04-17T04:44:57.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.512+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=YKJ1LZNCAM60 type=del size=4127001 2024-04-17T04:44:57.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.512+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~1WIFUQF0SW2Q.0 size=330152 2024-04-17T04:44:57.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.516+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=1WIFUQF0SW2Q tag=I7WX9YYWY2PL type=write size=1378728 2024-04-17T04:44:57.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.520+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~ESY80Z0U3FCS.0 size=1048576 2024-04-17T04:44:57.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.524+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~ESY80Z0U3FCS.1 size=1048576 2024-04-17T04:44:57.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.524+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~ESY80Z0U3FCS.2 size=839552 2024-04-17T04:44:57.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.524+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=ESY80Z0U3FCS tag=YO9WUIM67QKE type=write size=3985280 2024-04-17T04:44:57.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~GLME22WLYHOB.0 size=1048576 2024-04-17T04:44:57.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.532+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~GLME22WLYHOB.1 size=125307 2024-04-17T04:44:57.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.536+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=GLME22WLYHOB tag=W9F838NHHPX7 type=write size=2222459 2024-04-17T04:44:57.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.536+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=RK2I9HVY7ICG type=write size=475122 2024-04-17T04:44:57.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~4LHO5ZGGNKNE.0 size=1048576 2024-04-17T04:44:57.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~4LHO5ZGGNKNE.1 size=1048576 2024-04-17T04:44:57.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.540+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5EFSN2FPUQB6.2~4LHO5ZGGNKNE.2 size=1018142 2024-04-17T04:44:57.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.544+0000 7fd8ef2218c0 1 < canceled multipart upload key=5EFSN2FPUQB6 upload=4LHO5ZGGNKNE tag=S1I3MOTLIKJT type=write size=4163870 2024-04-17T04:44:57.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~ZZICPRWSUF32.0 size=1048576 2024-04-17T04:44:57.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.552+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~ZZICPRWSUF32.1 size=1048576 2024-04-17T04:44:57.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.552+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~ZZICPRWSUF32.2 size=952312 2024-04-17T04:44:57.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.552+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=ZZICPRWSUF32 tag=N0GU0YCNOHRD type=write size=4098040 2024-04-17T04:44:57.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.560+0000 7fd8ef2218c0 1 < suggested update operation key=N11Y8FPN9XES tag=OQAYZI62AD47 type=del size=3025228 2024-04-17T04:44:57.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.564+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=822Q0W8T7QDA type=write size=377744 2024-04-17T04:44:57.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.564+0000 7fd8ef2218c0 1 < completed write operation key=JVWRJBMFIRLL tag=UAKP8TO0B4HK type=write size=253256 2024-04-17T04:44:57.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.568+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=SJIVT1A6XE80 type=del size=3535647 2024-04-17T04:44:57.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.568+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~QSL5HRGG53I5.0 size=1048576 2024-04-17T04:44:57.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.572+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~QSL5HRGG53I5.1 size=929883 2024-04-17T04:44:57.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.576+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=QSL5HRGG53I5 tag=0QK6BOVTVLI1 type=write size=3027035 2024-04-17T04:44:57.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.576+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=6B30IUWD1NJ9 type=del size=1668707 2024-04-17T04:44:57.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.580+0000 7fd8ef2218c0 1 < completed write operation key=UYCKMXE6LBUH tag=V5JBKDB3141J type=write size=906327 2024-04-17T04:44:57.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.580+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=UCGKZV48TSPF type=del size=3052426 2024-04-17T04:44:57.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.584+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~5RCNQXHMLFJW.0 size=1048576 2024-04-17T04:44:57.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.584+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~5RCNQXHMLFJW.1 size=1048576 2024-04-17T04:44:57.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.588+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~5RCNQXHMLFJW.2 size=463455 2024-04-17T04:44:57.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.592+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=5RCNQXHMLFJW tag=T9A85178MAC7 type=write size=3609183 2024-04-17T04:44:57.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.596+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=0AAB09CHLVH2 type=del size=654208 2024-04-17T04:44:57.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.596+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=8311ISMAD8KR type=del 2024-04-17T04:44:57.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.600+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~T1B08IS7L7XW.0 size=326902 2024-04-17T04:44:57.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.600+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=T1B08IS7L7XW tag=V71Q01PKWJST type=write size=1375478 2024-04-17T04:44:57.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.604+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~TEUSZDAYTOY2.0 size=1048576 2024-04-17T04:44:57.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.604+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~TEUSZDAYTOY2.1 size=1048576 2024-04-17T04:44:57.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~TEUSZDAYTOY2.2 size=736369 2024-04-17T04:44:57.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.612+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=TEUSZDAYTOY2 tag=7GLPWMUJIHYA type=write size=3882097 2024-04-17T04:44:57.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.616+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=MU2NBKGYHM5C type=write size=869806 2024-04-17T04:44:57.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.616+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=78V4MKA3T90Z type=del 2024-04-17T04:44:57.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.620+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~OIAH85KZC7JE.0 size=1048576 2024-04-17T04:44:57.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~OIAH85KZC7JE.1 size=1048576 2024-04-17T04:44:57.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~OIAH85KZC7JE.2 size=437779 2024-04-17T04:44:57.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.624+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=OIAH85KZC7JE tag=NQ0NDBR57MEM type=write size=3583507 2024-04-17T04:44:57.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.628+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=DY2GOFYIDSXZ type=del 2024-04-17T04:44:57.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.632+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=B4OOO3E3SZ6E type=del size=209824 2024-04-17T04:44:57.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.636+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=M40K5905AHPW type=del size=1029368 2024-04-17T04:44:57.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.640+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=WZTOGVBL7D2R type=del size=730575 2024-04-17T04:44:57.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.640+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~965ZKFWR1D8R.0 size=1048576 2024-04-17T04:44:57.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.644+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~965ZKFWR1D8R.1 size=346383 2024-04-17T04:44:57.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.648+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=965ZKFWR1D8R tag=HAWUXCPCC3SI type=write size=2443535 2024-04-17T04:44:57.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.652+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~7YNB0WW11V26.0 size=1048576 2024-04-17T04:44:57.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.652+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~7YNB0WW11V26.1 size=1048576 2024-04-17T04:44:57.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~7YNB0WW11V26.2 size=245660 2024-04-17T04:44:57.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.656+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=7YNB0WW11V26 tag=728MFMCOA07S type=write size=3391388 2024-04-17T04:44:57.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.656+0000 7fd8ef2218c0 1 < canceled operation key=GX9X0D45GC2I tag=B9RFKXM8LE06 type=del size=3623289 2024-04-17T04:44:57.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.660+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~3KMZRRKTRJWZ.0 size=1048576 2024-04-17T04:44:57.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.664+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~3KMZRRKTRJWZ.1 size=540631 2024-04-17T04:44:57.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.664+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=3KMZRRKTRJWZ tag=66F1W3ME8KQD type=write size=2637783 2024-04-17T04:44:57.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.668+0000 7fd8ef2218c0 1 < suggested remove operation key=EWAW6EBW6W6U tag=YKJ1LZNCAM60 type=del 2024-04-17T04:44:57.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.672+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~N3K5LI5BCRKL.0 size=1048576 2024-04-17T04:44:57.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.676+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~N3K5LI5BCRKL.1 size=434856 2024-04-17T04:44:57.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.676+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=N3K5LI5BCRKL tag=DJ6QK503F0B7 type=write size=2532008 2024-04-17T04:44:57.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~ESY80Z0U3FCS.0 size=1048576 2024-04-17T04:44:57.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~ESY80Z0U3FCS.1 size=1048576 2024-04-17T04:44:57.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~ESY80Z0U3FCS.2 size=839552 2024-04-17T04:44:57.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.684+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=ESY80Z0U3FCS tag=YO9WUIM67QKE type=write size=3985280 2024-04-17T04:44:57.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.688+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=S0LVPZVWD1SU type=write size=786654 2024-04-17T04:44:57.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.692+0000 7fd8ef2218c0 1 < canceled operation key=7BV3V2XMLSPH tag=RK2I9HVY7ICG type=write size=475122 2024-04-17T04:44:57.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.692+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=MYSLCUW43I9D type=del size=3032800 2024-04-17T04:44:57.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.696+0000 7fd8ef2218c0 1 < canceled part key=_multipart_N11Y8FPN9XES.2~ZZICPRWSUF32.0 size=1048576 2024-04-17T04:44:57.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.696+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~ZZICPRWSUF32.1 size=1048576 2024-04-17T04:44:57.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.696+0000 7fd8ef2218c0 1 < canceled part key=_multipart_N11Y8FPN9XES.2~ZZICPRWSUF32.2 size=952312 2024-04-17T04:44:57.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.700+0000 7fd8ef2218c0 1 < canceled multipart upload key=N11Y8FPN9XES upload=ZZICPRWSUF32 tag=N0GU0YCNOHRD type=write size=4098040 2024-04-17T04:44:57.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.704+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=P7UMVKQ87LD0 type=del size=1922294 2024-04-17T04:44:57.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.708+0000 7fd8ef2218c0 1 < completed write operation key=G03VLS09F3BT tag=822Q0W8T7QDA type=write size=377744 2024-04-17T04:44:57.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~1JKL5VWR8VKJ.0 size=1048576 2024-04-17T04:44:57.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~1JKL5VWR8VKJ.1 size=253803 2024-04-17T04:44:57.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.716+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=1JKL5VWR8VKJ tag=KJXY8OVM4NUE type=write size=2350955 2024-04-17T04:44:57.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.720+0000 7fd8ef2218c0 1 < suggested update operation key=BCG05NIQFA53 tag=SJIVT1A6XE80 type=del size=3535647 2024-04-17T04:44:57.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.724+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=ULP2P8G5V778 type=del size=1938596 2024-04-17T04:44:57.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.728+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=6B30IUWD1NJ9 type=del 2024-04-17T04:44:57.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.728+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=Q1K74QG51KXF type=write size=735220 2024-04-17T04:44:57.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.732+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=UCGKZV48TSPF type=del 2024-04-17T04:44:57.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.732+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=OIZO0FMRAI4E type=write size=22425 2024-04-17T04:44:57.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.736+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=0AAB09CHLVH2 type=del 2024-04-17T04:44:57.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~HTI7OQ0M1L7W.0 size=1048576 2024-04-17T04:44:57.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~HTI7OQ0M1L7W.1 size=984938 2024-04-17T04:44:57.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=HTI7OQ0M1L7W tag=8TDUSUIW4BBZ type=write size=3082090 2024-04-17T04:44:57.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~T1B08IS7L7XW.0 size=326902 2024-04-17T04:44:57.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.752+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=T1B08IS7L7XW tag=V71Q01PKWJST type=write size=1375478 2024-04-17T04:44:57.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.756+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=9PAC5F8OBXLR type=del size=1646563 2024-04-17T04:44:57.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.760+0000 7fd8ef2218c0 1 < canceled operation key=6219DPY72LWC tag=MU2NBKGYHM5C type=write size=869806 2024-04-17T04:44:57.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.764+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=KT1XJPV7CWRJ type=del size=1892820 2024-04-17T04:44:57.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~OIAH85KZC7JE.0 size=1048576 2024-04-17T04:44:57.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.772+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~OIAH85KZC7JE.1 size=1048576 2024-04-17T04:44:57.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.772+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~OIAH85KZC7JE.2 size=437779 2024-04-17T04:44:57.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.780+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=OIAH85KZC7JE tag=NQ0NDBR57MEM type=write size=3583507 2024-04-17T04:44:57.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~FDL7M973IKEY.0 size=727621 2024-04-17T04:44:57.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.784+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=FDL7M973IKEY tag=3YJU6SONLPP4 type=write size=1776197 2024-04-17T04:44:57.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.788+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=B4OOO3E3SZ6E type=del 2024-04-17T04:44:57.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8BR82YFA11CI.0 size=1048576 2024-04-17T04:44:57.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.796+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8BR82YFA11CI.1 size=1048576 2024-04-17T04:44:57.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.796+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8BR82YFA11CI.2 size=1011455 2024-04-17T04:44:57.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.796+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=8BR82YFA11CI tag=BS9Z2T0ZCCH4 type=write size=4157183 2024-04-17T04:44:57.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.800+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=WZTOGVBL7D2R type=del 2024-04-17T04:44:57.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.804+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=WS83R4POURQH type=write size=161686 2024-04-17T04:44:57.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~7YNB0WW11V26.0 size=1048576 2024-04-17T04:44:57.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~7YNB0WW11V26.1 size=1048576 2024-04-17T04:44:57.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~7YNB0WW11V26.2 size=245660 2024-04-17T04:44:57.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.816+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=7YNB0WW11V26 tag=728MFMCOA07S type=write size=3391388 2024-04-17T04:44:57.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.820+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=V72VXAFG3UL0 type=write size=439727 2024-04-17T04:44:57.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~3KMZRRKTRJWZ.0 size=1048576 2024-04-17T04:44:57.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~3KMZRRKTRJWZ.1 size=540631 2024-04-17T04:44:57.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.828+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=3KMZRRKTRJWZ tag=66F1W3ME8KQD type=write size=2637783 2024-04-17T04:44:57.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~VL8AEEKDH6FM.0 size=1048576 2024-04-17T04:44:57.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~VL8AEEKDH6FM.1 size=66078 2024-04-17T04:44:57.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.836+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=VL8AEEKDH6FM tag=6UT6J5MCHL86 type=write size=2163230 2024-04-17T04:44:57.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.840+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~N3K5LI5BCRKL.0 size=1048576 2024-04-17T04:44:57.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.840+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~N3K5LI5BCRKL.1 size=434856 2024-04-17T04:44:57.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.844+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=N3K5LI5BCRKL tag=DJ6QK503F0B7 type=write size=2532008 2024-04-17T04:44:57.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.848+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=NYW3VH6VKFJT type=del size=872387 2024-04-17T04:44:57.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.852+0000 7fd8ef2218c0 1 < completed write operation key=5EFSN2FPUQB6 tag=S0LVPZVWD1SU type=write size=786654 2024-04-17T04:44:57.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.852+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=CBS7XJUQN4HD type=del size=888047 2024-04-17T04:44:57.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.856+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=MYSLCUW43I9D type=del 2024-04-17T04:44:57.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.860+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=F5VUYLCX8R7F type=del size=1373954 2024-04-17T04:44:57.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.864+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=P7UMVKQ87LD0 type=del 2024-04-17T04:44:57.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~9HACVDS5FTFX.0 size=826860 2024-04-17T04:44:57.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.868+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=9HACVDS5FTFX tag=301A0R7O47A4 type=write size=1875436 2024-04-17T04:44:57.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.868+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~1JKL5VWR8VKJ.0 size=1048576 2024-04-17T04:44:57.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.872+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~1JKL5VWR8VKJ.1 size=253803 2024-04-17T04:44:57.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.876+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=1JKL5VWR8VKJ tag=KJXY8OVM4NUE type=write size=2350955 2024-04-17T04:44:57.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.880+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=WWB9G32XQP5S type=del size=2223911 2024-04-17T04:44:57.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.880+0000 7fd8ef2218c0 1 < canceled operation key=5ZULD0PHLS4V tag=ULP2P8G5V778 type=del size=1938596 2024-04-17T04:44:57.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.884+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~QTXZ2BWIMC5R.0 size=203444 2024-04-17T04:44:57.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.884+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=QTXZ2BWIMC5R tag=GHWAYHV3IRPC type=write size=1252020 2024-04-17T04:44:57.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.892+0000 7fd8ef2218c0 1 < suggested update operation key=5ZULD0PHLS4V tag=Q1K74QG51KXF type=write size=735220 2024-04-17T04:44:57.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.892+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=CHBG6O339RU2 type=del size=486428 2024-04-17T04:44:57.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.896+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=OIZO0FMRAI4E type=write size=22425 2024-04-17T04:44:57.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~6R0EFYDJMDYC.0 size=1047636 2024-04-17T04:44:57.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.900+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=6R0EFYDJMDYC tag=P0PMAM3J2300 type=write size=2096212 2024-04-17T04:44:57.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~HTI7OQ0M1L7W.0 size=1048576 2024-04-17T04:44:57.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.904+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~HTI7OQ0M1L7W.1 size=984938 2024-04-17T04:44:57.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.908+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=HTI7OQ0M1L7W tag=8TDUSUIW4BBZ type=write size=3082090 2024-04-17T04:44:57.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.912+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~13OF77V4YB5P.0 size=1048576 2024-04-17T04:44:57.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.912+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~13OF77V4YB5P.1 size=1048576 2024-04-17T04:44:57.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~13OF77V4YB5P.2 size=34767 2024-04-17T04:44:57.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.916+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=13OF77V4YB5P tag=0OZ4THIFFC86 type=write size=3180495 2024-04-17T04:44:57.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.916+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=9PAC5F8OBXLR type=del 2024-04-17T04:44:57.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.920+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=DANHVYIIO2JF type=del size=2236629 2024-04-17T04:44:57.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.924+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=KT1XJPV7CWRJ type=del 2024-04-17T04:44:57.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.924+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=FNDRJ48WVVKP type=del size=2251307 2024-04-17T04:44:57.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.928+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~FDL7M973IKEY.0 size=727621 2024-04-17T04:44:57.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.932+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=FDL7M973IKEY tag=3YJU6SONLPP4 type=write size=1776197 2024-04-17T04:44:57.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.932+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=27UQ4ERDDBWM type=write size=343291 2024-04-17T04:44:57.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8BR82YFA11CI.0 size=1048576 2024-04-17T04:44:57.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8BR82YFA11CI.1 size=1048576 2024-04-17T04:44:57.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8BR82YFA11CI.2 size=1011455 2024-04-17T04:44:57.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.944+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=8BR82YFA11CI tag=BS9Z2T0ZCCH4 type=write size=4157183 2024-04-17T04:44:57.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.944+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=I4PV63H2M1D2 type=del size=1971509 2024-04-17T04:44:57.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.948+0000 7fd8ef2218c0 1 < completed write operation key=5EFSN2FPUQB6 tag=WS83R4POURQH type=write size=161686 2024-04-17T04:44:57.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~O6Q9HI7ME5YB.0 size=1048576 2024-04-17T04:44:57.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~O6Q9HI7ME5YB.1 size=1048576 2024-04-17T04:44:57.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.956+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~O6Q9HI7ME5YB.2 size=733071 2024-04-17T04:44:57.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.956+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=O6Q9HI7ME5YB tag=NY9SQR3BA23J type=write size=3878799 2024-04-17T04:44:57.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.956+0000 7fd8ef2218c0 1 < completed write operation key=1JOWGWFR7R6P tag=V72VXAFG3UL0 type=write size=439727 2024-04-17T04:44:57.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.960+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=VLARV0TWHC4S type=del size=2863465 2024-04-17T04:44:57.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.964+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~VL8AEEKDH6FM.0 size=1048576 2024-04-17T04:44:57.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.964+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~VL8AEEKDH6FM.1 size=66078 2024-04-17T04:44:57.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.968+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=VL8AEEKDH6FM tag=6UT6J5MCHL86 type=write size=2163230 2024-04-17T04:44:57.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~EF04OT3YUU6I.0 size=145460 2024-04-17T04:44:57.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.972+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=EF04OT3YUU6I tag=ZLZ2B6JD5ZMN type=write size=1194036 2024-04-17T04:44:57.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.976+0000 7fd8ef2218c0 1 < suggested remove operation key=R3C8PAVPVGBY tag=NYW3VH6VKFJT type=del 2024-04-17T04:44:57.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.976+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=495KX5RZVJKX type=del size=1206777 2024-04-17T04:44:57.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.984+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=CBS7XJUQN4HD type=del 2024-04-17T04:44:57.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~PF340UN8YGCH.0 size=215991 2024-04-17T04:44:57.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.988+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=PF340UN8YGCH tag=2RE1LY8MKYGW type=write size=1264567 2024-04-17T04:44:57.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.988+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=F5VUYLCX8R7F type=del 2024-04-17T04:44:57.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.992+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=TU2RLBQ61R78 type=write size=130893 2024-04-17T04:44:57.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~9HACVDS5FTFX.0 size=826860 2024-04-17T04:44:57.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:57.996+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=9HACVDS5FTFX tag=301A0R7O47A4 type=write size=1875436 2024-04-17T04:44:57.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.000+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=OW3Z2CTOEM6I type=del size=5920 2024-04-17T04:44:57.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.000+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=WWB9G32XQP5S type=del 2024-04-17T04:44:57.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.004+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=2U8AL2CZQI45 type=del size=4085953 2024-04-17T04:44:58.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.004+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~QTXZ2BWIMC5R.0 size=203444 2024-04-17T04:44:58.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.012+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=QTXZ2BWIMC5R tag=GHWAYHV3IRPC type=write size=1252020 2024-04-17T04:44:58.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.012+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=HZPR193TVFSO type=write size=188507 2024-04-17T04:44:58.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.016+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=CHBG6O339RU2 type=del 2024-04-17T04:44:58.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.016+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=J6D6T2GQQFQD type=del size=3799835 2024-04-17T04:44:58.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.020+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~6R0EFYDJMDYC.0 size=1047636 2024-04-17T04:44:58.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.024+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=6R0EFYDJMDYC tag=P0PMAM3J2300 type=write size=2096212 2024-04-17T04:44:58.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.024+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=DNE7WI0SW59E type=del size=564955 2024-04-17T04:44:58.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~13OF77V4YB5P.0 size=1048576 2024-04-17T04:44:58.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~13OF77V4YB5P.1 size=1048576 2024-04-17T04:44:58.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.028+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~13OF77V4YB5P.2 size=34767 2024-04-17T04:44:58.029 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.036+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=13OF77V4YB5P tag=0OZ4THIFFC86 type=write size=3180495 2024-04-17T04:44:58.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.036+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=YCWEU500XP4B type=del size=2009489 2024-04-17T04:44:58.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.040+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=DANHVYIIO2JF type=del 2024-04-17T04:44:58.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.040+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=8XZP47SVCOJ2 type=del size=2901740 2024-04-17T04:44:58.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.044+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=FNDRJ48WVVKP type=del 2024-04-17T04:44:58.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.044+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=1NL1VGWCC94L type=del size=462680 2024-04-17T04:44:58.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.048+0000 7fd8ef2218c0 1 < completed write operation key=A5ZUB3YK06J3 tag=27UQ4ERDDBWM type=write size=343291 2024-04-17T04:44:58.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.048+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=218LKTLVOF1T type=del size=2669021 2024-04-17T04:44:58.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.056+0000 7fd8ef2218c0 1 < suggested update operation key=WCIJICR0FT2C tag=I4PV63H2M1D2 type=del size=1971509 2024-04-17T04:44:58.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.056+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~S1OZ0AQ6JMW0.0 size=1048576 2024-04-17T04:44:58.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.060+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~S1OZ0AQ6JMW0.1 size=66585 2024-04-17T04:44:58.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.060+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=S1OZ0AQ6JMW0 tag=IEY7WSTGFNQ8 type=write size=2163737 2024-04-17T04:44:58.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~O6Q9HI7ME5YB.0 size=1048576 2024-04-17T04:44:58.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~O6Q9HI7ME5YB.1 size=1048576 2024-04-17T04:44:58.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~O6Q9HI7ME5YB.2 size=733071 2024-04-17T04:44:58.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.068+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=O6Q9HI7ME5YB tag=NY9SQR3BA23J type=write size=3878799 2024-04-17T04:44:58.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.072+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=NL0XNRWISIPO type=del size=1233184 2024-04-17T04:44:58.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.072+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=VLARV0TWHC4S type=del 2024-04-17T04:44:58.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.076+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=T1OIMY2D22Z8 type=del size=3165805 2024-04-17T04:44:58.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~EF04OT3YUU6I.0 size=145460 2024-04-17T04:44:58.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.080+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=EF04OT3YUU6I tag=ZLZ2B6JD5ZMN type=write size=1194036 2024-04-17T04:44:58.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~1B4F91G8T6LB.0 size=219418 2024-04-17T04:44:58.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.084+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=1B4F91G8T6LB tag=1KM4ZZNHJ43H type=write size=1267994 2024-04-17T04:44:58.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.088+0000 7fd8ef2218c0 1 < canceled operation key=82D6YWKJ1TFQ tag=495KX5RZVJKX type=del size=1206777 2024-04-17T04:44:58.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~RE0CRT4ZYJEO.0 size=1048576 2024-04-17T04:44:58.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~RE0CRT4ZYJEO.1 size=1048576 2024-04-17T04:44:58.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.096+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~RE0CRT4ZYJEO.2 size=859377 2024-04-17T04:44:58.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.096+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=RE0CRT4ZYJEO tag=YN8I08H3I34S type=write size=4005105 2024-04-17T04:44:58.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.096+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~PF340UN8YGCH.0 size=215991 2024-04-17T04:44:58.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.100+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=PF340UN8YGCH tag=2RE1LY8MKYGW type=write size=1264567 2024-04-17T04:44:58.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.104+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=UH6MQJI0DC8A type=write size=642031 2024-04-17T04:44:58.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.104+0000 7fd8ef2218c0 -1 < failed to complete operation key=WCIJICR0FT2C tag=TU2RLBQ61R78 type=write size=130893: (22) Invalid argument 2024-04-17T04:44:58.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.108+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=XJ3FY8OLWBGC type=del size=1840538 2024-04-17T04:44:58.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.112+0000 7fd8ef2218c0 -1 < failed to complete operation key=WCIJICR0FT2C tag=OW3Z2CTOEM6I type=del size=5920: (22) Invalid argument 2024-04-17T04:44:58.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.112+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=VBWWUGR171OS type=del size=435211 2024-04-17T04:44:58.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.116+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=2U8AL2CZQI45 type=del 2024-04-17T04:44:58.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.116+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=T6TZJHRA4UCF type=del size=2963683 2024-04-17T04:44:58.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.120+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=HZPR193TVFSO type=write size=188507 2024-04-17T04:44:58.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.120+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=RDHNZ4ZTT5D6 type=del size=3186391 2024-04-17T04:44:58.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.128+0000 7fd8ef2218c0 1 < suggested remove operation key=EWAW6EBW6W6U tag=J6D6T2GQQFQD type=del 2024-04-17T04:44:58.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.128+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=HTWMH1HUSOG4 type=write size=451293 2024-04-17T04:44:58.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.132+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=DNE7WI0SW59E type=del 2024-04-17T04:44:58.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.136+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~MYV45G7XBS89.0 size=25317 2024-04-17T04:44:58.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.136+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=MYV45G7XBS89 tag=XUH13VW95TEG type=write size=1073893 2024-04-17T04:44:58.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.136+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=YCWEU500XP4B type=del 2024-04-17T04:44:58.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.140+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=B85JAKZRUAAP type=del size=345088 2024-04-17T04:44:58.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.144+0000 7fd8ef2218c0 1 < suggested update operation key=B63EZS0LAC30 tag=8XZP47SVCOJ2 type=del size=2901740 2024-04-17T04:44:58.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~JVLMSXZFILU8.0 size=1048576 2024-04-17T04:44:58.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~JVLMSXZFILU8.1 size=999133 2024-04-17T04:44:58.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.148+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=JVLMSXZFILU8 tag=Z7C27W0HCAP3 type=write size=3096285 2024-04-17T04:44:58.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.152+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=1NL1VGWCC94L type=del 2024-04-17T04:44:58.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.156+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=4ADPOB5AI144 type=del size=621649 2024-04-17T04:44:58.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.160+0000 7fd8ef2218c0 1 < suggested update operation key=5EFSN2FPUQB6 tag=218LKTLVOF1T type=del size=2669021 2024-04-17T04:44:58.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.164+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~AY5IJW0QB69C.0 size=714327 2024-04-17T04:44:58.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.164+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=AY5IJW0QB69C tag=5AO4I2ZMXPIO type=write size=1762903 2024-04-17T04:44:58.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.168+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~S1OZ0AQ6JMW0.0 size=1048576 2024-04-17T04:44:58.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.168+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~S1OZ0AQ6JMW0.1 size=66585 2024-04-17T04:44:58.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.172+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=S1OZ0AQ6JMW0 tag=IEY7WSTGFNQ8 type=write size=2163737 2024-04-17T04:44:58.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.176+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=90ZDYWBOLHL4 type=del size=48887 2024-04-17T04:44:58.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.176+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=NL0XNRWISIPO type=del 2024-04-17T04:44:58.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.180+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=J24WWB4R09XI type=del size=2803361 2024-04-17T04:44:58.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.184+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=T1OIMY2D22Z8 type=del 2024-04-17T04:44:58.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~42OL8TRJPCC2.0 size=608319 2024-04-17T04:44:58.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.188+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=42OL8TRJPCC2 tag=61R9E7OPU7O9 type=write size=1656895 2024-04-17T04:44:58.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.192+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~1B4F91G8T6LB.0 size=219418 2024-04-17T04:44:58.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.196+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=1B4F91G8T6LB tag=1KM4ZZNHJ43H type=write size=1267994 2024-04-17T04:44:58.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.200+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=CPGL7I8PH7V8 type=del size=2702872 2024-04-17T04:44:58.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~RE0CRT4ZYJEO.0 size=1048576 2024-04-17T04:44:58.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~RE0CRT4ZYJEO.1 size=1048576 2024-04-17T04:44:58.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~RE0CRT4ZYJEO.2 size=859377 2024-04-17T04:44:58.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.212+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=RE0CRT4ZYJEO tag=YN8I08H3I34S type=write size=4005105 2024-04-17T04:44:58.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~T8LN1SMH72QN.0 size=1048576 2024-04-17T04:44:58.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~T8LN1SMH72QN.1 size=831596 2024-04-17T04:44:58.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.216+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=T8LN1SMH72QN tag=UUQ157AF95LH type=write size=2928748 2024-04-17T04:44:58.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.220+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=UH6MQJI0DC8A type=write size=642031 2024-04-17T04:44:58.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.224+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=375WF3J4O73L type=del size=1336120 2024-04-17T04:44:58.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.228+0000 7fd8ef2218c0 -1 < failed to complete operation key=B63EZS0LAC30 tag=XJ3FY8OLWBGC type=del size=1840538: (22) Invalid argument 2024-04-17T04:44:58.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.228+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=R47HXE86BWN8 type=del size=1551947 2024-04-17T04:44:58.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.232+0000 7fd8ef2218c0 -1 < failed to complete operation key=5EFSN2FPUQB6 tag=VBWWUGR171OS type=del size=435211: (22) Invalid argument 2024-04-17T04:44:58.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.236+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=WEJK3K9RSHE9 type=write size=557536 2024-04-17T04:44:58.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.240+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=T6TZJHRA4UCF type=del 2024-04-17T04:44:58.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.244+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=DSKTZW10CIG1 type=del size=3131396 2024-04-17T04:44:58.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.244+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=RDHNZ4ZTT5D6 type=del 2024-04-17T04:44:58.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~SWS1MQG4WKQD.0 size=97066 2024-04-17T04:44:58.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.252+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=SWS1MQG4WKQD tag=VXXL29FTUUXN type=write size=1145642 2024-04-17T04:44:58.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.256+0000 7fd8ef2218c0 1 < completed write operation key=1JOWGWFR7R6P tag=HTWMH1HUSOG4 type=write size=451293 2024-04-17T04:44:58.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.260+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~XSW6B259V9C0.0 size=87739 2024-04-17T04:44:58.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.260+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=XSW6B259V9C0 tag=I3J12YF300QK type=write size=1136315 2024-04-17T04:44:58.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~MYV45G7XBS89.0 size=25317 2024-04-17T04:44:58.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.276+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=MYV45G7XBS89 tag=XUH13VW95TEG type=write size=1073893 2024-04-17T04:44:58.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.276+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=XE49AF1EC1Q4 type=del size=2102143 2024-04-17T04:44:58.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.280+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=B85JAKZRUAAP type=del size=345088 2024-04-17T04:44:58.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.288+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=AYC83M2R90PA type=write size=199802 2024-04-17T04:44:58.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.288+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~JVLMSXZFILU8.0 size=1048576 2024-04-17T04:44:58.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.288+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~JVLMSXZFILU8.1 size=999133 2024-04-17T04:44:58.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.296+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=JVLMSXZFILU8 tag=Z7C27W0HCAP3 type=write size=3096285 2024-04-17T04:44:58.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.296+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=CA2HAQ4Z2B6R type=write size=479935 2024-04-17T04:44:58.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.300+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=4ADPOB5AI144 type=del 2024-04-17T04:44:58.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.304+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=3AFCQAFIV0PJ type=del size=3187332 2024-04-17T04:44:58.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.304+0000 7fd8ef2218c0 1 < canceled part key=_multipart_509DZ9DBJWC5.2~AY5IJW0QB69C.0 size=714327 2024-04-17T04:44:58.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.308+0000 7fd8ef2218c0 1 < canceled multipart upload key=509DZ9DBJWC5 upload=AY5IJW0QB69C tag=5AO4I2ZMXPIO type=write size=1762903 2024-04-17T04:44:58.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.312+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=UKWD3XTD9KXF type=write size=325607 2024-04-17T04:44:58.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.320+0000 7fd8ef2218c0 1 < suggested update operation key=VUYM3JC57QRP tag=90ZDYWBOLHL4 type=del size=48887 2024-04-17T04:44:58.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~GLV7SP7X0ZBW.0 size=1048576 2024-04-17T04:44:58.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~GLV7SP7X0ZBW.1 size=1048576 2024-04-17T04:44:58.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~GLV7SP7X0ZBW.2 size=980841 2024-04-17T04:44:58.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.328+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=GLV7SP7X0ZBW tag=FH5CV4ZWFZUI type=write size=4126569 2024-04-17T04:44:58.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.336+0000 7fd8ef2218c0 1 < suggested remove operation key=A5ZUB3YK06J3 tag=J24WWB4R09XI type=del 2024-04-17T04:44:58.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~L7KVVRCW6LKJ.0 size=1048576 2024-04-17T04:44:58.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~L7KVVRCW6LKJ.1 size=1048576 2024-04-17T04:44:58.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~L7KVVRCW6LKJ.2 size=516043 2024-04-17T04:44:58.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.344+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=L7KVVRCW6LKJ tag=WLD1595Q6W2V type=write size=3661771 2024-04-17T04:44:58.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.348+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~42OL8TRJPCC2.0 size=608319 2024-04-17T04:44:58.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.352+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=42OL8TRJPCC2 tag=61R9E7OPU7O9 type=write size=1656895 2024-04-17T04:44:58.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CWW05OK8H0I4.0 size=1048576 2024-04-17T04:44:58.355 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CWW05OK8H0I4.1 size=1048576 2024-04-17T04:44:58.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.364+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CWW05OK8H0I4.2 size=908096 2024-04-17T04:44:58.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.364+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=CWW05OK8H0I4 tag=IU8BP0MDUCBM type=write size=4053824 2024-04-17T04:44:58.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.364+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=CPGL7I8PH7V8 type=del 2024-04-17T04:44:58.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.368+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=IBVYZMDVA7NB type=write size=1039555 2024-04-17T04:44:58.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.372+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~T8LN1SMH72QN.0 size=1048576 2024-04-17T04:44:58.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.376+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~T8LN1SMH72QN.1 size=831596 2024-04-17T04:44:58.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.380+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=T8LN1SMH72QN tag=UUQ157AF95LH type=write size=2928748 2024-04-17T04:44:58.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.384+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=LHKSBOEX6LKI type=del size=3858314 2024-04-17T04:44:58.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.388+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=375WF3J4O73L type=del 2024-04-17T04:44:58.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~LXV76PJC8C5W.0 size=1048576 2024-04-17T04:44:58.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~LXV76PJC8C5W.1 size=1048576 2024-04-17T04:44:58.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~LXV76PJC8C5W.2 size=409998 2024-04-17T04:44:58.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.396+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=LXV76PJC8C5W tag=LF8O9Z3F04HX type=write size=3555726 2024-04-17T04:44:58.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.400+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=R47HXE86BWN8 type=del 2024-04-17T04:44:58.399 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.404+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=PQ5QCR7WN9L6 type=write size=393652 2024-04-17T04:44:58.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.408+0000 7fd8ef2218c0 1 < completed write operation key=G03VLS09F3BT tag=WEJK3K9RSHE9 type=write size=557536 2024-04-17T04:44:58.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.408+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=GJFJHRR5APYR type=write size=902575 2024-04-17T04:44:58.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.412+0000 7fd8ef2218c0 -1 < failed to complete operation key=A5ZUB3YK06J3 tag=DSKTZW10CIG1 type=del size=3131396: (22) Invalid argument 2024-04-17T04:44:58.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.416+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=D7TM2DSSIMLY type=del size=2127200 2024-04-17T04:44:58.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~SWS1MQG4WKQD.0 size=97066 2024-04-17T04:44:58.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.424+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=SWS1MQG4WKQD tag=VXXL29FTUUXN type=write size=1145642 2024-04-17T04:44:58.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.428+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~NPTN6GWOCIM3.0 size=306398 2024-04-17T04:44:58.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.428+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=NPTN6GWOCIM3 tag=O63TUHH0WQ1D type=write size=1354974 2024-04-17T04:44:58.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.432+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~XSW6B259V9C0.0 size=87739 2024-04-17T04:44:58.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.436+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=XSW6B259V9C0 tag=I3J12YF300QK type=write size=1136315 2024-04-17T04:44:58.438 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~Z4RYQ9DVV7FE.0 size=1048576 2024-04-17T04:44:58.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~Z4RYQ9DVV7FE.1 size=1048576 2024-04-17T04:44:58.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~Z4RYQ9DVV7FE.2 size=482276 2024-04-17T04:44:58.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.448+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=Z4RYQ9DVV7FE tag=NS445XZTV8XD type=write size=3628004 2024-04-17T04:44:58.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.452+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=XE49AF1EC1Q4 type=del 2024-04-17T04:44:58.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~SSA74U7KE6KU.0 size=489754 2024-04-17T04:44:58.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.456+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=SSA74U7KE6KU tag=B492CN671D01 type=write size=1538330 2024-04-17T04:44:58.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.464+0000 7fd8ef2218c0 1 < suggested remove operation key=7BV3V2XMLSPH tag=AYC83M2R90PA type=write 2024-04-17T04:44:58.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.468+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=YK0X9D47TS32 type=del size=3263868 2024-04-17T04:44:58.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.476+0000 7fd8ef2218c0 1 < suggested update operation key=G03VLS09F3BT tag=CA2HAQ4Z2B6R type=write size=479935 2024-04-17T04:44:58.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.480+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~XZ0Z6LI19Y7D.0 size=1048576 2024-04-17T04:44:58.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.484+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~XZ0Z6LI19Y7D.1 size=804477 2024-04-17T04:44:58.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.484+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=XZ0Z6LI19Y7D tag=DSK8OH4AI2MS type=write size=2901629 2024-04-17T04:44:58.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.488+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=3AFCQAFIV0PJ type=del 2024-04-17T04:44:58.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.492+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=BDUBNRHMRV02 type=del size=3043103 2024-04-17T04:44:58.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.496+0000 7fd8ef2218c0 1 < suggested remove operation key=2S1IT3TO1BPU tag=UKWD3XTD9KXF type=write 2024-04-17T04:44:58.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.500+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=PIHZE9WNYDTT type=write size=933774 2024-04-17T04:44:58.500 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~GLV7SP7X0ZBW.0 size=1048576 2024-04-17T04:44:58.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.508+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~GLV7SP7X0ZBW.1 size=1048576 2024-04-17T04:44:58.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.512+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~GLV7SP7X0ZBW.2 size=980841 2024-04-17T04:44:58.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.516+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=GLV7SP7X0ZBW tag=FH5CV4ZWFZUI type=write size=4126569 2024-04-17T04:44:58.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.520+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=WDV1CYI6W92U type=del size=1869437 2024-04-17T04:44:58.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.524+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~L7KVVRCW6LKJ.0 size=1048576 2024-04-17T04:44:58.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~L7KVVRCW6LKJ.1 size=1048576 2024-04-17T04:44:58.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~L7KVVRCW6LKJ.2 size=516043 2024-04-17T04:44:58.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.536+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=L7KVVRCW6LKJ tag=WLD1595Q6W2V type=write size=3661771 2024-04-17T04:44:58.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.540+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~W9VKJ0DKG005.0 size=831035 2024-04-17T04:44:58.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.540+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=W9VKJ0DKG005 tag=K4NM5YISLKCG type=write size=1879611 2024-04-17T04:44:58.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.544+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~CWW05OK8H0I4.0 size=1048576 2024-04-17T04:44:58.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~CWW05OK8H0I4.1 size=1048576 2024-04-17T04:44:58.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.548+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~CWW05OK8H0I4.2 size=908096 2024-04-17T04:44:58.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.552+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=CWW05OK8H0I4 tag=IU8BP0MDUCBM type=write size=4053824 2024-04-17T04:44:58.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~6SU5A9X6UKG1.0 size=1048576 2024-04-17T04:44:58.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~6SU5A9X6UKG1.1 size=1048576 2024-04-17T04:44:58.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~6SU5A9X6UKG1.2 size=991509 2024-04-17T04:44:58.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.564+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=6SU5A9X6UKG1 tag=CBP6GNHQYBFI type=write size=4137237 2024-04-17T04:44:58.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.568+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=IBVYZMDVA7NB type=write size=1039555 2024-04-17T04:44:58.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~XEP32H8035N8.0 size=1048576 2024-04-17T04:44:58.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~XEP32H8035N8.1 size=1048576 2024-04-17T04:44:58.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~XEP32H8035N8.2 size=1016496 2024-04-17T04:44:58.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.580+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=XEP32H8035N8 tag=IWXMNYATJCD4 type=write size=4162224 2024-04-17T04:44:58.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.584+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=LHKSBOEX6LKI type=del 2024-04-17T04:44:58.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~2ROXJ21UIEL6.0 size=1048576 2024-04-17T04:44:58.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~2ROXJ21UIEL6.1 size=504663 2024-04-17T04:44:58.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=2ROXJ21UIEL6 tag=EGX5K749KI0O type=write size=2601815 2024-04-17T04:44:58.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~LXV76PJC8C5W.0 size=1048576 2024-04-17T04:44:58.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~LXV76PJC8C5W.1 size=1048576 2024-04-17T04:44:58.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.600+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1JOWGWFR7R6P.2~LXV76PJC8C5W.2 size=409998 2024-04-17T04:44:58.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.608+0000 7fd8ef2218c0 1 < canceled multipart upload key=1JOWGWFR7R6P upload=LXV76PJC8C5W tag=LF8O9Z3F04HX type=write size=3555726 2024-04-17T04:44:58.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.608+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=LW5VHSAUS80K type=del size=981343 2024-04-17T04:44:58.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.612+0000 7fd8ef2218c0 -1 < failed to complete operation key=2S1IT3TO1BPU tag=PQ5QCR7WN9L6 type=write size=393652: (22) Invalid argument 2024-04-17T04:44:58.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.620+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~P8BZUPXU9FB3.0 size=289630 2024-04-17T04:44:58.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.620+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=P8BZUPXU9FB3 tag=GK0REYFKO9GZ type=write size=1338206 2024-04-17T04:44:58.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.624+0000 7fd8ef2218c0 1 < completed write operation key=UNOVYXDS5JEB tag=GJFJHRR5APYR type=write size=902575 2024-04-17T04:44:58.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CIB8V79WS320.0 size=1048576 2024-04-17T04:44:58.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.632+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CIB8V79WS320.1 size=514317 2024-04-17T04:44:58.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.632+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=CIB8V79WS320 tag=WZDIB163BW0G type=write size=2611469 2024-04-17T04:44:58.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.636+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=D7TM2DSSIMLY type=del 2024-04-17T04:44:58.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.636+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=207GWDB5OZCM type=del size=1054480 2024-04-17T04:44:58.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.640+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~NPTN6GWOCIM3.0 size=306398 2024-04-17T04:44:58.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.648+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=NPTN6GWOCIM3 tag=O63TUHH0WQ1D type=write size=1354974 2024-04-17T04:44:58.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.652+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=EXTRU4J6GLWI type=del size=1674360 2024-04-17T04:44:58.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.656+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~Z4RYQ9DVV7FE.0 size=1048576 2024-04-17T04:44:58.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.656+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~Z4RYQ9DVV7FE.1 size=1048576 2024-04-17T04:44:58.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.660+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~Z4RYQ9DVV7FE.2 size=482276 2024-04-17T04:44:58.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.668+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=Z4RYQ9DVV7FE tag=NS445XZTV8XD type=write size=3628004 2024-04-17T04:44:58.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.672+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~Y24JZS553Z8G.0 size=1048576 2024-04-17T04:44:58.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.676+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~Y24JZS553Z8G.1 size=1048576 2024-04-17T04:44:58.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~Y24JZS553Z8G.2 size=304629 2024-04-17T04:44:58.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=Y24JZS553Z8G tag=W1Y6D4PP6LYR type=write size=3450357 2024-04-17T04:44:58.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~SSA74U7KE6KU.0 size=489754 2024-04-17T04:44:58.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.688+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=SSA74U7KE6KU tag=B492CN671D01 type=write size=1538330 2024-04-17T04:44:58.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.696+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~RGTRMHV3L5IQ.0 size=1048576 2024-04-17T04:44:58.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.696+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~RGTRMHV3L5IQ.1 size=54681 2024-04-17T04:44:58.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.696+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=RGTRMHV3L5IQ tag=ZUW2UOLZUR4Y type=write size=2151833 2024-04-17T04:44:58.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.700+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=YK0X9D47TS32 type=del 2024-04-17T04:44:58.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.704+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=EKIK3CUPMNQG type=del size=352018 2024-04-17T04:44:58.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~XZ0Z6LI19Y7D.0 size=1048576 2024-04-17T04:44:58.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.712+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~XZ0Z6LI19Y7D.1 size=804477 2024-04-17T04:44:58.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.716+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=XZ0Z6LI19Y7D tag=DSK8OH4AI2MS type=write size=2901629 2024-04-17T04:44:58.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.720+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=S7IC7VVW2E7X type=del size=1054498 2024-04-17T04:44:58.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.728+0000 7fd8ef2218c0 1 < suggested remove operation key=UYCKMXE6LBUH tag=BDUBNRHMRV02 type=del 2024-04-17T04:44:58.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.728+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=7097VHRSBIO5 type=del size=831201 2024-04-17T04:44:58.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.736+0000 7fd8ef2218c0 1 < suggested update operation key=WCIJICR0FT2C tag=PIHZE9WNYDTT type=write size=933774 2024-04-17T04:44:58.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~CIIK3E3SEX70.0 size=1048576 2024-04-17T04:44:58.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~CIIK3E3SEX70.1 size=1048576 2024-04-17T04:44:58.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~CIIK3E3SEX70.2 size=582145 2024-04-17T04:44:58.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=CIIK3E3SEX70 tag=0C49E4WW4T2Q type=write size=3727873 2024-04-17T04:44:58.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.748+0000 7fd8ef2218c0 -1 < failed to complete operation key=WCIJICR0FT2C tag=WDV1CYI6W92U type=del size=1869437: (22) Invalid argument 2024-04-17T04:44:58.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.752+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=N4Z8S7075P05 type=write size=319582 2024-04-17T04:44:58.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.752+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5EFSN2FPUQB6.2~W9VKJ0DKG005.0 size=831035 2024-04-17T04:44:58.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.756+0000 7fd8ef2218c0 1 < canceled multipart upload key=5EFSN2FPUQB6 upload=W9VKJ0DKG005 tag=K4NM5YISLKCG type=write size=1879611 2024-04-17T04:44:58.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~JILJIQW9E110.0 size=948620 2024-04-17T04:44:58.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.764+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=JILJIQW9E110 tag=JDVKSDTLB313 type=write size=1997196 2024-04-17T04:44:58.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~6SU5A9X6UKG1.0 size=1048576 2024-04-17T04:44:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~6SU5A9X6UKG1.1 size=1048576 2024-04-17T04:44:58.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.772+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~6SU5A9X6UKG1.2 size=991509 2024-04-17T04:44:58.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.776+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=6SU5A9X6UKG1 tag=CBP6GNHQYBFI type=write size=4137237 2024-04-17T04:44:58.777 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.780+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=TRW5W3OL3NE9 type=del size=1779097 2024-04-17T04:44:58.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.784+0000 7fd8ef2218c0 1 < canceled part key=_multipart_EWAW6EBW6W6U.2~XEP32H8035N8.0 size=1048576 2024-04-17T04:44:58.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.784+0000 7fd8ef2218c0 1 < canceled part key=_multipart_EWAW6EBW6W6U.2~XEP32H8035N8.1 size=1048576 2024-04-17T04:44:58.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.784+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~XEP32H8035N8.2 size=1016496 2024-04-17T04:44:58.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.792+0000 7fd8ef2218c0 1 < canceled multipart upload key=EWAW6EBW6W6U upload=XEP32H8035N8 tag=IWXMNYATJCD4 type=write size=4162224 2024-04-17T04:44:58.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.796+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~BQVKC3GBFF1E.0 size=1048576 2024-04-17T04:44:58.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~BQVKC3GBFF1E.1 size=106066 2024-04-17T04:44:58.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.800+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=BQVKC3GBFF1E tag=2F2LIAZ69T7V type=write size=2203218 2024-04-17T04:44:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.804+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~2ROXJ21UIEL6.0 size=1048576 2024-04-17T04:44:58.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~2ROXJ21UIEL6.1 size=504663 2024-04-17T04:44:58.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.812+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=2ROXJ21UIEL6 tag=EGX5K749KI0O type=write size=2601815 2024-04-17T04:44:58.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.816+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=1RX6MW8UX2QH type=del size=2835559 2024-04-17T04:44:58.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.824+0000 7fd8ef2218c0 1 < suggested update operation key=EWAW6EBW6W6U tag=LW5VHSAUS80K type=del size=981343 2024-04-17T04:44:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.828+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~MHJOLBVCWVHI.0 size=1048576 2024-04-17T04:44:58.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~MHJOLBVCWVHI.1 size=146180 2024-04-17T04:44:58.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=MHJOLBVCWVHI tag=HBCVGTN4MTPA type=write size=2243332 2024-04-17T04:44:58.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.836+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~P8BZUPXU9FB3.0 size=289630 2024-04-17T04:44:58.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.840+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=P8BZUPXU9FB3 tag=GK0REYFKO9GZ type=write size=1338206 2024-04-17T04:44:58.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.844+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=2M7JOLNHUAS8 type=del size=2360440 2024-04-17T04:44:58.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.848+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~CIB8V79WS320.0 size=1048576 2024-04-17T04:44:58.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.848+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~CIB8V79WS320.1 size=514317 2024-04-17T04:44:58.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.852+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=CIB8V79WS320 tag=WZDIB163BW0G type=write size=2611469 2024-04-17T04:44:58.853 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.856+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=TB02DK7N5FZ4 type=del size=242471 2024-04-17T04:44:58.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.860+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=207GWDB5OZCM type=del 2024-04-17T04:44:58.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.864+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=P4Z2VH5L22LB type=del size=2686997 2024-04-17T04:44:58.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.868+0000 7fd8ef2218c0 1 < canceled operation key=AE4CYL334ONO tag=EXTRU4J6GLWI type=del size=1674360 2024-04-17T04:44:58.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.872+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=UQHQUBRXP90L type=del size=3041305 2024-04-17T04:44:58.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.872+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~Y24JZS553Z8G.0 size=1048576 2024-04-17T04:44:58.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~Y24JZS553Z8G.1 size=1048576 2024-04-17T04:44:58.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~Y24JZS553Z8G.2 size=304629 2024-04-17T04:44:58.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.884+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=Y24JZS553Z8G tag=W1Y6D4PP6LYR type=write size=3450357 2024-04-17T04:44:58.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.888+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=2T667AKJRDL8 type=del size=2534598 2024-04-17T04:44:58.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~RGTRMHV3L5IQ.0 size=1048576 2024-04-17T04:44:58.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.896+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~RGTRMHV3L5IQ.1 size=54681 2024-04-17T04:44:58.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.904+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=RGTRMHV3L5IQ tag=ZUW2UOLZUR4Y type=write size=2151833 2024-04-17T04:44:58.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.908+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=PVPIRR00LXNS type=del size=3112810 2024-04-17T04:44:58.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.912+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=EKIK3CUPMNQG type=del 2024-04-17T04:44:58.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~AXGNB0DII4GS.0 size=1048576 2024-04-17T04:44:58.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~AXGNB0DII4GS.1 size=1048576 2024-04-17T04:44:58.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~AXGNB0DII4GS.2 size=508306 2024-04-17T04:44:58.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.920+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=AXGNB0DII4GS tag=P6YI0P4YTVBE type=write size=3654034 2024-04-17T04:44:58.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.924+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=S7IC7VVW2E7X type=del 2024-04-17T04:44:58.923 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.928+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=BXD4IE7GE3V5 type=del size=1603163 2024-04-17T04:44:58.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.932+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=7097VHRSBIO5 type=del 2024-04-17T04:44:58.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.932+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=OPNYT5Z52SJJ type=del size=3822778 2024-04-17T04:44:58.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~CIIK3E3SEX70.0 size=1048576 2024-04-17T04:44:58.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~CIIK3E3SEX70.1 size=1048576 2024-04-17T04:44:58.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~CIIK3E3SEX70.2 size=582145 2024-04-17T04:44:58.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.948+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=CIIK3E3SEX70 tag=0C49E4WW4T2Q type=write size=3727873 2024-04-17T04:44:58.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.952+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=H1VMIQE91LQ2 type=write size=327231 2024-04-17T04:44:58.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.956+0000 7fd8ef2218c0 1 < completed write operation key=1ZTJP8DRITEK tag=N4Z8S7075P05 type=write size=319582 2024-04-17T04:44:58.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.956+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=H1WKSPWT9LM0 type=del size=2703245 2024-04-17T04:44:58.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.960+0000 7fd8ef2218c0 1 < canceled part key=_multipart_A5ZUB3YK06J3.2~JILJIQW9E110.0 size=948620 2024-04-17T04:44:58.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.964+0000 7fd8ef2218c0 1 < canceled multipart upload key=A5ZUB3YK06J3 upload=JILJIQW9E110 tag=JDVKSDTLB313 type=write size=1997196 2024-04-17T04:44:58.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~FAIL59X99KY1.0 size=1048576 2024-04-17T04:44:58.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~FAIL59X99KY1.1 size=1048576 2024-04-17T04:44:58.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.976+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~FAIL59X99KY1.2 size=84240 2024-04-17T04:44:58.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.976+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=FAIL59X99KY1 tag=2Y4O9LL8BC90 type=write size=3229968 2024-04-17T04:44:58.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.976+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=TRW5W3OL3NE9 type=del 2024-04-17T04:44:58.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8P4SUYLLQM41.0 size=1048576 2024-04-17T04:44:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8P4SUYLLQM41.1 size=290941 2024-04-17T04:44:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.988+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=8P4SUYLLQM41 tag=MNQ8A0LCEG16 type=write size=2388093 2024-04-17T04:44:58.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.988+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~BQVKC3GBFF1E.0 size=1048576 2024-04-17T04:44:58.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~BQVKC3GBFF1E.1 size=106066 2024-04-17T04:44:58.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:58.996+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=BQVKC3GBFF1E tag=2F2LIAZ69T7V type=write size=2203218 2024-04-17T04:44:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~LR1QKTOHLWTU.0 size=705290 2024-04-17T04:44:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.004+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=LR1QKTOHLWTU tag=JQXULIJP682S type=write size=1753866 2024-04-17T04:44:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.008+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=1RX6MW8UX2QH type=del size=2835559 2024-04-17T04:44:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~TBOEH14Q7QBP.0 size=1048576 2024-04-17T04:44:59.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~TBOEH14Q7QBP.1 size=1048576 2024-04-17T04:44:59.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.020+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~TBOEH14Q7QBP.2 size=374452 2024-04-17T04:44:59.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.020+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=TBOEH14Q7QBP tag=FWDZEU47WD1K type=write size=3520180 2024-04-17T04:44:59.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.024+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~MHJOLBVCWVHI.0 size=1048576 2024-04-17T04:44:59.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.024+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~MHJOLBVCWVHI.1 size=146180 2024-04-17T04:44:59.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.028+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=MHJOLBVCWVHI tag=HBCVGTN4MTPA type=write size=2243332 2024-04-17T04:44:59.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.032+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=J137FQBSMYFP type=del size=2053901 2024-04-17T04:44:59.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.036+0000 7fd8ef2218c0 1 < suggested update operation key=G03VLS09F3BT tag=2M7JOLNHUAS8 type=del size=2360440 2024-04-17T04:44:59.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.044+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~P01CXMVY204S.0 size=1048576 2024-04-17T04:44:59.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.044+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~P01CXMVY204S.1 size=164300 2024-04-17T04:44:59.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.044+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=P01CXMVY204S tag=YG1A509WHVRW type=write size=2261452 2024-04-17T04:44:59.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.052+0000 7fd8ef2218c0 1 < suggested update operation key=1ZTJP8DRITEK tag=TB02DK7N5FZ4 type=del size=242471 2024-04-17T04:44:59.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.052+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=ZBHTIXGUYE1I type=del size=941041 2024-04-17T04:44:59.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.056+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=P4Z2VH5L22LB type=del 2024-04-17T04:44:59.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.060+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=SNZ160VSUKMN type=write size=961617 2024-04-17T04:44:59.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.064+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=UQHQUBRXP90L type=del 2024-04-17T04:44:59.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.068+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=CZ2TFNCS7YMN type=del size=2862414 2024-04-17T04:44:59.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.072+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=2T667AKJRDL8 type=del 2024-04-17T04:44:59.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.076+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=8NNB2IF6SEB1 type=del size=3671141 2024-04-17T04:44:59.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.080+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=PVPIRR00LXNS type=del 2024-04-17T04:44:59.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.084+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=AVXLFM0UEATY type=del size=3598018 2024-04-17T04:44:59.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~AXGNB0DII4GS.0 size=1048576 2024-04-17T04:44:59.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~AXGNB0DII4GS.1 size=1048576 2024-04-17T04:44:59.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.092+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~AXGNB0DII4GS.2 size=508306 2024-04-17T04:44:59.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.100+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=AXGNB0DII4GS tag=P6YI0P4YTVBE type=write size=3654034 2024-04-17T04:44:59.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~RXSHFLLIR3DY.0 size=1048576 2024-04-17T04:44:59.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.108+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~RXSHFLLIR3DY.1 size=1048576 2024-04-17T04:44:59.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.108+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~RXSHFLLIR3DY.2 size=605386 2024-04-17T04:44:59.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.108+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=RXSHFLLIR3DY tag=L1N9T2570HKR type=write size=3751114 2024-04-17T04:44:59.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.112+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=BXD4IE7GE3V5 type=del 2024-04-17T04:44:59.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.116+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=IZM3BC5Q3XUR type=del size=685079 2024-04-17T04:44:59.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.120+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=OPNYT5Z52SJJ type=del 2024-04-17T04:44:59.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~AEDGKMF4M9YC.0 size=1048576 2024-04-17T04:44:59.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.128+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~AEDGKMF4M9YC.1 size=717674 2024-04-17T04:44:59.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.128+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=AEDGKMF4M9YC tag=BOBVWTG5PIXB type=write size=2814826 2024-04-17T04:44:59.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.136+0000 7fd8ef2218c0 1 < suggested remove operation key=JVWRJBMFIRLL tag=H1VMIQE91LQ2 type=write 2024-04-17T04:44:59.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.140+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=QUQG0PZLF9FP type=write size=1031123 2024-04-17T04:44:59.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.144+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=H1WKSPWT9LM0 type=del 2024-04-17T04:44:59.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.148+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=32RVIV3U0Y4H type=del size=2673645 2024-04-17T04:44:59.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~FAIL59X99KY1.0 size=1048576 2024-04-17T04:44:59.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.156+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~FAIL59X99KY1.1 size=1048576 2024-04-17T04:44:59.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.156+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~FAIL59X99KY1.2 size=84240 2024-04-17T04:44:59.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.164+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=FAIL59X99KY1 tag=2Y4O9LL8BC90 type=write size=3229968 2024-04-17T04:44:59.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.168+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=UQ4HL6Q09FBQ type=write size=901623 2024-04-17T04:44:59.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.172+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8P4SUYLLQM41.0 size=1048576 2024-04-17T04:44:59.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8P4SUYLLQM41.1 size=290941 2024-04-17T04:44:59.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.180+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=8P4SUYLLQM41 tag=MNQ8A0LCEG16 type=write size=2388093 2024-04-17T04:44:59.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~A6TP1NGZHDTW.0 size=1048576 2024-04-17T04:44:59.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~A6TP1NGZHDTW.1 size=50691 2024-04-17T04:44:59.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.188+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=A6TP1NGZHDTW tag=CQFK7QJS9FJE type=write size=2147843 2024-04-17T04:44:59.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.192+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~LR1QKTOHLWTU.0 size=705290 2024-04-17T04:44:59.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.200+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=LR1QKTOHLWTU tag=JQXULIJP682S type=write size=1753866 2024-04-17T04:44:59.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.204+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=IW2K9BZ6YV86 type=del size=1132271 2024-04-17T04:44:59.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.208+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~TBOEH14Q7QBP.0 size=1048576 2024-04-17T04:44:59.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.212+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~TBOEH14Q7QBP.1 size=1048576 2024-04-17T04:44:59.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.212+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~TBOEH14Q7QBP.2 size=374452 2024-04-17T04:44:59.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.220+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=TBOEH14Q7QBP tag=FWDZEU47WD1K type=write size=3520180 2024-04-17T04:44:59.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.224+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=BMG2W5B4B8WR type=del size=3084470 2024-04-17T04:44:59.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.228+0000 7fd8ef2218c0 -1 < failed to complete operation key=JVWRJBMFIRLL tag=J137FQBSMYFP type=del size=2053901: (22) Invalid argument 2024-04-17T04:44:59.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.228+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=KVDX6G9T03U5 type=del size=346338 2024-04-17T04:44:59.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~P01CXMVY204S.0 size=1048576 2024-04-17T04:44:59.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.236+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~P01CXMVY204S.1 size=164300 2024-04-17T04:44:59.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.244+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=P01CXMVY204S tag=YG1A509WHVRW type=write size=2261452 2024-04-17T04:44:59.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.244+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=UGLAM8RETZXZ type=del size=753500 2024-04-17T04:44:59.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.248+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=ZBHTIXGUYE1I type=del 2024-04-17T04:44:59.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.252+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=WTXN4F74H1QI type=del size=1766924 2024-04-17T04:44:59.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.256+0000 7fd8ef2218c0 1 < completed write operation key=EWAW6EBW6W6U tag=SNZ160VSUKMN type=write size=961617 2024-04-17T04:44:59.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.264+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~1SEQ64OLI5H1.0 size=1048576 2024-04-17T04:44:59.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.264+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~1SEQ64OLI5H1.1 size=21026 2024-04-17T04:44:59.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.264+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=1SEQ64OLI5H1 tag=M5DKSTR1UGPW type=write size=2118178 2024-04-17T04:44:59.268 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.272+0000 7fd8ef2218c0 1 < suggested update operation key=EWAW6EBW6W6U tag=CZ2TFNCS7YMN type=del size=2862414 2024-04-17T04:44:59.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.276+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=0TIV2OVYU6VI type=del size=2408573 2024-04-17T04:44:59.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.280+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=8NNB2IF6SEB1 type=del 2024-04-17T04:44:59.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~DV268H1OOD1D.0 size=1048576 2024-04-17T04:44:59.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~DV268H1OOD1D.1 size=1048576 2024-04-17T04:44:59.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.292+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~DV268H1OOD1D.2 size=273392 2024-04-17T04:44:59.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.292+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=DV268H1OOD1D tag=9SQ1QPSW62G1 type=write size=3419120 2024-04-17T04:44:59.288 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.292+0000 7fd8ef2218c0 -1 < failed to complete operation key=JVWRJBMFIRLL tag=AVXLFM0UEATY type=del size=3598018: (22) Invalid argument 2024-04-17T04:44:59.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.296+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=MM9C0QOOZ32G type=write size=788265 2024-04-17T04:44:59.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.300+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~RXSHFLLIR3DY.0 size=1048576 2024-04-17T04:44:59.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~RXSHFLLIR3DY.1 size=1048576 2024-04-17T04:44:59.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~RXSHFLLIR3DY.2 size=605386 2024-04-17T04:44:59.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.312+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=RXSHFLLIR3DY tag=L1N9T2570HKR type=write size=3751114 2024-04-17T04:44:59.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.316+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~5RQRP851FSF0.0 size=482702 2024-04-17T04:44:59.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.316+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=5RQRP851FSF0 tag=HQP0IJB9B4QF type=write size=1531278 2024-04-17T04:44:59.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.320+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=IZM3BC5Q3XUR type=del 2024-04-17T04:44:59.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~SSR9EJAJS9WC.0 size=358079 2024-04-17T04:44:59.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.324+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=SSR9EJAJS9WC tag=JUVO05APKRLO type=write size=1406655 2024-04-17T04:44:59.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.328+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~AEDGKMF4M9YC.0 size=1048576 2024-04-17T04:44:59.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.332+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~AEDGKMF4M9YC.1 size=717674 2024-04-17T04:44:59.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.336+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=AEDGKMF4M9YC tag=BOBVWTG5PIXB type=write size=2814826 2024-04-17T04:44:59.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.340+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=5F81I19XIJE8 type=del size=3425495 2024-04-17T04:44:59.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.340+0000 7fd8ef2218c0 1 < completed write operation key=R3C8PAVPVGBY tag=QUQG0PZLF9FP type=write size=1031123 2024-04-17T04:44:59.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~7JB6ZU63CVOX.0 size=1048576 2024-04-17T04:44:59.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~7JB6ZU63CVOX.1 size=500969 2024-04-17T04:44:59.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.348+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=7JB6ZU63CVOX tag=FIDV35HURAF8 type=write size=2598121 2024-04-17T04:44:59.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.352+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=32RVIV3U0Y4H type=del 2024-04-17T04:44:59.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.356+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=GRXVXIYPTXFC type=del size=769034 2024-04-17T04:44:59.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.360+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=UQ4HL6Q09FBQ type=write size=901623 2024-04-17T04:44:59.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.360+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=LGX05ZLEUUH6 type=del size=45352 2024-04-17T04:44:59.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.364+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~A6TP1NGZHDTW.0 size=1048576 2024-04-17T04:44:59.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.368+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~A6TP1NGZHDTW.1 size=50691 2024-04-17T04:44:59.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.372+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=A6TP1NGZHDTW tag=CQFK7QJS9FJE type=write size=2147843 2024-04-17T04:44:59.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.376+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~G7NY3R4LSSAU.0 size=1048576 2024-04-17T04:44:59.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.380+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~G7NY3R4LSSAU.1 size=1048576 2024-04-17T04:44:59.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.380+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~G7NY3R4LSSAU.2 size=26742 2024-04-17T04:44:59.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.380+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=G7NY3R4LSSAU tag=RZ6HXIKGUAFO type=write size=3172470 2024-04-17T04:44:59.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.384+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=IW2K9BZ6YV86 type=del 2024-04-17T04:44:59.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~JLDM18ZN35PY.0 size=1048576 2024-04-17T04:44:59.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~JLDM18ZN35PY.1 size=1048576 2024-04-17T04:44:59.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~JLDM18ZN35PY.2 size=59796 2024-04-17T04:44:59.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.392+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=JLDM18ZN35PY tag=19CEPTSSFCKH type=write size=3205524 2024-04-17T04:44:59.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.396+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=BMG2W5B4B8WR type=del 2024-04-17T04:44:59.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.400+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=FD5MJO2ZK0I3 type=del size=525068 2024-04-17T04:44:59.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.400+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=KVDX6G9T03U5 type=del 2024-04-17T04:44:59.399 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.404+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=9T6AB7Q69EQ4 type=del size=3757227 2024-04-17T04:44:59.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.408+0000 7fd8ef2218c0 1 < suggested update operation key=UYCKMXE6LBUH tag=UGLAM8RETZXZ type=del size=753500 2024-04-17T04:44:59.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.412+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=7R8A91CKP3D2 type=del size=3101042 2024-04-17T04:44:59.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.412+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=WTXN4F74H1QI type=del 2024-04-17T04:44:59.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.416+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~ZZEH4HHB9PK1.0 size=538568 2024-04-17T04:44:59.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.416+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=ZZEH4HHB9PK1 tag=IXXBMWU62HU0 type=write size=1587144 2024-04-17T04:44:59.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~1SEQ64OLI5H1.0 size=1048576 2024-04-17T04:44:59.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~1SEQ64OLI5H1.1 size=21026 2024-04-17T04:44:59.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.424+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=1SEQ64OLI5H1 tag=M5DKSTR1UGPW type=write size=2118178 2024-04-17T04:44:59.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.428+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=Z3HQ2CQI4OX9 type=del size=3897513 2024-04-17T04:44:59.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.432+0000 7fd8ef2218c0 1 < suggested remove operation key=MS3T476JOEON tag=0TIV2OVYU6VI type=del 2024-04-17T04:44:59.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.436+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=A85BU9L7LTFC type=write size=750574 2024-04-17T04:44:59.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.436+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~DV268H1OOD1D.0 size=1048576 2024-04-17T04:44:59.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~DV268H1OOD1D.1 size=1048576 2024-04-17T04:44:59.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~DV268H1OOD1D.2 size=273392 2024-04-17T04:44:59.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.444+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=DV268H1OOD1D tag=9SQ1QPSW62G1 type=write size=3419120 2024-04-17T04:44:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.448+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=BO7FWZR8738H type=del size=2868250 2024-04-17T04:44:59.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.452+0000 7fd8ef2218c0 1 < completed write operation key=AE4CYL334ONO tag=MM9C0QOOZ32G type=write size=788265 2024-04-17T04:44:59.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~DRF590C6XK83.0 size=1048576 2024-04-17T04:44:59.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~DRF590C6XK83.1 size=698897 2024-04-17T04:44:59.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.456+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=DRF590C6XK83 tag=4YLEP5W8YTCI type=write size=2796049 2024-04-17T04:44:59.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.460+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~5RQRP851FSF0.0 size=482702 2024-04-17T04:44:59.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.464+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=5RQRP851FSF0 tag=HQP0IJB9B4QF type=write size=1531278 2024-04-17T04:44:59.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.468+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~Z0V1EFA9OL6R.0 size=1048576 2024-04-17T04:44:59.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.472+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~Z0V1EFA9OL6R.1 size=1048576 2024-04-17T04:44:59.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.476+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~Z0V1EFA9OL6R.2 size=882987 2024-04-17T04:44:59.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.476+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=Z0V1EFA9OL6R tag=JESY95R68NCD type=write size=4028715 2024-04-17T04:44:59.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~SSR9EJAJS9WC.0 size=358079 2024-04-17T04:44:59.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.484+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=SSR9EJAJS9WC tag=JUVO05APKRLO type=write size=1406655 2024-04-17T04:44:59.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~DP0TMJ8Q81VB.0 size=1048576 2024-04-17T04:44:59.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~DP0TMJ8Q81VB.1 size=1048576 2024-04-17T04:44:59.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.492+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~DP0TMJ8Q81VB.2 size=522136 2024-04-17T04:44:59.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.492+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=DP0TMJ8Q81VB tag=57GIUN1L0VSX type=write size=3667864 2024-04-17T04:44:59.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.492+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=5F81I19XIJE8 type=del 2024-04-17T04:44:59.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.496+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=K1J33EIUSCO7 type=del size=2365705 2024-04-17T04:44:59.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~7JB6ZU63CVOX.0 size=1048576 2024-04-17T04:44:59.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~7JB6ZU63CVOX.1 size=500969 2024-04-17T04:44:59.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.504+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=7JB6ZU63CVOX tag=FIDV35HURAF8 type=write size=2598121 2024-04-17T04:44:59.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~0ZLCBG5S92DT.0 size=371140 2024-04-17T04:44:59.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=0ZLCBG5S92DT tag=SN79P9QMVJ5G type=write size=1419716 2024-04-17T04:44:59.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.512+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=GRXVXIYPTXFC type=del 2024-04-17T04:44:59.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.516+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=B5TW4CDNHGD5 type=write size=742930 2024-04-17T04:44:59.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.520+0000 7fd8ef2218c0 1 < suggested update operation key=3OFQH4KVYN18 tag=LGX05ZLEUUH6 type=del size=45352 2024-04-17T04:44:59.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.524+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=M0DZLFW28JTB type=del size=908541 2024-04-17T04:44:59.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~G7NY3R4LSSAU.0 size=1048576 2024-04-17T04:44:59.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~G7NY3R4LSSAU.1 size=1048576 2024-04-17T04:44:59.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.532+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~G7NY3R4LSSAU.2 size=26742 2024-04-17T04:44:59.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.536+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=G7NY3R4LSSAU tag=RZ6HXIKGUAFO type=write size=3172470 2024-04-17T04:44:59.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.540+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=MH2TY5322903 type=del size=1601399 2024-04-17T04:44:59.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~JLDM18ZN35PY.0 size=1048576 2024-04-17T04:44:59.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~JLDM18ZN35PY.1 size=1048576 2024-04-17T04:44:59.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~JLDM18ZN35PY.2 size=59796 2024-04-17T04:44:59.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.552+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=JLDM18ZN35PY tag=19CEPTSSFCKH type=write size=3205524 2024-04-17T04:44:59.550 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.552+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=1HII7EN2UQHQ type=del size=2890936 2024-04-17T04:44:59.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.556+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=FD5MJO2ZK0I3 type=del 2024-04-17T04:44:59.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.560+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=YYZ8YGZF3ZRP type=del size=1181609 2024-04-17T04:44:59.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.560+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=9T6AB7Q69EQ4 type=del 2024-04-17T04:44:59.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.564+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=ORZBMVZDA8V8 type=del size=1832998 2024-04-17T04:44:59.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.568+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=7R8A91CKP3D2 type=del 2024-04-17T04:44:59.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~CMBFG0CCWVEL.0 size=1048576 2024-04-17T04:44:59.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~CMBFG0CCWVEL.1 size=1048576 2024-04-17T04:44:59.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~CMBFG0CCWVEL.2 size=21430 2024-04-17T04:44:59.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.580+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=CMBFG0CCWVEL tag=SNQ9YZJUQFO0 type=write size=3167158 2024-04-17T04:44:59.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.580+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~ZZEH4HHB9PK1.0 size=538568 2024-04-17T04:44:59.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.588+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=ZZEH4HHB9PK1 tag=IXXBMWU62HU0 type=write size=1587144 2024-04-17T04:44:59.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.588+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=AGOHUQR3Y3O0 type=write size=168165 2024-04-17T04:44:59.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.592+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=Z3HQ2CQI4OX9 type=del 2024-04-17T04:44:59.591 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.596+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=ZG2MM93QO4Z6 type=del size=3799158 2024-04-17T04:44:59.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.600+0000 7fd8ef2218c0 1 < completed write operation key=2IY8DB37FI9H tag=A85BU9L7LTFC type=write size=750574 2024-04-17T04:44:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.600+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=W28P63SZA5QI type=del size=3352052 2024-04-17T04:44:59.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.604+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=BO7FWZR8738H type=del 2024-04-17T04:44:59.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~5ZURY5WEMYI1.0 size=1048576 2024-04-17T04:44:59.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~5ZURY5WEMYI1.1 size=1048576 2024-04-17T04:44:59.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~5ZURY5WEMYI1.2 size=281022 2024-04-17T04:44:59.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.616+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=5ZURY5WEMYI1 tag=XK1Y0M3CXQAY type=write size=3426750 2024-04-17T04:44:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~DRF590C6XK83.0 size=1048576 2024-04-17T04:44:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.620+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UYCKMXE6LBUH.2~DRF590C6XK83.1 size=698897 2024-04-17T04:44:59.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.624+0000 7fd8ef2218c0 1 < canceled multipart upload key=UYCKMXE6LBUH upload=DRF590C6XK83 tag=4YLEP5W8YTCI type=write size=2796049 2024-04-17T04:44:59.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~GDQZXNIBL7TM.0 size=537706 2024-04-17T04:44:59.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.628+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=GDQZXNIBL7TM tag=TK9DC7TFM19A type=write size=1586282 2024-04-17T04:44:59.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~Z0V1EFA9OL6R.0 size=1048576 2024-04-17T04:44:59.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~Z0V1EFA9OL6R.1 size=1048576 2024-04-17T04:44:59.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~Z0V1EFA9OL6R.2 size=882987 2024-04-17T04:44:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.644+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=Z0V1EFA9OL6R tag=JESY95R68NCD type=write size=4028715 2024-04-17T04:44:59.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.644+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=ZZW0NWBUEWJ9 type=write size=52974 2024-04-17T04:44:59.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.648+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~DP0TMJ8Q81VB.0 size=1048576 2024-04-17T04:44:59.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.652+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~DP0TMJ8Q81VB.1 size=1048576 2024-04-17T04:44:59.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.656+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~DP0TMJ8Q81VB.2 size=522136 2024-04-17T04:44:59.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.660+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=DP0TMJ8Q81VB tag=57GIUN1L0VSX type=write size=3667864 2024-04-17T04:44:59.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.664+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=MO2QD7HYWPAU type=del size=2205472 2024-04-17T04:44:59.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.668+0000 7fd8ef2218c0 1 < suggested remove operation key=EWAW6EBW6W6U tag=K1J33EIUSCO7 type=del 2024-04-17T04:44:59.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.676+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~ENVDK1X0RCGP.0 size=1048576 2024-04-17T04:44:59.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~ENVDK1X0RCGP.1 size=971772 2024-04-17T04:44:59.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=ENVDK1X0RCGP tag=YNJ985Q9IAR1 type=write size=3068924 2024-04-17T04:44:59.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~0ZLCBG5S92DT.0 size=371140 2024-04-17T04:44:59.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.688+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=0ZLCBG5S92DT tag=SN79P9QMVJ5G type=write size=1419716 2024-04-17T04:44:59.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.692+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=WMCLD032REQY type=del size=3022130 2024-04-17T04:44:59.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.696+0000 7fd8ef2218c0 1 < completed write operation key=5FR1NEWE0LD1 tag=B5TW4CDNHGD5 type=write size=742930 2024-04-17T04:44:59.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.700+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=P2EML1RD0G50 type=del size=2999277 2024-04-17T04:44:59.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.704+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=M0DZLFW28JTB type=del 2024-04-17T04:44:59.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.708+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5JE01RBC3ZP9.0 size=1048576 2024-04-17T04:44:59.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5JE01RBC3ZP9.1 size=1048576 2024-04-17T04:44:59.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5JE01RBC3ZP9.2 size=576875 2024-04-17T04:44:59.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.716+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=5JE01RBC3ZP9 tag=BZ2K671U5AW5 type=write size=3722603 2024-04-17T04:44:59.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.720+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=MH2TY5322903 type=del 2024-04-17T04:44:59.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.720+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=RVF6L2GRB5L4 type=del size=2182543 2024-04-17T04:44:59.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.724+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=1HII7EN2UQHQ type=del 2024-04-17T04:44:59.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.728+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=PBWZDG9AUPBK type=write size=35853 2024-04-17T04:44:59.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.732+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=YYZ8YGZF3ZRP type=del 2024-04-17T04:44:59.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.736+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=CRH3YRCQINCE type=del size=3689073 2024-04-17T04:44:59.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.740+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=ORZBMVZDA8V8 type=del 2024-04-17T04:44:59.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.744+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=O3UDKATT52YW type=del size=3096954 2024-04-17T04:44:59.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~CMBFG0CCWVEL.0 size=1048576 2024-04-17T04:44:59.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~CMBFG0CCWVEL.1 size=1048576 2024-04-17T04:44:59.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.752+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~CMBFG0CCWVEL.2 size=21430 2024-04-17T04:44:59.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.760+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=CMBFG0CCWVEL tag=SNQ9YZJUQFO0 type=write size=3167158 2024-04-17T04:44:59.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.764+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=JEPEX4ANS1OR type=del size=377759 2024-04-17T04:44:59.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.768+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=AGOHUQR3Y3O0 type=write size=168165 2024-04-17T04:44:59.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.772+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=VF8G33KWC109 type=del size=3775454 2024-04-17T04:44:59.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.776+0000 7fd8ef2218c0 1 < canceled operation key=UYCKMXE6LBUH tag=ZG2MM93QO4Z6 type=del size=3799158 2024-04-17T04:44:59.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.780+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~OF61UN1LOWCD.0 size=973130 2024-04-17T04:44:59.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.780+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=OF61UN1LOWCD tag=RPX1K5BLZNH9 type=write size=2021706 2024-04-17T04:44:59.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.784+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=W28P63SZA5QI type=del 2024-04-17T04:44:59.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.788+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=2WO1CFS7NHYN type=del size=588119 2024-04-17T04:44:59.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.788+0000 7fd8ef2218c0 1 < canceled part key=_multipart_G03VLS09F3BT.2~5ZURY5WEMYI1.0 size=1048576 2024-04-17T04:44:59.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.792+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~5ZURY5WEMYI1.1 size=1048576 2024-04-17T04:44:59.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.796+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~5ZURY5WEMYI1.2 size=281022 2024-04-17T04:44:59.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.800+0000 7fd8ef2218c0 1 < canceled multipart upload key=G03VLS09F3BT upload=5ZURY5WEMYI1 tag=XK1Y0M3CXQAY type=write size=3426750 2024-04-17T04:44:59.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.804+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=XHKRNT5RJUA1 type=del size=2987702 2024-04-17T04:44:59.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~GDQZXNIBL7TM.0 size=537706 2024-04-17T04:44:59.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.812+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=GDQZXNIBL7TM tag=TK9DC7TFM19A type=write size=1586282 2024-04-17T04:44:59.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~Y3B34DSTT8ED.0 size=1048576 2024-04-17T04:44:59.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.820+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~Y3B34DSTT8ED.1 size=536551 2024-04-17T04:44:59.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.820+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=Y3B34DSTT8ED tag=O9WQPR06E0Q5 type=write size=2633703 2024-04-17T04:44:59.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.824+0000 7fd8ef2218c0 1 < completed write operation key=82D6YWKJ1TFQ tag=ZZW0NWBUEWJ9 type=write size=52974 2024-04-17T04:44:59.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.828+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=I2T5R8EMIZQX type=del size=3207560 2024-04-17T04:44:59.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.832+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=MO2QD7HYWPAU type=del 2024-04-17T04:44:59.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.832+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=U9HMNSRMURW9 type=del size=2655532 2024-04-17T04:44:59.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.836+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~ENVDK1X0RCGP.0 size=1048576 2024-04-17T04:44:59.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.840+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~ENVDK1X0RCGP.1 size=971772 2024-04-17T04:44:59.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.844+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=ENVDK1X0RCGP tag=YNJ985Q9IAR1 type=write size=3068924 2024-04-17T04:44:59.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.848+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=MUJ0TEI44SG5 type=del size=907565 2024-04-17T04:44:59.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.852+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=WMCLD032REQY type=del 2024-04-17T04:44:59.853 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.856+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~S9RQ49OJW3AS.0 size=1048576 2024-04-17T04:44:59.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~S9RQ49OJW3AS.1 size=1048576 2024-04-17T04:44:59.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~S9RQ49OJW3AS.2 size=984372 2024-04-17T04:44:59.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.864+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=S9RQ49OJW3AS tag=U2OZCJBH7KSX type=write size=4130100 2024-04-17T04:44:59.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.868+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=P2EML1RD0G50 type=del 2024-04-17T04:44:59.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.868+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=CRFEBRO4GVI0 type=del size=2006116 2024-04-17T04:44:59.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.872+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~5JE01RBC3ZP9.0 size=1048576 2024-04-17T04:44:59.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.872+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5JE01RBC3ZP9.1 size=1048576 2024-04-17T04:44:59.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5JE01RBC3ZP9.2 size=576875 2024-04-17T04:44:59.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.880+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=5JE01RBC3ZP9 tag=BZ2K671U5AW5 type=write size=3722603 2024-04-17T04:44:59.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.888+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~9FBKXXC3KAMU.0 size=1048576 2024-04-17T04:44:59.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.888+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~9FBKXXC3KAMU.1 size=526814 2024-04-17T04:44:59.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.888+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=9FBKXXC3KAMU tag=YJ5JV37OHGAW type=write size=2623966 2024-04-17T04:44:59.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.892+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=RVF6L2GRB5L4 type=del 2024-04-17T04:44:59.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.896+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=DSML2ZENMASC type=del size=2624742 2024-04-17T04:44:59.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.900+0000 7fd8ef2218c0 1 < canceled operation key=WCIJICR0FT2C tag=PBWZDG9AUPBK type=write size=35853 2024-04-17T04:44:59.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~MNXNEPQ7JLDN.0 size=832013 2024-04-17T04:44:59.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.908+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=MNXNEPQ7JLDN tag=HGUHDSK4MI49 type=write size=1880589 2024-04-17T04:44:59.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.912+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=CRH3YRCQINCE type=del 2024-04-17T04:44:59.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.912+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=71KMF1V4R5IZ type=write size=270008 2024-04-17T04:44:59.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.916+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=O3UDKATT52YW type=del 2024-04-17T04:44:59.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.920+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=007GI101QUOU type=del size=1898408 2024-04-17T04:44:59.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.924+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=JEPEX4ANS1OR type=del 2024-04-17T04:44:59.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~EGAXGHSFJ8VD.0 size=1048576 2024-04-17T04:44:59.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~EGAXGHSFJ8VD.1 size=1048576 2024-04-17T04:44:59.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~EGAXGHSFJ8VD.2 size=448331 2024-04-17T04:44:59.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.936+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=EGAXGHSFJ8VD tag=Y1PV1825FC09 type=write size=3594059 2024-04-17T04:44:59.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.940+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=VF8G33KWC109 type=del 2024-04-17T04:44:59.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.944+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=50NC63N59AGL type=del size=1797310 2024-04-17T04:44:59.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~OF61UN1LOWCD.0 size=973130 2024-04-17T04:44:59.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.952+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=OF61UN1LOWCD tag=RPX1K5BLZNH9 type=write size=2021706 2024-04-17T04:44:59.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.956+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=J2CU5JN8RYIY type=del size=2020106 2024-04-17T04:44:59.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.960+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=2WO1CFS7NHYN type=del 2024-04-17T04:44:59.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.964+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~3XPI8RYMAISP.0 size=1048576 2024-04-17T04:44:59.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~3XPI8RYMAISP.1 size=61141 2024-04-17T04:44:59.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.968+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=3XPI8RYMAISP tag=IHA18Z7XSX23 type=write size=2158293 2024-04-17T04:44:59.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.972+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=XHKRNT5RJUA1 type=del 2024-04-17T04:44:59.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.976+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~ZMLFRGZAMPAX.0 size=1048576 2024-04-17T04:44:59.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~ZMLFRGZAMPAX.1 size=1048576 2024-04-17T04:44:59.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~ZMLFRGZAMPAX.2 size=186129 2024-04-17T04:44:59.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.980+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=ZMLFRGZAMPAX tag=LQLM5X2G2J7L type=write size=3331857 2024-04-17T04:44:59.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.980+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~Y3B34DSTT8ED.0 size=1048576 2024-04-17T04:44:59.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.984+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~Y3B34DSTT8ED.1 size=536551 2024-04-17T04:44:59.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.988+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=Y3B34DSTT8ED tag=O9WQPR06E0Q5 type=write size=2633703 2024-04-17T04:44:59.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.992+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=A3G4553996JB type=del size=1270320 2024-04-17T04:44:59.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:44:59.996+0000 7fd8ef2218c0 -1 < failed to complete operation key=M3U8L7M8JKG9 tag=I2T5R8EMIZQX type=del size=3207560: (22) Invalid argument 2024-04-17T04:44:59.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~V38SB16HKVTA.0 size=1048576 2024-04-17T04:44:59.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~V38SB16HKVTA.1 size=1048576 2024-04-17T04:45:00.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~V38SB16HKVTA.2 size=191136 2024-04-17T04:45:00.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=V38SB16HKVTA tag=LZEPFD91ZK68 type=write size=3336864 2024-04-17T04:45:00.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.008+0000 7fd8ef2218c0 1 < canceled operation key=B63EZS0LAC30 tag=U9HMNSRMURW9 type=del size=2655532 2024-04-17T04:45:00.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~TDONREPQYZKG.0 size=1048576 2024-04-17T04:45:00.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~TDONREPQYZKG.1 size=960516 2024-04-17T04:45:00.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.016+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=TDONREPQYZKG tag=JMF52RQ53Q0L type=write size=3057668 2024-04-17T04:45:00.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.016+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=MUJ0TEI44SG5 type=del 2024-04-17T04:45:00.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~C57CPPJW4V45.0 size=1048576 2024-04-17T04:45:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~C57CPPJW4V45.1 size=2584 2024-04-17T04:45:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.024+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=C57CPPJW4V45 tag=1LLPD4VZQM5G type=write size=2099736 2024-04-17T04:45:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~S9RQ49OJW3AS.0 size=1048576 2024-04-17T04:45:00.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~S9RQ49OJW3AS.1 size=1048576 2024-04-17T04:45:00.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.032+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~S9RQ49OJW3AS.2 size=984372 2024-04-17T04:45:00.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.036+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=S9RQ49OJW3AS tag=U2OZCJBH7KSX type=write size=4130100 2024-04-17T04:45:00.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.040+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=I9C0OEOGOLFM type=del size=3121988 2024-04-17T04:45:00.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.044+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=CRFEBRO4GVI0 type=del 2024-04-17T04:45:00.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.048+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=8ZJZCGZ49866 type=del size=2862561 2024-04-17T04:45:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.048+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~9FBKXXC3KAMU.0 size=1048576 2024-04-17T04:45:00.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~9FBKXXC3KAMU.1 size=526814 2024-04-17T04:45:00.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.060+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=9FBKXXC3KAMU tag=YJ5JV37OHGAW type=write size=2623966 2024-04-17T04:45:00.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.064+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=U04ZX89NP2Y1 type=del size=2800108 2024-04-17T04:45:00.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.068+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=DSML2ZENMASC type=del 2024-04-17T04:45:00.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~404VRFYEYMWT.0 size=315772 2024-04-17T04:45:00.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.072+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=404VRFYEYMWT tag=EIN4BCKOCCFG type=write size=1364348 2024-04-17T04:45:00.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.076+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~MNXNEPQ7JLDN.0 size=832013 2024-04-17T04:45:00.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.080+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=MNXNEPQ7JLDN tag=HGUHDSK4MI49 type=write size=1880589 2024-04-17T04:45:00.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.084+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=1KYK5RWA1FG2 type=del size=11811 2024-04-17T04:45:00.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.088+0000 7fd8ef2218c0 1 < suggested remove operation key=EWAW6EBW6W6U tag=71KMF1V4R5IZ type=write 2024-04-17T04:45:00.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.096+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~5GRF1I1GCYEC.0 size=1048576 2024-04-17T04:45:00.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~5GRF1I1GCYEC.1 size=645418 2024-04-17T04:45:00.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.100+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=5GRF1I1GCYEC tag=26B5906QUJMP type=write size=2742570 2024-04-17T04:45:00.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.104+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=007GI101QUOU type=del 2024-04-17T04:45:00.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~V4Y6J0I05D2J.0 size=1048576 2024-04-17T04:45:00.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~V4Y6J0I05D2J.1 size=696868 2024-04-17T04:45:00.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.112+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=V4Y6J0I05D2J tag=AQMGE6L68I6S type=write size=2794020 2024-04-17T04:45:00.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.116+0000 7fd8ef2218c0 1 < canceled part key=_multipart_OZ1IST8DOGXW.2~EGAXGHSFJ8VD.0 size=1048576 2024-04-17T04:45:00.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~EGAXGHSFJ8VD.1 size=1048576 2024-04-17T04:45:00.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~EGAXGHSFJ8VD.2 size=448331 2024-04-17T04:45:00.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.124+0000 7fd8ef2218c0 1 < canceled multipart upload key=OZ1IST8DOGXW upload=EGAXGHSFJ8VD tag=Y1PV1825FC09 type=write size=3594059 2024-04-17T04:45:00.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~APL30ZXULZWQ.0 size=225888 2024-04-17T04:45:00.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.132+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=APL30ZXULZWQ tag=135SHYH8UGFC type=write size=1274464 2024-04-17T04:45:00.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.136+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=50NC63N59AGL type=del 2024-04-17T04:45:00.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.136+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=3HG3V1NQO180 type=write size=589927 2024-04-17T04:45:00.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.144+0000 7fd8ef2218c0 1 < suggested remove operation key=1ZTJP8DRITEK tag=J2CU5JN8RYIY type=del 2024-04-17T04:45:00.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.148+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=WW661UHNZFNB type=write size=314041 2024-04-17T04:45:00.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~3XPI8RYMAISP.0 size=1048576 2024-04-17T04:45:00.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~3XPI8RYMAISP.1 size=61141 2024-04-17T04:45:00.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.160+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=3XPI8RYMAISP tag=IHA18Z7XSX23 type=write size=2158293 2024-04-17T04:45:00.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.164+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=WHVWSPYH9A23 type=write size=66926 2024-04-17T04:45:00.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.168+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~ZMLFRGZAMPAX.0 size=1048576 2024-04-17T04:45:00.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.168+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~ZMLFRGZAMPAX.1 size=1048576 2024-04-17T04:45:00.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.172+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~ZMLFRGZAMPAX.2 size=186129 2024-04-17T04:45:00.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.176+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=ZMLFRGZAMPAX tag=LQLM5X2G2J7L type=write size=3331857 2024-04-17T04:45:00.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.180+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=RWYZZ6KEPDPT type=write size=4073 2024-04-17T04:45:00.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.188+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=A3G4553996JB type=del 2024-04-17T04:45:00.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.192+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=QM3ZFE5LXC39 type=del size=1360606 2024-04-17T04:45:00.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~V38SB16HKVTA.0 size=1048576 2024-04-17T04:45:00.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~V38SB16HKVTA.1 size=1048576 2024-04-17T04:45:00.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~V38SB16HKVTA.2 size=191136 2024-04-17T04:45:00.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.204+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=V38SB16HKVTA tag=LZEPFD91ZK68 type=write size=3336864 2024-04-17T04:45:00.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.212+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~XFLFNPZTMTIU.0 size=349076 2024-04-17T04:45:00.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.212+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=XFLFNPZTMTIU tag=CWH5GE32C4D8 type=write size=1397652 2024-04-17T04:45:00.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.216+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~TDONREPQYZKG.0 size=1048576 2024-04-17T04:45:00.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.220+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~TDONREPQYZKG.1 size=960516 2024-04-17T04:45:00.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.224+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=TDONREPQYZKG tag=JMF52RQ53Q0L type=write size=3057668 2024-04-17T04:45:00.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.228+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=J7X0AHA9B812 type=del size=3288667 2024-04-17T04:45:00.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~C57CPPJW4V45.0 size=1048576 2024-04-17T04:45:00.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~C57CPPJW4V45.1 size=2584 2024-04-17T04:45:00.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.240+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=C57CPPJW4V45 tag=1LLPD4VZQM5G type=write size=2099736 2024-04-17T04:45:00.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.240+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=AICLSGRADZNU type=del size=2519245 2024-04-17T04:45:00.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.248+0000 7fd8ef2218c0 1 < suggested update operation key=2IY8DB37FI9H tag=I9C0OEOGOLFM type=del size=3121988 2024-04-17T04:45:00.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.252+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=EVWYATWEI02Z type=del size=3682744 2024-04-17T04:45:00.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.256+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=8ZJZCGZ49866 type=del 2024-04-17T04:45:00.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.260+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~OSSKID47848J.0 size=530258 2024-04-17T04:45:00.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.260+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=OSSKID47848J tag=NDHOAK7BHXRI type=write size=1578834 2024-04-17T04:45:00.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.268+0000 7fd8ef2218c0 1 < suggested update operation key=BCG05NIQFA53 tag=U04ZX89NP2Y1 type=del size=2800108 2024-04-17T04:45:00.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.272+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=OSL7GSEPEPWN type=del size=2764319 2024-04-17T04:45:00.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~404VRFYEYMWT.0 size=315772 2024-04-17T04:45:00.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.280+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=404VRFYEYMWT tag=EIN4BCKOCCFG type=write size=1364348 2024-04-17T04:45:00.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.284+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=B7CSN6FOJ0UR type=del size=2930394 2024-04-17T04:45:00.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.288+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=1KYK5RWA1FG2 type=del 2024-04-17T04:45:00.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.292+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=9BW8SP3MASR7 type=del size=3211659 2024-04-17T04:45:00.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~5GRF1I1GCYEC.0 size=1048576 2024-04-17T04:45:00.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~5GRF1I1GCYEC.1 size=645418 2024-04-17T04:45:00.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.304+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=5GRF1I1GCYEC tag=26B5906QUJMP type=write size=2742570 2024-04-17T04:45:00.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~OL9M3MELX4CW.0 size=1048576 2024-04-17T04:45:00.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.312+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~OL9M3MELX4CW.1 size=718219 2024-04-17T04:45:00.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.312+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=OL9M3MELX4CW tag=N72Z3OZAY35K type=write size=2815371 2024-04-17T04:45:00.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~V4Y6J0I05D2J.0 size=1048576 2024-04-17T04:45:00.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~V4Y6J0I05D2J.1 size=696868 2024-04-17T04:45:00.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.324+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=V4Y6J0I05D2J tag=AQMGE6L68I6S type=write size=2794020 2024-04-17T04:45:00.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~UBH1IAB190SH.0 size=1048576 2024-04-17T04:45:00.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~UBH1IAB190SH.1 size=1048576 2024-04-17T04:45:00.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.336+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~UBH1IAB190SH.2 size=673231 2024-04-17T04:45:00.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.336+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=UBH1IAB190SH tag=JKS5Q2U8BCYH type=write size=3818959 2024-04-17T04:45:00.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.336+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~APL30ZXULZWQ.0 size=225888 2024-04-17T04:45:00.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.340+0000 7fd8ef2218c0 1 < canceled multipart upload key=6WSZ5UL0V1Y1 upload=APL30ZXULZWQ tag=135SHYH8UGFC type=write size=1274464 2024-04-17T04:45:00.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~E0ARMEZAM0M2.0 size=276452 2024-04-17T04:45:00.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.348+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=E0ARMEZAM0M2 tag=13OJJ39UQ84P type=write size=1325028 2024-04-17T04:45:00.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.352+0000 7fd8ef2218c0 1 < suggested update operation key=A5ZUB3YK06J3 tag=3HG3V1NQO180 type=write size=589927 2024-04-17T04:45:00.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.356+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=VSJZT8AYVFMZ type=del size=1869383 2024-04-17T04:45:00.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.364+0000 7fd8ef2218c0 1 < suggested update operation key=WCIJICR0FT2C tag=WW661UHNZFNB type=write size=314041 2024-04-17T04:45:00.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.368+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=L5JWK42I704C type=write size=356056 2024-04-17T04:45:00.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.372+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=WHVWSPYH9A23 type=write size=66926 2024-04-17T04:45:00.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.376+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=11GSFXJKMOCN type=write size=960694 2024-04-17T04:45:00.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.380+0000 7fd8ef2218c0 1 < canceled operation key=N11Y8FPN9XES tag=RWYZZ6KEPDPT type=write size=4073 2024-04-17T04:45:00.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.380+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=6Z2CQUD1JJ0D type=del size=694779 2024-04-17T04:45:00.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.384+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=QM3ZFE5LXC39 type=del 2024-04-17T04:45:00.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.388+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=IZ7PW2WF7WMF type=del size=717759 2024-04-17T04:45:00.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.392+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~XFLFNPZTMTIU.0 size=349076 2024-04-17T04:45:00.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.396+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=XFLFNPZTMTIU tag=CWH5GE32C4D8 type=write size=1397652 2024-04-17T04:45:00.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.400+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=O891USEH75KI type=write size=788287 2024-04-17T04:45:00.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.404+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=J7X0AHA9B812 type=del 2024-04-17T04:45:00.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.408+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=0N5FRYISHB15 type=del size=1776069 2024-04-17T04:45:00.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.412+0000 7fd8ef2218c0 -1 < failed to complete operation key=BCG05NIQFA53 tag=AICLSGRADZNU type=del size=2519245: (22) Invalid argument 2024-04-17T04:45:00.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.416+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=FYM9PJVI3LIW type=del size=3352196 2024-04-17T04:45:00.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.420+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=EVWYATWEI02Z type=del 2024-04-17T04:45:00.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.420+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=BHGS2Z83RGK7 type=del size=2519969 2024-04-17T04:45:00.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.424+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~OSSKID47848J.0 size=530258 2024-04-17T04:45:00.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.428+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=OSSKID47848J tag=NDHOAK7BHXRI type=write size=1578834 2024-04-17T04:45:00.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.432+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=ZUZFW0214QAT type=del size=1409085 2024-04-17T04:45:00.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.432+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=OSL7GSEPEPWN type=del 2024-04-17T04:45:00.431 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.436+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=CHAKQ7QKAD8F type=del size=2836330 2024-04-17T04:45:00.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.440+0000 7fd8ef2218c0 1 < suggested remove operation key=BCG05NIQFA53 tag=B7CSN6FOJ0UR type=del 2024-04-17T04:45:00.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~S519XNW0AEXU.0 size=509128 2024-04-17T04:45:00.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.448+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=S519XNW0AEXU tag=14VVCUH7PVCQ type=write size=1557704 2024-04-17T04:45:00.445 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.448+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=9BW8SP3MASR7 type=del 2024-04-17T04:45:00.450 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~R9TAAI5RWRJD.0 size=1048576 2024-04-17T04:45:00.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~R9TAAI5RWRJD.1 size=279584 2024-04-17T04:45:00.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.456+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=R9TAAI5RWRJD tag=Z7BRJA6NDISY type=write size=2376736 2024-04-17T04:45:00.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.460+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~OL9M3MELX4CW.0 size=1048576 2024-04-17T04:45:00.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.464+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~OL9M3MELX4CW.1 size=718219 2024-04-17T04:45:00.464 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.468+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=OL9M3MELX4CW tag=N72Z3OZAY35K type=write size=2815371 2024-04-17T04:45:00.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.472+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=XG6G6JGJW7O2 type=del size=1918511 2024-04-17T04:45:00.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~UBH1IAB190SH.0 size=1048576 2024-04-17T04:45:00.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~UBH1IAB190SH.1 size=1048576 2024-04-17T04:45:00.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.480+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~UBH1IAB190SH.2 size=673231 2024-04-17T04:45:00.481 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.484+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=UBH1IAB190SH tag=JKS5Q2U8BCYH type=write size=3818959 2024-04-17T04:45:00.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.488+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=ADZSZN1VLPW6 type=del size=1874629 2024-04-17T04:45:00.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.492+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~E0ARMEZAM0M2.0 size=276452 2024-04-17T04:45:00.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.496+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=E0ARMEZAM0M2 tag=13OJJ39UQ84P type=write size=1325028 2024-04-17T04:45:00.500 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.504+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~2UTY13WGYU9Q.0 size=1048576 2024-04-17T04:45:00.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~2UTY13WGYU9Q.1 size=1048576 2024-04-17T04:45:00.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~2UTY13WGYU9Q.2 size=285907 2024-04-17T04:45:00.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=2UTY13WGYU9Q tag=EYPCJ6TF10PH type=write size=3431635 2024-04-17T04:45:00.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.512+0000 7fd8ef2218c0 1 < canceled operation key=EWAW6EBW6W6U tag=VSJZT8AYVFMZ type=del size=1869383 2024-04-17T04:45:00.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.516+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=A7VTYA9KN9AY type=write size=636420 2024-04-17T04:45:00.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.524+0000 7fd8ef2218c0 1 < suggested remove operation key=R3C8PAVPVGBY tag=L5JWK42I704C type=write 2024-04-17T04:45:00.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.528+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=K2AWCD5VSSIF type=write size=52966 2024-04-17T04:45:00.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.528+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=11GSFXJKMOCN type=write size=960694 2024-04-17T04:45:00.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.532+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=I9NWVXF1JC1O type=del size=2221701 2024-04-17T04:45:00.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.540+0000 7fd8ef2218c0 1 < suggested update operation key=5K50O96P7OGD tag=6Z2CQUD1JJ0D type=del size=694779 2024-04-17T04:45:00.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.544+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=ZRKM9H42JNNG type=write size=145994 2024-04-17T04:45:00.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.544+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=IZ7PW2WF7WMF type=del 2024-04-17T04:45:00.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.548+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=8SG9OLOX77OC type=del size=3194203 2024-04-17T04:45:00.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.552+0000 7fd8ef2218c0 1 < completed write operation key=WCIJICR0FT2C tag=O891USEH75KI type=write size=788287 2024-04-17T04:45:00.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.556+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~ZA6T0QY6PAPH.0 size=777384 2024-04-17T04:45:00.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.556+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=ZA6T0QY6PAPH tag=3ZV7TFQYIJ61 type=write size=1825960 2024-04-17T04:45:00.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.560+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=0N5FRYISHB15 type=del 2024-04-17T04:45:00.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.564+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=IE0H5UBB3LJA type=del size=2077224 2024-04-17T04:45:00.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.568+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=FYM9PJVI3LIW type=del 2024-04-17T04:45:00.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.572+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=HIMMAGS8GLQ0 type=del size=1352316 2024-04-17T04:45:00.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.576+0000 7fd8ef2218c0 1 < suggested remove operation key=OZ1IST8DOGXW tag=BHGS2Z83RGK7 type=del 2024-04-17T04:45:00.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~3NJQG98UU4VK.0 size=458822 2024-04-17T04:45:00.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.580+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=3NJQG98UU4VK tag=VG3EF5RO3YDT type=write size=1507398 2024-04-17T04:45:00.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.584+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=ZUZFW0214QAT type=del 2024-04-17T04:45:00.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.588+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=5I7O9MDF3015 type=del size=1051111 2024-04-17T04:45:00.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.588+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=CHAKQ7QKAD8F type=del 2024-04-17T04:45:00.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~5B1E6FPDWCCY.0 size=1048576 2024-04-17T04:45:00.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~5B1E6FPDWCCY.1 size=653372 2024-04-17T04:45:00.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=5B1E6FPDWCCY tag=J8E42UDKOXEF type=write size=2750524 2024-04-17T04:45:00.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.596+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~S519XNW0AEXU.0 size=509128 2024-04-17T04:45:00.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.604+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=S519XNW0AEXU tag=14VVCUH7PVCQ type=write size=1557704 2024-04-17T04:45:00.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.604+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=24UA5X0WS5UP type=write size=279994 2024-04-17T04:45:00.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~R9TAAI5RWRJD.0 size=1048576 2024-04-17T04:45:00.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.612+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~R9TAAI5RWRJD.1 size=279584 2024-04-17T04:45:00.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.616+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=R9TAAI5RWRJD tag=Z7BRJA6NDISY type=write size=2376736 2024-04-17T04:45:00.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.620+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~RPL40IQ2O1QI.0 size=1048576 2024-04-17T04:45:00.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~RPL40IQ2O1QI.1 size=1048576 2024-04-17T04:45:00.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~RPL40IQ2O1QI.2 size=737492 2024-04-17T04:45:00.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.628+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=RPL40IQ2O1QI tag=HFTKDYM39FIA type=write size=3883220 2024-04-17T04:45:00.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.628+0000 7fd8ef2218c0 1 < canceled operation key=2IY8DB37FI9H tag=XG6G6JGJW7O2 type=del size=1918511 2024-04-17T04:45:00.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.636+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~EPU9EAAECNP3.0 size=1048576 2024-04-17T04:45:00.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.636+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~EPU9EAAECNP3.1 size=837041 2024-04-17T04:45:00.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.636+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=EPU9EAAECNP3 tag=HFXTK93EGTOO type=write size=2934193 2024-04-17T04:45:00.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.640+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=ADZSZN1VLPW6 type=del 2024-04-17T04:45:00.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~4LY0E94AGQEE.0 size=1048576 2024-04-17T04:45:00.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.648+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~4LY0E94AGQEE.1 size=973392 2024-04-17T04:45:00.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.648+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=4LY0E94AGQEE tag=G3G3KFHU7VCH type=write size=3070544 2024-04-17T04:45:00.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.652+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~2UTY13WGYU9Q.0 size=1048576 2024-04-17T04:45:00.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.652+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~2UTY13WGYU9Q.1 size=1048576 2024-04-17T04:45:00.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.656+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~2UTY13WGYU9Q.2 size=285907 2024-04-17T04:45:00.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.664+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=2UTY13WGYU9Q tag=EYPCJ6TF10PH type=write size=3431635 2024-04-17T04:45:00.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.668+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~KD2K49AKVW0F.0 size=1048402 2024-04-17T04:45:00.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.668+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=KD2K49AKVW0F tag=I0KQI7CDO58P type=write size=2096978 2024-04-17T04:45:00.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.672+0000 7fd8ef2218c0 1 < completed write operation key=UYCKMXE6LBUH tag=A7VTYA9KN9AY type=write size=636420 2024-04-17T04:45:00.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.676+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=O20WGXAWNXW7 type=del size=1744130 2024-04-17T04:45:00.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.680+0000 7fd8ef2218c0 1 < suggested remove operation key=WCIJICR0FT2C tag=K2AWCD5VSSIF type=write 2024-04-17T04:45:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.684+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=L2NRIHOKNZMO type=del size=3987861 2024-04-17T04:45:00.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.688+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=I9NWVXF1JC1O type=del 2024-04-17T04:45:00.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.688+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=NLNJWGI0IKYW type=del size=3358256 2024-04-17T04:45:00.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.692+0000 7fd8ef2218c0 1 < completed write operation key=A5ZUB3YK06J3 tag=ZRKM9H42JNNG type=write size=145994 2024-04-17T04:45:00.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~UKUQQSTGH8UR.0 size=1048576 2024-04-17T04:45:00.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~UKUQQSTGH8UR.1 size=1048576 2024-04-17T04:45:00.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~UKUQQSTGH8UR.2 size=766043 2024-04-17T04:45:00.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.704+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=UKUQQSTGH8UR tag=C8YUQAVQ89LW type=write size=3911771 2024-04-17T04:45:00.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.712+0000 7fd8ef2218c0 1 < suggested remove operation key=2S1IT3TO1BPU tag=8SG9OLOX77OC type=del 2024-04-17T04:45:00.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.712+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=LXS8HKIQDY3U type=del size=2280544 2024-04-17T04:45:00.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.716+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~ZA6T0QY6PAPH.0 size=777384 2024-04-17T04:45:00.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.720+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=ZA6T0QY6PAPH tag=3ZV7TFQYIJ61 type=write size=1825960 2024-04-17T04:45:00.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.728+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~Q5SFNDPHZ77C.0 size=1048576 2024-04-17T04:45:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.728+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~Q5SFNDPHZ77C.1 size=369040 2024-04-17T04:45:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.728+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=Q5SFNDPHZ77C tag=XLOMVEQXE51T type=write size=2466192 2024-04-17T04:45:00.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.732+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=IE0H5UBB3LJA type=del 2024-04-17T04:45:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.736+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=L6DB4XO3QTRB type=del size=2320809 2024-04-17T04:45:00.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.740+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=HIMMAGS8GLQ0 type=del 2024-04-17T04:45:00.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~YSV3CBYIPPKG.0 size=59138 2024-04-17T04:45:00.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=YSV3CBYIPPKG tag=Z2W5NQE6SZEB type=write size=1107714 2024-04-17T04:45:00.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.744+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1ZTJP8DRITEK.2~3NJQG98UU4VK.0 size=458822 2024-04-17T04:45:00.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.752+0000 7fd8ef2218c0 1 < canceled multipart upload key=1ZTJP8DRITEK upload=3NJQG98UU4VK tag=VG3EF5RO3YDT type=write size=1507398 2024-04-17T04:45:00.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.756+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=7GR22WTYH5LI type=del size=41568 2024-04-17T04:45:00.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.760+0000 7fd8ef2218c0 1 < suggested remove operation key=JVWRJBMFIRLL tag=5I7O9MDF3015 type=del 2024-04-17T04:45:00.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.764+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=ANZ37ZPYB07J type=write size=354299 2024-04-17T04:45:00.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~5B1E6FPDWCCY.0 size=1048576 2024-04-17T04:45:00.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~5B1E6FPDWCCY.1 size=653372 2024-04-17T04:45:00.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.776+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=5B1E6FPDWCCY tag=J8E42UDKOXEF type=write size=2750524 2024-04-17T04:45:00.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.776+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=III5NUU1JGX4 type=del size=2387120 2024-04-17T04:45:00.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.780+0000 7fd8ef2218c0 1 < completed write operation key=82D6YWKJ1TFQ tag=24UA5X0WS5UP type=write size=279994 2024-04-17T04:45:00.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~LBWD94RQQ9XZ.0 size=1048576 2024-04-17T04:45:00.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~LBWD94RQQ9XZ.1 size=1048576 2024-04-17T04:45:00.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~LBWD94RQQ9XZ.2 size=575328 2024-04-17T04:45:00.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=LBWD94RQQ9XZ tag=U22NKH5NONS5 type=write size=3721056 2024-04-17T04:45:00.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.792+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~RPL40IQ2O1QI.0 size=1048576 2024-04-17T04:45:00.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.796+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~RPL40IQ2O1QI.1 size=1048576 2024-04-17T04:45:00.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~RPL40IQ2O1QI.2 size=737492 2024-04-17T04:45:00.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.804+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=RPL40IQ2O1QI tag=HFTKDYM39FIA type=write size=3883220 2024-04-17T04:45:00.805 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.812+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~OI959BGNS7XW.0 size=1048576 2024-04-17T04:45:00.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.812+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~OI959BGNS7XW.1 size=1048576 2024-04-17T04:45:00.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~OI959BGNS7XW.2 size=568011 2024-04-17T04:45:00.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.816+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=OI959BGNS7XW tag=3YXORXMQLP58 type=write size=3713739 2024-04-17T04:45:00.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.816+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~EPU9EAAECNP3.0 size=1048576 2024-04-17T04:45:00.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.820+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~EPU9EAAECNP3.1 size=837041 2024-04-17T04:45:00.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.824+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=EPU9EAAECNP3 tag=HFXTK93EGTOO type=write size=2934193 2024-04-17T04:45:00.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.828+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=J03DGBUILAMF type=del size=3663473 2024-04-17T04:45:00.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.832+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~4LY0E94AGQEE.0 size=1048576 2024-04-17T04:45:00.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.836+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~4LY0E94AGQEE.1 size=973392 2024-04-17T04:45:00.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.840+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=4LY0E94AGQEE tag=G3G3KFHU7VCH type=write size=3070544 2024-04-17T04:45:00.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~3XQJBFYY94X5.0 size=644081 2024-04-17T04:45:00.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=3XQJBFYY94X5 tag=YLLDXGVWLGGQ type=write size=1692657 2024-04-17T04:45:00.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~KD2K49AKVW0F.0 size=1048402 2024-04-17T04:45:00.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.852+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=KD2K49AKVW0F tag=I0KQI7CDO58P type=write size=2096978 2024-04-17T04:45:00.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.856+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=L04NJ7BEOCTP type=del size=1917109 2024-04-17T04:45:00.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.860+0000 7fd8ef2218c0 1 < canceled operation key=GX9X0D45GC2I tag=O20WGXAWNXW7 type=del size=1744130 2024-04-17T04:45:00.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.864+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=SDJ9THQTP26Q type=del size=1037343 2024-04-17T04:45:00.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.864+0000 7fd8ef2218c0 -1 < failed to complete operation key=2S1IT3TO1BPU tag=L2NRIHOKNZMO type=del size=3987861: (22) Invalid argument 2024-04-17T04:45:00.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.868+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=R0HX2C2Z93KS type=del size=54746 2024-04-17T04:45:00.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.872+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=NLNJWGI0IKYW type=del 2024-04-17T04:45:00.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.876+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~5X4YOCNMA63T.0 size=549223 2024-04-17T04:45:00.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.876+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=5X4YOCNMA63T tag=ZZ3IQSQD4HHU type=write size=1597799 2024-04-17T04:45:00.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.876+0000 7fd8ef2218c0 1 < canceled part key=_multipart_EWAW6EBW6W6U.2~UKUQQSTGH8UR.0 size=1048576 2024-04-17T04:45:00.875 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.880+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~UKUQQSTGH8UR.1 size=1048576 2024-04-17T04:45:00.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.884+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~UKUQQSTGH8UR.2 size=766043 2024-04-17T04:45:00.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.888+0000 7fd8ef2218c0 1 < canceled multipart upload key=EWAW6EBW6W6U upload=UKUQQSTGH8UR tag=C8YUQAVQ89LW type=write size=3911771 2024-04-17T04:45:00.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.892+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=O2ON1ZFPCAX0 type=del size=2593219 2024-04-17T04:45:00.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.896+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=LXS8HKIQDY3U type=del 2024-04-17T04:45:00.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.900+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=Q7FV0HEFTIRS type=del size=3161345 2024-04-17T04:45:00.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.900+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~Q5SFNDPHZ77C.0 size=1048576 2024-04-17T04:45:00.897 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~Q5SFNDPHZ77C.1 size=369040 2024-04-17T04:45:00.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.908+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=Q5SFNDPHZ77C tag=XLOMVEQXE51T type=write size=2466192 2024-04-17T04:45:00.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.912+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=Z3LYTI7AORPO type=write size=875572 2024-04-17T04:45:00.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.916+0000 7fd8ef2218c0 -1 < failed to complete operation key=JVWRJBMFIRLL tag=L6DB4XO3QTRB type=del size=2320809: (22) Invalid argument 2024-04-17T04:45:00.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~IY5FQNUWFLPE.0 size=1048576 2024-04-17T04:45:00.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~IY5FQNUWFLPE.1 size=1048576 2024-04-17T04:45:00.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~IY5FQNUWFLPE.2 size=360419 2024-04-17T04:45:00.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.924+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=IY5FQNUWFLPE tag=7D37QZU99L6Z type=write size=3506147 2024-04-17T04:45:00.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.928+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~YSV3CBYIPPKG.0 size=59138 2024-04-17T04:45:00.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.932+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=YSV3CBYIPPKG tag=Z2W5NQE6SZEB type=write size=1107714 2024-04-17T04:45:00.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~CLOWTX2UWYQV.0 size=603071 2024-04-17T04:45:00.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.936+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=CLOWTX2UWYQV tag=EIEBIDLQKS5Y type=write size=1651647 2024-04-17T04:45:00.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.940+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=7GR22WTYH5LI type=del 2024-04-17T04:45:00.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.944+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=Y9C53PV6T468 type=del size=2202197 2024-04-17T04:45:00.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.952+0000 7fd8ef2218c0 1 < suggested update operation key=AE4CYL334ONO tag=ANZ37ZPYB07J type=write size=354299 2024-04-17T04:45:00.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.956+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=O168MBO5LQC4 type=del size=1229001 2024-04-17T04:45:00.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.960+0000 7fd8ef2218c0 1 < suggested update operation key=R3C8PAVPVGBY tag=III5NUU1JGX4 type=del size=2387120 2024-04-17T04:45:00.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.964+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=OASCY7FTUMJB type=del size=1635237 2024-04-17T04:45:00.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.968+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~LBWD94RQQ9XZ.0 size=1048576 2024-04-17T04:45:00.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~LBWD94RQQ9XZ.1 size=1048576 2024-04-17T04:45:00.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~LBWD94RQQ9XZ.2 size=575328 2024-04-17T04:45:00.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.976+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=LBWD94RQQ9XZ tag=U22NKH5NONS5 type=write size=3721056 2024-04-17T04:45:00.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~L87FOYAW935G.0 size=1048576 2024-04-17T04:45:00.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~L87FOYAW935G.1 size=798945 2024-04-17T04:45:00.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.984+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=L87FOYAW935G tag=TO947QQBIRPH type=write size=2896097 2024-04-17T04:45:00.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.988+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~OI959BGNS7XW.0 size=1048576 2024-04-17T04:45:00.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~OI959BGNS7XW.1 size=1048576 2024-04-17T04:45:00.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:00.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~OI959BGNS7XW.2 size=568011 2024-04-17T04:45:00.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.000+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=OI959BGNS7XW tag=3YXORXMQLP58 type=write size=3713739 2024-04-17T04:45:00.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.000+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=M2333TC0Y893 type=write size=789629 2024-04-17T04:45:01.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.004+0000 7fd8ef2218c0 -1 < failed to complete operation key=R3C8PAVPVGBY tag=J03DGBUILAMF type=del size=3663473: (22) Invalid argument 2024-04-17T04:45:01.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.008+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=PJZO1CIXAKUJ type=del size=2519234 2024-04-17T04:45:01.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~3XQJBFYY94X5.0 size=644081 2024-04-17T04:45:01.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.016+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=3XQJBFYY94X5 tag=YLLDXGVWLGGQ type=write size=1692657 2024-04-17T04:45:01.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.020+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=5FX0S4RKY5HD type=del size=873 2024-04-17T04:45:01.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.024+0000 7fd8ef2218c0 -1 < failed to complete operation key=R3C8PAVPVGBY tag=L04NJ7BEOCTP type=del size=1917109: (22) Invalid argument 2024-04-17T04:45:01.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.024+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=CNDCWNZWP4JP type=write size=931596 2024-04-17T04:45:01.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.028+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=SDJ9THQTP26Q type=del 2024-04-17T04:45:01.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.032+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=GF5LO8K0IDH8 type=del size=3727772 2024-04-17T04:45:01.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.036+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=R0HX2C2Z93KS type=del 2024-04-17T04:45:01.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.040+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~R210QDBMPSO1.0 size=779738 2024-04-17T04:45:01.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.040+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=R210QDBMPSO1 tag=2R4CDC1YJF65 type=write size=1828314 2024-04-17T04:45:01.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.044+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~5X4YOCNMA63T.0 size=549223 2024-04-17T04:45:01.044 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.048+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=5X4YOCNMA63T tag=ZZ3IQSQD4HHU type=write size=1597799 2024-04-17T04:45:01.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.052+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=C6L0E857G5WA type=write size=350952 2024-04-17T04:45:01.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.056+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=O2ON1ZFPCAX0 type=del 2024-04-17T04:45:01.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.064+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~ZOJLHTZ4GR2Q.0 size=1048576 2024-04-17T04:45:01.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.064+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~ZOJLHTZ4GR2Q.1 size=236460 2024-04-17T04:45:01.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.064+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=ZOJLHTZ4GR2Q tag=EMJQMQLMSCD3 type=write size=2333612 2024-04-17T04:45:01.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.068+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=Q7FV0HEFTIRS type=del 2024-04-17T04:45:01.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.072+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=BIH1JY5RUR2Q type=del size=29114 2024-04-17T04:45:01.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.076+0000 7fd8ef2218c0 -1 < failed to complete operation key=AE4CYL334ONO tag=Z3LYTI7AORPO type=write size=875572: (22) Invalid argument 2024-04-17T04:45:01.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.080+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=A64JHHLRKDDL type=del size=1473232 2024-04-17T04:45:01.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~IY5FQNUWFLPE.0 size=1048576 2024-04-17T04:45:01.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~IY5FQNUWFLPE.1 size=1048576 2024-04-17T04:45:01.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~IY5FQNUWFLPE.2 size=360419 2024-04-17T04:45:01.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.092+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=IY5FQNUWFLPE tag=7D37QZU99L6Z type=write size=3506147 2024-04-17T04:45:01.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~FSWH661BDL3K.0 size=1048576 2024-04-17T04:45:01.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~FSWH661BDL3K.1 size=1048576 2024-04-17T04:45:01.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~FSWH661BDL3K.2 size=442557 2024-04-17T04:45:01.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.104+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=FSWH661BDL3K tag=QYR7JYJY0S1K type=write size=3588285 2024-04-17T04:45:01.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~CLOWTX2UWYQV.0 size=603071 2024-04-17T04:45:01.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.112+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=CLOWTX2UWYQV tag=EIEBIDLQKS5Y type=write size=1651647 2024-04-17T04:45:01.111 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.116+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=2OA2RG79DCKP type=write size=1022808 2024-04-17T04:45:01.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.120+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=Y9C53PV6T468 type=del 2024-04-17T04:45:01.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.124+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=J2GXTBXSZBS7 type=del size=65457 2024-04-17T04:45:01.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.128+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=O168MBO5LQC4 type=del 2024-04-17T04:45:01.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.128+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=HPBHM3V6F4IS type=del size=1216122 2024-04-17T04:45:01.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.132+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=OASCY7FTUMJB type=del 2024-04-17T04:45:01.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.136+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=XA879MX7OJX2 type=del size=3250570 2024-04-17T04:45:01.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.140+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~L87FOYAW935G.0 size=1048576 2024-04-17T04:45:01.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.144+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~L87FOYAW935G.1 size=798945 2024-04-17T04:45:01.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.148+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=L87FOYAW935G tag=TO947QQBIRPH type=write size=2896097 2024-04-17T04:45:01.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.156+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~6MH2JFMQUFB8.0 size=1000811 2024-04-17T04:45:01.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.156+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=6MH2JFMQUFB8 tag=VFHDOOXQZLAF type=write size=2049387 2024-04-17T04:45:01.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.160+0000 7fd8ef2218c0 1 < canceled operation key=VUYM3JC57QRP tag=M2333TC0Y893 type=write size=789629 2024-04-17T04:45:01.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.160+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=ML4LF9FJSLUQ type=write size=316853 2024-04-17T04:45:01.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.168+0000 7fd8ef2218c0 1 < suggested update operation key=509DZ9DBJWC5 tag=PJZO1CIXAKUJ type=del size=2519234 2024-04-17T04:45:01.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.172+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=L2JPU7LQZH6O type=write size=980093 2024-04-17T04:45:01.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.176+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=5FX0S4RKY5HD type=del 2024-04-17T04:45:01.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.176+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=ZPWCKBTC7U6T type=del size=971080 2024-04-17T04:45:01.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.180+0000 7fd8ef2218c0 1 < completed write operation key=6219DPY72LWC tag=CNDCWNZWP4JP type=write size=931596 2024-04-17T04:45:01.182 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~VS6LF3RXFWOP.0 size=1048576 2024-04-17T04:45:01.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~VS6LF3RXFWOP.1 size=1048576 2024-04-17T04:45:01.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~VS6LF3RXFWOP.2 size=215992 2024-04-17T04:45:01.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.192+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=VS6LF3RXFWOP tag=1PSUMDDVRSDY type=write size=3361720 2024-04-17T04:45:01.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.196+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=GF5LO8K0IDH8 type=del 2024-04-17T04:45:01.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.200+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=CQU3WZUKO3I3 type=write size=548743 2024-04-17T04:45:01.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~R210QDBMPSO1.0 size=779738 2024-04-17T04:45:01.203 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.208+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=R210QDBMPSO1 tag=2R4CDC1YJF65 type=write size=1828314 2024-04-17T04:45:01.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.212+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=IKGQ17AOO1J1 type=del size=265333 2024-04-17T04:45:01.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.216+0000 7fd8ef2218c0 1 < suggested update operation key=5K50O96P7OGD tag=C6L0E857G5WA type=write size=350952 2024-04-17T04:45:01.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.220+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=UW3XPCVWUHBI type=del size=2314774 2024-04-17T04:45:01.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.224+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~ZOJLHTZ4GR2Q.0 size=1048576 2024-04-17T04:45:01.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.224+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~ZOJLHTZ4GR2Q.1 size=236460 2024-04-17T04:45:01.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.232+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=ZOJLHTZ4GR2Q tag=EMJQMQLMSCD3 type=write size=2333612 2024-04-17T04:45:01.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.236+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~MG2UNYF1WPUR.0 size=1048576 2024-04-17T04:45:01.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.236+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~MG2UNYF1WPUR.1 size=1007727 2024-04-17T04:45:01.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.236+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=MG2UNYF1WPUR tag=5OA15WSA91T2 type=write size=3104879 2024-04-17T04:45:01.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.240+0000 7fd8ef2218c0 1 < suggested remove operation key=7BV3V2XMLSPH tag=BIH1JY5RUR2Q type=del 2024-04-17T04:45:01.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.244+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=UWVX988EWSYV type=del size=2779469 2024-04-17T04:45:01.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.244+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=A64JHHLRKDDL type=del 2024-04-17T04:45:01.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.248+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=39XXT211R40D type=del size=3326235 2024-04-17T04:45:01.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.248+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~FSWH661BDL3K.0 size=1048576 2024-04-17T04:45:01.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.252+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~FSWH661BDL3K.1 size=1048576 2024-04-17T04:45:01.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.252+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~FSWH661BDL3K.2 size=442557 2024-04-17T04:45:01.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.256+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=FSWH661BDL3K tag=QYR7JYJY0S1K type=write size=3588285 2024-04-17T04:45:01.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.256+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=G2JFMIBB1227 type=del size=2378077 2024-04-17T04:45:01.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.260+0000 7fd8ef2218c0 1 < completed write operation key=B63EZS0LAC30 tag=2OA2RG79DCKP type=write size=1022808 2024-04-17T04:45:01.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.260+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=JP032TO0JN93 type=write size=105253 2024-04-17T04:45:01.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.264+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=J2GXTBXSZBS7 type=del 2024-04-17T04:45:01.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.268+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=CBDCKRJYNMF3 type=del size=3241829 2024-04-17T04:45:01.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.268+0000 7fd8ef2218c0 -1 < failed to complete operation key=7BV3V2XMLSPH tag=HPBHM3V6F4IS type=del size=1216122: (22) Invalid argument 2024-04-17T04:45:01.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.268+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=89FSXZYWCE34 type=del size=3612386 2024-04-17T04:45:01.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.272+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=XA879MX7OJX2 type=del 2024-04-17T04:45:01.268 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.272+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=8X4N1RW5ES3G type=del size=3426023 2024-04-17T04:45:01.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~6MH2JFMQUFB8.0 size=1000811 2024-04-17T04:45:01.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.276+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=6MH2JFMQUFB8 tag=VFHDOOXQZLAF type=write size=2049387 2024-04-17T04:45:01.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~DB0Q9AHYC8W3.0 size=323060 2024-04-17T04:45:01.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.280+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=DB0Q9AHYC8W3 tag=70EEMNP7949M type=write size=1371636 2024-04-17T04:45:01.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.280+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=ML4LF9FJSLUQ type=write size=316853 2024-04-17T04:45:01.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~C1H1M9GAGPT1.0 size=1048576 2024-04-17T04:45:01.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~C1H1M9GAGPT1.1 size=1048576 2024-04-17T04:45:01.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~C1H1M9GAGPT1.2 size=824954 2024-04-17T04:45:01.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.288+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=C1H1M9GAGPT1 tag=6PCM39IDQFQQ type=write size=3970682 2024-04-17T04:45:01.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.288+0000 7fd8ef2218c0 1 < completed write operation key=R3C8PAVPVGBY tag=L2JPU7LQZH6O type=write size=980093 2024-04-17T04:45:01.288 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.292+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~T2YX0TC8GBUS.0 size=1048576 2024-04-17T04:45:01.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.296+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~T2YX0TC8GBUS.1 size=1000940 2024-04-17T04:45:01.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.296+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=T2YX0TC8GBUS tag=77SORT4US0LU type=write size=3098092 2024-04-17T04:45:01.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.296+0000 7fd8ef2218c0 1 < suggested update operation key=B63EZS0LAC30 tag=ZPWCKBTC7U6T type=del size=971080 2024-04-17T04:45:01.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.300+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=OG8QHMB72RX9 type=del size=1718124 2024-04-17T04:45:01.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.300+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~VS6LF3RXFWOP.0 size=1048576 2024-04-17T04:45:01.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~VS6LF3RXFWOP.1 size=1048576 2024-04-17T04:45:01.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.304+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~VS6LF3RXFWOP.2 size=215992 2024-04-17T04:45:01.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.304+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=VS6LF3RXFWOP tag=1PSUMDDVRSDY type=write size=3361720 2024-04-17T04:45:01.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.308+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=39F44G2MHYY3 type=del size=3348293 2024-04-17T04:45:01.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.308+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=CQU3WZUKO3I3 type=write size=548743 2024-04-17T04:45:01.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.312+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=BMHA79031NC2 type=del size=482986 2024-04-17T04:45:01.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.312+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=IKGQ17AOO1J1 type=del 2024-04-17T04:45:01.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.316+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=HD1T624BO82Z type=del size=1705418 2024-04-17T04:45:01.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.316+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=UW3XPCVWUHBI type=del 2024-04-17T04:45:01.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.320+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~L603CLIUM9O0.0 size=1048576 2024-04-17T04:45:01.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~L603CLIUM9O0.1 size=1048576 2024-04-17T04:45:01.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~L603CLIUM9O0.2 size=943832 2024-04-17T04:45:01.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.324+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=L603CLIUM9O0 tag=JW0TJPT6AU4R type=write size=4089560 2024-04-17T04:45:01.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.324+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~MG2UNYF1WPUR.0 size=1048576 2024-04-17T04:45:01.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.328+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~MG2UNYF1WPUR.1 size=1007727 2024-04-17T04:45:01.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.328+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=MG2UNYF1WPUR tag=5OA15WSA91T2 type=write size=3104879 2024-04-17T04:45:01.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.332+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=SOQML6W73311 type=del size=386797 2024-04-17T04:45:01.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.332+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=UWVX988EWSYV type=del 2024-04-17T04:45:01.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.336+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~ZXP6308DSQIC.0 size=1048576 2024-04-17T04:45:01.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~ZXP6308DSQIC.1 size=1048576 2024-04-17T04:45:01.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~ZXP6308DSQIC.2 size=512993 2024-04-17T04:45:01.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.340+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=ZXP6308DSQIC tag=AOUMCPQ0RU7B type=write size=3658721 2024-04-17T04:45:01.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.344+0000 7fd8ef2218c0 1 < suggested remove operation key=WCIJICR0FT2C tag=39XXT211R40D type=del 2024-04-17T04:45:01.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5E1K14M30J8X.0 size=1048576 2024-04-17T04:45:01.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5E1K14M30J8X.1 size=1048576 2024-04-17T04:45:01.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5E1K14M30J8X.2 size=925 2024-04-17T04:45:01.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.348+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=5E1K14M30J8X tag=SD6Y5TETOR6O type=write size=3146653 2024-04-17T04:45:01.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.352+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=G2JFMIBB1227 type=del 2024-04-17T04:45:01.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~BI83RXMJFBKT.0 size=1048576 2024-04-17T04:45:01.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~BI83RXMJFBKT.1 size=1048576 2024-04-17T04:45:01.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~BI83RXMJFBKT.2 size=563514 2024-04-17T04:45:01.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.360+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=BI83RXMJFBKT tag=WVDJ8LDLLMW3 type=write size=3709242 2024-04-17T04:45:01.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.364+0000 7fd8ef2218c0 1 < suggested remove operation key=3OFQH4KVYN18 tag=JP032TO0JN93 type=write 2024-04-17T04:45:01.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.364+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=TOZSO8RZD9EY type=del size=214988 2024-04-17T04:45:01.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.368+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=CBDCKRJYNMF3 type=del 2024-04-17T04:45:01.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.368+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=T9VUBZ2EP46D type=del size=2336295 2024-04-17T04:45:01.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.372+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=89FSXZYWCE34 type=del 2024-04-17T04:45:01.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.376+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=N7J25IKG5DI9 type=del size=4042162 2024-04-17T04:45:01.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.376+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=8X4N1RW5ES3G type=del 2024-04-17T04:45:01.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.380+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~ZCY8E1XME3GK.0 size=1048576 2024-04-17T04:45:01.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~ZCY8E1XME3GK.1 size=1048576 2024-04-17T04:45:01.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~ZCY8E1XME3GK.2 size=387711 2024-04-17T04:45:01.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.384+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=ZCY8E1XME3GK tag=GGYJQN08LOM9 type=write size=3533439 2024-04-17T04:45:01.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~DB0Q9AHYC8W3.0 size=323060 2024-04-17T04:45:01.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.392+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=DB0Q9AHYC8W3 tag=70EEMNP7949M type=write size=1371636 2024-04-17T04:45:01.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~SLI7TYN0OD1L.0 size=16905 2024-04-17T04:45:01.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.392+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=SLI7TYN0OD1L tag=GZSV2Z109OT8 type=write size=1065481 2024-04-17T04:45:01.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.396+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~C1H1M9GAGPT1.0 size=1048576 2024-04-17T04:45:01.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.396+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~C1H1M9GAGPT1.1 size=1048576 2024-04-17T04:45:01.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.400+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~C1H1M9GAGPT1.2 size=824954 2024-04-17T04:45:01.399 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.404+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=C1H1M9GAGPT1 tag=6PCM39IDQFQQ type=write size=3970682 2024-04-17T04:45:01.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~U1JB78J0PDTL.0 size=773554 2024-04-17T04:45:01.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.408+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=U1JB78J0PDTL tag=TO3W5X8UT103 type=write size=1822130 2024-04-17T04:45:01.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.408+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~T2YX0TC8GBUS.0 size=1048576 2024-04-17T04:45:01.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~T2YX0TC8GBUS.1 size=1000940 2024-04-17T04:45:01.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.416+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=T2YX0TC8GBUS tag=77SORT4US0LU type=write size=3098092 2024-04-17T04:45:01.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.420+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5Y01FYDU76ZE.0 size=1048576 2024-04-17T04:45:01.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.420+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5Y01FYDU76ZE.1 size=1048576 2024-04-17T04:45:01.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.424+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5Y01FYDU76ZE.2 size=1002472 2024-04-17T04:45:01.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.424+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=5Y01FYDU76ZE tag=MTBMYV72O4L6 type=write size=4148200 2024-04-17T04:45:01.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.428+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=OG8QHMB72RX9 type=del 2024-04-17T04:45:01.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.428+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=DQB37CGLPI6S type=write size=818851 2024-04-17T04:45:01.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.432+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=39F44G2MHYY3 type=del 2024-04-17T04:45:01.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.432+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=FP0ZKFPNH9Q5 type=del size=199817 2024-04-17T04:45:01.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.436+0000 7fd8ef2218c0 1 < suggested update operation key=GX9X0D45GC2I tag=BMHA79031NC2 type=del size=482986 2024-04-17T04:45:01.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~CR32AVTFLATO.0 size=1048576 2024-04-17T04:45:01.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~CR32AVTFLATO.1 size=185562 2024-04-17T04:45:01.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.444+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=CR32AVTFLATO tag=MRLJSOK4BHXT type=write size=2282714 2024-04-17T04:45:01.443 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.448+0000 7fd8ef2218c0 1 < suggested update operation key=G03VLS09F3BT tag=HD1T624BO82Z type=del size=1705418 2024-04-17T04:45:01.445 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.448+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=UL72HAKF4VDK type=del size=1503873 2024-04-17T04:45:01.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.452+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~L603CLIUM9O0.0 size=1048576 2024-04-17T04:45:01.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.452+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~L603CLIUM9O0.1 size=1048576 2024-04-17T04:45:01.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~L603CLIUM9O0.2 size=943832 2024-04-17T04:45:01.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.460+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=L603CLIUM9O0 tag=JW0TJPT6AU4R type=write size=4089560 2024-04-17T04:45:01.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.464+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~SS6J60AIIAQ8.0 size=1048576 2024-04-17T04:45:01.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.464+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~SS6J60AIIAQ8.1 size=462007 2024-04-17T04:45:01.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.464+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=SS6J60AIIAQ8 tag=U6R38Q82IL0O type=write size=2559159 2024-04-17T04:45:01.464 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.468+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=SOQML6W73311 type=del 2024-04-17T04:45:01.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.472+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=6BXLMQ2RTLLE type=write size=652395 2024-04-17T04:45:01.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~ZXP6308DSQIC.0 size=1048576 2024-04-17T04:45:01.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~ZXP6308DSQIC.1 size=1048576 2024-04-17T04:45:01.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~ZXP6308DSQIC.2 size=512993 2024-04-17T04:45:01.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.480+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=ZXP6308DSQIC tag=AOUMCPQ0RU7B type=write size=3658721 2024-04-17T04:45:01.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.484+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=LKCBL34XYOLQ type=del size=1426371 2024-04-17T04:45:01.481 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.484+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5E1K14M30J8X.0 size=1048576 2024-04-17T04:45:01.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.488+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5E1K14M30J8X.1 size=1048576 2024-04-17T04:45:01.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.492+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5E1K14M30J8X.2 size=925 2024-04-17T04:45:01.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.496+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=5E1K14M30J8X tag=SD6Y5TETOR6O type=write size=3146653 2024-04-17T04:45:01.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.496+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=0HEKFLIHX4K3 type=del size=3614887 2024-04-17T04:45:01.495 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~BI83RXMJFBKT.0 size=1048576 2024-04-17T04:45:01.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~BI83RXMJFBKT.1 size=1048576 2024-04-17T04:45:01.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~BI83RXMJFBKT.2 size=563514 2024-04-17T04:45:01.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.508+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=BI83RXMJFBKT tag=WVDJ8LDLLMW3 type=write size=3709242 2024-04-17T04:45:01.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.508+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=0QJA1CHI52A3 type=del size=1924416 2024-04-17T04:45:01.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.516+0000 7fd8ef2218c0 1 < suggested remove operation key=7BV3V2XMLSPH tag=TOZSO8RZD9EY type=del 2024-04-17T04:45:01.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.520+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~74JY5O8D7HUP.0 size=1048576 2024-04-17T04:45:01.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.524+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~74JY5O8D7HUP.1 size=507919 2024-04-17T04:45:01.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.524+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=74JY5O8D7HUP tag=YT6N5GQU4OPT type=write size=2605071 2024-04-17T04:45:01.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.524+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=T9VUBZ2EP46D type=del 2024-04-17T04:45:01.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.528+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~77MUEEQMJVXS.0 size=1048576 2024-04-17T04:45:01.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~77MUEEQMJVXS.1 size=1048576 2024-04-17T04:45:01.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~77MUEEQMJVXS.2 size=863406 2024-04-17T04:45:01.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.536+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=77MUEEQMJVXS tag=0S0IBS7PKIRU type=write size=4009134 2024-04-17T04:45:01.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.536+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=N7J25IKG5DI9 type=del 2024-04-17T04:45:01.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.540+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~VLBBU0OJPHOJ.0 size=1048576 2024-04-17T04:45:01.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.544+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~VLBBU0OJPHOJ.1 size=1048576 2024-04-17T04:45:01.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~VLBBU0OJPHOJ.2 size=66935 2024-04-17T04:45:01.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.548+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=VLBBU0OJPHOJ tag=A5N7MOD1DYQM type=write size=3212663 2024-04-17T04:45:01.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.548+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~ZCY8E1XME3GK.0 size=1048576 2024-04-17T04:45:01.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~ZCY8E1XME3GK.1 size=1048576 2024-04-17T04:45:01.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~ZCY8E1XME3GK.2 size=387711 2024-04-17T04:45:01.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.560+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=ZCY8E1XME3GK tag=GGYJQN08LOM9 type=write size=3533439 2024-04-17T04:45:01.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~ACSWV1MNTD51.0 size=355720 2024-04-17T04:45:01.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.564+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=ACSWV1MNTD51 tag=I688W45P0GNZ type=write size=1404296 2024-04-17T04:45:01.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.568+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~SLI7TYN0OD1L.0 size=16905 2024-04-17T04:45:01.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.572+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=SLI7TYN0OD1L tag=GZSV2Z109OT8 type=write size=1065481 2024-04-17T04:45:01.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.572+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=HV77JSVKQKZS type=del size=3143223 2024-04-17T04:45:01.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~U1JB78J0PDTL.0 size=773554 2024-04-17T04:45:01.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.580+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=U1JB78J0PDTL tag=TO3W5X8UT103 type=write size=1822130 2024-04-17T04:45:01.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.584+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=9ENYH90BJO3H type=del size=3282987 2024-04-17T04:45:01.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.584+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5Y01FYDU76ZE.0 size=1048576 2024-04-17T04:45:01.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.588+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5Y01FYDU76ZE.1 size=1048576 2024-04-17T04:45:01.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.588+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5Y01FYDU76ZE.2 size=1002472 2024-04-17T04:45:01.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.596+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=5Y01FYDU76ZE tag=MTBMYV72O4L6 type=write size=4148200 2024-04-17T04:45:01.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.600+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~6QNAU8ANSXVG.0 size=1048576 2024-04-17T04:45:01.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.600+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~6QNAU8ANSXVG.1 size=1048576 2024-04-17T04:45:01.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~6QNAU8ANSXVG.2 size=129232 2024-04-17T04:45:01.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.604+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=6QNAU8ANSXVG tag=OJDIKE3AGR6U type=write size=3274960 2024-04-17T04:45:01.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.604+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=DQB37CGLPI6S type=write size=818851 2024-04-17T04:45:01.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.608+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=XM4T2VVKW14M type=del size=3566858 2024-04-17T04:45:01.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.612+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=FP0ZKFPNH9Q5 type=del 2024-04-17T04:45:01.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.612+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=VNNUG9FQ321U type=del size=2905524 2024-04-17T04:45:01.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.616+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~CR32AVTFLATO.0 size=1048576 2024-04-17T04:45:01.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~CR32AVTFLATO.1 size=185562 2024-04-17T04:45:01.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.624+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=CR32AVTFLATO tag=MRLJSOK4BHXT type=write size=2282714 2024-04-17T04:45:01.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.624+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=286AS061ES03 type=del size=3429759 2024-04-17T04:45:01.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.628+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=UL72HAKF4VDK type=del 2024-04-17T04:45:01.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.632+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=9TTG0BUR93YA type=del size=3438370 2024-04-17T04:45:01.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~SS6J60AIIAQ8.0 size=1048576 2024-04-17T04:45:01.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.632+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2S1IT3TO1BPU.2~SS6J60AIIAQ8.1 size=462007 2024-04-17T04:45:01.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.636+0000 7fd8ef2218c0 1 < canceled multipart upload key=2S1IT3TO1BPU upload=SS6J60AIIAQ8 tag=U6R38Q82IL0O type=write size=2559159 2024-04-17T04:45:01.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.640+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~T5NPW08Y0J3C.0 size=1048576 2024-04-17T04:45:01.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~T5NPW08Y0J3C.1 size=1048576 2024-04-17T04:45:01.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~T5NPW08Y0J3C.2 size=725728 2024-04-17T04:45:01.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.644+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=T5NPW08Y0J3C tag=PXKNB2545GCA type=write size=3871456 2024-04-17T04:45:01.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.648+0000 7fd8ef2218c0 1 < canceled operation key=UEWLJ3E9ODUL tag=6BXLMQ2RTLLE type=write size=652395 2024-04-17T04:45:01.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.648+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=8FSDX2BQ1QNT type=del size=1176488 2024-04-17T04:45:01.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.656+0000 7fd8ef2218c0 1 < canceled operation key=2IY8DB37FI9H tag=LKCBL34XYOLQ type=del size=1426371 2024-04-17T04:45:01.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.660+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~8MCA9GSGN86P.0 size=1048576 2024-04-17T04:45:01.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.664+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~8MCA9GSGN86P.1 size=1048576 2024-04-17T04:45:01.660 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.664+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~8MCA9GSGN86P.2 size=158103 2024-04-17T04:45:01.660 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.664+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=8MCA9GSGN86P tag=XBMW2P2C9DEY type=write size=3303831 2024-04-17T04:45:01.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.668+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=0HEKFLIHX4K3 type=del 2024-04-17T04:45:01.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.668+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=F18ACTXMFF3K type=write size=622139 2024-04-17T04:45:01.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.676+0000 7fd8ef2218c0 1 < suggested update operation key=MS3T476JOEON tag=0QJA1CHI52A3 type=del size=1924416 2024-04-17T04:45:01.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.676+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=0NEC0TJSVVQS type=del size=880737 2024-04-17T04:45:01.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~74JY5O8D7HUP.0 size=1048576 2024-04-17T04:45:01.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~74JY5O8D7HUP.1 size=507919 2024-04-17T04:45:01.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.688+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=74JY5O8D7HUP tag=YT6N5GQU4OPT type=write size=2605071 2024-04-17T04:45:01.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.688+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=WEM2OQFDTYWI type=del size=1595665 2024-04-17T04:45:01.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.692+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~77MUEEQMJVXS.0 size=1048576 2024-04-17T04:45:01.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.692+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UNOVYXDS5JEB.2~77MUEEQMJVXS.1 size=1048576 2024-04-17T04:45:01.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.692+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~77MUEEQMJVXS.2 size=863406 2024-04-17T04:45:01.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.696+0000 7fd8ef2218c0 1 < canceled multipart upload key=UNOVYXDS5JEB upload=77MUEEQMJVXS tag=0S0IBS7PKIRU type=write size=4009134 2024-04-17T04:45:01.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.700+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=7X1G69GYRUX9 type=del size=1805392 2024-04-17T04:45:01.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.704+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~VLBBU0OJPHOJ.0 size=1048576 2024-04-17T04:45:01.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.704+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~VLBBU0OJPHOJ.1 size=1048576 2024-04-17T04:45:01.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~VLBBU0OJPHOJ.2 size=66935 2024-04-17T04:45:01.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.712+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=VLBBU0OJPHOJ tag=A5N7MOD1DYQM type=write size=3212663 2024-04-17T04:45:01.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.716+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=CI57WLO12TT5 type=write size=959946 2024-04-17T04:45:01.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.716+0000 7fd8ef2218c0 1 < canceled part key=_multipart_7BV3V2XMLSPH.2~ACSWV1MNTD51.0 size=355720 2024-04-17T04:45:01.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.720+0000 7fd8ef2218c0 1 < canceled multipart upload key=7BV3V2XMLSPH upload=ACSWV1MNTD51 tag=I688W45P0GNZ type=write size=1404296 2024-04-17T04:45:01.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.724+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~28YL41S0I1P6.0 size=762324 2024-04-17T04:45:01.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.724+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=28YL41S0I1P6 tag=Q3SH2WWFCEX5 type=write size=1810900 2024-04-17T04:45:01.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.728+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=HV77JSVKQKZS type=del 2024-04-17T04:45:01.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.732+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=RRWIG8WNE536 type=write size=969076 2024-04-17T04:45:01.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.736+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=9ENYH90BJO3H type=del 2024-04-17T04:45:01.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.736+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=RQVZUVC3FM2P type=del size=112735 2024-04-17T04:45:01.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~6QNAU8ANSXVG.0 size=1048576 2024-04-17T04:45:01.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~6QNAU8ANSXVG.1 size=1048576 2024-04-17T04:45:01.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.740+0000 7fd8ef2218c0 1 < canceled part key=_multipart_OZ1IST8DOGXW.2~6QNAU8ANSXVG.2 size=129232 2024-04-17T04:45:01.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.744+0000 7fd8ef2218c0 1 < canceled multipart upload key=OZ1IST8DOGXW upload=6QNAU8ANSXVG tag=OJDIKE3AGR6U type=write size=3274960 2024-04-17T04:45:01.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.748+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=9GAXX93OPX7U type=del size=3116754 2024-04-17T04:45:01.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.748+0000 7fd8ef2218c0 -1 < failed to complete operation key=UEWLJ3E9ODUL tag=XM4T2VVKW14M type=del size=3566858: (22) Invalid argument 2024-04-17T04:45:01.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.752+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=4GJ1GJ838AKV type=del size=818254 2024-04-17T04:45:01.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.752+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=VNNUG9FQ321U type=del 2024-04-17T04:45:01.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.756+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=606G9VLB7X2O type=del size=1444872 2024-04-17T04:45:01.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.760+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=286AS061ES03 type=del 2024-04-17T04:45:01.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~KLHFGADB95WM.0 size=1048576 2024-04-17T04:45:01.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~KLHFGADB95WM.1 size=395697 2024-04-17T04:45:01.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.764+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=KLHFGADB95WM tag=EGX41G2EU0DI type=write size=2492849 2024-04-17T04:45:01.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.768+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=9TTG0BUR93YA type=del 2024-04-17T04:45:01.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.772+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~VJHWWGEAG0TD.0 size=1048576 2024-04-17T04:45:01.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.772+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~VJHWWGEAG0TD.1 size=250094 2024-04-17T04:45:01.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.772+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=VJHWWGEAG0TD tag=RZ2699L412NC type=write size=2347246 2024-04-17T04:45:01.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.776+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~T5NPW08Y0J3C.0 size=1048576 2024-04-17T04:45:01.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.776+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~T5NPW08Y0J3C.1 size=1048576 2024-04-17T04:45:01.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.780+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~T5NPW08Y0J3C.2 size=725728 2024-04-17T04:45:01.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.784+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=T5NPW08Y0J3C tag=PXKNB2545GCA type=write size=3871456 2024-04-17T04:45:01.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~3LVRIH42GJY6.0 size=1048576 2024-04-17T04:45:01.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~3LVRIH42GJY6.1 size=1009121 2024-04-17T04:45:01.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.788+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=3LVRIH42GJY6 tag=GL3TP8N7L3IW type=write size=3106273 2024-04-17T04:45:01.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.792+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=8FSDX2BQ1QNT type=del 2024-04-17T04:45:01.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.796+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~EV564RJMHZ01.0 size=404217 2024-04-17T04:45:01.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.796+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=EV564RJMHZ01 tag=PWRY3CNTG670 type=write size=1452793 2024-04-17T04:45:01.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~8MCA9GSGN86P.0 size=1048576 2024-04-17T04:45:01.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.800+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~8MCA9GSGN86P.1 size=1048576 2024-04-17T04:45:01.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~8MCA9GSGN86P.2 size=158103 2024-04-17T04:45:01.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.804+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=8MCA9GSGN86P tag=XBMW2P2C9DEY type=write size=3303831 2024-04-17T04:45:01.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.808+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=2WZXR5R0L9HR type=del size=2500904 2024-04-17T04:45:01.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.808+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=F18ACTXMFF3K type=write size=622139 2024-04-17T04:45:01.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.812+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~LL1EAU4JPP3I.0 size=1048576 2024-04-17T04:45:01.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.812+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~LL1EAU4JPP3I.1 size=1048576 2024-04-17T04:45:01.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~LL1EAU4JPP3I.2 size=479958 2024-04-17T04:45:01.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.816+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=LL1EAU4JPP3I tag=WL10D17DGNOW type=write size=3625686 2024-04-17T04:45:01.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.820+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=0NEC0TJSVVQS type=del size=880737 2024-04-17T04:45:01.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.824+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=DYQIITXZ006M type=write size=535643 2024-04-17T04:45:01.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.828+0000 7fd8ef2218c0 1 < suggested update operation key=5K50O96P7OGD tag=WEM2OQFDTYWI type=del size=1595665 2024-04-17T04:45:01.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~CWS0HA03MMZ8.0 size=1048576 2024-04-17T04:45:01.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~CWS0HA03MMZ8.1 size=1048576 2024-04-17T04:45:01.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~CWS0HA03MMZ8.2 size=286614 2024-04-17T04:45:01.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.836+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=CWS0HA03MMZ8 tag=5V5NKXC638Y3 type=write size=3432342 2024-04-17T04:45:01.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.840+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=7X1G69GYRUX9 type=del 2024-04-17T04:45:01.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.840+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=76QO1GAF53ZW type=del size=2621412 2024-04-17T04:45:01.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.844+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=CI57WLO12TT5 type=write size=959946 2024-04-17T04:45:01.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.848+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=0CSPBAYUT44S type=write size=533050 2024-04-17T04:45:01.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~28YL41S0I1P6.0 size=762324 2024-04-17T04:45:01.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.856+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=28YL41S0I1P6 tag=Q3SH2WWFCEX5 type=write size=1810900 2024-04-17T04:45:01.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.856+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=ZP63LFJMKMXE type=del size=1485394 2024-04-17T04:45:01.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.860+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=RRWIG8WNE536 type=write size=969076 2024-04-17T04:45:01.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.860+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=51ZCBYB980DB type=del size=509129 2024-04-17T04:45:01.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.868+0000 7fd8ef2218c0 1 < suggested update operation key=7BV3V2XMLSPH tag=RQVZUVC3FM2P type=del size=112735 2024-04-17T04:45:01.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.868+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=E194F49MYUBL type=del size=90390 2024-04-17T04:45:01.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.872+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=9GAXX93OPX7U type=del 2024-04-17T04:45:01.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.872+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=D53N9ZXYN4SM type=del size=2450635 2024-04-17T04:45:01.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.876+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=4GJ1GJ838AKV type=del 2024-04-17T04:45:01.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.880+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=XN62XNSOCOGO type=write size=1031088 2024-04-17T04:45:01.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.880+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=606G9VLB7X2O type=del 2024-04-17T04:45:01.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.884+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=J6HECS13OMNC type=del size=1116024 2024-04-17T04:45:01.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~KLHFGADB95WM.0 size=1048576 2024-04-17T04:45:01.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~KLHFGADB95WM.1 size=395697 2024-04-17T04:45:01.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.896+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=KLHFGADB95WM tag=EGX41G2EU0DI type=write size=2492849 2024-04-17T04:45:01.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.900+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=F7LD83TV0UKZ type=del size=1537002 2024-04-17T04:45:01.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~VJHWWGEAG0TD.0 size=1048576 2024-04-17T04:45:01.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~VJHWWGEAG0TD.1 size=250094 2024-04-17T04:45:01.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.908+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=VJHWWGEAG0TD tag=RZ2699L412NC type=write size=2347246 2024-04-17T04:45:01.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.912+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=2CQMLEOGVF5W type=del size=238542 2024-04-17T04:45:01.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~3LVRIH42GJY6.0 size=1048576 2024-04-17T04:45:01.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~3LVRIH42GJY6.1 size=1009121 2024-04-17T04:45:01.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.920+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=3LVRIH42GJY6 tag=GL3TP8N7L3IW type=write size=3106273 2024-04-17T04:45:01.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~DPJUGWGKSLWO.0 size=1048576 2024-04-17T04:45:01.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~DPJUGWGKSLWO.1 size=882226 2024-04-17T04:45:01.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.928+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=DPJUGWGKSLWO tag=R73Q1MRDVHNH type=write size=2979378 2024-04-17T04:45:01.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~EV564RJMHZ01.0 size=404217 2024-04-17T04:45:01.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.936+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=EV564RJMHZ01 tag=PWRY3CNTG670 type=write size=1452793 2024-04-17T04:45:01.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~2PS82AUBUV88.0 size=1048576 2024-04-17T04:45:01.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.944+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~2PS82AUBUV88.1 size=1048576 2024-04-17T04:45:01.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.948+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~2PS82AUBUV88.2 size=538172 2024-04-17T04:45:01.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.948+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=2PS82AUBUV88 tag=X3MSCHQSI3YT type=write size=3683900 2024-04-17T04:45:01.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.948+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=2WZXR5R0L9HR type=del 2024-04-17T04:45:01.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.952+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=F6F9H7JUBJE0 type=del size=797169 2024-04-17T04:45:01.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.952+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UNOVYXDS5JEB.2~LL1EAU4JPP3I.0 size=1048576 2024-04-17T04:45:01.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.956+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~LL1EAU4JPP3I.1 size=1048576 2024-04-17T04:45:01.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.956+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~LL1EAU4JPP3I.2 size=479958 2024-04-17T04:45:01.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.964+0000 7fd8ef2218c0 1 < canceled multipart upload key=UNOVYXDS5JEB upload=LL1EAU4JPP3I tag=WL10D17DGNOW type=write size=3625686 2024-04-17T04:45:01.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.964+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=QIT1L9B3VYOY type=del size=251812 2024-04-17T04:45:01.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.968+0000 7fd8ef2218c0 1 < canceled operation key=5EFSN2FPUQB6 tag=DYQIITXZ006M type=write size=535643 2024-04-17T04:45:01.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.972+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=W1VBWXIM57OJ type=del size=1763367 2024-04-17T04:45:01.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~CWS0HA03MMZ8.0 size=1048576 2024-04-17T04:45:01.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.976+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~CWS0HA03MMZ8.1 size=1048576 2024-04-17T04:45:01.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.976+0000 7fd8ef2218c0 1 < canceled part key=_multipart_R3C8PAVPVGBY.2~CWS0HA03MMZ8.2 size=286614 2024-04-17T04:45:01.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.980+0000 7fd8ef2218c0 1 < canceled multipart upload key=R3C8PAVPVGBY upload=CWS0HA03MMZ8 tag=5V5NKXC638Y3 type=write size=3432342 2024-04-17T04:45:01.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~41RWT835IBZ3.0 size=1048576 2024-04-17T04:45:01.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~41RWT835IBZ3.1 size=976538 2024-04-17T04:45:01.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.984+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=41RWT835IBZ3 tag=873ZO4KMG5HP type=write size=3073690 2024-04-17T04:45:01.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.988+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=76QO1GAF53ZW type=del 2024-04-17T04:45:01.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.992+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=N30KPLUBQQ5U type=del size=329240 2024-04-17T04:45:01.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:01.992+0000 7fd8ef2218c0 1 < completed write operation key=JVWRJBMFIRLL tag=0CSPBAYUT44S type=write size=533050 2024-04-17T04:45:01.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~JN8D7IATVL7P.0 size=1048576 2024-04-17T04:45:01.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~JN8D7IATVL7P.1 size=1048576 2024-04-17T04:45:01.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~JN8D7IATVL7P.2 size=409043 2024-04-17T04:45:01.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.004+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=JN8D7IATVL7P tag=4U95YFWWP4TA type=write size=3554771 2024-04-17T04:45:02.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.004+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=ZP63LFJMKMXE type=del 2024-04-17T04:45:02.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.008+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=2LZBLB6BOQW0 type=del size=1717389 2024-04-17T04:45:02.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.008+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=51ZCBYB980DB type=del 2024-04-17T04:45:02.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.012+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=EH2VYG952FPZ type=write size=74972 2024-04-17T04:45:02.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.016+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=E194F49MYUBL type=del 2024-04-17T04:45:02.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.016+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=V4XMK3O1SQD8 type=write size=534360 2024-04-17T04:45:02.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.020+0000 7fd8ef2218c0 1 < canceled operation key=2IY8DB37FI9H tag=D53N9ZXYN4SM type=del size=2450635 2024-04-17T04:45:02.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~AJRVAVXY254Z.0 size=1048576 2024-04-17T04:45:02.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~AJRVAVXY254Z.1 size=686573 2024-04-17T04:45:02.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.028+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=AJRVAVXY254Z tag=WYFAG8PO4GXO type=write size=2783725 2024-04-17T04:45:02.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.028+0000 7fd8ef2218c0 1 < completed write operation key=82D6YWKJ1TFQ tag=XN62XNSOCOGO type=write size=1031088 2024-04-17T04:45:02.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.036+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~FHREFVB9OINY.0 size=1048576 2024-04-17T04:45:02.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.036+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~FHREFVB9OINY.1 size=1048576 2024-04-17T04:45:02.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.040+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~FHREFVB9OINY.2 size=239863 2024-04-17T04:45:02.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.040+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=FHREFVB9OINY tag=NRX04LFTR1X2 type=write size=3385591 2024-04-17T04:45:02.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.044+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=J6HECS13OMNC type=del 2024-04-17T04:45:02.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.048+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=APWHVK7YT46P type=del size=330173 2024-04-17T04:45:02.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.048+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=F7LD83TV0UKZ type=del 2024-04-17T04:45:02.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.052+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=KDCUOUI5RHK0 type=del size=585837 2024-04-17T04:45:02.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.056+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=2CQMLEOGVF5W type=del 2024-04-17T04:45:02.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.060+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~MRC8XRRWJSZA.0 size=1048576 2024-04-17T04:45:02.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.064+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~MRC8XRRWJSZA.1 size=1048576 2024-04-17T04:45:02.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.064+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~MRC8XRRWJSZA.2 size=488 2024-04-17T04:45:02.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.064+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=MRC8XRRWJSZA tag=K16RQ2ZBIU1G type=write size=3146216 2024-04-17T04:45:02.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~DPJUGWGKSLWO.0 size=1048576 2024-04-17T04:45:02.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~DPJUGWGKSLWO.1 size=882226 2024-04-17T04:45:02.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.076+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=DPJUGWGKSLWO tag=R73Q1MRDVHNH type=write size=2979378 2024-04-17T04:45:02.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.080+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ZDVQ1805719S.0 size=1048576 2024-04-17T04:45:02.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ZDVQ1805719S.1 size=1048576 2024-04-17T04:45:02.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.088+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ZDVQ1805719S.2 size=487947 2024-04-17T04:45:02.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.088+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=ZDVQ1805719S tag=X5K06FSNT1PE type=write size=3633675 2024-04-17T04:45:02.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.088+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~2PS82AUBUV88.0 size=1048576 2024-04-17T04:45:02.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~2PS82AUBUV88.1 size=1048576 2024-04-17T04:45:02.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.092+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~2PS82AUBUV88.2 size=538172 2024-04-17T04:45:02.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.096+0000 7fd8ef2218c0 1 < canceled multipart upload key=6WSZ5UL0V1Y1 upload=2PS82AUBUV88 tag=X3MSCHQSI3YT type=write size=3683900 2024-04-17T04:45:02.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.100+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=OCPPVWUKTFTR type=del size=2617813 2024-04-17T04:45:02.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.104+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=F6F9H7JUBJE0 type=del 2024-04-17T04:45:02.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~4I7L3ANDSTBB.0 size=1048576 2024-04-17T04:45:02.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~4I7L3ANDSTBB.1 size=1048576 2024-04-17T04:45:02.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~4I7L3ANDSTBB.2 size=267880 2024-04-17T04:45:02.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=4I7L3ANDSTBB tag=NK1NIGMGREFR type=write size=3413608 2024-04-17T04:45:02.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.120+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=QIT1L9B3VYOY type=del 2024-04-17T04:45:02.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.124+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=8E7QXWXQC2P6 type=del size=3983772 2024-04-17T04:45:02.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.128+0000 7fd8ef2218c0 1 < suggested update operation key=6219DPY72LWC tag=W1VBWXIM57OJ type=del size=1763367 2024-04-17T04:45:02.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.132+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=QDSJ5J60JG4E type=write size=327845 2024-04-17T04:45:02.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~41RWT835IBZ3.0 size=1048576 2024-04-17T04:45:02.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~41RWT835IBZ3.1 size=976538 2024-04-17T04:45:02.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.144+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=41RWT835IBZ3 tag=873ZO4KMG5HP type=write size=3073690 2024-04-17T04:45:02.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.144+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=LZQ9W10ZUELQ type=del size=3852953 2024-04-17T04:45:02.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.152+0000 7fd8ef2218c0 1 < canceled operation key=A5ZUB3YK06J3 tag=N30KPLUBQQ5U type=del size=329240 2024-04-17T04:45:02.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.156+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=EWYN6KWPE6T8 type=del size=430649 2024-04-17T04:45:02.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~JN8D7IATVL7P.0 size=1048576 2024-04-17T04:45:02.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~JN8D7IATVL7P.1 size=1048576 2024-04-17T04:45:02.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.164+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~JN8D7IATVL7P.2 size=409043 2024-04-17T04:45:02.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.168+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=JN8D7IATVL7P tag=4U95YFWWP4TA type=write size=3554771 2024-04-17T04:45:02.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.172+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=JB79HKC08NDW type=del size=1395047 2024-04-17T04:45:02.172 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.176+0000 7fd8ef2218c0 1 < suggested update operation key=JVWRJBMFIRLL tag=2LZBLB6BOQW0 type=del size=1717389 2024-04-17T04:45:02.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.180+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=OP4QI3LAUJ4J type=del size=3472017 2024-04-17T04:45:02.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.184+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=EH2VYG952FPZ type=write size=74972 2024-04-17T04:45:02.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.188+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=GNCEBLM1MTJ9 type=del size=2115672 2024-04-17T04:45:02.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.196+0000 7fd8ef2218c0 1 < suggested update operation key=A5ZUB3YK06J3 tag=V4XMK3O1SQD8 type=write size=534360 2024-04-17T04:45:02.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.196+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=8FWLBA5APGNI type=del size=2639816 2024-04-17T04:45:02.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~AJRVAVXY254Z.0 size=1048576 2024-04-17T04:45:02.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~AJRVAVXY254Z.1 size=686573 2024-04-17T04:45:02.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.208+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=AJRVAVXY254Z tag=WYFAG8PO4GXO type=write size=2783725 2024-04-17T04:45:02.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~AHQWQLJ7UBY1.0 size=1048576 2024-04-17T04:45:02.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~AHQWQLJ7UBY1.1 size=308201 2024-04-17T04:45:02.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.220+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=AHQWQLJ7UBY1 tag=8N15VECU5TR4 type=write size=2405353 2024-04-17T04:45:02.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.220+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~FHREFVB9OINY.0 size=1048576 2024-04-17T04:45:02.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.224+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~FHREFVB9OINY.1 size=1048576 2024-04-17T04:45:02.223 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~FHREFVB9OINY.2 size=239863 2024-04-17T04:45:02.230 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.236+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=FHREFVB9OINY tag=NRX04LFTR1X2 type=write size=3385591 2024-04-17T04:45:02.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.236+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=AY0QYJ6KBHNE type=del size=2234239 2024-04-17T04:45:02.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.240+0000 7fd8ef2218c0 -1 < failed to complete operation key=A5ZUB3YK06J3 tag=APWHVK7YT46P type=del size=330173: (22) Invalid argument 2024-04-17T04:45:02.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.244+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=2A1DS08FGCU4 type=write size=320924 2024-04-17T04:45:02.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.248+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=KDCUOUI5RHK0 type=del 2024-04-17T04:45:02.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.252+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=6IKAI22CLR5I type=del size=2326945 2024-04-17T04:45:02.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.252+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~MRC8XRRWJSZA.0 size=1048576 2024-04-17T04:45:02.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.256+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~MRC8XRRWJSZA.1 size=1048576 2024-04-17T04:45:02.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.260+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~MRC8XRRWJSZA.2 size=488 2024-04-17T04:45:02.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.264+0000 7fd8ef2218c0 1 < canceled multipart upload key=6219DPY72LWC upload=MRC8XRRWJSZA tag=K16RQ2ZBIU1G type=write size=3146216 2024-04-17T04:45:02.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.272+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5SUJYV819VA5.0 size=1048576 2024-04-17T04:45:02.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.272+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5SUJYV819VA5.1 size=1022896 2024-04-17T04:45:02.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.272+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=5SUJYV819VA5 tag=ER5F6RLNOHCX type=write size=3120048 2024-04-17T04:45:02.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~ZDVQ1805719S.0 size=1048576 2024-04-17T04:45:02.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.280+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~ZDVQ1805719S.1 size=1048576 2024-04-17T04:45:02.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.284+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~ZDVQ1805719S.2 size=487947 2024-04-17T04:45:02.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.288+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=ZDVQ1805719S tag=X5K06FSNT1PE type=write size=3633675 2024-04-17T04:45:02.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.296+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~S9RQI81SM7N9.0 size=42673 2024-04-17T04:45:02.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.296+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=S9RQI81SM7N9 tag=Y5WZ1P1PL7ES type=write size=1091249 2024-04-17T04:45:02.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.300+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=OCPPVWUKTFTR type=del 2024-04-17T04:45:02.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.304+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=M8S5L9SRPP1U type=del size=1817482 2024-04-17T04:45:02.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.308+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~4I7L3ANDSTBB.0 size=1048576 2024-04-17T04:45:02.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~4I7L3ANDSTBB.1 size=1048576 2024-04-17T04:45:02.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~4I7L3ANDSTBB.2 size=267880 2024-04-17T04:45:02.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.320+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=4I7L3ANDSTBB tag=NK1NIGMGREFR type=write size=3413608 2024-04-17T04:45:02.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.324+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=TJFJJSRHMNNE type=del size=113152 2024-04-17T04:45:02.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.328+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=8E7QXWXQC2P6 type=del 2024-04-17T04:45:02.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.332+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=1LCO70NC6Z15 type=del size=3906667 2024-04-17T04:45:02.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.336+0000 7fd8ef2218c0 1 < completed write operation key=1JOWGWFR7R6P tag=QDSJ5J60JG4E type=write size=327845 2024-04-17T04:45:02.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.340+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=4MDBHYMJD24O type=del size=817043 2024-04-17T04:45:02.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.340+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=LZQ9W10ZUELQ type=del 2024-04-17T04:45:02.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~YYWTOJ9APHR1.0 size=1048576 2024-04-17T04:45:02.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~YYWTOJ9APHR1.1 size=1048576 2024-04-17T04:45:02.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~YYWTOJ9APHR1.2 size=471790 2024-04-17T04:45:02.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.352+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=YYWTOJ9APHR1 tag=42F4LNK9A8S0 type=write size=3617518 2024-04-17T04:45:02.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.356+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=EWYN6KWPE6T8 type=del 2024-04-17T04:45:02.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.360+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=F4CKO8OEZEKE type=del size=1849930 2024-04-17T04:45:02.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.364+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=JB79HKC08NDW type=del 2024-04-17T04:45:02.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~K42MI8YMT6GT.0 size=1048576 2024-04-17T04:45:02.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~K42MI8YMT6GT.1 size=1048576 2024-04-17T04:45:02.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.376+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~K42MI8YMT6GT.2 size=602002 2024-04-17T04:45:02.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.376+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=K42MI8YMT6GT tag=21EJX5DQD2DX type=write size=3747730 2024-04-17T04:45:02.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.380+0000 7fd8ef2218c0 1 < suggested remove operation key=MS3T476JOEON tag=OP4QI3LAUJ4J type=del 2024-04-17T04:45:02.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4X253ACR4LP9.0 size=1048576 2024-04-17T04:45:02.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4X253ACR4LP9.1 size=440866 2024-04-17T04:45:02.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.388+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=4X253ACR4LP9 tag=I6H9QB3ZVL05 type=write size=2538018 2024-04-17T04:45:02.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.396+0000 7fd8ef2218c0 1 < suggested remove operation key=5K50O96P7OGD tag=GNCEBLM1MTJ9 type=del 2024-04-17T04:45:02.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.404+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~JOPOU989Z2UC.0 size=238291 2024-04-17T04:45:02.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.404+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=JOPOU989Z2UC tag=O1VRLLUMEF69 type=write size=1286867 2024-04-17T04:45:02.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.404+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=8FWLBA5APGNI type=del 2024-04-17T04:45:02.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.408+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=5H21K35405H4 type=del size=2114013 2024-04-17T04:45:02.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~AHQWQLJ7UBY1.0 size=1048576 2024-04-17T04:45:02.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.416+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~AHQWQLJ7UBY1.1 size=308201 2024-04-17T04:45:02.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.420+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=AHQWQLJ7UBY1 tag=8N15VECU5TR4 type=write size=2405353 2024-04-17T04:45:02.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.424+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=ULQ1AWZMX5IL type=del size=2620835 2024-04-17T04:45:02.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.428+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=AY0QYJ6KBHNE type=del 2024-04-17T04:45:02.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.428+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=YUUN91F0O1PC type=del size=1016535 2024-04-17T04:45:02.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.432+0000 7fd8ef2218c0 -1 < failed to complete operation key=82D6YWKJ1TFQ tag=2A1DS08FGCU4 type=write size=320924: (22) Invalid argument 2024-04-17T04:45:02.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.436+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=H864RJ7S7H4E type=del size=799444 2024-04-17T04:45:02.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.444+0000 7fd8ef2218c0 1 < suggested update operation key=1ZTJP8DRITEK tag=6IKAI22CLR5I type=del size=2326945 2024-04-17T04:45:02.445 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~KQF2XJE0M3L0.0 size=1048576 2024-04-17T04:45:02.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~KQF2XJE0M3L0.1 size=961207 2024-04-17T04:45:02.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.452+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=KQF2XJE0M3L0 tag=GUIQGAGF9WEY type=write size=3058359 2024-04-17T04:45:02.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5SUJYV819VA5.0 size=1048576 2024-04-17T04:45:02.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.460+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5SUJYV819VA5.1 size=1022896 2024-04-17T04:45:02.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.468+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=5SUJYV819VA5 tag=ER5F6RLNOHCX type=write size=3120048 2024-04-17T04:45:02.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.472+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=QJNET1X3YNQW type=del size=2537732 2024-04-17T04:45:02.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.472+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~S9RQI81SM7N9.0 size=42673 2024-04-17T04:45:02.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.476+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=S9RQI81SM7N9 tag=Y5WZ1P1PL7ES type=write size=1091249 2024-04-17T04:45:02.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.480+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=N1GAT98LZF0Z type=del size=1248640 2024-04-17T04:45:02.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.484+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=M8S5L9SRPP1U type=del 2024-04-17T04:45:02.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.488+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=JHYEVIZASFE1 type=del size=2556625 2024-04-17T04:45:02.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.488+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=TJFJJSRHMNNE type=del 2024-04-17T04:45:02.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.492+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=YMLHTZKDH1A2 type=del size=2726725 2024-04-17T04:45:02.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.496+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=1LCO70NC6Z15 type=del 2024-04-17T04:45:02.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.500+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=FMF9V0H3RBAD type=del size=1196700 2024-04-17T04:45:02.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.504+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=4MDBHYMJD24O type=del 2024-04-17T04:45:02.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.508+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=JOG2RC2PCZ4Z type=write size=300700 2024-04-17T04:45:02.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.512+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~YYWTOJ9APHR1.0 size=1048576 2024-04-17T04:45:02.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.512+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~YYWTOJ9APHR1.1 size=1048576 2024-04-17T04:45:02.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~YYWTOJ9APHR1.2 size=471790 2024-04-17T04:45:02.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.520+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=YYWTOJ9APHR1 tag=42F4LNK9A8S0 type=write size=3617518 2024-04-17T04:45:02.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.524+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=W5GP57XI8HNQ type=write size=435706 2024-04-17T04:45:02.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.528+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=F4CKO8OEZEKE type=del 2024-04-17T04:45:02.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~XYXX0MYHC1O1.0 size=1048576 2024-04-17T04:45:02.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~XYXX0MYHC1O1.1 size=463136 2024-04-17T04:45:02.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.536+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=XYXX0MYHC1O1 tag=UXBMJNUQ49SI type=write size=2560288 2024-04-17T04:45:02.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~K42MI8YMT6GT.0 size=1048576 2024-04-17T04:45:02.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~K42MI8YMT6GT.1 size=1048576 2024-04-17T04:45:02.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~K42MI8YMT6GT.2 size=602002 2024-04-17T04:45:02.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.552+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=K42MI8YMT6GT tag=21EJX5DQD2DX type=write size=3747730 2024-04-17T04:45:02.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.556+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~4ML5T19AKB14.0 size=1048576 2024-04-17T04:45:02.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~4ML5T19AKB14.1 size=97662 2024-04-17T04:45:02.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.560+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=4ML5T19AKB14 tag=FIJNA9S0ER89 type=write size=2194814 2024-04-17T04:45:02.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4X253ACR4LP9.0 size=1048576 2024-04-17T04:45:02.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.564+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4X253ACR4LP9.1 size=440866 2024-04-17T04:45:02.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.572+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=4X253ACR4LP9 tag=I6H9QB3ZVL05 type=write size=2538018 2024-04-17T04:45:02.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.572+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=H5FBU17B4WWH type=del size=3027142 2024-04-17T04:45:02.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~JOPOU989Z2UC.0 size=238291 2024-04-17T04:45:02.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.580+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=JOPOU989Z2UC tag=O1VRLLUMEF69 type=write size=1286867 2024-04-17T04:45:02.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.584+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=WFMOFJEAOX7B type=write size=545403 2024-04-17T04:45:02.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.588+0000 7fd8ef2218c0 1 < suggested remove operation key=5K50O96P7OGD tag=5H21K35405H4 type=del 2024-04-17T04:45:02.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.592+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=CKT9J7XIFDQ6 type=write size=199399 2024-04-17T04:45:02.591 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.596+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=ULQ1AWZMX5IL type=del 2024-04-17T04:45:02.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.600+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=VTAY5XB6HOG8 type=del size=3551352 2024-04-17T04:45:02.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.600+0000 7fd8ef2218c0 -1 < failed to complete operation key=1ZTJP8DRITEK tag=YUUN91F0O1PC type=del size=1016535: (22) Invalid argument 2024-04-17T04:45:02.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.604+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=72D4V7XJDW69 type=del size=4086197 2024-04-17T04:45:02.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.608+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=H864RJ7S7H4E type=del 2024-04-17T04:45:02.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.612+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=1KH9J0ZUZ4A0 type=del size=1496221 2024-04-17T04:45:02.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.612+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~KQF2XJE0M3L0.0 size=1048576 2024-04-17T04:45:02.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.616+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~KQF2XJE0M3L0.1 size=961207 2024-04-17T04:45:02.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.620+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=KQF2XJE0M3L0 tag=GUIQGAGF9WEY type=write size=3058359 2024-04-17T04:45:02.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~JD07HIRYG0DH.0 size=1048576 2024-04-17T04:45:02.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~JD07HIRYG0DH.1 size=1048576 2024-04-17T04:45:02.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.632+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~JD07HIRYG0DH.2 size=38377 2024-04-17T04:45:02.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.632+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=JD07HIRYG0DH tag=KDWASBFWXW2R type=write size=3184105 2024-04-17T04:45:02.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.636+0000 7fd8ef2218c0 1 < suggested update operation key=MS3T476JOEON tag=QJNET1X3YNQW type=del size=2537732 2024-04-17T04:45:02.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.640+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=9XI43ULPEMOR type=del size=2944050 2024-04-17T04:45:02.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.644+0000 7fd8ef2218c0 1 < canceled operation key=1JOWGWFR7R6P tag=N1GAT98LZF0Z type=del size=1248640 2024-04-17T04:45:02.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.648+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=BBXDQQK3EQGN type=del size=2126339 2024-04-17T04:45:02.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.648+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=JHYEVIZASFE1 type=del 2024-04-17T04:45:02.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.652+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=OEYKYB4NOQDR type=del size=2437316 2024-04-17T04:45:02.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.656+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=YMLHTZKDH1A2 type=del 2024-04-17T04:45:02.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.660+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=F8ZOQ733FISC type=del size=1561014 2024-04-17T04:45:02.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.664+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=FMF9V0H3RBAD type=del 2024-04-17T04:45:02.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.664+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=PMMH5GRG57FE type=del size=315286 2024-04-17T04:45:02.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.668+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=JOG2RC2PCZ4Z type=write size=300700 2024-04-17T04:45:02.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.672+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=CONV8L0V7V1H type=del size=119219 2024-04-17T04:45:02.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.676+0000 7fd8ef2218c0 1 < completed write operation key=JVWRJBMFIRLL tag=W5GP57XI8HNQ type=write size=435706 2024-04-17T04:45:02.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~8W2VN4X6RHCQ.0 size=476002 2024-04-17T04:45:02.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=8W2VN4X6RHCQ tag=Z05MBLHGMYOY type=write size=1524578 2024-04-17T04:45:02.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.684+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~XYXX0MYHC1O1.0 size=1048576 2024-04-17T04:45:02.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~XYXX0MYHC1O1.1 size=463136 2024-04-17T04:45:02.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.692+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=XYXX0MYHC1O1 tag=UXBMJNUQ49SI type=write size=2560288 2024-04-17T04:45:02.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.696+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=72UNSUTVC2VQ type=del size=1616342 2024-04-17T04:45:02.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.700+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~4ML5T19AKB14.0 size=1048576 2024-04-17T04:45:02.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.700+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~4ML5T19AKB14.1 size=97662 2024-04-17T04:45:02.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.708+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=4ML5T19AKB14 tag=FIJNA9S0ER89 type=write size=2194814 2024-04-17T04:45:02.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~0L13D5PN5X3M.0 size=865961 2024-04-17T04:45:02.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.712+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=0L13D5PN5X3M tag=HHCNREM2FHWC type=write size=1914537 2024-04-17T04:45:02.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.716+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=H5FBU17B4WWH type=del 2024-04-17T04:45:02.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.720+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=6TF25KK1DLZX type=del size=1684341 2024-04-17T04:45:02.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.728+0000 7fd8ef2218c0 1 < suggested update operation key=JVWRJBMFIRLL tag=WFMOFJEAOX7B type=write size=545403 2024-04-17T04:45:02.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.732+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~3ZVDP4JJPLUZ.0 size=1048576 2024-04-17T04:45:02.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~3ZVDP4JJPLUZ.1 size=865195 2024-04-17T04:45:02.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.736+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=3ZVDP4JJPLUZ tag=9OKC0B2QI221 type=write size=2962347 2024-04-17T04:45:02.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.740+0000 7fd8ef2218c0 1 < suggested remove operation key=WCIJICR0FT2C tag=CKT9J7XIFDQ6 type=write 2024-04-17T04:45:02.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.748+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~2XWQCOL9WG98.0 size=1048576 2024-04-17T04:45:02.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.748+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~2XWQCOL9WG98.1 size=1048576 2024-04-17T04:45:02.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.752+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~2XWQCOL9WG98.2 size=39748 2024-04-17T04:45:02.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.752+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=2XWQCOL9WG98 tag=6EF3WY3165WJ type=write size=3185476 2024-04-17T04:45:02.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.756+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=VTAY5XB6HOG8 type=del 2024-04-17T04:45:02.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.756+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=BU26AUY5D684 type=write size=527671 2024-04-17T04:45:02.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.764+0000 7fd8ef2218c0 1 < suggested remove operation key=2S1IT3TO1BPU tag=72D4V7XJDW69 type=del 2024-04-17T04:45:02.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.772+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~PLIPJ3HY6QS4.0 size=1048576 2024-04-17T04:45:02.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.772+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~PLIPJ3HY6QS4.1 size=142316 2024-04-17T04:45:02.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.772+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=PLIPJ3HY6QS4 tag=Y2RI07NSUVLO type=write size=2239468 2024-04-17T04:45:02.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.780+0000 7fd8ef2218c0 1 < suggested remove operation key=JVWRJBMFIRLL tag=1KH9J0ZUZ4A0 type=del 2024-04-17T04:45:02.777 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.780+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=2GUSI4P1IFK8 type=write size=901829 2024-04-17T04:45:02.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.784+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~JD07HIRYG0DH.0 size=1048576 2024-04-17T04:45:02.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~JD07HIRYG0DH.1 size=1048576 2024-04-17T04:45:02.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~JD07HIRYG0DH.2 size=38377 2024-04-17T04:45:02.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.792+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=JD07HIRYG0DH tag=KDWASBFWXW2R type=write size=3184105 2024-04-17T04:45:02.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~W679X3A528FV.0 size=1048576 2024-04-17T04:45:02.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~W679X3A528FV.1 size=162918 2024-04-17T04:45:02.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.800+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=W679X3A528FV tag=8KRFCAQQTY3M type=write size=2260070 2024-04-17T04:45:02.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.804+0000 7fd8ef2218c0 -1 < failed to complete operation key=JVWRJBMFIRLL tag=9XI43ULPEMOR type=del size=2944050: (22) Invalid argument 2024-04-17T04:45:02.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.808+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=2AZI0BMCW0T2 type=del size=4027999 2024-04-17T04:45:02.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.812+0000 7fd8ef2218c0 1 < canceled operation key=OZ1IST8DOGXW tag=BBXDQQK3EQGN type=del size=2126339 2024-04-17T04:45:02.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.812+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=I0ON6B09COFW type=write size=279960 2024-04-17T04:45:02.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.816+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=OEYKYB4NOQDR type=del 2024-04-17T04:45:02.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.820+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=JHP545X6YUV4 type=del size=814560 2024-04-17T04:45:02.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.824+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=F8ZOQ733FISC type=del 2024-04-17T04:45:02.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.824+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=SWHFN681CNVD type=write size=951760 2024-04-17T04:45:02.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.828+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=PMMH5GRG57FE type=del 2024-04-17T04:45:02.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.832+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=D1LJWCYJQDW8 type=del size=650676 2024-04-17T04:45:02.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.836+0000 7fd8ef2218c0 1 < suggested remove operation key=5ZULD0PHLS4V tag=CONV8L0V7V1H type=del 2024-04-17T04:45:02.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~PG30CPHBDKX3.0 size=874047 2024-04-17T04:45:02.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=PG30CPHBDKX3 tag=2AOCPS6ERBR9 type=write size=1922623 2024-04-17T04:45:02.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.844+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UYCKMXE6LBUH.2~8W2VN4X6RHCQ.0 size=476002 2024-04-17T04:45:02.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.848+0000 7fd8ef2218c0 1 < canceled multipart upload key=UYCKMXE6LBUH upload=8W2VN4X6RHCQ tag=Z05MBLHGMYOY type=write size=1524578 2024-04-17T04:45:02.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.852+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=I0Z065EHEP0E type=del size=3825824 2024-04-17T04:45:02.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.852+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=72UNSUTVC2VQ type=del 2024-04-17T04:45:02.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~HZDTSOJU0RUB.0 size=722890 2024-04-17T04:45:02.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.860+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=HZDTSOJU0RUB tag=4Q428W149R03 type=write size=1771466 2024-04-17T04:45:02.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.860+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~0L13D5PN5X3M.0 size=865961 2024-04-17T04:45:02.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.868+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=0L13D5PN5X3M tag=HHCNREM2FHWC type=write size=1914537 2024-04-17T04:45:02.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~J3NPUBLRNBKS.0 size=1048576 2024-04-17T04:45:02.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~J3NPUBLRNBKS.1 size=1048576 2024-04-17T04:45:02.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.876+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~J3NPUBLRNBKS.2 size=909518 2024-04-17T04:45:02.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.876+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=J3NPUBLRNBKS tag=IL1PJC33B8V4 type=write size=4055246 2024-04-17T04:45:02.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.880+0000 7fd8ef2218c0 1 < canceled operation key=BCG05NIQFA53 tag=6TF25KK1DLZX type=del size=1684341 2024-04-17T04:45:02.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.880+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=B269F7JCV02V type=del size=1283319 2024-04-17T04:45:02.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.884+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~3ZVDP4JJPLUZ.0 size=1048576 2024-04-17T04:45:02.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~3ZVDP4JJPLUZ.1 size=865195 2024-04-17T04:45:02.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.892+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=3ZVDP4JJPLUZ tag=9OKC0B2QI221 type=write size=2962347 2024-04-17T04:45:02.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.896+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=SRET8LIXH80V type=del size=2407663 2024-04-17T04:45:02.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.896+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~2XWQCOL9WG98.0 size=1048576 2024-04-17T04:45:02.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~2XWQCOL9WG98.1 size=1048576 2024-04-17T04:45:02.897 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~2XWQCOL9WG98.2 size=39748 2024-04-17T04:45:02.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.904+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=2XWQCOL9WG98 tag=6EF3WY3165WJ type=write size=3185476 2024-04-17T04:45:02.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.908+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=LBXCQZB7MLQB type=write size=1011350 2024-04-17T04:45:02.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.912+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=BU26AUY5D684 type=write size=527671 2024-04-17T04:45:02.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.916+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=O9UFWN3SBVNW type=del size=2194442 2024-04-17T04:45:02.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~PLIPJ3HY6QS4.0 size=1048576 2024-04-17T04:45:02.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.920+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~PLIPJ3HY6QS4.1 size=142316 2024-04-17T04:45:02.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.924+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=PLIPJ3HY6QS4 tag=Y2RI07NSUVLO type=write size=2239468 2024-04-17T04:45:02.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.924+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=PIKSOXMPI20H type=del size=1355435 2024-04-17T04:45:02.923 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.928+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=2GUSI4P1IFK8 type=write size=901829 2024-04-17T04:45:02.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.932+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=ATRS5H16WOX1 type=del size=150956 2024-04-17T04:45:02.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~W679X3A528FV.0 size=1048576 2024-04-17T04:45:02.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~W679X3A528FV.1 size=162918 2024-04-17T04:45:02.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.940+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=W679X3A528FV tag=8KRFCAQQTY3M type=write size=2260070 2024-04-17T04:45:02.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.944+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~P2X6W7GFH4A6.0 size=1048576 2024-04-17T04:45:02.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.948+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~P2X6W7GFH4A6.1 size=1048576 2024-04-17T04:45:02.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~P2X6W7GFH4A6.2 size=411335 2024-04-17T04:45:02.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.952+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=P2X6W7GFH4A6 tag=H13T6YMFMFQR type=write size=3557063 2024-04-17T04:45:02.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.952+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=2AZI0BMCW0T2 type=del 2024-04-17T04:45:02.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~JJYS434884RU.0 size=1048576 2024-04-17T04:45:02.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~JJYS434884RU.1 size=1048576 2024-04-17T04:45:02.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.964+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~JJYS434884RU.2 size=450488 2024-04-17T04:45:02.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.964+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=JJYS434884RU tag=TC9VERN9YB67 type=write size=3596216 2024-04-17T04:45:02.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.968+0000 7fd8ef2218c0 1 < suggested remove operation key=EWAW6EBW6W6U tag=I0ON6B09COFW type=write 2024-04-17T04:45:02.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.972+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=6XQI9Y6AA46K type=del size=31830 2024-04-17T04:45:02.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.976+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=JHP545X6YUV4 type=del 2024-04-17T04:45:02.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.980+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=YQCSANU8RUC3 type=del size=938000 2024-04-17T04:45:02.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.984+0000 7fd8ef2218c0 1 < completed write operation key=MS3T476JOEON tag=SWHFN681CNVD type=write size=951760 2024-04-17T04:45:02.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.988+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=5I1BJYDI9AAC type=del size=1393454 2024-04-17T04:45:02.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.992+0000 7fd8ef2218c0 -1 < failed to complete operation key=5ZULD0PHLS4V tag=D1LJWCYJQDW8 type=del size=650676: (22) Invalid argument 2024-04-17T04:45:02.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.992+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=08FKCCO8I305 type=del size=4142565 2024-04-17T04:45:02.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:02.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~PG30CPHBDKX3.0 size=874047 2024-04-17T04:45:02.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.000+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=PG30CPHBDKX3 tag=2AOCPS6ERBR9 type=write size=1922623 2024-04-17T04:45:03.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.004+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=NW2OKU45PZOH type=del size=2104275 2024-04-17T04:45:03.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.008+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=I0Z065EHEP0E type=del 2024-04-17T04:45:03.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.012+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=8DBTV7JZ2YKJ type=del size=2296304 2024-04-17T04:45:03.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~HZDTSOJU0RUB.0 size=722890 2024-04-17T04:45:03.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.020+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=HZDTSOJU0RUB tag=4Q428W149R03 type=write size=1771466 2024-04-17T04:45:03.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.020+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=5YL3WUV59VAG type=del size=3051327 2024-04-17T04:45:03.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.024+0000 7fd8ef2218c0 1 < canceled part key=_multipart_A5ZUB3YK06J3.2~J3NPUBLRNBKS.0 size=1048576 2024-04-17T04:45:03.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.024+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~J3NPUBLRNBKS.1 size=1048576 2024-04-17T04:45:03.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~J3NPUBLRNBKS.2 size=909518 2024-04-17T04:45:03.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.036+0000 7fd8ef2218c0 1 < canceled multipart upload key=A5ZUB3YK06J3 upload=J3NPUBLRNBKS tag=IL1PJC33B8V4 type=write size=4055246 2024-04-17T04:45:03.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.040+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=SLF7D14Z9SXR type=del size=1063987 2024-04-17T04:45:03.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.048+0000 7fd8ef2218c0 1 < suggested update operation key=JVWRJBMFIRLL tag=B269F7JCV02V type=del size=1283319 2024-04-17T04:45:03.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.052+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=JNDOO27OW4QI type=del size=2853275 2024-04-17T04:45:03.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.052+0000 7fd8ef2218c0 1 < canceled operation key=WCIJICR0FT2C tag=SRET8LIXH80V type=del size=2407663 2024-04-17T04:45:03.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.056+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=QGTRSD88DWNW type=del size=119288 2024-04-17T04:45:03.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.060+0000 7fd8ef2218c0 1 < canceled operation key=UYCKMXE6LBUH tag=LBXCQZB7MLQB type=write size=1011350 2024-04-17T04:45:03.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.068+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~XWRKV8KVO2K0.0 size=773025 2024-04-17T04:45:03.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.068+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=XWRKV8KVO2K0 tag=BUCBHQCGLFN8 type=write size=1821601 2024-04-17T04:45:03.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.068+0000 7fd8ef2218c0 1 < canceled operation key=5ZULD0PHLS4V tag=O9UFWN3SBVNW type=del size=2194442 2024-04-17T04:45:03.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.072+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=E7LFG2U59025 type=write size=418843 2024-04-17T04:45:03.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.076+0000 7fd8ef2218c0 1 < canceled operation key=BCG05NIQFA53 tag=PIKSOXMPI20H type=del size=1355435 2024-04-17T04:45:03.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~WEK7JNHP3X5G.0 size=1048576 2024-04-17T04:45:03.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~WEK7JNHP3X5G.1 size=1048576 2024-04-17T04:45:03.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.088+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~WEK7JNHP3X5G.2 size=331447 2024-04-17T04:45:03.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.088+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=WEK7JNHP3X5G tag=FLUNS2AG7465 type=write size=3477175 2024-04-17T04:45:03.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.096+0000 7fd8ef2218c0 1 < suggested update operation key=R3C8PAVPVGBY tag=ATRS5H16WOX1 type=del size=150956 2024-04-17T04:45:03.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~0A1J52GINKY0.0 size=1048576 2024-04-17T04:45:03.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~0A1J52GINKY0.1 size=1048576 2024-04-17T04:45:03.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~0A1J52GINKY0.2 size=308770 2024-04-17T04:45:03.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.104+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=0A1J52GINKY0 tag=E3T4KT6XPW2K type=write size=3454498 2024-04-17T04:45:03.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~P2X6W7GFH4A6.0 size=1048576 2024-04-17T04:45:03.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.112+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~P2X6W7GFH4A6.1 size=1048576 2024-04-17T04:45:03.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~P2X6W7GFH4A6.2 size=411335 2024-04-17T04:45:03.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.120+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=P2X6W7GFH4A6 tag=H13T6YMFMFQR type=write size=3557063 2024-04-17T04:45:03.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.124+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=XZUJYAL9YKQG type=del size=2653175 2024-04-17T04:45:03.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.128+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~JJYS434884RU.0 size=1048576 2024-04-17T04:45:03.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.128+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~JJYS434884RU.1 size=1048576 2024-04-17T04:45:03.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~JJYS434884RU.2 size=450488 2024-04-17T04:45:03.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.136+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=JJYS434884RU tag=TC9VERN9YB67 type=write size=3596216 2024-04-17T04:45:03.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~FFYTOTUPNMUE.0 size=1048576 2024-04-17T04:45:03.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~FFYTOTUPNMUE.1 size=426927 2024-04-17T04:45:03.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.144+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=FFYTOTUPNMUE tag=0CFXT3YKJ5LE type=write size=2524079 2024-04-17T04:45:03.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.148+0000 7fd8ef2218c0 -1 < failed to complete operation key=JVWRJBMFIRLL tag=6XQI9Y6AA46K type=del size=31830: (22) Invalid argument 2024-04-17T04:45:03.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.152+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=NRJMN1M5YRV4 type=del size=176402 2024-04-17T04:45:03.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.156+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=YQCSANU8RUC3 type=del 2024-04-17T04:45:03.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.160+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=6A5M74D2RRKD type=del size=4014283 2024-04-17T04:45:03.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.164+0000 7fd8ef2218c0 1 < suggested remove operation key=3OFQH4KVYN18 tag=5I1BJYDI9AAC type=del 2024-04-17T04:45:03.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.172+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~XJOTYTAV7W26.0 size=1048576 2024-04-17T04:45:03.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.176+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~XJOTYTAV7W26.1 size=653729 2024-04-17T04:45:03.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.176+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=XJOTYTAV7W26 tag=HEUTO6ZYAMLV type=write size=2750881 2024-04-17T04:45:03.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.176+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=08FKCCO8I305 type=del 2024-04-17T04:45:03.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.180+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=P07KA1Y0VBRT type=del size=1739 2024-04-17T04:45:03.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.184+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=NW2OKU45PZOH type=del 2024-04-17T04:45:03.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.188+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=XCTFMWRIHZEY type=del size=3489863 2024-04-17T04:45:03.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.192+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=8DBTV7JZ2YKJ type=del 2024-04-17T04:45:03.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.196+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~VMIUZ9O6ZA49.0 size=900290 2024-04-17T04:45:03.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.196+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=VMIUZ9O6ZA49 tag=SBA8XK2D92KH type=write size=1948866 2024-04-17T04:45:03.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.200+0000 7fd8ef2218c0 1 < canceled operation key=509DZ9DBJWC5 tag=5YL3WUV59VAG type=del size=3051327 2024-04-17T04:45:03.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.204+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=SKOBHNKLVBJ1 type=del size=3222889 2024-04-17T04:45:03.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.212+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=SLF7D14Z9SXR type=del 2024-04-17T04:45:03.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4W8GWMHUWPZE.0 size=1048576 2024-04-17T04:45:03.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4W8GWMHUWPZE.1 size=598304 2024-04-17T04:45:03.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.220+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=4W8GWMHUWPZE tag=QGTV15NU9UX4 type=write size=2695456 2024-04-17T04:45:03.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.228+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=JNDOO27OW4QI type=del 2024-04-17T04:45:03.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.232+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=35W456E1QJ1Y type=del size=3670085 2024-04-17T04:45:03.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.236+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=QGTRSD88DWNW type=del 2024-04-17T04:45:03.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.240+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=UECQ1HRNMOF0 type=del size=400278 2024-04-17T04:45:03.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.240+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~XWRKV8KVO2K0.0 size=773025 2024-04-17T04:45:03.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.248+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=XWRKV8KVO2K0 tag=BUCBHQCGLFN8 type=write size=1821601 2024-04-17T04:45:03.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~7XTHECWFJCN4.0 size=903458 2024-04-17T04:45:03.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.252+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=7XTHECWFJCN4 tag=HIM0T09UPSO8 type=write size=1952034 2024-04-17T04:45:03.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.256+0000 7fd8ef2218c0 1 < completed write operation key=UYCKMXE6LBUH tag=E7LFG2U59025 type=write size=418843 2024-04-17T04:45:03.256 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.260+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=0DAMF6FGNSLS type=del size=3310434 2024-04-17T04:45:03.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~WEK7JNHP3X5G.0 size=1048576 2024-04-17T04:45:03.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~WEK7JNHP3X5G.1 size=1048576 2024-04-17T04:45:03.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~WEK7JNHP3X5G.2 size=331447 2024-04-17T04:45:03.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.276+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=WEK7JNHP3X5G tag=FLUNS2AG7465 type=write size=3477175 2024-04-17T04:45:03.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.280+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=GT9DH9UZ8XF3 type=write size=641644 2024-04-17T04:45:03.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.284+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~0A1J52GINKY0.0 size=1048576 2024-04-17T04:45:03.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.284+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~0A1J52GINKY0.1 size=1048576 2024-04-17T04:45:03.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.288+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~0A1J52GINKY0.2 size=308770 2024-04-17T04:45:03.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.296+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=0A1J52GINKY0 tag=E3T4KT6XPW2K type=write size=3454498 2024-04-17T04:45:03.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.300+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=XZHKJCNNVZHD type=del size=3730066 2024-04-17T04:45:03.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.304+0000 7fd8ef2218c0 -1 < failed to complete operation key=3OFQH4KVYN18 tag=XZUJYAL9YKQG type=del size=2653175: (22) Invalid argument 2024-04-17T04:45:03.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~AG320J6GD8LA.0 size=816600 2024-04-17T04:45:03.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.308+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=AG320J6GD8LA tag=XHLQ1TNQEVOY type=write size=1865176 2024-04-17T04:45:03.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~FFYTOTUPNMUE.0 size=1048576 2024-04-17T04:45:03.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~FFYTOTUPNMUE.1 size=426927 2024-04-17T04:45:03.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.320+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=FFYTOTUPNMUE tag=0CFXT3YKJ5LE type=write size=2524079 2024-04-17T04:45:03.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~PWY8009069LY.0 size=1048576 2024-04-17T04:45:03.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~PWY8009069LY.1 size=1048576 2024-04-17T04:45:03.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~PWY8009069LY.2 size=18702 2024-04-17T04:45:03.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.332+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=PWY8009069LY tag=YK3B41W14T6A type=write size=3164430 2024-04-17T04:45:03.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.336+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=NRJMN1M5YRV4 type=del 2024-04-17T04:45:03.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.340+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=L0P864QOOUF1 type=del size=3269506 2024-04-17T04:45:03.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.344+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=6A5M74D2RRKD type=del 2024-04-17T04:45:03.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.348+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=9UOJJQCRKTQF type=del size=3315577 2024-04-17T04:45:03.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~XJOTYTAV7W26.0 size=1048576 2024-04-17T04:45:03.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~XJOTYTAV7W26.1 size=653729 2024-04-17T04:45:03.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.364+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=XJOTYTAV7W26 tag=HEUTO6ZYAMLV type=write size=2750881 2024-04-17T04:45:03.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.368+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=KVK0E2XIX3SK type=del size=3677290 2024-04-17T04:45:03.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.372+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=P07KA1Y0VBRT type=del 2024-04-17T04:45:03.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.372+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=TQ88MC0CIH8P type=del size=286878 2024-04-17T04:45:03.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.376+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=XCTFMWRIHZEY type=del 2024-04-17T04:45:03.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~VSGXQ7XQ20S0.0 size=1048576 2024-04-17T04:45:03.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~VSGXQ7XQ20S0.1 size=1048576 2024-04-17T04:45:03.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~VSGXQ7XQ20S0.2 size=791132 2024-04-17T04:45:03.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.388+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=VSGXQ7XQ20S0 tag=697REJR8REBU type=write size=3936860 2024-04-17T04:45:03.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.392+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~VMIUZ9O6ZA49.0 size=900290 2024-04-17T04:45:03.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.400+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=VMIUZ9O6ZA49 tag=SBA8XK2D92KH type=write size=1948866 2024-04-17T04:45:03.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.404+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=VMP1F5SPXSAI type=del size=106071 2024-04-17T04:45:03.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.412+0000 7fd8ef2218c0 1 < suggested remove operation key=5ZULD0PHLS4V tag=SKOBHNKLVBJ1 type=del 2024-04-17T04:45:03.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.412+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=N15MM8LZ5JRT type=del size=2778137 2024-04-17T04:45:03.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.416+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~4W8GWMHUWPZE.0 size=1048576 2024-04-17T04:45:03.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~4W8GWMHUWPZE.1 size=598304 2024-04-17T04:45:03.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.424+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=4W8GWMHUWPZE tag=QGTV15NU9UX4 type=write size=2695456 2024-04-17T04:45:03.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~B76N4KXNEVW8.0 size=1048576 2024-04-17T04:45:03.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~B76N4KXNEVW8.1 size=189985 2024-04-17T04:45:03.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.432+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=B76N4KXNEVW8 tag=F9SXD8Q8ZI0W type=write size=2287137 2024-04-17T04:45:03.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.436+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=35W456E1QJ1Y type=del 2024-04-17T04:45:03.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.440+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=TOQIEK05Q6EQ type=del size=2129581 2024-04-17T04:45:03.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.444+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=UECQ1HRNMOF0 type=del 2024-04-17T04:45:03.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.444+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=0DHIUWA440KP type=del size=2579504 2024-04-17T04:45:03.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.448+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~7XTHECWFJCN4.0 size=903458 2024-04-17T04:45:03.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.452+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=7XTHECWFJCN4 tag=HIM0T09UPSO8 type=write size=1952034 2024-04-17T04:45:03.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.460+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~XZ1YQ65RRT7Z.0 size=1048576 2024-04-17T04:45:03.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.460+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~XZ1YQ65RRT7Z.1 size=886480 2024-04-17T04:45:03.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.460+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=XZ1YQ65RRT7Z tag=K2CWPT6UVMUZ type=write size=2983632 2024-04-17T04:45:03.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.464+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=0DAMF6FGNSLS type=del 2024-04-17T04:45:03.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.472+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~T9965T4KTATG.0 size=1048576 2024-04-17T04:45:03.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.472+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~T9965T4KTATG.1 size=939222 2024-04-17T04:45:03.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.472+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=T9965T4KTATG tag=K8VGBC3WK2G7 type=write size=3036374 2024-04-17T04:45:03.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.480+0000 7fd8ef2218c0 1 < suggested remove operation key=AE4CYL334ONO tag=GT9DH9UZ8XF3 type=write 2024-04-17T04:45:03.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.484+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=AJNUE1CX2JKF type=write size=218667 2024-04-17T04:45:03.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.488+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=XZHKJCNNVZHD type=del 2024-04-17T04:45:03.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.492+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~3A4JKRC6BFM9.0 size=1048576 2024-04-17T04:45:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.496+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~3A4JKRC6BFM9.1 size=1048576 2024-04-17T04:45:03.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.496+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~3A4JKRC6BFM9.2 size=654327 2024-04-17T04:45:03.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.496+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=3A4JKRC6BFM9 tag=EE6PF00QK3RW type=write size=3800055 2024-04-17T04:45:03.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~AG320J6GD8LA.0 size=816600 2024-04-17T04:45:03.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.508+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=AG320J6GD8LA tag=XHLQ1TNQEVOY type=write size=1865176 2024-04-17T04:45:03.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~23PZ3VF252W5.0 size=861098 2024-04-17T04:45:03.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=23PZ3VF252W5 tag=9W9RHADLWA7E type=write size=1909674 2024-04-17T04:45:03.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~PWY8009069LY.0 size=1048576 2024-04-17T04:45:03.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~PWY8009069LY.1 size=1048576 2024-04-17T04:45:03.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.520+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~PWY8009069LY.2 size=18702 2024-04-17T04:45:03.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.524+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=PWY8009069LY tag=YK3B41W14T6A type=write size=3164430 2024-04-17T04:45:03.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.528+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=I16BWMRU8DZF type=write size=1014022 2024-04-17T04:45:03.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.528+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=L0P864QOOUF1 type=del 2024-04-17T04:45:03.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~9T5DIHD19AWW.0 size=1048576 2024-04-17T04:45:03.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~9T5DIHD19AWW.1 size=1048576 2024-04-17T04:45:03.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.540+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~9T5DIHD19AWW.2 size=955898 2024-04-17T04:45:03.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.540+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=9T5DIHD19AWW tag=Z7UCMD7AJXH0 type=write size=4101626 2024-04-17T04:45:03.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.540+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=9UOJJQCRKTQF type=del 2024-04-17T04:45:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~0ZF79364G7AH.0 size=930413 2024-04-17T04:45:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.548+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=0ZF79364G7AH tag=J44LTHOWNS2U type=write size=1978989 2024-04-17T04:45:03.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.548+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=KVK0E2XIX3SK type=del 2024-04-17T04:45:03.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.552+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=GSJFPZ6OFP3Q type=del size=1859488 2024-04-17T04:45:03.550 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.556+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=TQ88MC0CIH8P type=del 2024-04-17T04:45:03.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.556+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=AID56PRW5YOH type=del size=444142 2024-04-17T04:45:03.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~VSGXQ7XQ20S0.0 size=1048576 2024-04-17T04:45:03.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.564+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~VSGXQ7XQ20S0.1 size=1048576 2024-04-17T04:45:03.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.564+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~VSGXQ7XQ20S0.2 size=791132 2024-04-17T04:45:03.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.568+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=VSGXQ7XQ20S0 tag=697REJR8REBU type=write size=3936860 2024-04-17T04:45:03.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.572+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=ZM22BN6BLYA4 type=del size=3098423 2024-04-17T04:45:03.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.576+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=VMP1F5SPXSAI type=del 2024-04-17T04:45:03.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.580+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=I7MNH61HMQW4 type=del size=3744044 2024-04-17T04:45:03.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.584+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=N15MM8LZ5JRT type=del 2024-04-17T04:45:03.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.588+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=GYLNFNFG0V1K type=del size=1626531 2024-04-17T04:45:03.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.588+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~B76N4KXNEVW8.0 size=1048576 2024-04-17T04:45:03.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.592+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~B76N4KXNEVW8.1 size=189985 2024-04-17T04:45:03.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.596+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=B76N4KXNEVW8 tag=F9SXD8Q8ZI0W type=write size=2287137 2024-04-17T04:45:03.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~8UVK8YFKFTUG.0 size=504515 2024-04-17T04:45:03.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.604+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=8UVK8YFKFTUG tag=B8VAMJG8MSMB type=write size=1553091 2024-04-17T04:45:03.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.608+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=TOQIEK05Q6EQ type=del 2024-04-17T04:45:03.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.612+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=9VQ1G7YK122B type=del size=1073709 2024-04-17T04:45:03.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.616+0000 7fd8ef2218c0 1 < canceled operation key=3OFQH4KVYN18 tag=0DHIUWA440KP type=del size=2579504 2024-04-17T04:45:03.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.620+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=0KSPQOHUMDIY type=del size=1169017 2024-04-17T04:45:03.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~XZ1YQ65RRT7Z.0 size=1048576 2024-04-17T04:45:03.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.624+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~XZ1YQ65RRT7Z.1 size=886480 2024-04-17T04:45:03.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.628+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=XZ1YQ65RRT7Z tag=K2CWPT6UVMUZ type=write size=2983632 2024-04-17T04:45:03.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.632+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=L6MCBMK6C8WF type=write size=140259 2024-04-17T04:45:03.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~T9965T4KTATG.0 size=1048576 2024-04-17T04:45:03.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~T9965T4KTATG.1 size=939222 2024-04-17T04:45:03.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.644+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=T9965T4KTATG tag=K8VGBC3WK2G7 type=write size=3036374 2024-04-17T04:45:03.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.652+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~FQIZIIQR7K17.0 size=1048576 2024-04-17T04:45:03.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.652+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~FQIZIIQR7K17.1 size=1048576 2024-04-17T04:45:03.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~FQIZIIQR7K17.2 size=258032 2024-04-17T04:45:03.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.656+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=FQIZIIQR7K17 tag=YMIU011621SU type=write size=3403760 2024-04-17T04:45:03.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.660+0000 7fd8ef2218c0 1 < completed write operation key=A5ZUB3YK06J3 tag=AJNUE1CX2JKF type=write size=218667 2024-04-17T04:45:03.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.664+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=6W7DG0E4H78G type=del size=3550515 2024-04-17T04:45:03.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.664+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~3A4JKRC6BFM9.0 size=1048576 2024-04-17T04:45:03.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~3A4JKRC6BFM9.1 size=1048576 2024-04-17T04:45:03.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.672+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~3A4JKRC6BFM9.2 size=654327 2024-04-17T04:45:03.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.676+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=3A4JKRC6BFM9 tag=EE6PF00QK3RW type=write size=3800055 2024-04-17T04:45:03.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~LJ0PTGOOU1M6.0 size=1048576 2024-04-17T04:45:03.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.684+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~LJ0PTGOOU1M6.1 size=1048576 2024-04-17T04:45:03.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.684+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~LJ0PTGOOU1M6.2 size=350363 2024-04-17T04:45:03.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.684+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=LJ0PTGOOU1M6 tag=6ZT5HHST6MCP type=write size=3496091 2024-04-17T04:45:03.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~23PZ3VF252W5.0 size=861098 2024-04-17T04:45:03.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.692+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=23PZ3VF252W5 tag=9W9RHADLWA7E type=write size=1909674 2024-04-17T04:45:03.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.696+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=MB8TFMVZVYWW type=del size=2516833 2024-04-17T04:45:03.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.700+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=I16BWMRU8DZF type=write size=1014022 2024-04-17T04:45:03.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~E8M4B4XEFVK7.0 size=1048576 2024-04-17T04:45:03.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.708+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~E8M4B4XEFVK7.1 size=690597 2024-04-17T04:45:03.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.708+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=E8M4B4XEFVK7 tag=6M1IZNR4SPZ7 type=write size=2787749 2024-04-17T04:45:03.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~9T5DIHD19AWW.0 size=1048576 2024-04-17T04:45:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.712+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~9T5DIHD19AWW.1 size=1048576 2024-04-17T04:45:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.712+0000 7fd8ef2218c0 1 < canceled part key=_multipart_G03VLS09F3BT.2~9T5DIHD19AWW.2 size=955898 2024-04-17T04:45:03.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.716+0000 7fd8ef2218c0 1 < canceled multipart upload key=G03VLS09F3BT upload=9T5DIHD19AWW tag=Z7UCMD7AJXH0 type=write size=4101626 2024-04-17T04:45:03.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~ECCLEZ7PX54A.0 size=1048576 2024-04-17T04:45:03.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.724+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~ECCLEZ7PX54A.1 size=1048576 2024-04-17T04:45:03.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.724+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~ECCLEZ7PX54A.2 size=486358 2024-04-17T04:45:03.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.724+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=ECCLEZ7PX54A tag=NGZGEA3EY7O4 type=write size=3632086 2024-04-17T04:45:03.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.728+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~0ZF79364G7AH.0 size=930413 2024-04-17T04:45:03.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.732+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=0ZF79364G7AH tag=J44LTHOWNS2U type=write size=1978989 2024-04-17T04:45:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~3W2HX4PO2LB3.0 size=1048576 2024-04-17T04:45:03.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~3W2HX4PO2LB3.1 size=1048576 2024-04-17T04:45:03.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~3W2HX4PO2LB3.2 size=646846 2024-04-17T04:45:03.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=3W2HX4PO2LB3 tag=4EH92AG4KNMX type=write size=3792574 2024-04-17T04:45:03.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.748+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=GSJFPZ6OFP3Q type=del 2024-04-17T04:45:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.752+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~33OPLGMDH9EU.0 size=279961 2024-04-17T04:45:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.752+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=33OPLGMDH9EU tag=Z4L785KMF5AO type=write size=1328537 2024-04-17T04:45:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.760+0000 7fd8ef2218c0 1 < suggested update operation key=6WSZ5UL0V1Y1 tag=AID56PRW5YOH type=del size=444142 2024-04-17T04:45:03.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~AZUD13OZDKUR.0 size=1048576 2024-04-17T04:45:03.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~AZUD13OZDKUR.1 size=1048576 2024-04-17T04:45:03.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.768+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~AZUD13OZDKUR.2 size=851809 2024-04-17T04:45:03.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.768+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=AZUD13OZDKUR tag=STL083REUNZ8 type=write size=3997537 2024-04-17T04:45:03.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.776+0000 7fd8ef2218c0 1 < suggested remove operation key=JVWRJBMFIRLL tag=ZM22BN6BLYA4 type=del 2024-04-17T04:45:03.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.780+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~ZARXV0YTYZHD.0 size=1048576 2024-04-17T04:45:03.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~ZARXV0YTYZHD.1 size=1048576 2024-04-17T04:45:03.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~ZARXV0YTYZHD.2 size=808959 2024-04-17T04:45:03.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.784+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=ZARXV0YTYZHD tag=YVG3TZRFG1I9 type=write size=3954687 2024-04-17T04:45:03.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.788+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=I7MNH61HMQW4 type=del 2024-04-17T04:45:03.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.792+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=HD81VUEK8YON type=del size=49484 2024-04-17T04:45:03.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.792+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=GYLNFNFG0V1K type=del 2024-04-17T04:45:03.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~O3ME2ZD3WE92.0 size=1048576 2024-04-17T04:45:03.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~O3ME2ZD3WE92.1 size=1048576 2024-04-17T04:45:03.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.804+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~O3ME2ZD3WE92.2 size=28395 2024-04-17T04:45:03.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.804+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=O3ME2ZD3WE92 tag=1DVMACXK129M type=write size=3174123 2024-04-17T04:45:03.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~8UVK8YFKFTUG.0 size=504515 2024-04-17T04:45:03.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.812+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=8UVK8YFKFTUG tag=B8VAMJG8MSMB type=write size=1553091 2024-04-17T04:45:03.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~2MBPIG62TY0L.0 size=8872 2024-04-17T04:45:03.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.816+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=2MBPIG62TY0L tag=JROHU07EDQGI type=write size=1057448 2024-04-17T04:45:03.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.820+0000 7fd8ef2218c0 1 < canceled operation key=EWAW6EBW6W6U tag=9VQ1G7YK122B type=del size=1073709 2024-04-17T04:45:03.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.820+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=PSKT0ZRKCRE4 type=del size=2653761 2024-04-17T04:45:03.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.828+0000 7fd8ef2218c0 1 < suggested update operation key=EWAW6EBW6W6U tag=0KSPQOHUMDIY type=del size=1169017 2024-04-17T04:45:03.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~USG4F3I9NPNN.0 size=299633 2024-04-17T04:45:03.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=USG4F3I9NPNN tag=JA0E444Q9VD8 type=write size=1348209 2024-04-17T04:45:03.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.836+0000 7fd8ef2218c0 1 < completed write operation key=WCIJICR0FT2C tag=L6MCBMK6C8WF type=write size=140259 2024-04-17T04:45:03.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~GG3YUQCONIPB.0 size=1048576 2024-04-17T04:45:03.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~GG3YUQCONIPB.1 size=922127 2024-04-17T04:45:03.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=GG3YUQCONIPB tag=BNQLOAUUJLF4 type=write size=3019279 2024-04-17T04:45:03.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~FQIZIIQR7K17.0 size=1048576 2024-04-17T04:45:03.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~FQIZIIQR7K17.1 size=1048576 2024-04-17T04:45:03.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.848+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~FQIZIIQR7K17.2 size=258032 2024-04-17T04:45:03.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.852+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=FQIZIIQR7K17 tag=YMIU011621SU type=write size=3403760 2024-04-17T04:45:03.853 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.856+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~IQ7RH8RQ1WK8.0 size=1048576 2024-04-17T04:45:03.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~IQ7RH8RQ1WK8.1 size=1048576 2024-04-17T04:45:03.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~IQ7RH8RQ1WK8.2 size=78851 2024-04-17T04:45:03.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.864+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=IQ7RH8RQ1WK8 tag=TBOIFXXKM2TP type=write size=3224579 2024-04-17T04:45:03.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.864+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=6W7DG0E4H78G type=del 2024-04-17T04:45:03.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~XN32R9EC9B73.0 size=1048576 2024-04-17T04:45:03.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~XN32R9EC9B73.1 size=848436 2024-04-17T04:45:03.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.872+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=XN32R9EC9B73 tag=LM79IIMNAGQM type=write size=2945588 2024-04-17T04:45:03.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~LJ0PTGOOU1M6.0 size=1048576 2024-04-17T04:45:03.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~LJ0PTGOOU1M6.1 size=1048576 2024-04-17T04:45:03.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.876+0000 7fd8ef2218c0 1 < canceled part key=_multipart_OZ1IST8DOGXW.2~LJ0PTGOOU1M6.2 size=350363 2024-04-17T04:45:03.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.880+0000 7fd8ef2218c0 1 < canceled multipart upload key=OZ1IST8DOGXW upload=LJ0PTGOOU1M6 tag=6ZT5HHST6MCP type=write size=3496091 2024-04-17T04:45:03.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.884+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=3GNV2J70I7FJ type=del size=4165237 2024-04-17T04:45:03.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.888+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=MB8TFMVZVYWW type=del 2024-04-17T04:45:03.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.892+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=60SVEP2Q1GPA type=del size=581341 2024-04-17T04:45:03.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~E8M4B4XEFVK7.0 size=1048576 2024-04-17T04:45:03.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.896+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~E8M4B4XEFVK7.1 size=690597 2024-04-17T04:45:03.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.900+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=E8M4B4XEFVK7 tag=6M1IZNR4SPZ7 type=write size=2787749 2024-04-17T04:45:03.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~AZN1AJ246J7M.0 size=734564 2024-04-17T04:45:03.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.904+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=AZN1AJ246J7M tag=03DI2ZHHWUW7 type=write size=1783140 2024-04-17T04:45:03.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.908+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~ECCLEZ7PX54A.0 size=1048576 2024-04-17T04:45:03.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~ECCLEZ7PX54A.1 size=1048576 2024-04-17T04:45:03.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~ECCLEZ7PX54A.2 size=486358 2024-04-17T04:45:03.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.920+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=ECCLEZ7PX54A tag=NGZGEA3EY7O4 type=write size=3632086 2024-04-17T04:45:03.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~AQ06LUJNT7ST.0 size=1048576 2024-04-17T04:45:03.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~AQ06LUJNT7ST.1 size=1048576 2024-04-17T04:45:03.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~AQ06LUJNT7ST.2 size=581351 2024-04-17T04:45:03.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.928+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=AQ06LUJNT7ST tag=LYPLOI29ZISA type=write size=3727079 2024-04-17T04:45:03.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~3W2HX4PO2LB3.0 size=1048576 2024-04-17T04:45:03.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~3W2HX4PO2LB3.1 size=1048576 2024-04-17T04:45:03.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~3W2HX4PO2LB3.2 size=646846 2024-04-17T04:45:03.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.940+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=3W2HX4PO2LB3 tag=4EH92AG4KNMX type=write size=3792574 2024-04-17T04:45:03.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.944+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~HX7ED3JH1A0P.0 size=820291 2024-04-17T04:45:03.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.944+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=HX7ED3JH1A0P tag=B9622OBUUIU1 type=write size=1868867 2024-04-17T04:45:03.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~33OPLGMDH9EU.0 size=279961 2024-04-17T04:45:03.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.952+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=33OPLGMDH9EU tag=Z4L785KMF5AO type=write size=1328537 2024-04-17T04:45:03.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.956+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~6G51WVMLEXJG.0 size=1048576 2024-04-17T04:45:03.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.956+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~6G51WVMLEXJG.1 size=806197 2024-04-17T04:45:03.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.956+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=6G51WVMLEXJG tag=BDLT9X1IQV0K type=write size=2903349 2024-04-17T04:45:03.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.960+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~AZUD13OZDKUR.0 size=1048576 2024-04-17T04:45:03.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.960+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~AZUD13OZDKUR.1 size=1048576 2024-04-17T04:45:03.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.964+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~AZUD13OZDKUR.2 size=851809 2024-04-17T04:45:03.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.968+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=AZUD13OZDKUR tag=STL083REUNZ8 type=write size=3997537 2024-04-17T04:45:03.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.968+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=09VNHE08O52R type=del size=1578160 2024-04-17T04:45:03.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~ZARXV0YTYZHD.0 size=1048576 2024-04-17T04:45:03.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.976+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~ZARXV0YTYZHD.1 size=1048576 2024-04-17T04:45:03.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.976+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~ZARXV0YTYZHD.2 size=808959 2024-04-17T04:45:03.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.980+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=ZARXV0YTYZHD tag=YVG3TZRFG1I9 type=write size=3954687 2024-04-17T04:45:03.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~TVTJIAWO26AP.0 size=1048576 2024-04-17T04:45:03.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~TVTJIAWO26AP.1 size=1048576 2024-04-17T04:45:03.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~TVTJIAWO26AP.2 size=547743 2024-04-17T04:45:03.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.988+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=TVTJIAWO26AP tag=31N1LMBWXK3U type=write size=3693471 2024-04-17T04:45:03.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.996+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=HD81VUEK8YON type=del 2024-04-17T04:45:03.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:03.996+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=K36RYDKEB3GV type=del size=3032620 2024-04-17T04:45:03.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.000+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~O3ME2ZD3WE92.0 size=1048576 2024-04-17T04:45:03.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.000+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~O3ME2ZD3WE92.1 size=1048576 2024-04-17T04:45:03.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.004+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~O3ME2ZD3WE92.2 size=28395 2024-04-17T04:45:04.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.008+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=O3ME2ZD3WE92 tag=1DVMACXK129M type=write size=3174123 2024-04-17T04:45:04.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~BFDDNPG7BEKL.0 size=1048576 2024-04-17T04:45:04.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~BFDDNPG7BEKL.1 size=1048576 2024-04-17T04:45:04.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~BFDDNPG7BEKL.2 size=35004 2024-04-17T04:45:04.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.016+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=BFDDNPG7BEKL tag=Q65L2LQ2KS3I type=write size=3180732 2024-04-17T04:45:04.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.020+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~2MBPIG62TY0L.0 size=8872 2024-04-17T04:45:04.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.024+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=2MBPIG62TY0L tag=JROHU07EDQGI type=write size=1057448 2024-04-17T04:45:04.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.028+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=D90Y9AKUS7Y2 type=write size=673072 2024-04-17T04:45:04.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.028+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=PSKT0ZRKCRE4 type=del 2024-04-17T04:45:04.029 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.036+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~74AU1C0EGZLI.0 size=1048576 2024-04-17T04:45:04.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.036+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~74AU1C0EGZLI.1 size=855869 2024-04-17T04:45:04.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.036+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=74AU1C0EGZLI tag=TZKRD543PF13 type=write size=2953021 2024-04-17T04:45:04.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~USG4F3I9NPNN.0 size=299633 2024-04-17T04:45:04.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.044+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=USG4F3I9NPNN tag=JA0E444Q9VD8 type=write size=1348209 2024-04-17T04:45:04.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.052+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=250D7EO9RP4U type=del size=557852 2024-04-17T04:45:04.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.052+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~GG3YUQCONIPB.0 size=1048576 2024-04-17T04:45:04.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.052+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~GG3YUQCONIPB.1 size=922127 2024-04-17T04:45:04.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.060+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=GG3YUQCONIPB tag=BNQLOAUUJLF4 type=write size=3019279 2024-04-17T04:45:04.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.060+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=URT8GO010YAT type=del size=1767577 2024-04-17T04:45:04.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.060+0000 7fd8ef2218c0 1 < canceled part key=_multipart_509DZ9DBJWC5.2~IQ7RH8RQ1WK8.0 size=1048576 2024-04-17T04:45:04.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~IQ7RH8RQ1WK8.1 size=1048576 2024-04-17T04:45:04.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~IQ7RH8RQ1WK8.2 size=78851 2024-04-17T04:45:04.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.072+0000 7fd8ef2218c0 1 < canceled multipart upload key=509DZ9DBJWC5 upload=IQ7RH8RQ1WK8 tag=TBOIFXXKM2TP type=write size=3224579 2024-04-17T04:45:04.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~CM56Y4DI8E4J.0 size=1048576 2024-04-17T04:45:04.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~CM56Y4DI8E4J.1 size=1048576 2024-04-17T04:45:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.080+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~CM56Y4DI8E4J.2 size=536460 2024-04-17T04:45:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.080+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=CM56Y4DI8E4J tag=3RTOTER2ENPQ type=write size=3682188 2024-04-17T04:45:04.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.080+0000 7fd8ef2218c0 1 < canceled part key=_multipart_N11Y8FPN9XES.2~XN32R9EC9B73.0 size=1048576 2024-04-17T04:45:04.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~XN32R9EC9B73.1 size=848436 2024-04-17T04:45:04.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.084+0000 7fd8ef2218c0 1 < canceled multipart upload key=N11Y8FPN9XES upload=XN32R9EC9B73 tag=LM79IIMNAGQM type=write size=2945588 2024-04-17T04:45:04.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~2S3E6FWT0SA6.0 size=1048576 2024-04-17T04:45:04.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~2S3E6FWT0SA6.1 size=260374 2024-04-17T04:45:04.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.092+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=2S3E6FWT0SA6 tag=T05LWDJWG4UX type=write size=2357526 2024-04-17T04:45:04.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.096+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=3GNV2J70I7FJ type=del 2024-04-17T04:45:04.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.096+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=TXM9D39XA5JK type=del size=3264631 2024-04-17T04:45:04.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.104+0000 7fd8ef2218c0 1 < suggested update operation key=UEWLJ3E9ODUL tag=60SVEP2Q1GPA type=del size=581341 2024-04-17T04:45:04.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.104+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=LU5UUD5KRKBP type=del size=766869 2024-04-17T04:45:04.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~AZN1AJ246J7M.0 size=734564 2024-04-17T04:45:04.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.112+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=AZN1AJ246J7M tag=03DI2ZHHWUW7 type=write size=1783140 2024-04-17T04:45:04.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.116+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=6YKK02LLWG49 type=del size=2053726 2024-04-17T04:45:04.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~AQ06LUJNT7ST.0 size=1048576 2024-04-17T04:45:04.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.116+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~AQ06LUJNT7ST.1 size=1048576 2024-04-17T04:45:04.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~AQ06LUJNT7ST.2 size=581351 2024-04-17T04:45:04.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.124+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=AQ06LUJNT7ST tag=LYPLOI29ZISA type=write size=3727079 2024-04-17T04:45:04.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.128+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=G4G2A55WBHYZ type=del size=3628160 2024-04-17T04:45:04.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.128+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~HX7ED3JH1A0P.0 size=820291 2024-04-17T04:45:04.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.132+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=HX7ED3JH1A0P tag=B9622OBUUIU1 type=write size=1868867 2024-04-17T04:45:04.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.136+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=B5C5NNGJ78M4 type=del size=1390968 2024-04-17T04:45:04.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.140+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~6G51WVMLEXJG.0 size=1048576 2024-04-17T04:45:04.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.140+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~6G51WVMLEXJG.1 size=806197 2024-04-17T04:45:04.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.144+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=6G51WVMLEXJG tag=BDLT9X1IQV0K type=write size=2903349 2024-04-17T04:45:04.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~52RHHTCHHQ88.0 size=1048576 2024-04-17T04:45:04.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.156+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~52RHHTCHHQ88.1 size=1048576 2024-04-17T04:45:04.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.156+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~52RHHTCHHQ88.2 size=1038101 2024-04-17T04:45:04.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.156+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=52RHHTCHHQ88 tag=47AQF6XO9W8L type=write size=4183829 2024-04-17T04:45:04.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.164+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=09VNHE08O52R type=del 2024-04-17T04:45:04.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.168+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~TDIN4QLEHGZX.0 size=1048576 2024-04-17T04:45:04.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.168+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~TDIN4QLEHGZX.1 size=1048576 2024-04-17T04:45:04.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.172+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~TDIN4QLEHGZX.2 size=970668 2024-04-17T04:45:04.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.172+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=TDIN4QLEHGZX tag=GV4BYJQJ402R type=write size=4116396 2024-04-17T04:45:04.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.172+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~TVTJIAWO26AP.0 size=1048576 2024-04-17T04:45:04.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~TVTJIAWO26AP.1 size=1048576 2024-04-17T04:45:04.172 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~TVTJIAWO26AP.2 size=547743 2024-04-17T04:45:04.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.180+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=TVTJIAWO26AP tag=31N1LMBWXK3U type=write size=3693471 2024-04-17T04:45:04.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.184+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=V4JBTK0ZPEU1 type=write size=275044 2024-04-17T04:45:04.182 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.188+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=K36RYDKEB3GV type=del 2024-04-17T04:45:04.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.188+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=L237WIIU6T6X type=del size=927490 2024-04-17T04:45:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.192+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~BFDDNPG7BEKL.0 size=1048576 2024-04-17T04:45:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.192+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~BFDDNPG7BEKL.1 size=1048576 2024-04-17T04:45:04.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.192+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~BFDDNPG7BEKL.2 size=35004 2024-04-17T04:45:04.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.200+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=BFDDNPG7BEKL tag=Q65L2LQ2KS3I type=write size=3180732 2024-04-17T04:45:04.198 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.204+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~GVAHD194MDIP.0 size=1048576 2024-04-17T04:45:04.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.204+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~GVAHD194MDIP.1 size=1048576 2024-04-17T04:45:04.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~GVAHD194MDIP.2 size=218371 2024-04-17T04:45:04.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.208+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=GVAHD194MDIP tag=KG81IKEQDW75 type=write size=3364099 2024-04-17T04:45:04.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.208+0000 7fd8ef2218c0 -1 < failed to complete operation key=UEWLJ3E9ODUL tag=D90Y9AKUS7Y2 type=write size=673072: (22) Invalid argument 2024-04-17T04:45:04.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.212+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=SRFFLEK6MLB4 type=del size=2200958 2024-04-17T04:45:04.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.216+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~74AU1C0EGZLI.0 size=1048576 2024-04-17T04:45:04.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.216+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~74AU1C0EGZLI.1 size=855869 2024-04-17T04:45:04.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.220+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=74AU1C0EGZLI tag=TZKRD543PF13 type=write size=2953021 2024-04-17T04:45:04.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.224+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~LVAOVC5D3VG9.0 size=1048576 2024-04-17T04:45:04.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.228+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~LVAOVC5D3VG9.1 size=1048576 2024-04-17T04:45:04.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.228+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~LVAOVC5D3VG9.2 size=637516 2024-04-17T04:45:04.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.228+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=LVAOVC5D3VG9 tag=8FK4QUZA46OC type=write size=3783244 2024-04-17T04:45:04.230 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.236+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=250D7EO9RP4U type=del 2024-04-17T04:45:04.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.236+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=8GLVKNT8SKUF type=del size=2210812 2024-04-17T04:45:04.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.240+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=URT8GO010YAT type=del 2024-04-17T04:45:04.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.244+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=GJ23N1ICL2SM type=del size=96566 2024-04-17T04:45:04.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.244+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~CM56Y4DI8E4J.0 size=1048576 2024-04-17T04:45:04.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.248+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~CM56Y4DI8E4J.1 size=1048576 2024-04-17T04:45:04.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.252+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~CM56Y4DI8E4J.2 size=536460 2024-04-17T04:45:04.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.256+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=CM56Y4DI8E4J tag=3RTOTER2ENPQ type=write size=3682188 2024-04-17T04:45:04.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.260+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=O7UCZ1X4BTW8 type=del size=4124092 2024-04-17T04:45:04.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.260+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~2S3E6FWT0SA6.0 size=1048576 2024-04-17T04:45:04.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~2S3E6FWT0SA6.1 size=260374 2024-04-17T04:45:04.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.268+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=2S3E6FWT0SA6 tag=T05LWDJWG4UX type=write size=2357526 2024-04-17T04:45:04.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.272+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=8XBNMFYRXRBN type=del size=775333 2024-04-17T04:45:04.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.276+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=TXM9D39XA5JK type=del 2024-04-17T04:45:04.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~BTKMBCGTB5JR.0 size=1048576 2024-04-17T04:45:04.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~BTKMBCGTB5JR.1 size=1048576 2024-04-17T04:45:04.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~BTKMBCGTB5JR.2 size=876300 2024-04-17T04:45:04.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.284+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=BTKMBCGTB5JR tag=6BRBIOFUJ0FF type=write size=4022028 2024-04-17T04:45:04.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.292+0000 7fd8ef2218c0 1 < suggested update operation key=AE4CYL334ONO tag=LU5UUD5KRKBP type=del size=766869 2024-04-17T04:45:04.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.292+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=Z3HDRKQ6ZU9R type=del size=1876949 2024-04-17T04:45:04.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.296+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=6YKK02LLWG49 type=del 2024-04-17T04:45:04.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.300+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=Q5YBP0WXMCJJ type=del size=3722995 2024-04-17T04:45:04.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.304+0000 7fd8ef2218c0 1 < suggested remove operation key=7BV3V2XMLSPH tag=G4G2A55WBHYZ type=del 2024-04-17T04:45:04.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.312+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~F3SGSCXQJ7DR.0 size=1048576 2024-04-17T04:45:04.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.312+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~F3SGSCXQJ7DR.1 size=1048576 2024-04-17T04:45:04.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.316+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~F3SGSCXQJ7DR.2 size=410074 2024-04-17T04:45:04.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.316+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=F3SGSCXQJ7DR tag=P81JTB0I0F1F type=write size=3555802 2024-04-17T04:45:04.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.320+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=B5C5NNGJ78M4 type=del 2024-04-17T04:45:04.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~CNV5C633NJD4.0 size=422196 2024-04-17T04:45:04.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.324+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=CNV5C633NJD4 tag=E4LJF62U49H7 type=write size=1470772 2024-04-17T04:45:04.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.324+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~52RHHTCHHQ88.0 size=1048576 2024-04-17T04:45:04.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.328+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~52RHHTCHHQ88.1 size=1048576 2024-04-17T04:45:04.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.328+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~52RHHTCHHQ88.2 size=1038101 2024-04-17T04:45:04.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.336+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=52RHHTCHHQ88 tag=47AQF6XO9W8L type=write size=4183829 2024-04-17T04:45:04.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~LHSOAAE3DKDZ.0 size=1048576 2024-04-17T04:45:04.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~LHSOAAE3DKDZ.1 size=1048576 2024-04-17T04:45:04.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~LHSOAAE3DKDZ.2 size=833861 2024-04-17T04:45:04.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.344+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=LHSOAAE3DKDZ tag=MMO1RVF0O3LF type=write size=3979589 2024-04-17T04:45:04.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.348+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~TDIN4QLEHGZX.0 size=1048576 2024-04-17T04:45:04.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~TDIN4QLEHGZX.1 size=1048576 2024-04-17T04:45:04.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.356+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~TDIN4QLEHGZX.2 size=970668 2024-04-17T04:45:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.360+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=TDIN4QLEHGZX tag=GV4BYJQJ402R type=write size=4116396 2024-04-17T04:45:04.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.364+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=SPQQ5SK6Y159 type=del size=1527613 2024-04-17T04:45:04.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.368+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=V4JBTK0ZPEU1 type=write size=275044 2024-04-17T04:45:04.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.372+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=7AB42SF7EFLM type=del size=3596422 2024-04-17T04:45:04.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.376+0000 7fd8ef2218c0 1 < canceled operation key=UNOVYXDS5JEB tag=L237WIIU6T6X type=del size=927490 2024-04-17T04:45:04.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.376+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=89O4DLS0NJZG type=write size=391498 2024-04-17T04:45:04.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~GVAHD194MDIP.0 size=1048576 2024-04-17T04:45:04.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.384+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~GVAHD194MDIP.1 size=1048576 2024-04-17T04:45:04.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~GVAHD194MDIP.2 size=218371 2024-04-17T04:45:04.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.392+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=GVAHD194MDIP tag=KG81IKEQDW75 type=write size=3364099 2024-04-17T04:45:04.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.396+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=DLTQZ00HC3PZ type=write size=447049 2024-04-17T04:45:04.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.400+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=SRFFLEK6MLB4 type=del 2024-04-17T04:45:04.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.404+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=4EZ0V6ACAN8A type=write size=477661 2024-04-17T04:45:04.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.408+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~LVAOVC5D3VG9.0 size=1048576 2024-04-17T04:45:04.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~LVAOVC5D3VG9.1 size=1048576 2024-04-17T04:45:04.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~LVAOVC5D3VG9.2 size=637516 2024-04-17T04:45:04.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.420+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=LVAOVC5D3VG9 tag=8FK4QUZA46OC type=write size=3783244 2024-04-17T04:45:04.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.428+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~3XI0VJF134QF.0 size=820228 2024-04-17T04:45:04.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.428+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=3XI0VJF134QF tag=GLY78525BTPL type=write size=1868804 2024-04-17T04:45:04.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.432+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=8GLVKNT8SKUF type=del 2024-04-17T04:45:04.431 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.436+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=QNR335927W37 type=write size=754023 2024-04-17T04:45:04.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.440+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=GJ23N1ICL2SM type=del 2024-04-17T04:45:04.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~N79QTK9QE41B.0 size=215493 2024-04-17T04:45:04.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.444+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=N79QTK9QE41B tag=XFBC322J89DE type=write size=1264069 2024-04-17T04:45:04.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.448+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=O7UCZ1X4BTW8 type=del 2024-04-17T04:45:04.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~1S31O5KMI4QO.0 size=1048576 2024-04-17T04:45:04.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~1S31O5KMI4QO.1 size=1048576 2024-04-17T04:45:04.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.460+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~1S31O5KMI4QO.2 size=778508 2024-04-17T04:45:04.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.460+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=1S31O5KMI4QO tag=XM3BX7Z9G5J8 type=write size=3924236 2024-04-17T04:45:04.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.464+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=8XBNMFYRXRBN type=del size=775333 2024-04-17T04:45:04.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.468+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=D1VEU93SGVZ9 type=del size=2820141 2024-04-17T04:45:04.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~BTKMBCGTB5JR.0 size=1048576 2024-04-17T04:45:04.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~BTKMBCGTB5JR.1 size=1048576 2024-04-17T04:45:04.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~BTKMBCGTB5JR.2 size=876300 2024-04-17T04:45:04.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.484+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=BTKMBCGTB5JR tag=6BRBIOFUJ0FF type=write size=4022028 2024-04-17T04:45:04.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~6NKXZM58WYDU.0 size=1048576 2024-04-17T04:45:04.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.492+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~6NKXZM58WYDU.1 size=1048576 2024-04-17T04:45:04.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.492+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~6NKXZM58WYDU.2 size=939808 2024-04-17T04:45:04.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.492+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=6NKXZM58WYDU tag=STATDR34R741 type=write size=4085536 2024-04-17T04:45:04.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.500+0000 7fd8ef2218c0 1 < suggested update operation key=3OFQH4KVYN18 tag=Z3HDRKQ6ZU9R type=del size=1876949 2024-04-17T04:45:04.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~MIQQTUJMSN80.0 size=1048576 2024-04-17T04:45:04.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~MIQQTUJMSN80.1 size=378540 2024-04-17T04:45:04.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.508+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=MIQQTUJMSN80 tag=M2PH50IZIN3J type=write size=2475692 2024-04-17T04:45:04.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.512+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=Q5YBP0WXMCJJ type=del 2024-04-17T04:45:04.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.520+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~TLDJVN30CBF6.0 size=248912 2024-04-17T04:45:04.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.520+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=TLDJVN30CBF6 tag=3D29RUV0I5JA type=write size=1297488 2024-04-17T04:45:04.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.520+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~F3SGSCXQJ7DR.0 size=1048576 2024-04-17T04:45:04.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.524+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~F3SGSCXQJ7DR.1 size=1048576 2024-04-17T04:45:04.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.524+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~F3SGSCXQJ7DR.2 size=410074 2024-04-17T04:45:04.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.532+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=F3SGSCXQJ7DR tag=P81JTB0I0F1F type=write size=3555802 2024-04-17T04:45:04.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.540+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~TUAQT2LA85ZN.0 size=1048576 2024-04-17T04:45:04.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.540+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~TUAQT2LA85ZN.1 size=1048576 2024-04-17T04:45:04.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.544+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~TUAQT2LA85ZN.2 size=210002 2024-04-17T04:45:04.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.544+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=TUAQT2LA85ZN tag=VPST9AMMFR5Y type=write size=3355730 2024-04-17T04:45:04.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.548+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~CNV5C633NJD4.0 size=422196 2024-04-17T04:45:04.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.552+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=CNV5C633NJD4 tag=E4LJF62U49H7 type=write size=1470772 2024-04-17T04:45:04.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.556+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=Z0RW3S7Q9LDG type=del size=1239369 2024-04-17T04:45:04.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~LHSOAAE3DKDZ.0 size=1048576 2024-04-17T04:45:04.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.564+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~LHSOAAE3DKDZ.1 size=1048576 2024-04-17T04:45:04.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.568+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~LHSOAAE3DKDZ.2 size=833861 2024-04-17T04:45:04.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.572+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=LHSOAAE3DKDZ tag=MMO1RVF0O3LF type=write size=3979589 2024-04-17T04:45:04.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.576+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=WVOLK4ZS24Y9 type=del size=863802 2024-04-17T04:45:04.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.580+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=SPQQ5SK6Y159 type=del 2024-04-17T04:45:04.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.584+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=FI16Y62AGRQZ type=write size=217207 2024-04-17T04:45:04.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.588+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=7AB42SF7EFLM type=del 2024-04-17T04:45:04.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~GZ06Y2OZR226.0 size=439176 2024-04-17T04:45:04.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.592+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=GZ06Y2OZR226 tag=UPXM0O692A96 type=write size=1487752 2024-04-17T04:45:04.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.596+0000 7fd8ef2218c0 1 < completed write operation key=M3U8L7M8JKG9 tag=89O4DLS0NJZG type=write size=391498 2024-04-17T04:45:04.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~38L2MADHD49K.0 size=1048576 2024-04-17T04:45:04.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~38L2MADHD49K.1 size=1048576 2024-04-17T04:45:04.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.608+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~38L2MADHD49K.2 size=534219 2024-04-17T04:45:04.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.608+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=38L2MADHD49K tag=CC2MNC5ANQ9Q type=write size=3679947 2024-04-17T04:45:04.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.616+0000 7fd8ef2218c0 1 < suggested update operation key=MS3T476JOEON tag=DLTQZ00HC3PZ type=write size=447049 2024-04-17T04:45:04.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.620+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~HPBNMW0U899D.0 size=1048576 2024-04-17T04:45:04.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~HPBNMW0U899D.1 size=1048576 2024-04-17T04:45:04.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.628+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~HPBNMW0U899D.2 size=514308 2024-04-17T04:45:04.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.628+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=HPBNMW0U899D tag=50KEGCQ9H3X3 type=write size=3660036 2024-04-17T04:45:04.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.632+0000 7fd8ef2218c0 1 < completed write operation key=B63EZS0LAC30 tag=4EZ0V6ACAN8A type=write size=477661 2024-04-17T04:45:04.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.636+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=TR26TBNTHDOK type=write size=24186 2024-04-17T04:45:04.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~3XI0VJF134QF.0 size=820228 2024-04-17T04:45:04.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.644+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=3XI0VJF134QF tag=GLY78525BTPL type=write size=1868804 2024-04-17T04:45:04.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.648+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~YJ3Q4A714U34.0 size=1048576 2024-04-17T04:45:04.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.652+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~YJ3Q4A714U34.1 size=912693 2024-04-17T04:45:04.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.652+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=YJ3Q4A714U34 tag=Y7UULOT7CVLU type=write size=3009845 2024-04-17T04:45:04.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.660+0000 7fd8ef2218c0 1 < suggested update operation key=2S1IT3TO1BPU tag=QNR335927W37 type=write size=754023 2024-04-17T04:45:04.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.660+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=ZHQJ16QZU5OR type=del size=3973235 2024-04-17T04:45:04.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.664+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~N79QTK9QE41B.0 size=215493 2024-04-17T04:45:04.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.668+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=N79QTK9QE41B tag=XFBC322J89DE type=write size=1264069 2024-04-17T04:45:04.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.672+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~TIJ8VFXJ1JF0.0 size=1048576 2024-04-17T04:45:04.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.676+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~TIJ8VFXJ1JF0.1 size=1048576 2024-04-17T04:45:04.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.680+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~TIJ8VFXJ1JF0.2 size=897700 2024-04-17T04:45:04.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.680+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=TIJ8VFXJ1JF0 tag=O4TOJFUBP47F type=write size=4043428 2024-04-17T04:45:04.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.684+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~1S31O5KMI4QO.0 size=1048576 2024-04-17T04:45:04.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.684+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~1S31O5KMI4QO.1 size=1048576 2024-04-17T04:45:04.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~1S31O5KMI4QO.2 size=778508 2024-04-17T04:45:04.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.692+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=1S31O5KMI4QO tag=XM3BX7Z9G5J8 type=write size=3924236 2024-04-17T04:45:04.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.696+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=W25ML9C7W4MQ type=del size=2153125 2024-04-17T04:45:04.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.700+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=D1VEU93SGVZ9 type=del 2024-04-17T04:45:04.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.704+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=XLUU31PU3C5T type=del size=1790556 2024-04-17T04:45:04.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.704+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~6NKXZM58WYDU.0 size=1048576 2024-04-17T04:45:04.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.704+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5ZULD0PHLS4V.2~6NKXZM58WYDU.1 size=1048576 2024-04-17T04:45:04.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~6NKXZM58WYDU.2 size=939808 2024-04-17T04:45:04.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.716+0000 7fd8ef2218c0 1 < canceled multipart upload key=5ZULD0PHLS4V upload=6NKXZM58WYDU tag=STATDR34R741 type=write size=4085536 2024-04-17T04:45:04.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.716+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=BB3SLH70FWED type=del size=201768 2024-04-17T04:45:04.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.720+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~MIQQTUJMSN80.0 size=1048576 2024-04-17T04:45:04.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.724+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~MIQQTUJMSN80.1 size=378540 2024-04-17T04:45:04.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.732+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=MIQQTUJMSN80 tag=M2PH50IZIN3J type=write size=2475692 2024-04-17T04:45:04.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.732+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=UFBQ7LK6OI6H type=del size=2997120 2024-04-17T04:45:04.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.736+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~TLDJVN30CBF6.0 size=248912 2024-04-17T04:45:04.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.748+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=TLDJVN30CBF6 tag=3D29RUV0I5JA type=write size=1297488 2024-04-17T04:45:04.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.752+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=KP2HXBOP2A08 type=del size=873464 2024-04-17T04:45:04.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.756+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~TUAQT2LA85ZN.0 size=1048576 2024-04-17T04:45:04.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.756+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~TUAQT2LA85ZN.1 size=1048576 2024-04-17T04:45:04.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.760+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~TUAQT2LA85ZN.2 size=210002 2024-04-17T04:45:04.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.768+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=TUAQT2LA85ZN tag=VPST9AMMFR5Y type=write size=3355730 2024-04-17T04:45:04.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.776+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~GMIB5TXS5TF4.0 size=270096 2024-04-17T04:45:04.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.776+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=GMIB5TXS5TF4 tag=MA9OFPRYGGEI type=write size=1318672 2024-04-17T04:45:04.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.780+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=Z0RW3S7Q9LDG type=del 2024-04-17T04:45:04.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~JLZBXTOT8ZK2.0 size=1048576 2024-04-17T04:45:04.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~JLZBXTOT8ZK2.1 size=1048576 2024-04-17T04:45:04.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~JLZBXTOT8ZK2.2 size=794215 2024-04-17T04:45:04.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=JLZBXTOT8ZK2 tag=JTMSNGCB3F9M type=write size=3939943 2024-04-17T04:45:04.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.796+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=WVOLK4ZS24Y9 type=del 2024-04-17T04:45:04.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~M87UA5LLZ9PG.0 size=1048576 2024-04-17T04:45:04.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.804+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~M87UA5LLZ9PG.1 size=1048576 2024-04-17T04:45:04.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.804+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~M87UA5LLZ9PG.2 size=549878 2024-04-17T04:45:04.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.804+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=M87UA5LLZ9PG tag=ZAYL6L6IK9LU type=write size=3695606 2024-04-17T04:45:04.805 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.808+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=FI16Y62AGRQZ type=write size=217207 2024-04-17T04:45:04.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.812+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=62XA04CNJKZJ type=del size=948224 2024-04-17T04:45:04.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.816+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~GZ06Y2OZR226.0 size=439176 2024-04-17T04:45:04.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.820+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=GZ06Y2OZR226 tag=UPXM0O692A96 type=write size=1487752 2024-04-17T04:45:04.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.824+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=VZ09ABHS7GM5 type=del size=3749148 2024-04-17T04:45:04.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.828+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~38L2MADHD49K.0 size=1048576 2024-04-17T04:45:04.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.832+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~38L2MADHD49K.1 size=1048576 2024-04-17T04:45:04.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.832+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5EFSN2FPUQB6.2~38L2MADHD49K.2 size=534219 2024-04-17T04:45:04.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.836+0000 7fd8ef2218c0 1 < canceled multipart upload key=5EFSN2FPUQB6 upload=38L2MADHD49K tag=CC2MNC5ANQ9Q type=write size=3679947 2024-04-17T04:45:04.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.840+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=KFP6RFM0LNXZ type=del size=4104421 2024-04-17T04:45:04.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.840+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~HPBNMW0U899D.0 size=1048576 2024-04-17T04:45:04.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.844+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~HPBNMW0U899D.1 size=1048576 2024-04-17T04:45:04.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~HPBNMW0U899D.2 size=514308 2024-04-17T04:45:04.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.852+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=HPBNMW0U899D tag=50KEGCQ9H3X3 type=write size=3660036 2024-04-17T04:45:04.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.856+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~FKQEIDQDOSM0.0 size=1048576 2024-04-17T04:45:04.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~FKQEIDQDOSM0.1 size=1048576 2024-04-17T04:45:04.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~FKQEIDQDOSM0.2 size=76480 2024-04-17T04:45:04.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.860+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=FKQEIDQDOSM0 tag=6ZV7T1YFG1FY type=write size=3222208 2024-04-17T04:45:04.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.864+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=TR26TBNTHDOK type=write size=24186 2024-04-17T04:45:04.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~U38LJUQND3ZW.0 size=1048576 2024-04-17T04:45:04.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~U38LJUQND3ZW.1 size=1048576 2024-04-17T04:45:04.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.876+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~U38LJUQND3ZW.2 size=537666 2024-04-17T04:45:04.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.876+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=U38LJUQND3ZW tag=J8U1WS0KWV5L type=write size=3683394 2024-04-17T04:45:04.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.880+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~YJ3Q4A714U34.0 size=1048576 2024-04-17T04:45:04.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.880+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~YJ3Q4A714U34.1 size=912693 2024-04-17T04:45:04.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.888+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=YJ3Q4A714U34 tag=Y7UULOT7CVLU type=write size=3009845 2024-04-17T04:45:04.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.892+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~74SDMASFRDNZ.0 size=1048576 2024-04-17T04:45:04.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~74SDMASFRDNZ.1 size=1048576 2024-04-17T04:45:04.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~74SDMASFRDNZ.2 size=91621 2024-04-17T04:45:04.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.896+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=74SDMASFRDNZ tag=UPOUP22GVREE type=write size=3237349 2024-04-17T04:45:04.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.900+0000 7fd8ef2218c0 1 < canceled operation key=7BV3V2XMLSPH tag=ZHQJ16QZU5OR type=del size=3973235 2024-04-17T04:45:04.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.904+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=EDR02HKCF3AP type=write size=97488 2024-04-17T04:45:04.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~TIJ8VFXJ1JF0.0 size=1048576 2024-04-17T04:45:04.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.908+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~TIJ8VFXJ1JF0.1 size=1048576 2024-04-17T04:45:04.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.908+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~TIJ8VFXJ1JF0.2 size=897700 2024-04-17T04:45:04.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.912+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=TIJ8VFXJ1JF0 tag=O4TOJFUBP47F type=write size=4043428 2024-04-17T04:45:04.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~4KQJ3E3QSQ9Y.0 size=247079 2024-04-17T04:45:04.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.920+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=4KQJ3E3QSQ9Y tag=MI9WRC5XLRID type=write size=1295655 2024-04-17T04:45:04.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.920+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=W25ML9C7W4MQ type=del 2024-04-17T04:45:04.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~C8V6QQGOIX4S.0 size=1048576 2024-04-17T04:45:04.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~C8V6QQGOIX4S.1 size=414730 2024-04-17T04:45:04.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.928+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=C8V6QQGOIX4S tag=432UDB214M7F type=write size=2511882 2024-04-17T04:45:04.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.932+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=XLUU31PU3C5T type=del 2024-04-17T04:45:04.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~K6J0XBN5N1HM.0 size=1048576 2024-04-17T04:45:04.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~K6J0XBN5N1HM.1 size=1048576 2024-04-17T04:45:04.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.944+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~K6J0XBN5N1HM.2 size=248692 2024-04-17T04:45:04.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.944+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=K6J0XBN5N1HM tag=NNX67SMVJL39 type=write size=3394420 2024-04-17T04:45:04.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.952+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=BB3SLH70FWED type=del 2024-04-17T04:45:04.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.956+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~J72ZG5JJQ252.0 size=1048576 2024-04-17T04:45:04.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~J72ZG5JJQ252.1 size=587145 2024-04-17T04:45:04.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.960+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=J72ZG5JJQ252 tag=4D3ZH5SS1WF0 type=write size=2684297 2024-04-17T04:45:04.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.964+0000 7fd8ef2218c0 1 < suggested remove operation key=509DZ9DBJWC5 tag=UFBQ7LK6OI6H type=del 2024-04-17T04:45:04.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.968+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=U6D657P51IJ6 type=del size=3099122 2024-04-17T04:45:04.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.972+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=KP2HXBOP2A08 type=del 2024-04-17T04:45:04.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.976+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=XEDJEH58BU1Z type=del size=1090043 2024-04-17T04:45:04.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.980+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~GMIB5TXS5TF4.0 size=270096 2024-04-17T04:45:04.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.984+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=GMIB5TXS5TF4 tag=MA9OFPRYGGEI type=write size=1318672 2024-04-17T04:45:04.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.988+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=5F0Y50UZDXLO type=del size=777626 2024-04-17T04:45:04.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~JLZBXTOT8ZK2.0 size=1048576 2024-04-17T04:45:04.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~JLZBXTOT8ZK2.1 size=1048576 2024-04-17T04:45:04.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:04.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~JLZBXTOT8ZK2.2 size=794215 2024-04-17T04:45:04.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.000+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=JLZBXTOT8ZK2 tag=JTMSNGCB3F9M type=write size=3939943 2024-04-17T04:45:05.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~0ISDCDY6HVQM.0 size=455584 2024-04-17T04:45:05.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.004+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=0ISDCDY6HVQM tag=WC3BPPAYXV9J type=write size=1504160 2024-04-17T04:45:05.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.008+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~M87UA5LLZ9PG.0 size=1048576 2024-04-17T04:45:05.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~M87UA5LLZ9PG.1 size=1048576 2024-04-17T04:45:05.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.016+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~M87UA5LLZ9PG.2 size=549878 2024-04-17T04:45:05.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.020+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=M87UA5LLZ9PG tag=ZAYL6L6IK9LU type=write size=3695606 2024-04-17T04:45:05.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.024+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=36WNU1G3F3T0 type=del size=3041765 2024-04-17T04:45:05.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.024+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=62XA04CNJKZJ type=del 2024-04-17T04:45:05.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.028+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=GPER28U0PN28 type=del size=1386602 2024-04-17T04:45:05.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.036+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=VZ09ABHS7GM5 type=del 2024-04-17T04:45:05.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.040+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=C3G9DMTNY99T type=del size=5275 2024-04-17T04:45:05.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.040+0000 7fd8ef2218c0 1 < canceled operation key=WCIJICR0FT2C tag=KFP6RFM0LNXZ type=del size=4104421 2024-04-17T04:45:05.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.044+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=TQPPPLW25W05 type=del size=3982878 2024-04-17T04:45:05.044 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.048+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~FKQEIDQDOSM0.0 size=1048576 2024-04-17T04:45:05.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.052+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~FKQEIDQDOSM0.1 size=1048576 2024-04-17T04:45:05.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~FKQEIDQDOSM0.2 size=76480 2024-04-17T04:45:05.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.060+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=FKQEIDQDOSM0 tag=6ZV7T1YFG1FY type=write size=3222208 2024-04-17T04:45:05.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.064+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=G6YYIBBLI6ZM type=del size=2210361 2024-04-17T04:45:05.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~U38LJUQND3ZW.0 size=1048576 2024-04-17T04:45:05.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~U38LJUQND3ZW.1 size=1048576 2024-04-17T04:45:05.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~U38LJUQND3ZW.2 size=537666 2024-04-17T04:45:05.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.080+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=U38LJUQND3ZW tag=J8U1WS0KWV5L type=write size=3683394 2024-04-17T04:45:05.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.080+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=LEUL2OHXFS4M type=del size=3583246 2024-04-17T04:45:05.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~74SDMASFRDNZ.0 size=1048576 2024-04-17T04:45:05.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~74SDMASFRDNZ.1 size=1048576 2024-04-17T04:45:05.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.092+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~74SDMASFRDNZ.2 size=91621 2024-04-17T04:45:05.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.096+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=74SDMASFRDNZ tag=UPOUP22GVREE type=write size=3237349 2024-04-17T04:45:05.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~E3TPQ0YKW60E.0 size=1048576 2024-04-17T04:45:05.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~E3TPQ0YKW60E.1 size=800260 2024-04-17T04:45:05.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.104+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=E3TPQ0YKW60E tag=LR6UWFMNUFDP type=write size=2897412 2024-04-17T04:45:05.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.108+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=EDR02HKCF3AP type=write size=97488 2024-04-17T04:45:05.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~BEPN378PRWD4.0 size=1048576 2024-04-17T04:45:05.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~BEPN378PRWD4.1 size=1048576 2024-04-17T04:45:05.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~BEPN378PRWD4.2 size=256628 2024-04-17T04:45:05.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=BEPN378PRWD4 tag=SOY6L1O060P5 type=write size=3402356 2024-04-17T04:45:05.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~4KQJ3E3QSQ9Y.0 size=247079 2024-04-17T04:45:05.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.124+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=4KQJ3E3QSQ9Y tag=MI9WRC5XLRID type=write size=1295655 2024-04-17T04:45:05.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.128+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=H8FLI7VW3U8A type=del size=1199250 2024-04-17T04:45:05.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~C8V6QQGOIX4S.0 size=1048576 2024-04-17T04:45:05.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.132+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UNOVYXDS5JEB.2~C8V6QQGOIX4S.1 size=414730 2024-04-17T04:45:05.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.136+0000 7fd8ef2218c0 1 < canceled multipart upload key=UNOVYXDS5JEB upload=C8V6QQGOIX4S tag=432UDB214M7F type=write size=2511882 2024-04-17T04:45:05.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~8JC4E7HPRZTD.0 size=167002 2024-04-17T04:45:05.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.144+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=8JC4E7HPRZTD tag=OE7OH6KBK4Q7 type=write size=1215578 2024-04-17T04:45:05.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~K6J0XBN5N1HM.0 size=1048576 2024-04-17T04:45:05.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~K6J0XBN5N1HM.1 size=1048576 2024-04-17T04:45:05.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~K6J0XBN5N1HM.2 size=248692 2024-04-17T04:45:05.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.156+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=K6J0XBN5N1HM tag=NNX67SMVJL39 type=write size=3394420 2024-04-17T04:45:05.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.160+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=Y4ZWD7G08PP9 type=del size=2999678 2024-04-17T04:45:05.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.164+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~J72ZG5JJQ252.0 size=1048576 2024-04-17T04:45:05.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.164+0000 7fd8ef2218c0 1 < canceled part key=_multipart_EWAW6EBW6W6U.2~J72ZG5JJQ252.1 size=587145 2024-04-17T04:45:05.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.168+0000 7fd8ef2218c0 1 < canceled multipart upload key=EWAW6EBW6W6U upload=J72ZG5JJQ252 tag=4D3ZH5SS1WF0 type=write size=2684297 2024-04-17T04:45:05.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.172+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=LP57D1UJJMWE type=del size=2058186 2024-04-17T04:45:05.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.180+0000 7fd8ef2218c0 1 < suggested remove operation key=UNOVYXDS5JEB tag=U6D657P51IJ6 type=del 2024-04-17T04:45:05.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.184+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=R56I32G8CIK3 type=del size=4024952 2024-04-17T04:45:05.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.188+0000 7fd8ef2218c0 -1 < failed to complete operation key=UNOVYXDS5JEB tag=XEDJEH58BU1Z type=del size=1090043: (22) Invalid argument 2024-04-17T04:45:05.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.192+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=F5PGG3459ZOT type=del size=2656027 2024-04-17T04:45:05.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.196+0000 7fd8ef2218c0 1 < suggested update operation key=GX9X0D45GC2I tag=5F0Y50UZDXLO type=del size=777626 2024-04-17T04:45:05.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.200+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=VY2CUDWY46MT type=del size=3473953 2024-04-17T04:45:05.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~0ISDCDY6HVQM.0 size=455584 2024-04-17T04:45:05.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.212+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=0ISDCDY6HVQM tag=WC3BPPAYXV9J type=write size=1504160 2024-04-17T04:45:05.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~NJ2GDOJKSICT.0 size=329749 2024-04-17T04:45:05.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.216+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=NJ2GDOJKSICT tag=JZJ9IS3PL9E0 type=write size=1378325 2024-04-17T04:45:05.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.224+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=36WNU1G3F3T0 type=del 2024-04-17T04:45:05.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.228+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8ZFI3RU5XS69.0 size=1048576 2024-04-17T04:45:05.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.232+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8ZFI3RU5XS69.1 size=1048576 2024-04-17T04:45:05.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.232+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~8ZFI3RU5XS69.2 size=1045757 2024-04-17T04:45:05.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.232+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=8ZFI3RU5XS69 tag=ZLJKOUKYTI87 type=write size=4191485 2024-04-17T04:45:05.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.236+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=GPER28U0PN28 type=del 2024-04-17T04:45:05.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~OWKBG7SR6PRG.0 size=1048576 2024-04-17T04:45:05.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.248+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~OWKBG7SR6PRG.1 size=1048576 2024-04-17T04:45:05.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.248+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~OWKBG7SR6PRG.2 size=311559 2024-04-17T04:45:05.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.248+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=OWKBG7SR6PRG tag=XPP955BKKBN6 type=write size=3457287 2024-04-17T04:45:05.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.252+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=C3G9DMTNY99T type=del 2024-04-17T04:45:05.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.260+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~3G3V71MV347F.0 size=1048576 2024-04-17T04:45:05.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.260+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~3G3V71MV347F.1 size=395065 2024-04-17T04:45:05.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.260+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=3G3V71MV347F tag=150UCDZ2D59J type=write size=2492217 2024-04-17T04:45:05.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.264+0000 7fd8ef2218c0 1 < canceled operation key=5K50O96P7OGD tag=TQPPPLW25W05 type=del size=3982878 2024-04-17T04:45:05.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.272+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~ZZR2DAU0T2C8.0 size=1048576 2024-04-17T04:45:05.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.276+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~ZZR2DAU0T2C8.1 size=1006226 2024-04-17T04:45:05.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.276+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=ZZR2DAU0T2C8 tag=WX1BS735WFSP type=write size=3103378 2024-04-17T04:45:05.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.280+0000 7fd8ef2218c0 1 < suggested remove operation key=AE4CYL334ONO tag=G6YYIBBLI6ZM type=del 2024-04-17T04:45:05.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.284+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=07TRYV3I9NAH type=del size=829876 2024-04-17T04:45:05.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.288+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=LEUL2OHXFS4M type=del 2024-04-17T04:45:05.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.292+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=HRXZTM8XO2CO type=write size=581776 2024-04-17T04:45:05.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~E3TPQ0YKW60E.0 size=1048576 2024-04-17T04:45:05.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.300+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~E3TPQ0YKW60E.1 size=800260 2024-04-17T04:45:05.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.304+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=E3TPQ0YKW60E tag=LR6UWFMNUFDP type=write size=2897412 2024-04-17T04:45:05.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.308+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=VJPPCLGPYTJ7 type=del size=3054532 2024-04-17T04:45:05.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~BEPN378PRWD4.0 size=1048576 2024-04-17T04:45:05.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~BEPN378PRWD4.1 size=1048576 2024-04-17T04:45:05.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.320+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~BEPN378PRWD4.2 size=256628 2024-04-17T04:45:05.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.324+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=BEPN378PRWD4 tag=SOY6L1O060P5 type=write size=3402356 2024-04-17T04:45:05.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.328+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=77IGQQBUBS6H type=del size=903098 2024-04-17T04:45:05.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.332+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=H8FLI7VW3U8A type=del 2024-04-17T04:45:05.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~IZG27HLW2X4Z.0 size=38474 2024-04-17T04:45:05.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.340+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=IZG27HLW2X4Z tag=14B9TW3FK236 type=write size=1087050 2024-04-17T04:45:05.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.344+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~8JC4E7HPRZTD.0 size=167002 2024-04-17T04:45:05.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.352+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=8JC4E7HPRZTD tag=OE7OH6KBK4Q7 type=write size=1215578 2024-04-17T04:45:05.355 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~P2W7XCU7A6QK.0 size=1048576 2024-04-17T04:45:05.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.364+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~P2W7XCU7A6QK.1 size=602518 2024-04-17T04:45:05.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.364+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=P2W7XCU7A6QK tag=0DJESPIMDQCN type=write size=2699670 2024-04-17T04:45:05.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.364+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=Y4ZWD7G08PP9 type=del 2024-04-17T04:45:05.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.368+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=EKJFAVDIUT5J type=del size=1203938 2024-04-17T04:45:05.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.372+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=LP57D1UJJMWE type=del 2024-04-17T04:45:05.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.376+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=EFRDN5078E92 type=del size=1903430 2024-04-17T04:45:05.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.380+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=R56I32G8CIK3 type=del 2024-04-17T04:45:05.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.384+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=6V1643ER4XVG type=del size=1216815 2024-04-17T04:45:05.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.388+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=F5PGG3459ZOT type=del 2024-04-17T04:45:05.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.388+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=VF8SW1SRHL0Q type=del size=3138072 2024-04-17T04:45:05.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.392+0000 7fd8ef2218c0 1 < canceled operation key=MS3T476JOEON tag=VY2CUDWY46MT type=del size=3473953 2024-04-17T04:45:05.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.400+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~V3YT2ZBG1XMG.0 size=1048576 2024-04-17T04:45:05.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.404+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~V3YT2ZBG1XMG.1 size=750986 2024-04-17T04:45:05.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.404+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=V3YT2ZBG1XMG tag=3C7ND3LQ2A7R type=write size=2848138 2024-04-17T04:45:05.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.408+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~NJ2GDOJKSICT.0 size=329749 2024-04-17T04:45:05.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.412+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=NJ2GDOJKSICT tag=JZJ9IS3PL9E0 type=write size=1378325 2024-04-17T04:45:05.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.416+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=A41UJCCXTBNU type=del size=3959136 2024-04-17T04:45:05.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8ZFI3RU5XS69.0 size=1048576 2024-04-17T04:45:05.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.420+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8ZFI3RU5XS69.1 size=1048576 2024-04-17T04:45:05.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.424+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~8ZFI3RU5XS69.2 size=1045757 2024-04-17T04:45:05.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.428+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=8ZFI3RU5XS69 tag=ZLJKOUKYTI87 type=write size=4191485 2024-04-17T04:45:05.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~1Z0TF92HXEAA.0 size=1048576 2024-04-17T04:45:05.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~1Z0TF92HXEAA.1 size=591855 2024-04-17T04:45:05.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.436+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=1Z0TF92HXEAA tag=FQ1A1SXJ5WDC type=write size=2689007 2024-04-17T04:45:05.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.436+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~OWKBG7SR6PRG.0 size=1048576 2024-04-17T04:45:05.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~OWKBG7SR6PRG.1 size=1048576 2024-04-17T04:45:05.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~OWKBG7SR6PRG.2 size=311559 2024-04-17T04:45:05.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.448+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=OWKBG7SR6PRG tag=XPP955BKKBN6 type=write size=3457287 2024-04-17T04:45:05.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~SDYVB5TBHQB2.0 size=1048576 2024-04-17T04:45:05.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~SDYVB5TBHQB2.1 size=166863 2024-04-17T04:45:05.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.452+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=SDYVB5TBHQB2 tag=AH78CVQIUKEL type=write size=2264015 2024-04-17T04:45:05.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~3G3V71MV347F.0 size=1048576 2024-04-17T04:45:05.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.460+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~3G3V71MV347F.1 size=395065 2024-04-17T04:45:05.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.464+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=3G3V71MV347F tag=150UCDZ2D59J type=write size=2492217 2024-04-17T04:45:05.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.468+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=AQPUCVNY7WLL type=del size=1071635 2024-04-17T04:45:05.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.468+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~ZZR2DAU0T2C8.0 size=1048576 2024-04-17T04:45:05.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~ZZR2DAU0T2C8.1 size=1006226 2024-04-17T04:45:05.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.476+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=ZZR2DAU0T2C8 tag=WX1BS735WFSP type=write size=3103378 2024-04-17T04:45:05.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.484+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~YF1KSR6V2PGM.0 size=275302 2024-04-17T04:45:05.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.484+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=YF1KSR6V2PGM tag=LSAUUVF9T31J type=write size=1323878 2024-04-17T04:45:05.481 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.484+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=07TRYV3I9NAH type=del 2024-04-17T04:45:05.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.488+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=DD2H7K8G9YMK type=del size=4186214 2024-04-17T04:45:05.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.492+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=HRXZTM8XO2CO type=write size=581776 2024-04-17T04:45:05.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.496+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=XWA0DRHO2YBF type=write size=882280 2024-04-17T04:45:05.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.500+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=VJPPCLGPYTJ7 type=del 2024-04-17T04:45:05.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.504+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=R2V38D7AVQAM type=del size=1441489 2024-04-17T04:45:05.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.504+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=77IGQQBUBS6H type=del 2024-04-17T04:45:05.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~MTM9GKYNTYN4.0 size=1048576 2024-04-17T04:45:05.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.516+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~MTM9GKYNTYN4.1 size=668701 2024-04-17T04:45:05.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.516+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=MTM9GKYNTYN4 tag=4V812KSNDMZA type=write size=2765853 2024-04-17T04:45:05.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~IZG27HLW2X4Z.0 size=38474 2024-04-17T04:45:05.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.524+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=IZG27HLW2X4Z tag=14B9TW3FK236 type=write size=1087050 2024-04-17T04:45:05.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.528+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=QWXARG4C0VHL type=del size=665500 2024-04-17T04:45:05.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.532+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~P2W7XCU7A6QK.0 size=1048576 2024-04-17T04:45:05.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.532+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~P2W7XCU7A6QK.1 size=602518 2024-04-17T04:45:05.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.536+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=P2W7XCU7A6QK tag=0DJESPIMDQCN type=write size=2699670 2024-04-17T04:45:05.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.540+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=ZBIB1O0TJ3K9 type=del size=2859402 2024-04-17T04:45:05.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.544+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=EKJFAVDIUT5J type=del 2024-04-17T04:45:05.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.548+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=O1O84C1X0DTM type=del size=1498927 2024-04-17T04:45:05.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.548+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=EFRDN5078E92 type=del 2024-04-17T04:45:05.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.552+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=WB675162QGMZ type=del size=1532384 2024-04-17T04:45:05.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.556+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=6V1643ER4XVG type=del 2024-04-17T04:45:05.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.560+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=KXDGVIDXUO38 type=del size=2015456 2024-04-17T04:45:05.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.560+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=VF8SW1SRHL0Q type=del 2024-04-17T04:45:05.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.568+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~WZ9J14A9OC9G.0 size=1048576 2024-04-17T04:45:05.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.568+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~WZ9J14A9OC9G.1 size=1048576 2024-04-17T04:45:05.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.572+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~WZ9J14A9OC9G.2 size=788008 2024-04-17T04:45:05.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.572+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=WZ9J14A9OC9G tag=IBGLRSSJCXFI type=write size=3933736 2024-04-17T04:45:05.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.572+0000 7fd8ef2218c0 1 < canceled part key=_multipart_R3C8PAVPVGBY.2~V3YT2ZBG1XMG.0 size=1048576 2024-04-17T04:45:05.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~V3YT2ZBG1XMG.1 size=750986 2024-04-17T04:45:05.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.580+0000 7fd8ef2218c0 1 < canceled multipart upload key=R3C8PAVPVGBY upload=V3YT2ZBG1XMG tag=3C7ND3LQ2A7R type=write size=2848138 2024-04-17T04:45:05.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.584+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=R2DIDGALH3XU type=del size=2156270 2024-04-17T04:45:05.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.584+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=A41UJCCXTBNU type=del 2024-04-17T04:45:05.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.588+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=WI1VVUQTOAEX type=del size=2438133 2024-04-17T04:45:05.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.592+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~1Z0TF92HXEAA.0 size=1048576 2024-04-17T04:45:05.589 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.592+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~1Z0TF92HXEAA.1 size=591855 2024-04-17T04:45:05.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.600+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=1Z0TF92HXEAA tag=FQ1A1SXJ5WDC type=write size=2689007 2024-04-17T04:45:05.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~FAFNJ5Z5LG4A.0 size=719953 2024-04-17T04:45:05.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.604+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=FAFNJ5Z5LG4A tag=WVK22F9U1B1R type=write size=1768529 2024-04-17T04:45:05.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~SDYVB5TBHQB2.0 size=1048576 2024-04-17T04:45:05.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~SDYVB5TBHQB2.1 size=166863 2024-04-17T04:45:05.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.616+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=SDYVB5TBHQB2 tag=AH78CVQIUKEL type=write size=2264015 2024-04-17T04:45:05.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.620+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=W3ZBV1Z880F2 type=write size=657497 2024-04-17T04:45:05.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.620+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=AQPUCVNY7WLL type=del 2024-04-17T04:45:05.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.624+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=HXE40KJELPJZ type=write size=547025 2024-04-17T04:45:05.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.628+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~YF1KSR6V2PGM.0 size=275302 2024-04-17T04:45:05.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.636+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=YF1KSR6V2PGM tag=LSAUUVF9T31J type=write size=1323878 2024-04-17T04:45:05.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~QO5X349S71GF.0 size=1048576 2024-04-17T04:45:05.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~QO5X349S71GF.1 size=268445 2024-04-17T04:45:05.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.644+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=QO5X349S71GF tag=ZAR3W4VOQV0T type=write size=2365597 2024-04-17T04:45:05.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.648+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=DD2H7K8G9YMK type=del 2024-04-17T04:45:05.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.652+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=W54C4B2L1WRA type=del size=1315271 2024-04-17T04:45:05.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.660+0000 7fd8ef2218c0 1 < suggested update operation key=N11Y8FPN9XES tag=XWA0DRHO2YBF type=write size=882280 2024-04-17T04:45:05.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.664+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=GGB4VHDX4OEY type=del size=1996020 2024-04-17T04:45:05.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.664+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=R2V38D7AVQAM type=del 2024-04-17T04:45:05.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.672+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~F1KY4JD4SKCL.0 size=52854 2024-04-17T04:45:05.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.672+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=F1KY4JD4SKCL tag=3CBFY12AO9XH type=write size=1101430 2024-04-17T04:45:05.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~MTM9GKYNTYN4.0 size=1048576 2024-04-17T04:45:05.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~MTM9GKYNTYN4.1 size=668701 2024-04-17T04:45:05.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.684+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=MTM9GKYNTYN4 tag=4V812KSNDMZA type=write size=2765853 2024-04-17T04:45:05.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.688+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=QTG633LOR3X7 type=del size=3053960 2024-04-17T04:45:05.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.692+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=QWXARG4C0VHL type=del 2024-04-17T04:45:05.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.696+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=02KV0OX3UOWW type=del size=1186892 2024-04-17T04:45:05.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.700+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=ZBIB1O0TJ3K9 type=del 2024-04-17T04:45:05.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.700+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=6II53NV2FGV1 type=write size=827279 2024-04-17T04:45:05.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.704+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=O1O84C1X0DTM type=del 2024-04-17T04:45:05.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~WWZMD6G4TS7N.0 size=1048576 2024-04-17T04:45:05.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~WWZMD6G4TS7N.1 size=175384 2024-04-17T04:45:05.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.716+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=WWZMD6G4TS7N tag=7TGY1UVUZUQI type=write size=2272536 2024-04-17T04:45:05.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.716+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=WB675162QGMZ type=del 2024-04-17T04:45:05.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.720+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=N2KP5U4JCF1K type=del size=1417548 2024-04-17T04:45:05.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.728+0000 7fd8ef2218c0 1 < suggested update operation key=1JOWGWFR7R6P tag=KXDGVIDXUO38 type=del size=2015456 2024-04-17T04:45:05.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.732+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=Z73AF2PJPJGH type=write size=276701 2024-04-17T04:45:05.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.736+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~WZ9J14A9OC9G.0 size=1048576 2024-04-17T04:45:05.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~WZ9J14A9OC9G.1 size=1048576 2024-04-17T04:45:05.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~WZ9J14A9OC9G.2 size=788008 2024-04-17T04:45:05.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.748+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=WZ9J14A9OC9G tag=IBGLRSSJCXFI type=write size=3933736 2024-04-17T04:45:05.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.752+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~Z62NHQDOKNXN.0 size=1048576 2024-04-17T04:45:05.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~Z62NHQDOKNXN.1 size=728397 2024-04-17T04:45:05.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.756+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=Z62NHQDOKNXN tag=FUBGB0O8325S type=write size=2825549 2024-04-17T04:45:05.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.764+0000 7fd8ef2218c0 1 < suggested update operation key=UNOVYXDS5JEB tag=R2DIDGALH3XU type=del size=2156270 2024-04-17T04:45:05.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.768+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=6RX5EIMKYL7I type=del size=2547363 2024-04-17T04:45:05.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.768+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=WI1VVUQTOAEX type=del 2024-04-17T04:45:05.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.776+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=MI11N4HTXGHE type=del size=4061855 2024-04-17T04:45:05.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.776+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~FAFNJ5Z5LG4A.0 size=719953 2024-04-17T04:45:05.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.784+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=FAFNJ5Z5LG4A tag=WVK22F9U1B1R type=write size=1768529 2024-04-17T04:45:05.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~DSAKHTMLAVZO.0 size=1048576 2024-04-17T04:45:05.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~DSAKHTMLAVZO.1 size=1048576 2024-04-17T04:45:05.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~DSAKHTMLAVZO.2 size=131961 2024-04-17T04:45:05.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=DSAKHTMLAVZO tag=ELS1GGHKJ383 type=write size=3277689 2024-04-17T04:45:05.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.796+0000 7fd8ef2218c0 1 < completed write operation key=WCIJICR0FT2C tag=W3ZBV1Z880F2 type=write size=657497 2024-04-17T04:45:05.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.800+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=3Z1VTPU2BQVR type=del size=157782 2024-04-17T04:45:05.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.804+0000 7fd8ef2218c0 1 < canceled operation key=AE4CYL334ONO tag=HXE40KJELPJZ type=write size=547025 2024-04-17T04:45:05.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.808+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=G17O6TFS4PV2 type=del size=2575969 2024-04-17T04:45:05.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.808+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~QO5X349S71GF.0 size=1048576 2024-04-17T04:45:05.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.808+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~QO5X349S71GF.1 size=268445 2024-04-17T04:45:05.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.812+0000 7fd8ef2218c0 1 < canceled multipart upload key=6219DPY72LWC upload=QO5X349S71GF tag=ZAR3W4VOQV0T type=write size=2365597 2024-04-17T04:45:05.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.816+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=DKA9XQY1GBVB type=del size=1658068 2024-04-17T04:45:05.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.820+0000 7fd8ef2218c0 1 < canceled operation key=509DZ9DBJWC5 tag=W54C4B2L1WRA type=del size=1315271 2024-04-17T04:45:05.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.824+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~F4GLV0WDSOLX.0 size=1048576 2024-04-17T04:45:05.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.828+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~F4GLV0WDSOLX.1 size=1048576 2024-04-17T04:45:05.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~F4GLV0WDSOLX.2 size=828713 2024-04-17T04:45:05.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=F4GLV0WDSOLX tag=IRSF9O52FLVK type=write size=3974441 2024-04-17T04:45:05.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.832+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=GGB4VHDX4OEY type=del 2024-04-17T04:45:05.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~9JGV3WK6AV3S.0 size=1048576 2024-04-17T04:45:05.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~9JGV3WK6AV3S.1 size=337498 2024-04-17T04:45:05.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=9JGV3WK6AV3S tag=32SKFR3VFM3H type=write size=2434650 2024-04-17T04:45:05.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.844+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~F1KY4JD4SKCL.0 size=52854 2024-04-17T04:45:05.846 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.852+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=F1KY4JD4SKCL tag=3CBFY12AO9XH type=write size=1101430 2024-04-17T04:45:05.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.856+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~L8AH10D131QT.0 size=1048576 2024-04-17T04:45:05.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~L8AH10D131QT.1 size=1048576 2024-04-17T04:45:05.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~L8AH10D131QT.2 size=631926 2024-04-17T04:45:05.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.860+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=L8AH10D131QT tag=DXNGGQBUQM7V type=write size=3777654 2024-04-17T04:45:05.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.864+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=QTG633LOR3X7 type=del 2024-04-17T04:45:05.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.868+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=4EJ7KP2DPK5W type=del size=1405297 2024-04-17T04:45:05.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.876+0000 7fd8ef2218c0 1 < suggested update operation key=JVWRJBMFIRLL tag=02KV0OX3UOWW type=del size=1186892 2024-04-17T04:45:05.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.880+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~X1IIK9OP4Q5Q.0 size=267766 2024-04-17T04:45:05.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.880+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=X1IIK9OP4Q5Q tag=ZAN36202FJHS type=write size=1316342 2024-04-17T04:45:05.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.884+0000 7fd8ef2218c0 1 < completed write operation key=VUYM3JC57QRP tag=6II53NV2FGV1 type=write size=827279 2024-04-17T04:45:05.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.892+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~O4CBMM6CC4P5.0 size=1048576 2024-04-17T04:45:05.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.892+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~O4CBMM6CC4P5.1 size=1048576 2024-04-17T04:45:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~O4CBMM6CC4P5.2 size=897652 2024-04-17T04:45:05.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.896+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=O4CBMM6CC4P5 tag=QT1YS66Y0SJK type=write size=4043380 2024-04-17T04:45:05.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~WWZMD6G4TS7N.0 size=1048576 2024-04-17T04:45:05.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~WWZMD6G4TS7N.1 size=175384 2024-04-17T04:45:05.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.908+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=WWZMD6G4TS7N tag=7TGY1UVUZUQI type=write size=2272536 2024-04-17T04:45:05.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~1LTPOFE4DHB0.0 size=1048576 2024-04-17T04:45:05.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~1LTPOFE4DHB0.1 size=1048576 2024-04-17T04:45:05.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~1LTPOFE4DHB0.2 size=978746 2024-04-17T04:45:05.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.924+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=1LTPOFE4DHB0 tag=DWZH3MOCV5OF type=write size=4124474 2024-04-17T04:45:05.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.928+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=N2KP5U4JCF1K type=del 2024-04-17T04:45:05.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~DILXD56XFMPG.0 size=1048576 2024-04-17T04:45:05.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~DILXD56XFMPG.1 size=1048576 2024-04-17T04:45:05.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~DILXD56XFMPG.2 size=500190 2024-04-17T04:45:05.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.940+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=DILXD56XFMPG tag=ORJ4JEHT3213 type=write size=3645918 2024-04-17T04:45:05.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.944+0000 7fd8ef2218c0 1 < canceled operation key=UEWLJ3E9ODUL tag=Z73AF2PJPJGH type=write size=276701 2024-04-17T04:45:05.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.948+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=LBAU41T19TA0 type=del size=464180 2024-04-17T04:45:05.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~Z62NHQDOKNXN.0 size=1048576 2024-04-17T04:45:05.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~Z62NHQDOKNXN.1 size=728397 2024-04-17T04:45:05.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.964+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=Z62NHQDOKNXN tag=FUBGB0O8325S type=write size=2825549 2024-04-17T04:45:05.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.968+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=MIUTPYHNQ0IJ type=del size=3629278 2024-04-17T04:45:05.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.972+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=6RX5EIMKYL7I type=del 2024-04-17T04:45:05.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.972+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=SV6X7IBS5EJN type=write size=955586 2024-04-17T04:45:05.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.976+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=MI11N4HTXGHE type=del 2024-04-17T04:45:05.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.980+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=YA1ID3I7W81F type=del size=2259926 2024-04-17T04:45:05.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.980+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~DSAKHTMLAVZO.0 size=1048576 2024-04-17T04:45:05.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.984+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~DSAKHTMLAVZO.1 size=1048576 2024-04-17T04:45:05.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.984+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~DSAKHTMLAVZO.2 size=131961 2024-04-17T04:45:05.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.988+0000 7fd8ef2218c0 1 < canceled multipart upload key=6WSZ5UL0V1Y1 upload=DSAKHTMLAVZO tag=ELS1GGHKJ383 type=write size=3277689 2024-04-17T04:45:05.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.992+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=Y3E7UAG8JL81 type=del size=3954832 2024-04-17T04:45:05.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:05.996+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=3Z1VTPU2BQVR type=del 2024-04-17T04:45:05.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~3EQQMJKS0NX4.0 size=1048576 2024-04-17T04:45:05.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~3EQQMJKS0NX4.1 size=1048576 2024-04-17T04:45:06.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~3EQQMJKS0NX4.2 size=776595 2024-04-17T04:45:06.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.004+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=3EQQMJKS0NX4 tag=3WVI78KJX3IV type=write size=3922323 2024-04-17T04:45:06.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.008+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=G17O6TFS4PV2 type=del 2024-04-17T04:45:06.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~13HPJE9VN3E2.0 size=1048576 2024-04-17T04:45:06.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~13HPJE9VN3E2.1 size=346774 2024-04-17T04:45:06.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.016+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=13HPJE9VN3E2 tag=SCXX2N5OL6NN type=write size=2443926 2024-04-17T04:45:06.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.020+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=DKA9XQY1GBVB type=del 2024-04-17T04:45:06.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~CSQGED15JWHX.0 size=1048576 2024-04-17T04:45:06.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~CSQGED15JWHX.1 size=1024653 2024-04-17T04:45:06.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.028+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=CSQGED15JWHX tag=HQRVZYG3F7H0 type=write size=3121805 2024-04-17T04:45:06.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~F4GLV0WDSOLX.0 size=1048576 2024-04-17T04:45:06.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.032+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~F4GLV0WDSOLX.1 size=1048576 2024-04-17T04:45:06.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.032+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~F4GLV0WDSOLX.2 size=828713 2024-04-17T04:45:06.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.036+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=F4GLV0WDSOLX tag=IRSF9O52FLVK type=write size=3974441 2024-04-17T04:45:06.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.040+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=W43MLL3TO10U type=del size=846881 2024-04-17T04:45:06.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~9JGV3WK6AV3S.0 size=1048576 2024-04-17T04:45:06.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.044+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~9JGV3WK6AV3S.1 size=337498 2024-04-17T04:45:06.044 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.048+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=9JGV3WK6AV3S tag=32SKFR3VFM3H type=write size=2434650 2024-04-17T04:45:06.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.052+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~JTVWNWH3Q9FL.0 size=486890 2024-04-17T04:45:06.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.052+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=JTVWNWH3Q9FL tag=IPB81GDEVYBO type=write size=1535466 2024-04-17T04:45:06.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~L8AH10D131QT.0 size=1048576 2024-04-17T04:45:06.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~L8AH10D131QT.1 size=1048576 2024-04-17T04:45:06.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~L8AH10D131QT.2 size=631926 2024-04-17T04:45:06.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.064+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=L8AH10D131QT tag=DXNGGQBUQM7V type=write size=3777654 2024-04-17T04:45:06.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.068+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~MU90VH8X944R.0 size=1048576 2024-04-17T04:45:06.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~MU90VH8X944R.1 size=1048576 2024-04-17T04:45:06.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~MU90VH8X944R.2 size=922558 2024-04-17T04:45:06.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.072+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=MU90VH8X944R tag=2P44R57PBF0A type=write size=4068286 2024-04-17T04:45:06.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.080+0000 7fd8ef2218c0 1 < suggested update operation key=2S1IT3TO1BPU tag=4EJ7KP2DPK5W type=del size=1405297 2024-04-17T04:45:06.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.084+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=3INTFJUYHERP type=del size=2941496 2024-04-17T04:45:06.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~X1IIK9OP4Q5Q.0 size=267766 2024-04-17T04:45:06.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.092+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=X1IIK9OP4Q5Q tag=ZAN36202FJHS type=write size=1316342 2024-04-17T04:45:06.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.096+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=Y2TGLQUXZJ9K type=del size=1201167 2024-04-17T04:45:06.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.096+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~O4CBMM6CC4P5.0 size=1048576 2024-04-17T04:45:06.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.100+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~O4CBMM6CC4P5.1 size=1048576 2024-04-17T04:45:06.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.104+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~O4CBMM6CC4P5.2 size=897652 2024-04-17T04:45:06.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.108+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=O4CBMM6CC4P5 tag=QT1YS66Y0SJK type=write size=4043380 2024-04-17T04:45:06.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~M9OYJXKYWGDK.0 size=336486 2024-04-17T04:45:06.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.112+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=M9OYJXKYWGDK tag=6J9P2X6G2077 type=write size=1385062 2024-04-17T04:45:06.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~1LTPOFE4DHB0.0 size=1048576 2024-04-17T04:45:06.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.116+0000 7fd8ef2218c0 1 < canceled part key=_multipart_7BV3V2XMLSPH.2~1LTPOFE4DHB0.1 size=1048576 2024-04-17T04:45:06.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~1LTPOFE4DHB0.2 size=978746 2024-04-17T04:45:06.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.124+0000 7fd8ef2218c0 1 < canceled multipart upload key=7BV3V2XMLSPH upload=1LTPOFE4DHB0 tag=DWZH3MOCV5OF type=write size=4124474 2024-04-17T04:45:06.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~ZRAA1CBR4W29.0 size=706251 2024-04-17T04:45:06.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.132+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=ZRAA1CBR4W29 tag=U4E54NING2BY type=write size=1754827 2024-04-17T04:45:06.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.132+0000 7fd8ef2218c0 1 < canceled part key=_multipart_3OFQH4KVYN18.2~DILXD56XFMPG.0 size=1048576 2024-04-17T04:45:06.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~DILXD56XFMPG.1 size=1048576 2024-04-17T04:45:06.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~DILXD56XFMPG.2 size=500190 2024-04-17T04:45:06.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.144+0000 7fd8ef2218c0 1 < canceled multipart upload key=3OFQH4KVYN18 upload=DILXD56XFMPG tag=ORJ4JEHT3213 type=write size=3645918 2024-04-17T04:45:06.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.144+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=OQSK02WMXM4I type=write size=274506 2024-04-17T04:45:06.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.148+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=LBAU41T19TA0 type=del 2024-04-17T04:45:06.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.152+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=N1N9PYT5B02B type=del size=749098 2024-04-17T04:45:06.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.156+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=MIUTPYHNQ0IJ type=del 2024-04-17T04:45:06.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.160+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=ZSY63DUEDS62 type=write size=137780 2024-04-17T04:45:06.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.164+0000 7fd8ef2218c0 1 < canceled operation key=UNOVYXDS5JEB tag=SV6X7IBS5EJN type=write size=955586 2024-04-17T04:45:06.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.168+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=HUEV6LO91J6D type=del size=3499532 2024-04-17T04:45:06.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.168+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=YA1ID3I7W81F type=del 2024-04-17T04:45:06.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.176+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~8M5M911JDW40.0 size=1048576 2024-04-17T04:45:06.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.176+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~8M5M911JDW40.1 size=644818 2024-04-17T04:45:06.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.176+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=8M5M911JDW40 tag=2811DSR63CHY type=write size=2741970 2024-04-17T04:45:06.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.180+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=Y3E7UAG8JL81 type=del 2024-04-17T04:45:06.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.184+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=8YI1JNORP72X type=del size=2183872 2024-04-17T04:45:06.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.188+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~3EQQMJKS0NX4.0 size=1048576 2024-04-17T04:45:06.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.188+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~3EQQMJKS0NX4.1 size=1048576 2024-04-17T04:45:06.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.188+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~3EQQMJKS0NX4.2 size=776595 2024-04-17T04:45:06.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.196+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=3EQQMJKS0NX4 tag=3WVI78KJX3IV type=write size=3922323 2024-04-17T04:45:06.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.200+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=IUGZQOBIAV5D type=del size=3825392 2024-04-17T04:45:06.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~13HPJE9VN3E2.0 size=1048576 2024-04-17T04:45:06.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~13HPJE9VN3E2.1 size=346774 2024-04-17T04:45:06.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.208+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=13HPJE9VN3E2 tag=SCXX2N5OL6NN type=write size=2443926 2024-04-17T04:45:06.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~IR84TM39R20M.0 size=997622 2024-04-17T04:45:06.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.216+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=IR84TM39R20M tag=RPURWMO88B0C type=write size=2046198 2024-04-17T04:45:06.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.216+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~CSQGED15JWHX.0 size=1048576 2024-04-17T04:45:06.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.220+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~CSQGED15JWHX.1 size=1024653 2024-04-17T04:45:06.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.224+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=CSQGED15JWHX tag=HQRVZYG3F7H0 type=write size=3121805 2024-04-17T04:45:06.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.232+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~85GDO42ZPL2I.0 size=1048576 2024-04-17T04:45:06.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.232+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~85GDO42ZPL2I.1 size=945096 2024-04-17T04:45:06.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.232+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=85GDO42ZPL2I tag=77OTCQ2H69QI type=write size=3042248 2024-04-17T04:45:06.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.236+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=W43MLL3TO10U type=del 2024-04-17T04:45:06.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.240+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=CT34QRWCOV4S type=del size=3832958 2024-04-17T04:45:06.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.240+0000 7fd8ef2218c0 1 < canceled part key=_multipart_AE4CYL334ONO.2~JTVWNWH3Q9FL.0 size=486890 2024-04-17T04:45:06.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.248+0000 7fd8ef2218c0 1 < canceled multipart upload key=AE4CYL334ONO upload=JTVWNWH3Q9FL tag=IPB81GDEVYBO type=write size=1535466 2024-04-17T04:45:06.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~Q280TFN0QEB6.0 size=200891 2024-04-17T04:45:06.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.252+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=Q280TFN0QEB6 tag=9HD6Z9JH343M type=write size=1249467 2024-04-17T04:45:06.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.256+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~MU90VH8X944R.0 size=1048576 2024-04-17T04:45:06.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.256+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~MU90VH8X944R.1 size=1048576 2024-04-17T04:45:06.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.260+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~MU90VH8X944R.2 size=922558 2024-04-17T04:45:06.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.264+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=MU90VH8X944R tag=2P44R57PBF0A type=write size=4068286 2024-04-17T04:45:06.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.268+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=HEEPOAZU6S35 type=del size=3577267 2024-04-17T04:45:06.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.272+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=3INTFJUYHERP type=del 2024-04-17T04:45:06.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.276+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~OVA5WSDUKC2Y.0 size=1048576 2024-04-17T04:45:06.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.276+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~OVA5WSDUKC2Y.1 size=1048576 2024-04-17T04:45:06.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~OVA5WSDUKC2Y.2 size=863877 2024-04-17T04:45:06.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.280+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=OVA5WSDUKC2Y tag=MAN0BR362Z2T type=write size=4009605 2024-04-17T04:45:06.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.284+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=Y2TGLQUXZJ9K type=del 2024-04-17T04:45:06.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~VQQ5229BFTTF.0 size=317547 2024-04-17T04:45:06.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.288+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=VQQ5229BFTTF tag=CZ961HYZGYXS type=write size=1366123 2024-04-17T04:45:06.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.292+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~M9OYJXKYWGDK.0 size=336486 2024-04-17T04:45:06.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.296+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=M9OYJXKYWGDK tag=6J9P2X6G2077 type=write size=1385062 2024-04-17T04:45:06.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.300+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=IIJHGB5NG2MR type=write size=646871 2024-04-17T04:45:06.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.300+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~ZRAA1CBR4W29.0 size=706251 2024-04-17T04:45:06.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.308+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=ZRAA1CBR4W29 tag=U4E54NING2BY type=write size=1754827 2024-04-17T04:45:06.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.308+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=Z98FR973L3XH type=del size=2134567 2024-04-17T04:45:06.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.312+0000 7fd8ef2218c0 1 < completed write operation key=1ZTJP8DRITEK tag=OQSK02WMXM4I type=write size=274506 2024-04-17T04:45:06.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.316+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=EN3WEINTV9W8 type=write size=973748 2024-04-17T04:45:06.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.324+0000 7fd8ef2218c0 1 < suggested remove operation key=AE4CYL334ONO tag=N1N9PYT5B02B type=del 2024-04-17T04:45:06.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.328+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=2MZU3SRMPINO type=del size=305944 2024-04-17T04:45:06.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.328+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=ZSY63DUEDS62 type=write size=137780 2024-04-17T04:45:06.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.332+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=LVBQEN5Z2BNE type=del size=1827906 2024-04-17T04:45:06.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.340+0000 7fd8ef2218c0 1 < suggested remove operation key=EWAW6EBW6W6U tag=HUEV6LO91J6D type=del 2024-04-17T04:45:06.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.344+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=CGCUARPVALNE type=write size=777459 2024-04-17T04:45:06.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.344+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~8M5M911JDW40.0 size=1048576 2024-04-17T04:45:06.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.348+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~8M5M911JDW40.1 size=644818 2024-04-17T04:45:06.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.352+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=8M5M911JDW40 tag=2811DSR63CHY type=write size=2741970 2024-04-17T04:45:06.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~OMLCSTA6OOFC.0 size=131958 2024-04-17T04:45:06.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.360+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=OMLCSTA6OOFC tag=2S530QP7NJXB type=write size=1180534 2024-04-17T04:45:06.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.364+0000 7fd8ef2218c0 1 < suggested remove operation key=R3C8PAVPVGBY tag=8YI1JNORP72X type=del 2024-04-17T04:45:06.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.368+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=OIONC07OZXJW type=del size=3860855 2024-04-17T04:45:06.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.368+0000 7fd8ef2218c0 -1 < failed to complete operation key=EWAW6EBW6W6U tag=IUGZQOBIAV5D type=del size=3825392: (22) Invalid argument 2024-04-17T04:45:06.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.372+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=1EGDUTQ8DC4X type=del size=1532483 2024-04-17T04:45:06.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.376+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~IR84TM39R20M.0 size=997622 2024-04-17T04:45:06.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.380+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=IR84TM39R20M tag=RPURWMO88B0C type=write size=2046198 2024-04-17T04:45:06.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.384+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=WYSCVK0U2AQX type=del size=2634296 2024-04-17T04:45:06.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.384+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~85GDO42ZPL2I.0 size=1048576 2024-04-17T04:45:06.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~85GDO42ZPL2I.1 size=945096 2024-04-17T04:45:06.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.392+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=85GDO42ZPL2I tag=77OTCQ2H69QI type=write size=3042248 2024-04-17T04:45:06.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.396+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=S9HUWLHGJXIA type=del size=2168927 2024-04-17T04:45:06.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.400+0000 7fd8ef2218c0 1 < suggested update operation key=N11Y8FPN9XES tag=CT34QRWCOV4S type=del size=3832958 2024-04-17T04:45:06.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~I4X62TR6KKS2.0 size=1048576 2024-04-17T04:45:06.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~I4X62TR6KKS2.1 size=966522 2024-04-17T04:45:06.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.408+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=I4X62TR6KKS2 tag=H340OSTZKE51 type=write size=3063674 2024-04-17T04:45:06.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~Q280TFN0QEB6.0 size=200891 2024-04-17T04:45:06.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.416+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=Q280TFN0QEB6 tag=9HD6Z9JH343M type=write size=1249467 2024-04-17T04:45:06.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.420+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=YZLNO9PJNO1Z type=write size=166803 2024-04-17T04:45:06.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.424+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=HEEPOAZU6S35 type=del size=3577267 2024-04-17T04:45:06.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.428+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~GWXI7589MO0I.0 size=1048576 2024-04-17T04:45:06.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~GWXI7589MO0I.1 size=1048576 2024-04-17T04:45:06.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~GWXI7589MO0I.2 size=765728 2024-04-17T04:45:06.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.432+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=GWXI7589MO0I tag=7NDZWJZZOAXV type=write size=3911456 2024-04-17T04:45:06.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.436+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~OVA5WSDUKC2Y.0 size=1048576 2024-04-17T04:45:06.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~OVA5WSDUKC2Y.1 size=1048576 2024-04-17T04:45:06.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~OVA5WSDUKC2Y.2 size=863877 2024-04-17T04:45:06.443 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.448+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=OVA5WSDUKC2Y tag=MAN0BR362Z2T type=write size=4009605 2024-04-17T04:45:06.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.452+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=LT4C1MQLNAEU type=del size=980877 2024-04-17T04:45:06.450 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~VQQ5229BFTTF.0 size=317547 2024-04-17T04:45:06.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.460+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=VQQ5229BFTTF tag=CZ961HYZGYXS type=write size=1366123 2024-04-17T04:45:06.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.464+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=IAPCZZTSJLIH type=del size=1488275 2024-04-17T04:45:06.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.468+0000 7fd8ef2218c0 1 < canceled operation key=3OFQH4KVYN18 tag=IIJHGB5NG2MR type=write size=646871 2024-04-17T04:45:06.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.472+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~4UQU0ENZT0R4.0 size=958068 2024-04-17T04:45:06.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.472+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=4UQU0ENZT0R4 tag=LXY4NY7S5JVA type=write size=2006644 2024-04-17T04:45:06.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.476+0000 7fd8ef2218c0 1 < suggested update operation key=JVWRJBMFIRLL tag=Z98FR973L3XH type=del size=2134567 2024-04-17T04:45:06.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.480+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=0X1J7IAF4T0S type=write size=852474 2024-04-17T04:45:06.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.484+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=R3C8PAVPVGBY tag=EN3WEINTV9W8 type=write size=973748: (2) No such file or directory 2024-04-17T04:45:06.481 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.484+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=QY0NH26KP5UF type=del size=1497000 2024-04-17T04:45:06.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.488+0000 7fd8ef2218c0 -1 < failed to complete operation key=EWAW6EBW6W6U tag=2MZU3SRMPINO type=del size=305944: (22) Invalid argument 2024-04-17T04:45:06.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.492+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=MVRTJZOI40DN type=del size=1865110 2024-04-17T04:45:06.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.496+0000 7fd8ef2218c0 -1 < failed to complete operation key=EWAW6EBW6W6U tag=LVBQEN5Z2BNE type=del size=1827906: (22) Invalid argument 2024-04-17T04:45:06.495 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~S6Q1ZQEQKEQ4.0 size=1048576 2024-04-17T04:45:06.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~S6Q1ZQEQKEQ4.1 size=850240 2024-04-17T04:45:06.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.504+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=S6Q1ZQEQKEQ4 tag=92ER4S5ZEWAO type=write size=2947392 2024-04-17T04:45:06.500 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.504+0000 7fd8ef2218c0 1 < completed write operation key=EWAW6EBW6W6U tag=CGCUARPVALNE type=write size=777459 2024-04-17T04:45:06.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~4GUOFC5XRR5K.0 size=680131 2024-04-17T04:45:06.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=4GUOFC5XRR5K tag=JV8ZX52LK3KP type=write size=1728707 2024-04-17T04:45:06.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.512+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~OMLCSTA6OOFC.0 size=131958 2024-04-17T04:45:06.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.520+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=OMLCSTA6OOFC tag=2S530QP7NJXB type=write size=1180534 2024-04-17T04:45:06.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.520+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=C8G7S67VDCVQ type=del size=867734 2024-04-17T04:45:06.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.524+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=OIONC07OZXJW type=del 2024-04-17T04:45:06.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.532+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=TSPMQXMW9SMR type=del size=862366 2024-04-17T04:45:06.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.532+0000 7fd8ef2218c0 -1 < failed to complete operation key=N11Y8FPN9XES tag=1EGDUTQ8DC4X type=del size=1532483: (22) Invalid argument 2024-04-17T04:45:06.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.536+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=BTA2PW4W2EK7 type=del size=1357313 2024-04-17T04:45:06.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.544+0000 7fd8ef2218c0 1 < suggested remove operation key=509DZ9DBJWC5 tag=WYSCVK0U2AQX type=del 2024-04-17T04:45:06.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.548+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=0AZ83C7GY39L type=del size=3275778 2024-04-17T04:45:06.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.548+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=S9HUWLHGJXIA type=del 2024-04-17T04:45:06.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.556+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5WUDIO2SPX4B.0 size=1048576 2024-04-17T04:45:06.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~5WUDIO2SPX4B.1 size=312886 2024-04-17T04:45:06.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.560+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=5WUDIO2SPX4B tag=Q7LBAV2F901Q type=write size=2410038 2024-04-17T04:45:06.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~I4X62TR6KKS2.0 size=1048576 2024-04-17T04:45:06.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.564+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~I4X62TR6KKS2.1 size=966522 2024-04-17T04:45:06.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.572+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=I4X62TR6KKS2 tag=H340OSTZKE51 type=write size=3063674 2024-04-17T04:45:06.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.576+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=RRAAFLLJ3B8U type=del size=2817047 2024-04-17T04:45:06.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.580+0000 7fd8ef2218c0 1 < suggested update operation key=WCIJICR0FT2C tag=YZLNO9PJNO1Z type=write size=166803 2024-04-17T04:45:06.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.588+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~Y97CKVY0PDZM.0 size=1048576 2024-04-17T04:45:06.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.588+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~Y97CKVY0PDZM.1 size=248367 2024-04-17T04:45:06.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.588+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=Y97CKVY0PDZM tag=FWV10OVO7YM9 type=write size=2345519 2024-04-17T04:45:06.589 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.592+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~GWXI7589MO0I.0 size=1048576 2024-04-17T04:45:06.591 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.596+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~GWXI7589MO0I.1 size=1048576 2024-04-17T04:45:06.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~GWXI7589MO0I.2 size=765728 2024-04-17T04:45:06.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.604+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=GWXI7589MO0I tag=7NDZWJZZOAXV type=write size=3911456 2024-04-17T04:45:06.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.608+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=9DZIKYAOOG2F type=del size=3852364 2024-04-17T04:45:06.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.612+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=LT4C1MQLNAEU type=del 2024-04-17T04:45:06.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~NCD396MWW7AC.0 size=1048576 2024-04-17T04:45:06.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.620+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~NCD396MWW7AC.1 size=492354 2024-04-17T04:45:06.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.620+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=NCD396MWW7AC tag=EBVERZBJN3HC type=write size=2589506 2024-04-17T04:45:06.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.628+0000 7fd8ef2218c0 1 < suggested update operation key=EWAW6EBW6W6U tag=IAPCZZTSJLIH type=del size=1488275 2024-04-17T04:45:06.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.632+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=9MZOVMGY5G7V type=del size=3259703 2024-04-17T04:45:06.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~4UQU0ENZT0R4.0 size=958068 2024-04-17T04:45:06.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.640+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=4UQU0ENZT0R4 tag=LXY4NY7S5JVA type=write size=2006644 2024-04-17T04:45:06.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.644+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=91ZKU5GDN7H1 type=del size=2111767 2024-04-17T04:45:06.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.648+0000 7fd8ef2218c0 1 < completed write operation key=MS3T476JOEON tag=0X1J7IAF4T0S type=write size=852474 2024-04-17T04:45:06.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.652+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=YM99WL1GQRW0 type=del size=2991918 2024-04-17T04:45:06.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.652+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=QY0NH26KP5UF type=del 2024-04-17T04:45:06.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.656+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=6SAT7XN817QL type=del size=2929731 2024-04-17T04:45:06.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.660+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=MVRTJZOI40DN type=del 2024-04-17T04:45:06.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.664+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=1VS5HDSVN0KI type=del size=2251858 2024-04-17T04:45:06.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~S6Q1ZQEQKEQ4.0 size=1048576 2024-04-17T04:45:06.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.672+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~S6Q1ZQEQKEQ4.1 size=850240 2024-04-17T04:45:06.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.676+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=S6Q1ZQEQKEQ4 tag=92ER4S5ZEWAO type=write size=2947392 2024-04-17T04:45:06.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.680+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=AAIJ0O47GB8F type=del size=689235 2024-04-17T04:45:06.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.684+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~4GUOFC5XRR5K.0 size=680131 2024-04-17T04:45:06.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.688+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=4GUOFC5XRR5K tag=JV8ZX52LK3KP type=write size=1728707 2024-04-17T04:45:06.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.696+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~O0O99IT42ZWQ.0 size=1048576 2024-04-17T04:45:06.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~O0O99IT42ZWQ.1 size=529830 2024-04-17T04:45:06.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.700+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=O0O99IT42ZWQ tag=KS3J2EIBCWVL type=write size=2626982 2024-04-17T04:45:06.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.704+0000 7fd8ef2218c0 1 < canceled operation key=MS3T476JOEON tag=C8G7S67VDCVQ type=del size=867734 2024-04-17T04:45:06.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.708+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=931QOSR8H3DP type=del size=1065355 2024-04-17T04:45:06.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.712+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=TSPMQXMW9SMR type=del 2024-04-17T04:45:06.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~17PSRP8EJAAY.0 size=1048576 2024-04-17T04:45:06.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~17PSRP8EJAAY.1 size=27841 2024-04-17T04:45:06.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.720+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=17PSRP8EJAAY tag=OHWJTE82RN8P type=write size=2124993 2024-04-17T04:45:06.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.724+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=BTA2PW4W2EK7 type=del 2024-04-17T04:45:06.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.728+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=YMHK160SDVZ5 type=del size=1566856 2024-04-17T04:45:06.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.732+0000 7fd8ef2218c0 1 < suggested remove operation key=UNOVYXDS5JEB tag=0AZ83C7GY39L type=del 2024-04-17T04:45:06.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.736+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=RMR0ABCWJBEO type=del size=2801091 2024-04-17T04:45:06.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5WUDIO2SPX4B.0 size=1048576 2024-04-17T04:45:06.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.744+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~5WUDIO2SPX4B.1 size=312886 2024-04-17T04:45:06.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.748+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=5WUDIO2SPX4B tag=Q7LBAV2F901Q type=write size=2410038 2024-04-17T04:45:06.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.752+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=LBDF02TUE0MA type=del size=550027 2024-04-17T04:45:06.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.756+0000 7fd8ef2218c0 -1 < failed to complete operation key=EWAW6EBW6W6U tag=RRAAFLLJ3B8U type=del size=2817047: (22) Invalid argument 2024-04-17T04:45:06.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.760+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=LCTBMXXXYJHF type=write size=822023 2024-04-17T04:45:06.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.764+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~Y97CKVY0PDZM.0 size=1048576 2024-04-17T04:45:06.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.764+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~Y97CKVY0PDZM.1 size=248367 2024-04-17T04:45:06.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.772+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=Y97CKVY0PDZM tag=FWV10OVO7YM9 type=write size=2345519 2024-04-17T04:45:06.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.776+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=OIN1FK1NR9CP type=write size=150182 2024-04-17T04:45:06.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.776+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=9DZIKYAOOG2F type=del 2024-04-17T04:45:06.777 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.780+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=V0RMN26CZSUX type=del size=1284673 2024-04-17T04:45:06.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.784+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~NCD396MWW7AC.0 size=1048576 2024-04-17T04:45:06.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.788+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~NCD396MWW7AC.1 size=492354 2024-04-17T04:45:06.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.792+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=NCD396MWW7AC tag=EBVERZBJN3HC type=write size=2589506 2024-04-17T04:45:06.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~O7VADN1O2QY0.0 size=430067 2024-04-17T04:45:06.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.800+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=O7VADN1O2QY0 tag=E6KEG3CE5EKJ type=write size=1478643 2024-04-17T04:45:06.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.804+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=9MZOVMGY5G7V type=del 2024-04-17T04:45:06.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.804+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=GVDXFF9UVP4X type=write size=543803 2024-04-17T04:45:06.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.812+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=91ZKU5GDN7H1 type=del 2024-04-17T04:45:06.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.812+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=GHAYX4OCA7TS type=del size=636970 2024-04-17T04:45:06.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.816+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=YM99WL1GQRW0 type=del 2024-04-17T04:45:06.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.820+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=62B37PBH5H10 type=del size=70052 2024-04-17T04:45:06.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.824+0000 7fd8ef2218c0 1 < suggested update operation key=VUYM3JC57QRP tag=6SAT7XN817QL type=del size=2929731 2024-04-17T04:45:06.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~1QIS3GHOYU60.0 size=484072 2024-04-17T04:45:06.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=1QIS3GHOYU60 tag=7M04RZGKJ9MC type=write size=1532648 2024-04-17T04:45:06.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.836+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=1VS5HDSVN0KI type=del 2024-04-17T04:45:06.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.840+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=VN8R222KI23T type=del size=819937 2024-04-17T04:45:06.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.844+0000 7fd8ef2218c0 1 < suggested update operation key=UYCKMXE6LBUH tag=AAIJ0O47GB8F type=del size=689235 2024-04-17T04:45:06.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.848+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=OA61O37DQV23 type=del size=847073 2024-04-17T04:45:06.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.856+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~O0O99IT42ZWQ.0 size=1048576 2024-04-17T04:45:06.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.860+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~O0O99IT42ZWQ.1 size=529830 2024-04-17T04:45:06.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.864+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=O0O99IT42ZWQ tag=KS3J2EIBCWVL type=write size=2626982 2024-04-17T04:45:06.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~GTEW3O3ND3RI.0 size=83664 2024-04-17T04:45:06.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.872+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=GTEW3O3ND3RI tag=G5YU78NYB0NV type=write size=1132240 2024-04-17T04:45:06.872 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.876+0000 7fd8ef2218c0 1 < suggested update operation key=MS3T476JOEON tag=931QOSR8H3DP type=del size=1065355 2024-04-17T04:45:06.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.880+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~6NW7OHJBLLZ4.0 size=1048576 2024-04-17T04:45:06.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.884+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~6NW7OHJBLLZ4.1 size=86687 2024-04-17T04:45:06.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.884+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=6NW7OHJBLLZ4 tag=XS2J41XRG71I type=write size=2183839 2024-04-17T04:45:06.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~17PSRP8EJAAY.0 size=1048576 2024-04-17T04:45:06.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~17PSRP8EJAAY.1 size=27841 2024-04-17T04:45:06.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.896+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=17PSRP8EJAAY tag=OHWJTE82RN8P type=write size=2124993 2024-04-17T04:45:06.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.900+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=0U36M85RZEKX type=del size=171131 2024-04-17T04:45:06.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.904+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=YMHK160SDVZ5 type=del 2024-04-17T04:45:06.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.908+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=FI1LTPPUI6TP type=del size=2170251 2024-04-17T04:45:06.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.908+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=RMR0ABCWJBEO type=del 2024-04-17T04:45:06.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~1FE835XTNB1O.0 size=1048576 2024-04-17T04:45:06.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~1FE835XTNB1O.1 size=255689 2024-04-17T04:45:06.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.920+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=1FE835XTNB1O tag=GX8SXZ2N6Q91 type=write size=2352841 2024-04-17T04:45:06.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.924+0000 7fd8ef2218c0 -1 < failed to complete operation key=MS3T476JOEON tag=LBDF02TUE0MA type=del size=550027: (22) Invalid argument 2024-04-17T04:45:06.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.924+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=M9191ZHRIRB8 type=del size=1988549 2024-04-17T04:45:06.925 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.928+0000 7fd8ef2218c0 1 < completed write operation key=2IY8DB37FI9H tag=LCTBMXXXYJHF type=write size=822023 2024-04-17T04:45:06.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.932+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=YBPHQ4IN4CHF type=del size=1200089 2024-04-17T04:45:06.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.936+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=OIN1FK1NR9CP type=write size=150182 2024-04-17T04:45:06.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.940+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=KLK3S0P4U3EW type=del size=1489133 2024-04-17T04:45:06.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.944+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=V0RMN26CZSUX type=del 2024-04-17T04:45:06.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.948+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=FD9H3MC2R9ZJ type=del size=3173788 2024-04-17T04:45:06.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~O7VADN1O2QY0.0 size=430067 2024-04-17T04:45:06.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.956+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=O7VADN1O2QY0 tag=E6KEG3CE5EKJ type=write size=1478643 2024-04-17T04:45:06.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.960+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=L0OG8KWLTDFP type=del size=1087335 2024-04-17T04:45:06.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.964+0000 7fd8ef2218c0 1 < completed write operation key=5ZULD0PHLS4V tag=GVDXFF9UVP4X type=write size=543803 2024-04-17T04:45:06.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.968+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=7DHDO15OL51K type=write size=239201 2024-04-17T04:45:06.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.972+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=GHAYX4OCA7TS type=del 2024-04-17T04:45:06.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.976+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~1YZA1UY1YJLJ.0 size=1048576 2024-04-17T04:45:06.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~1YZA1UY1YJLJ.1 size=1048576 2024-04-17T04:45:06.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~1YZA1UY1YJLJ.2 size=261468 2024-04-17T04:45:06.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.980+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=1YZA1UY1YJLJ tag=6PKRCLN9ZLM5 type=write size=3407196 2024-04-17T04:45:06.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.984+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=62B37PBH5H10 type=del 2024-04-17T04:45:06.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.992+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~U5M3HKPGZHJV.0 size=598569 2024-04-17T04:45:06.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.992+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=U5M3HKPGZHJV tag=ZZKGI172VIP3 type=write size=1647145 2024-04-17T04:45:06.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:06.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~1QIS3GHOYU60.0 size=484072 2024-04-17T04:45:06.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.000+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=1QIS3GHOYU60 tag=7M04RZGKJ9MC type=write size=1532648 2024-04-17T04:45:06.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.000+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=2VE953SLDLAF type=del size=3859330 2024-04-17T04:45:07.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.004+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=VN8R222KI23T type=del 2024-04-17T04:45:07.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~GQETWL7293YO.0 size=1048576 2024-04-17T04:45:07.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~GQETWL7293YO.1 size=1048576 2024-04-17T04:45:07.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~GQETWL7293YO.2 size=592610 2024-04-17T04:45:07.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.016+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=GQETWL7293YO tag=YTUM9U783FAS type=write size=3738338 2024-04-17T04:45:07.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.020+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=OA61O37DQV23 type=del 2024-04-17T04:45:07.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.024+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=WAQI6Z7ZKLWY type=del size=1778275 2024-04-17T04:45:07.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~GTEW3O3ND3RI.0 size=83664 2024-04-17T04:45:07.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.032+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=GTEW3O3ND3RI tag=G5YU78NYB0NV type=write size=1132240 2024-04-17T04:45:07.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.036+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=E7PD80NKWJHH type=del size=3859500 2024-04-17T04:45:07.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~6NW7OHJBLLZ4.0 size=1048576 2024-04-17T04:45:07.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~6NW7OHJBLLZ4.1 size=86687 2024-04-17T04:45:07.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.048+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=6NW7OHJBLLZ4 tag=XS2J41XRG71I type=write size=2183839 2024-04-17T04:45:07.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.052+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~X2X19EF0OZ7M.0 size=207666 2024-04-17T04:45:07.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.052+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=X2X19EF0OZ7M tag=79AOSLQZ1A5R type=write size=1256242 2024-04-17T04:45:07.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.056+0000 7fd8ef2218c0 1 < canceled operation key=M3U8L7M8JKG9 tag=0U36M85RZEKX type=del size=171131 2024-04-17T04:45:07.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.060+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~6OP3E5GD9XHK.0 size=1048576 2024-04-17T04:45:07.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.064+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~6OP3E5GD9XHK.1 size=1048576 2024-04-17T04:45:07.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.064+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~6OP3E5GD9XHK.2 size=771993 2024-04-17T04:45:07.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.064+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=6OP3E5GD9XHK tag=O1IMBOYV2H7I type=write size=3917721 2024-04-17T04:45:07.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.068+0000 7fd8ef2218c0 1 < canceled operation key=1JOWGWFR7R6P tag=FI1LTPPUI6TP type=del size=2170251 2024-04-17T04:45:07.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~R68DAF2VDQDV.0 size=1048576 2024-04-17T04:45:07.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~R68DAF2VDQDV.1 size=168912 2024-04-17T04:45:07.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.076+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=R68DAF2VDQDV tag=FHQENXDLRJ0Y type=write size=2266064 2024-04-17T04:45:07.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~1FE835XTNB1O.0 size=1048576 2024-04-17T04:45:07.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.084+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~1FE835XTNB1O.1 size=255689 2024-04-17T04:45:07.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.088+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=1FE835XTNB1O tag=GX8SXZ2N6Q91 type=write size=2352841 2024-04-17T04:45:07.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.092+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=2M5LRMEAB4K2 type=write size=786852 2024-04-17T04:45:07.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.096+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=M9191ZHRIRB8 type=del 2024-04-17T04:45:07.095 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.100+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=I9WK3DNMSMXH type=write size=43875 2024-04-17T04:45:07.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.104+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=YBPHQ4IN4CHF type=del 2024-04-17T04:45:07.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.108+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~TEI7BMXGRHES.0 size=1048576 2024-04-17T04:45:07.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~TEI7BMXGRHES.1 size=1048576 2024-04-17T04:45:07.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~TEI7BMXGRHES.2 size=152072 2024-04-17T04:45:07.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=TEI7BMXGRHES tag=4660T4O5144L type=write size=3297800 2024-04-17T04:45:07.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.120+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=KLK3S0P4U3EW type=del 2024-04-17T04:45:07.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~1LHAA0A4E1N8.0 size=526918 2024-04-17T04:45:07.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.124+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=1LHAA0A4E1N8 tag=O9LI62QZ96DY type=write size=1575494 2024-04-17T04:45:07.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.128+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=FD9H3MC2R9ZJ type=del 2024-04-17T04:45:07.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.132+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=BA8UKGCW13C2 type=del size=2830124 2024-04-17T04:45:07.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.136+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=L0OG8KWLTDFP type=del 2024-04-17T04:45:07.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.136+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=OF9ZZ1CX6DIC type=del size=3112257 2024-04-17T04:45:07.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.140+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=7DHDO15OL51K type=write size=239201 2024-04-17T04:45:07.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~UY9Q4B95IO3R.0 size=1048576 2024-04-17T04:45:07.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~UY9Q4B95IO3R.1 size=543377 2024-04-17T04:45:07.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.148+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=UY9Q4B95IO3R tag=41YU5G36TTAU type=write size=2640529 2024-04-17T04:45:07.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~1YZA1UY1YJLJ.0 size=1048576 2024-04-17T04:45:07.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.156+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~1YZA1UY1YJLJ.1 size=1048576 2024-04-17T04:45:07.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.160+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~1YZA1UY1YJLJ.2 size=261468 2024-04-17T04:45:07.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.168+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=1YZA1UY1YJLJ tag=6PKRCLN9ZLM5 type=write size=3407196 2024-04-17T04:45:07.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.172+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=PTVAQZFETXML type=del size=2752252 2024-04-17T04:45:07.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.172+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~U5M3HKPGZHJV.0 size=598569 2024-04-17T04:45:07.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.176+0000 7fd8ef2218c0 1 < canceled multipart upload key=6WSZ5UL0V1Y1 upload=U5M3HKPGZHJV tag=ZZKGI172VIP3 type=write size=1647145 2024-04-17T04:45:07.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.180+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=QGVCJJS6DORH type=write size=435533 2024-04-17T04:45:07.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.184+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=2VE953SLDLAF type=del 2024-04-17T04:45:07.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.188+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=AG8Q5QO8Y2H8 type=del size=1882875 2024-04-17T04:45:07.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.192+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~GQETWL7293YO.0 size=1048576 2024-04-17T04:45:07.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.192+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~GQETWL7293YO.1 size=1048576 2024-04-17T04:45:07.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.192+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UNOVYXDS5JEB.2~GQETWL7293YO.2 size=592610 2024-04-17T04:45:07.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.200+0000 7fd8ef2218c0 1 < canceled multipart upload key=UNOVYXDS5JEB upload=GQETWL7293YO tag=YTUM9U783FAS type=write size=3738338 2024-04-17T04:45:07.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.204+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~AFHHRPWNRT8E.0 size=1048576 2024-04-17T04:45:07.203 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~AFHHRPWNRT8E.1 size=806344 2024-04-17T04:45:07.203 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.208+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=AFHHRPWNRT8E tag=9VZ7T0LTDMRY type=write size=2903496 2024-04-17T04:45:07.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.212+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=WAQI6Z7ZKLWY type=del 2024-04-17T04:45:07.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.216+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=Q1IU3EXE062Y type=del size=3165428 2024-04-17T04:45:07.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.220+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=E7PD80NKWJHH type=del 2024-04-17T04:45:07.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.224+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=3KD3IPGGKOEA type=del size=540099 2024-04-17T04:45:07.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.224+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5ZULD0PHLS4V.2~X2X19EF0OZ7M.0 size=207666 2024-04-17T04:45:07.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.228+0000 7fd8ef2218c0 1 < canceled multipart upload key=5ZULD0PHLS4V upload=X2X19EF0OZ7M tag=79AOSLQZ1A5R type=write size=1256242 2024-04-17T04:45:07.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.232+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=DF897QYQPEPN type=del size=2591982 2024-04-17T04:45:07.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.232+0000 7fd8ef2218c0 1 < canceled part key=_multipart_R3C8PAVPVGBY.2~6OP3E5GD9XHK.0 size=1048576 2024-04-17T04:45:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.236+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~6OP3E5GD9XHK.1 size=1048576 2024-04-17T04:45:07.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.240+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~6OP3E5GD9XHK.2 size=771993 2024-04-17T04:45:07.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.248+0000 7fd8ef2218c0 1 < canceled multipart upload key=R3C8PAVPVGBY upload=6OP3E5GD9XHK tag=O1IMBOYV2H7I type=write size=3917721 2024-04-17T04:45:07.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.248+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=WXPZ336E6WY3 type=del size=4166370 2024-04-17T04:45:07.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.252+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~R68DAF2VDQDV.0 size=1048576 2024-04-17T04:45:07.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.252+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~R68DAF2VDQDV.1 size=168912 2024-04-17T04:45:07.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.260+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=R68DAF2VDQDV tag=FHQENXDLRJ0Y type=write size=2266064 2024-04-17T04:45:07.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.264+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~ZFZWS1IP5VHE.0 size=1048576 2024-04-17T04:45:07.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.268+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~ZFZWS1IP5VHE.1 size=19 2024-04-17T04:45:07.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.268+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=ZFZWS1IP5VHE tag=VUB3A051GDMN type=write size=2097171 2024-04-17T04:45:07.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.272+0000 7fd8ef2218c0 1 < suggested update operation key=2IY8DB37FI9H tag=2M5LRMEAB4K2 type=write size=786852 2024-04-17T04:45:07.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~3CTLPBI6IM67.0 size=1048576 2024-04-17T04:45:07.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~3CTLPBI6IM67.1 size=126174 2024-04-17T04:45:07.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.280+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=3CTLPBI6IM67 tag=E0LU6HFM0MX8 type=write size=2223326 2024-04-17T04:45:07.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.284+0000 7fd8ef2218c0 1 < completed write operation key=5K50O96P7OGD tag=I9WK3DNMSMXH type=write size=43875 2024-04-17T04:45:07.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.288+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=7N9PFWBBLGMS type=del size=214608 2024-04-17T04:45:07.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.292+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~TEI7BMXGRHES.0 size=1048576 2024-04-17T04:45:07.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~TEI7BMXGRHES.1 size=1048576 2024-04-17T04:45:07.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~TEI7BMXGRHES.2 size=152072 2024-04-17T04:45:07.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.300+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=TEI7BMXGRHES tag=4660T4O5144L type=write size=3297800 2024-04-17T04:45:07.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~JQJ54S6J4KDH.0 size=1048576 2024-04-17T04:45:07.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.312+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~JQJ54S6J4KDH.1 size=1034645 2024-04-17T04:45:07.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.312+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=JQJ54S6J4KDH tag=NBAIGYZBDVMH type=write size=3131797 2024-04-17T04:45:07.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~1LHAA0A4E1N8.0 size=526918 2024-04-17T04:45:07.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.320+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=1LHAA0A4E1N8 tag=O9LI62QZ96DY type=write size=1575494 2024-04-17T04:45:07.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.324+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=U11QQJA3MYSA type=del size=1325294 2024-04-17T04:45:07.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.332+0000 7fd8ef2218c0 1 < suggested remove operation key=5FR1NEWE0LD1 tag=BA8UKGCW13C2 type=del 2024-04-17T04:45:07.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.336+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~B9JH0J9FPOMA.0 size=334135 2024-04-17T04:45:07.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.336+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=B9JH0J9FPOMA tag=XFEVYQZK8B1E type=write size=1382711 2024-04-17T04:45:07.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.340+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=OF9ZZ1CX6DIC type=del 2024-04-17T04:45:07.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~VP0A08O3VP6O.0 size=1048576 2024-04-17T04:45:07.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.348+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~VP0A08O3VP6O.1 size=667414 2024-04-17T04:45:07.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.348+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=VP0A08O3VP6O tag=X5APZ6994DIB type=write size=2764566 2024-04-17T04:45:07.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~UY9Q4B95IO3R.0 size=1048576 2024-04-17T04:45:07.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.356+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~UY9Q4B95IO3R.1 size=543377 2024-04-17T04:45:07.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.364+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=UY9Q4B95IO3R tag=41YU5G36TTAU type=write size=2640529 2024-04-17T04:45:07.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.364+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=GF1LFPG7JNPQ type=del size=583281 2024-04-17T04:45:07.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.368+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=PTVAQZFETXML type=del 2024-04-17T04:45:07.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.372+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=K0Z7SINMSOLO type=del size=2968253 2024-04-17T04:45:07.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.376+0000 7fd8ef2218c0 -1 < failed to complete operation key=5FR1NEWE0LD1 tag=QGVCJJS6DORH type=write size=435533: (22) Invalid argument 2024-04-17T04:45:07.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.380+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=L5K0BEXOOQWE type=write size=182440 2024-04-17T04:45:07.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.384+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=AG8Q5QO8Y2H8 type=del 2024-04-17T04:45:07.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.384+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=V65DJ5FGW024 type=del size=1789663 2024-04-17T04:45:07.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~AFHHRPWNRT8E.0 size=1048576 2024-04-17T04:45:07.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.392+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~AFHHRPWNRT8E.1 size=806344 2024-04-17T04:45:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.396+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=AFHHRPWNRT8E tag=9VZ7T0LTDMRY type=write size=2903496 2024-04-17T04:45:07.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.400+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=JEUD1RSBGO17 type=write size=550917 2024-04-17T04:45:07.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.408+0000 7fd8ef2218c0 1 < suggested remove operation key=509DZ9DBJWC5 tag=Q1IU3EXE062Y type=del 2024-04-17T04:45:07.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.412+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=ZO9KKC2E4R9T type=del size=2584287 2024-04-17T04:45:07.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.412+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=5FR1NEWE0LD1 tag=3KD3IPGGKOEA type=del size=540099: (2) No such file or directory 2024-04-17T04:45:07.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.416+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=QPFCMPQR0T97 type=del size=764040 2024-04-17T04:45:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.420+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=DF897QYQPEPN type=del 2024-04-17T04:45:07.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.424+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=7FDMPBAZZGHA type=del size=4174927 2024-04-17T04:45:07.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.428+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=WXPZ336E6WY3 type=del 2024-04-17T04:45:07.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~562ERW2ZDIWS.0 size=1048576 2024-04-17T04:45:07.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~562ERW2ZDIWS.1 size=820739 2024-04-17T04:45:07.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.436+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=562ERW2ZDIWS tag=NA5UNDHYJKSR type=write size=2917891 2024-04-17T04:45:07.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~ZFZWS1IP5VHE.0 size=1048576 2024-04-17T04:45:07.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.444+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~ZFZWS1IP5VHE.1 size=19 2024-04-17T04:45:07.443 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.448+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=ZFZWS1IP5VHE tag=VUB3A051GDMN type=write size=2097171 2024-04-17T04:45:07.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.452+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=OQLXE8LD5H6I type=del size=2934290 2024-04-17T04:45:07.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.452+0000 7fd8ef2218c0 1 < canceled part key=_multipart_AE4CYL334ONO.2~3CTLPBI6IM67.0 size=1048576 2024-04-17T04:45:07.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.452+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~3CTLPBI6IM67.1 size=126174 2024-04-17T04:45:07.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.460+0000 7fd8ef2218c0 1 < canceled multipart upload key=AE4CYL334ONO upload=3CTLPBI6IM67 tag=E0LU6HFM0MX8 type=write size=2223326 2024-04-17T04:45:07.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.464+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~4F9P5H8FFMRW.0 size=1048576 2024-04-17T04:45:07.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.468+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~4F9P5H8FFMRW.1 size=1048576 2024-04-17T04:45:07.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.468+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~4F9P5H8FFMRW.2 size=861495 2024-04-17T04:45:07.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.468+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=4F9P5H8FFMRW tag=GTOOSBM9PM3F type=write size=4007223 2024-04-17T04:45:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.472+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=7N9PFWBBLGMS type=del 2024-04-17T04:45:07.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.476+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=XKTWQIOGY65S type=write size=986625 2024-04-17T04:45:07.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.480+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~JQJ54S6J4KDH.0 size=1048576 2024-04-17T04:45:07.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.484+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~JQJ54S6J4KDH.1 size=1034645 2024-04-17T04:45:07.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.488+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=JQJ54S6J4KDH tag=NBAIGYZBDVMH type=write size=3131797 2024-04-17T04:45:07.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.492+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=A6P16XYMPEPI type=del size=402433 2024-04-17T04:45:07.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.496+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=U11QQJA3MYSA type=del 2024-04-17T04:45:07.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.500+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=OULP6ZAPI7CC type=del size=1788618 2024-04-17T04:45:07.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.500+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~B9JH0J9FPOMA.0 size=334135 2024-04-17T04:45:07.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.504+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=B9JH0J9FPOMA tag=XFEVYQZK8B1E type=write size=1382711 2024-04-17T04:45:07.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~925Q5FEFNE08.0 size=693566 2024-04-17T04:45:07.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=925Q5FEFNE08 tag=6YKEUIUJ9RUD type=write size=1742142 2024-04-17T04:45:07.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~VP0A08O3VP6O.0 size=1048576 2024-04-17T04:45:07.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~VP0A08O3VP6O.1 size=667414 2024-04-17T04:45:07.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.524+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=VP0A08O3VP6O tag=X5APZ6994DIB type=write size=2764566 2024-04-17T04:45:07.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.524+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=VTWFFURQ6AWN type=del size=1796953 2024-04-17T04:45:07.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.528+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=GF1LFPG7JNPQ type=del 2024-04-17T04:45:07.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~862CVC02RV1F.0 size=1048576 2024-04-17T04:45:07.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~862CVC02RV1F.1 size=1048576 2024-04-17T04:45:07.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~862CVC02RV1F.2 size=630817 2024-04-17T04:45:07.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.536+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=862CVC02RV1F tag=I8FD602XKHKS type=write size=3776545 2024-04-17T04:45:07.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.540+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=K0Z7SINMSOLO type=del 2024-04-17T04:45:07.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.544+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=V108ME87TJ59 type=del size=1977284 2024-04-17T04:45:07.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.548+0000 7fd8ef2218c0 1 < completed write operation key=1ZTJP8DRITEK tag=L5K0BEXOOQWE type=write size=182440 2024-04-17T04:45:07.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.552+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~328TY9HSI33N.0 size=9308 2024-04-17T04:45:07.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.552+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=328TY9HSI33N tag=U0Z1RW90JBC8 type=write size=1057884 2024-04-17T04:45:07.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.552+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=V65DJ5FGW024 type=del 2024-04-17T04:45:07.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~YWPOO78O3O7J.0 size=1048576 2024-04-17T04:45:07.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~YWPOO78O3O7J.1 size=881568 2024-04-17T04:45:07.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.560+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=YWPOO78O3O7J tag=Y5ILATYBC42Q type=write size=2978720 2024-04-17T04:45:07.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.564+0000 7fd8ef2218c0 1 < completed write operation key=G03VLS09F3BT tag=JEUD1RSBGO17 type=write size=550917 2024-04-17T04:45:07.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.572+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~6HGLS7O3NG9I.0 size=550265 2024-04-17T04:45:07.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.572+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=6HGLS7O3NG9I tag=N1GIXZTAXOML type=write size=1598841 2024-04-17T04:45:07.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.576+0000 7fd8ef2218c0 1 < suggested remove operation key=UYCKMXE6LBUH tag=ZO9KKC2E4R9T type=del 2024-04-17T04:45:07.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.580+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=QMBC8AN3631C type=del size=1943958 2024-04-17T04:45:07.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.584+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=QPFCMPQR0T97 type=del 2024-04-17T04:45:07.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.588+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~MYH0ZOL2DS70.0 size=167017 2024-04-17T04:45:07.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.588+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=MYH0ZOL2DS70 tag=J7P3A2K6QZVF type=write size=1215593 2024-04-17T04:45:07.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.592+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=7FDMPBAZZGHA type=del 2024-04-17T04:45:07.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~8U7ED4R9TZ2D.0 size=1048576 2024-04-17T04:45:07.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.600+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~8U7ED4R9TZ2D.1 size=1048576 2024-04-17T04:45:07.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~8U7ED4R9TZ2D.2 size=975920 2024-04-17T04:45:07.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.604+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=8U7ED4R9TZ2D tag=VS49ALM3TLWZ type=write size=4121648 2024-04-17T04:45:07.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~562ERW2ZDIWS.0 size=1048576 2024-04-17T04:45:07.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~562ERW2ZDIWS.1 size=820739 2024-04-17T04:45:07.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.616+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=562ERW2ZDIWS tag=NA5UNDHYJKSR type=write size=2917891 2024-04-17T04:45:07.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.616+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=D5FAFS3ONBR7 type=del size=804699 2024-04-17T04:45:07.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.624+0000 7fd8ef2218c0 1 < suggested remove operation key=5FR1NEWE0LD1 tag=OQLXE8LD5H6I type=del 2024-04-17T04:45:07.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.628+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=RFGNNO8H2F1D type=del size=3863774 2024-04-17T04:45:07.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~4F9P5H8FFMRW.0 size=1048576 2024-04-17T04:45:07.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.632+0000 7fd8ef2218c0 1 < canceled part key=_multipart_R3C8PAVPVGBY.2~4F9P5H8FFMRW.1 size=1048576 2024-04-17T04:45:07.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~4F9P5H8FFMRW.2 size=861495 2024-04-17T04:45:07.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.640+0000 7fd8ef2218c0 1 < canceled multipart upload key=R3C8PAVPVGBY upload=4F9P5H8FFMRW tag=GTOOSBM9PM3F type=write size=4007223 2024-04-17T04:45:07.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.644+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=ZA59VTIKUYX5 type=del size=2048715 2024-04-17T04:45:07.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.648+0000 7fd8ef2218c0 1 < completed write operation key=WCIJICR0FT2C tag=XKTWQIOGY65S type=write size=986625 2024-04-17T04:45:07.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.648+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=EECNLABZ7IOP type=del size=1311621 2024-04-17T04:45:07.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.652+0000 7fd8ef2218c0 -1 < failed to complete operation key=5FR1NEWE0LD1 tag=A6P16XYMPEPI type=del size=402433: (22) Invalid argument 2024-04-17T04:45:07.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.656+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=BXSFPMI6PH9T type=del size=1675209 2024-04-17T04:45:07.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.660+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=OULP6ZAPI7CC type=del 2024-04-17T04:45:07.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.664+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=87URF29EF4O3 type=del size=849803 2024-04-17T04:45:07.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~925Q5FEFNE08.0 size=693566 2024-04-17T04:45:07.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.672+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=925Q5FEFNE08 tag=6YKEUIUJ9RUD type=write size=1742142 2024-04-17T04:45:07.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.676+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=JIDU0F09K7G2 type=del size=544458 2024-04-17T04:45:07.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.680+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=VTWFFURQ6AWN type=del 2024-04-17T04:45:07.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.684+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~TUHU63GWCG0S.0 size=1048576 2024-04-17T04:45:07.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.688+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~TUHU63GWCG0S.1 size=1048576 2024-04-17T04:45:07.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.692+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~TUHU63GWCG0S.2 size=441495 2024-04-17T04:45:07.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.692+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=TUHU63GWCG0S tag=UQEXHCQLO2N3 type=write size=3587223 2024-04-17T04:45:07.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.692+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~862CVC02RV1F.0 size=1048576 2024-04-17T04:45:07.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.696+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~862CVC02RV1F.1 size=1048576 2024-04-17T04:45:07.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.700+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~862CVC02RV1F.2 size=630817 2024-04-17T04:45:07.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.704+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=862CVC02RV1F tag=I8FD602XKHKS type=write size=3776545 2024-04-17T04:45:07.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~WJ4EMEDHAGHS.0 size=1048576 2024-04-17T04:45:07.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~WJ4EMEDHAGHS.1 size=1048576 2024-04-17T04:45:07.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~WJ4EMEDHAGHS.2 size=1042573 2024-04-17T04:45:07.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.716+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=WJ4EMEDHAGHS tag=F58664EK8AXQ type=write size=4188301 2024-04-17T04:45:07.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.720+0000 7fd8ef2218c0 1 < suggested update operation key=1ZTJP8DRITEK tag=V108ME87TJ59 type=del size=1977284 2024-04-17T04:45:07.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.724+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=CNMJ8L0KAODL type=del size=1913780 2024-04-17T04:45:07.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.728+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~328TY9HSI33N.0 size=9308 2024-04-17T04:45:07.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.736+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=328TY9HSI33N tag=U0Z1RW90JBC8 type=write size=1057884 2024-04-17T04:45:07.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~TXGT69S33WCW.0 size=1048576 2024-04-17T04:45:07.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~TXGT69S33WCW.1 size=1040096 2024-04-17T04:45:07.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=TXGT69S33WCW tag=2NVRIGXSJUAL type=write size=3137248 2024-04-17T04:45:07.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~YWPOO78O3O7J.0 size=1048576 2024-04-17T04:45:07.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~YWPOO78O3O7J.1 size=881568 2024-04-17T04:45:07.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.756+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=YWPOO78O3O7J tag=Y5ILATYBC42Q type=write size=2978720 2024-04-17T04:45:07.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~NLJBUE9YUISL.0 size=666434 2024-04-17T04:45:07.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.764+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=NLJBUE9YUISL tag=7326A20UPICG type=write size=1715010 2024-04-17T04:45:07.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~6HGLS7O3NG9I.0 size=550265 2024-04-17T04:45:07.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.772+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=6HGLS7O3NG9I tag=N1GIXZTAXOML type=write size=1598841 2024-04-17T04:45:07.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~T5X5BQVAMPXU.0 size=627946 2024-04-17T04:45:07.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.784+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=T5X5BQVAMPXU tag=QY83AY0JQ5DC type=write size=1676522 2024-04-17T04:45:07.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.784+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=QMBC8AN3631C type=del 2024-04-17T04:45:07.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~E1KAEAHU52E3.0 size=1048576 2024-04-17T04:45:07.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~E1KAEAHU52E3.1 size=989391 2024-04-17T04:45:07.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=E1KAEAHU52E3 tag=J6G108JPZH3H type=write size=3086543 2024-04-17T04:45:07.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.796+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~MYH0ZOL2DS70.0 size=167017 2024-04-17T04:45:07.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.800+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=MYH0ZOL2DS70 tag=J7P3A2K6QZVF type=write size=1215593 2024-04-17T04:45:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.804+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=DHFRSPW907HQ type=write size=159296 2024-04-17T04:45:07.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.804+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~8U7ED4R9TZ2D.0 size=1048576 2024-04-17T04:45:07.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~8U7ED4R9TZ2D.1 size=1048576 2024-04-17T04:45:07.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.812+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~8U7ED4R9TZ2D.2 size=975920 2024-04-17T04:45:07.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.816+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=8U7ED4R9TZ2D tag=VS49ALM3TLWZ type=write size=4121648 2024-04-17T04:45:07.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.816+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=4SALV1EHVNNM type=del size=1175607 2024-04-17T04:45:07.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.820+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=D5FAFS3ONBR7 type=del 2024-04-17T04:45:07.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.824+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=NCGHSTPOAC3N type=del size=2277407 2024-04-17T04:45:07.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.828+0000 7fd8ef2218c0 -1 < failed to complete operation key=1ZTJP8DRITEK tag=RFGNNO8H2F1D type=del size=3863774: (22) Invalid argument 2024-04-17T04:45:07.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~UI0FA1G0Z8A2.0 size=1048576 2024-04-17T04:45:07.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~UI0FA1G0Z8A2.1 size=928731 2024-04-17T04:45:07.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.836+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=UI0FA1G0Z8A2 tag=VAW21I2J48K7 type=write size=3025883 2024-04-17T04:45:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.836+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=ZA59VTIKUYX5 type=del 2024-04-17T04:45:07.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~7VXZ468B6RK8.0 size=1048576 2024-04-17T04:45:07.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~7VXZ468B6RK8.1 size=728823 2024-04-17T04:45:07.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=7VXZ468B6RK8 tag=VD69VOER8KHH type=write size=2825975 2024-04-17T04:45:07.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.848+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=EECNLABZ7IOP type=del 2024-04-17T04:45:07.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.852+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~K0OXY87I4NFU.0 size=1048576 2024-04-17T04:45:07.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.856+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~K0OXY87I4NFU.1 size=690681 2024-04-17T04:45:07.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.856+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=K0OXY87I4NFU tag=J10LDY3OTN9V type=write size=2787833 2024-04-17T04:45:07.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.860+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=BXSFPMI6PH9T type=del 2024-04-17T04:45:07.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.864+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=SLV0VAQIQPL7 type=del size=2516688 2024-04-17T04:45:07.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.864+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=87URF29EF4O3 type=del 2024-04-17T04:45:07.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.868+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=9F7YJTBHM26E type=del size=4126145 2024-04-17T04:45:07.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.872+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=JIDU0F09K7G2 type=del 2024-04-17T04:45:07.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.876+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=2P4WTMAYNJ3L type=write size=804620 2024-04-17T04:45:07.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~TUHU63GWCG0S.0 size=1048576 2024-04-17T04:45:07.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.880+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~TUHU63GWCG0S.1 size=1048576 2024-04-17T04:45:07.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.884+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~TUHU63GWCG0S.2 size=441495 2024-04-17T04:45:07.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.888+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=TUHU63GWCG0S tag=UQEXHCQLO2N3 type=write size=3587223 2024-04-17T04:45:07.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.892+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=PWAFXXON6U78 type=del size=1184791 2024-04-17T04:45:07.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.896+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~WJ4EMEDHAGHS.0 size=1048576 2024-04-17T04:45:07.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~WJ4EMEDHAGHS.1 size=1048576 2024-04-17T04:45:07.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.900+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~WJ4EMEDHAGHS.2 size=1042573 2024-04-17T04:45:07.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.904+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=WJ4EMEDHAGHS tag=F58664EK8AXQ type=write size=4188301 2024-04-17T04:45:07.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.908+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=I6O2Z5DSM5AI type=del size=1693517 2024-04-17T04:45:07.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.912+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=CNMJ8L0KAODL type=del 2024-04-17T04:45:07.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~3MEP1BI6CWJ3.0 size=1048576 2024-04-17T04:45:07.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~3MEP1BI6CWJ3.1 size=337715 2024-04-17T04:45:07.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.920+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=3MEP1BI6CWJ3 tag=YBHUMZNLMH4X type=write size=2434867 2024-04-17T04:45:07.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.924+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~TXGT69S33WCW.0 size=1048576 2024-04-17T04:45:07.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.928+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~TXGT69S33WCW.1 size=1040096 2024-04-17T04:45:07.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.932+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=TXGT69S33WCW tag=2NVRIGXSJUAL type=write size=3137248 2024-04-17T04:45:07.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~RN781G77BO8S.0 size=384258 2024-04-17T04:45:07.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.940+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=RN781G77BO8S tag=HYOI1FI4BCHY type=write size=1432834 2024-04-17T04:45:07.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~NLJBUE9YUISL.0 size=666434 2024-04-17T04:45:07.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.948+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=NLJBUE9YUISL tag=7326A20UPICG type=write size=1715010 2024-04-17T04:45:07.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.948+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=HU1S037Y8FTW type=del size=2329391 2024-04-17T04:45:07.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~T5X5BQVAMPXU.0 size=627946 2024-04-17T04:45:07.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.956+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=T5X5BQVAMPXU tag=QY83AY0JQ5DC type=write size=1676522 2024-04-17T04:45:07.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.960+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=ODJLU8MU4QSZ type=del size=3742433 2024-04-17T04:45:07.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.960+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~E1KAEAHU52E3.0 size=1048576 2024-04-17T04:45:07.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.964+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~E1KAEAHU52E3.1 size=989391 2024-04-17T04:45:07.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.968+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=E1KAEAHU52E3 tag=J6G108JPZH3H type=write size=3086543 2024-04-17T04:45:07.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~J0W25EGWBEMO.0 size=1048576 2024-04-17T04:45:07.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.976+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~J0W25EGWBEMO.1 size=289930 2024-04-17T04:45:07.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.976+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=J0W25EGWBEMO tag=7ZCXJOF7GHKA type=write size=2387082 2024-04-17T04:45:07.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.976+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=DHFRSPW907HQ type=write size=159296 2024-04-17T04:45:07.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.980+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=8LBOF65THPRZ type=del size=2774974 2024-04-17T04:45:07.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.984+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=4SALV1EHVNNM type=del 2024-04-17T04:45:07.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.988+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=33Q6QR3XF9TN type=del size=3426495 2024-04-17T04:45:07.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.992+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=NCGHSTPOAC3N type=del 2024-04-17T04:45:07.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:07.996+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~OQZRTZKNIN7S.0 size=1048576 2024-04-17T04:45:07.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~OQZRTZKNIN7S.1 size=246165 2024-04-17T04:45:07.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.000+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=OQZRTZKNIN7S tag=BWF09D47DCNI type=write size=2343317 2024-04-17T04:45:07.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.000+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~UI0FA1G0Z8A2.0 size=1048576 2024-04-17T04:45:07.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.004+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~UI0FA1G0Z8A2.1 size=928731 2024-04-17T04:45:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.008+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=UI0FA1G0Z8A2 tag=VAW21I2J48K7 type=write size=3025883 2024-04-17T04:45:08.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~QB73X6T6VKM6.0 size=5912 2024-04-17T04:45:08.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.016+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=QB73X6T6VKM6 tag=CWKZHY0LCD34 type=write size=1054488 2024-04-17T04:45:08.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.020+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~7VXZ468B6RK8.0 size=1048576 2024-04-17T04:45:08.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.020+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~7VXZ468B6RK8.1 size=728823 2024-04-17T04:45:08.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.024+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=7VXZ468B6RK8 tag=VD69VOER8KHH type=write size=2825975 2024-04-17T04:45:08.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.028+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=6L07T9SYO82N type=write size=455921 2024-04-17T04:45:08.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.032+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~K0OXY87I4NFU.0 size=1048576 2024-04-17T04:45:08.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~K0OXY87I4NFU.1 size=690681 2024-04-17T04:45:08.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.040+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=K0OXY87I4NFU tag=J10LDY3OTN9V type=write size=2787833 2024-04-17T04:45:08.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.044+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=TQ1766LB2678 type=del size=36682 2024-04-17T04:45:08.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.044+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=SLV0VAQIQPL7 type=del 2024-04-17T04:45:08.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.052+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~YKVGMFWLT8I1.0 size=1048576 2024-04-17T04:45:08.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.056+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~YKVGMFWLT8I1.1 size=1048576 2024-04-17T04:45:08.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.056+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~YKVGMFWLT8I1.2 size=1020962 2024-04-17T04:45:08.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.056+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=YKVGMFWLT8I1 tag=MV31Z2W1JPD1 type=write size=4166690 2024-04-17T04:45:08.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.064+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=9F7YJTBHM26E type=del 2024-04-17T04:45:08.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~CF61NW3VK8UL.0 size=959096 2024-04-17T04:45:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.072+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=CF61NW3VK8UL tag=QURDWQSK8Z0D type=write size=2007672 2024-04-17T04:45:08.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.076+0000 7fd8ef2218c0 1 < suggested remove operation key=B63EZS0LAC30 tag=2P4WTMAYNJ3L type=write 2024-04-17T04:45:08.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.080+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=4Y2VUPJEKTO0 type=del size=3210749 2024-04-17T04:45:08.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.084+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=PWAFXXON6U78 type=del 2024-04-17T04:45:08.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.088+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~56FR44T9FTCH.0 size=1048576 2024-04-17T04:45:08.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~56FR44T9FTCH.1 size=1048576 2024-04-17T04:45:08.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.092+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~56FR44T9FTCH.2 size=357704 2024-04-17T04:45:08.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.092+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=56FR44T9FTCH tag=E4ZPLKTZWTGK type=write size=3503432 2024-04-17T04:45:08.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.100+0000 7fd8ef2218c0 1 < suggested update operation key=3OFQH4KVYN18 tag=I6O2Z5DSM5AI type=del size=1693517 2024-04-17T04:45:08.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.104+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=PZRO1A47BHLK type=del size=2957666 2024-04-17T04:45:08.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~3MEP1BI6CWJ3.0 size=1048576 2024-04-17T04:45:08.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~3MEP1BI6CWJ3.1 size=337715 2024-04-17T04:45:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.112+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=3MEP1BI6CWJ3 tag=YBHUMZNLMH4X type=write size=2434867 2024-04-17T04:45:08.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.116+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=S45V9VGU3VJJ type=del size=2137464 2024-04-17T04:45:08.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~RN781G77BO8S.0 size=384258 2024-04-17T04:45:08.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.124+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=RN781G77BO8S tag=HYOI1FI4BCHY type=write size=1432834 2024-04-17T04:45:08.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~ROWL4WA6Y4AL.0 size=1048576 2024-04-17T04:45:08.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~ROWL4WA6Y4AL.1 size=1048576 2024-04-17T04:45:08.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.136+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~ROWL4WA6Y4AL.2 size=50522 2024-04-17T04:45:08.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.136+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=ROWL4WA6Y4AL tag=MZBC3O7WUJOO type=write size=3196250 2024-04-17T04:45:08.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.140+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=HU1S037Y8FTW type=del 2024-04-17T04:45:08.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.140+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=2GUQ0OQ35QDD type=del size=659340 2024-04-17T04:45:08.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.144+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=ODJLU8MU4QSZ type=del 2024-04-17T04:45:08.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.148+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=YF0612448O4R type=del size=19121 2024-04-17T04:45:08.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~J0W25EGWBEMO.0 size=1048576 2024-04-17T04:45:08.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~J0W25EGWBEMO.1 size=289930 2024-04-17T04:45:08.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.156+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=J0W25EGWBEMO tag=7ZCXJOF7GHKA type=write size=2387082 2024-04-17T04:45:08.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.160+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=4LUV6FQURQK7 type=del size=2384768 2024-04-17T04:45:08.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.164+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=8LBOF65THPRZ type=del 2024-04-17T04:45:08.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.168+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~VTZNK7DHI9WV.0 size=367792 2024-04-17T04:45:08.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.168+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=VTZNK7DHI9WV tag=0ZT7IYUEIVWJ type=write size=1416368 2024-04-17T04:45:08.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.172+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=33Q6QR3XF9TN type=del 2024-04-17T04:45:08.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.172+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=QZWRMFPEHU59 type=del size=1632667 2024-04-17T04:45:08.172 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~OQZRTZKNIN7S.0 size=1048576 2024-04-17T04:45:08.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.180+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~OQZRTZKNIN7S.1 size=246165 2024-04-17T04:45:08.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.184+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=OQZRTZKNIN7S tag=BWF09D47DCNI type=write size=2343317 2024-04-17T04:45:08.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~0RHRBJEO9XKY.0 size=1048576 2024-04-17T04:45:08.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~0RHRBJEO9XKY.1 size=1048576 2024-04-17T04:45:08.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.196+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~0RHRBJEO9XKY.2 size=373344 2024-04-17T04:45:08.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.196+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=0RHRBJEO9XKY tag=541DWWJVTAUF type=write size=3519072 2024-04-17T04:45:08.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~QB73X6T6VKM6.0 size=5912 2024-04-17T04:45:08.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.204+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=QB73X6T6VKM6 tag=CWKZHY0LCD34 type=write size=1054488 2024-04-17T04:45:08.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~QS8WK4ZVKL7M.0 size=625 2024-04-17T04:45:08.203 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.208+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=QS8WK4ZVKL7M tag=H7MZOAIR48NW type=write size=1049201 2024-04-17T04:45:08.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.208+0000 7fd8ef2218c0 1 < canceled operation key=EWAW6EBW6W6U tag=6L07T9SYO82N type=write size=455921 2024-04-17T04:45:08.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~4W5RIT7RBB0H.0 size=1048576 2024-04-17T04:45:08.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.216+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~4W5RIT7RBB0H.1 size=422651 2024-04-17T04:45:08.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.216+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=4W5RIT7RBB0H tag=9UHG8O1DB4Q7 type=write size=2519803 2024-04-17T04:45:08.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.220+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=TQ1766LB2678 type=del 2024-04-17T04:45:08.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.224+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~GZQGBJ13TJ25.0 size=495146 2024-04-17T04:45:08.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.224+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=GZQGBJ13TJ25 tag=ARTHGI384A2O type=write size=1543722 2024-04-17T04:45:08.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~YKVGMFWLT8I1.0 size=1048576 2024-04-17T04:45:08.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~YKVGMFWLT8I1.1 size=1048576 2024-04-17T04:45:08.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~YKVGMFWLT8I1.2 size=1020962 2024-04-17T04:45:08.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.240+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=YKVGMFWLT8I1 tag=MV31Z2W1JPD1 type=write size=4166690 2024-04-17T04:45:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~7BM2W069SGOA.0 size=524853 2024-04-17T04:45:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.244+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=7BM2W069SGOA tag=GEGXP1ZV3MX5 type=write size=1573429 2024-04-17T04:45:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.248+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~CF61NW3VK8UL.0 size=959096 2024-04-17T04:45:08.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.252+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=CF61NW3VK8UL tag=QURDWQSK8Z0D type=write size=2007672 2024-04-17T04:45:08.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.256+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=P9Z4N4ASXZHC type=del size=2469017 2024-04-17T04:45:08.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.260+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=4Y2VUPJEKTO0 type=del 2024-04-17T04:45:08.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.264+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=9FSV4S9QJ2KE type=del size=1810047 2024-04-17T04:45:08.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~56FR44T9FTCH.0 size=1048576 2024-04-17T04:45:08.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~56FR44T9FTCH.1 size=1048576 2024-04-17T04:45:08.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.272+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~56FR44T9FTCH.2 size=357704 2024-04-17T04:45:08.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.276+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=56FR44T9FTCH tag=E4ZPLKTZWTGK type=write size=3503432 2024-04-17T04:45:08.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~G0AGIAWKV6LD.0 size=1048576 2024-04-17T04:45:08.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~G0AGIAWKV6LD.1 size=97594 2024-04-17T04:45:08.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.284+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=G0AGIAWKV6LD tag=I69Y5U6RUN55 type=write size=2194746 2024-04-17T04:45:08.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.288+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=PZRO1A47BHLK type=del 2024-04-17T04:45:08.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.288+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=IQ1FPVDEETBW type=del size=1313298 2024-04-17T04:45:08.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.296+0000 7fd8ef2218c0 1 < suggested remove operation key=WCIJICR0FT2C tag=S45V9VGU3VJJ type=del 2024-04-17T04:45:08.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.296+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=J7CZ3KTM7D3M type=write size=115422 2024-04-17T04:45:08.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.300+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~ROWL4WA6Y4AL.0 size=1048576 2024-04-17T04:45:08.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~ROWL4WA6Y4AL.1 size=1048576 2024-04-17T04:45:08.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~ROWL4WA6Y4AL.2 size=50522 2024-04-17T04:45:08.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.308+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=ROWL4WA6Y4AL tag=MZBC3O7WUJOO type=write size=3196250 2024-04-17T04:45:08.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.312+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=AHUJP23Y9073 type=write size=743583 2024-04-17T04:45:08.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.316+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=2GUQ0OQ35QDD type=del 2024-04-17T04:45:08.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.320+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=G9DWCUTYF12G type=del size=3749182 2024-04-17T04:45:08.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.320+0000 7fd8ef2218c0 1 < canceled operation key=5K50O96P7OGD tag=YF0612448O4R type=del size=19121 2024-04-17T04:45:08.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.324+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=A3HZYEV66IUB type=del size=574915 2024-04-17T04:45:08.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.332+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=4LUV6FQURQK7 type=del 2024-04-17T04:45:08.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.336+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=NQXZ5YJR90ED type=del size=166665 2024-04-17T04:45:08.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.340+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~VTZNK7DHI9WV.0 size=367792 2024-04-17T04:45:08.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.348+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=VTZNK7DHI9WV tag=0ZT7IYUEIVWJ type=write size=1416368 2024-04-17T04:45:08.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~N0X1ZSCIIY89.0 size=1048576 2024-04-17T04:45:08.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~N0X1ZSCIIY89.1 size=43914 2024-04-17T04:45:08.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.356+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=N0X1ZSCIIY89 tag=ICT9CKE890CI type=write size=2141066 2024-04-17T04:45:08.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.356+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=QZWRMFPEHU59 type=del 2024-04-17T04:45:08.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.360+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=OLMJCS7M4RT5 type=del size=1501861 2024-04-17T04:45:08.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.364+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~0RHRBJEO9XKY.0 size=1048576 2024-04-17T04:45:08.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.364+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~0RHRBJEO9XKY.1 size=1048576 2024-04-17T04:45:08.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.364+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6WSZ5UL0V1Y1.2~0RHRBJEO9XKY.2 size=373344 2024-04-17T04:45:08.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.368+0000 7fd8ef2218c0 1 < canceled multipart upload key=6WSZ5UL0V1Y1 upload=0RHRBJEO9XKY tag=541DWWJVTAUF type=write size=3519072 2024-04-17T04:45:08.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.372+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=0KMI0E2YTI8D type=write size=321641 2024-04-17T04:45:08.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.372+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~QS8WK4ZVKL7M.0 size=625 2024-04-17T04:45:08.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.376+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=QS8WK4ZVKL7M tag=H7MZOAIR48NW type=write size=1049201 2024-04-17T04:45:08.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.380+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~2CK6GIGQ5ETB.0 size=1048576 2024-04-17T04:45:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~2CK6GIGQ5ETB.1 size=653469 2024-04-17T04:45:08.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.384+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=2CK6GIGQ5ETB tag=APO09SPW0ARS type=write size=2750621 2024-04-17T04:45:08.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~4W5RIT7RBB0H.0 size=1048576 2024-04-17T04:45:08.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.388+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~4W5RIT7RBB0H.1 size=422651 2024-04-17T04:45:08.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.392+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=4W5RIT7RBB0H tag=9UHG8O1DB4Q7 type=write size=2519803 2024-04-17T04:45:08.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.396+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=3UVAUL7SC15J type=del size=827478 2024-04-17T04:45:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.400+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~GZQGBJ13TJ25.0 size=495146 2024-04-17T04:45:08.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.404+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=GZQGBJ13TJ25 tag=ARTHGI384A2O type=write size=1543722 2024-04-17T04:45:08.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.404+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=K3GU5O22EJDS type=del size=995286 2024-04-17T04:45:08.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.408+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~7BM2W069SGOA.0 size=524853 2024-04-17T04:45:08.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.412+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=7BM2W069SGOA tag=GEGXP1ZV3MX5 type=write size=1573429 2024-04-17T04:45:08.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.416+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~26GGGYHNLTCF.0 size=169157 2024-04-17T04:45:08.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.416+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=26GGGYHNLTCF tag=30XSOU6N2LCS type=write size=1217733 2024-04-17T04:45:08.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.420+0000 7fd8ef2218c0 1 < canceled operation key=MS3T476JOEON tag=P9Z4N4ASXZHC type=del size=2469017 2024-04-17T04:45:08.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.420+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=1RLSYMLE50EK type=del size=1661168 2024-04-17T04:45:08.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.424+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=9FSV4S9QJ2KE type=del 2024-04-17T04:45:08.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.428+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~JGE34PIPO0YB.0 size=1048576 2024-04-17T04:45:08.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~JGE34PIPO0YB.1 size=918637 2024-04-17T04:45:08.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.432+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=JGE34PIPO0YB tag=F476MQZPO5AN type=write size=3015789 2024-04-17T04:45:08.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.436+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~G0AGIAWKV6LD.0 size=1048576 2024-04-17T04:45:08.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.436+0000 7fd8ef2218c0 1 < canceled part key=_multipart_GX9X0D45GC2I.2~G0AGIAWKV6LD.1 size=97594 2024-04-17T04:45:08.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.440+0000 7fd8ef2218c0 1 < canceled multipart upload key=GX9X0D45GC2I upload=G0AGIAWKV6LD tag=I69Y5U6RUN55 type=write size=2194746 2024-04-17T04:45:08.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~S71HON021WFQ.0 size=1048576 2024-04-17T04:45:08.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~S71HON021WFQ.1 size=75564 2024-04-17T04:45:08.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.448+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=S71HON021WFQ tag=EY822M1872HW type=write size=2172716 2024-04-17T04:45:08.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.452+0000 7fd8ef2218c0 1 < suggested remove operation key=OZ1IST8DOGXW tag=IQ1FPVDEETBW type=del 2024-04-17T04:45:08.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.456+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~YV59272R21BK.0 size=221346 2024-04-17T04:45:08.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.456+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=YV59272R21BK tag=SN7QYMRWMQZA type=write size=1269922 2024-04-17T04:45:08.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.460+0000 7fd8ef2218c0 1 < completed write operation key=MS3T476JOEON tag=J7CZ3KTM7D3M type=write size=115422 2024-04-17T04:45:08.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.464+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=WR6RZ81SNQCJ type=del size=3270753 2024-04-17T04:45:08.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.468+0000 7fd8ef2218c0 1 < completed write operation key=R3C8PAVPVGBY tag=AHUJP23Y9073 type=write size=743583 2024-04-17T04:45:08.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.468+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=R7710U5AG8R0 type=write size=623132 2024-04-17T04:45:08.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.472+0000 7fd8ef2218c0 1 < canceled operation key=M3U8L7M8JKG9 tag=G9DWCUTYF12G type=del size=3749182 2024-04-17T04:45:08.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.476+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~5BOY8AFPN096.0 size=974466 2024-04-17T04:45:08.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.476+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=5BOY8AFPN096 tag=KOVS3VJ7UERE type=write size=2023042 2024-04-17T04:45:08.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.480+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=A3HZYEV66IUB type=del 2024-04-17T04:45:08.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.484+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~NKPGAUBPFNSB.0 size=1048576 2024-04-17T04:45:08.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~NKPGAUBPFNSB.1 size=838675 2024-04-17T04:45:08.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.488+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=NKPGAUBPFNSB tag=H00ZIJL0C9VV type=write size=2935827 2024-04-17T04:45:08.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.492+0000 7fd8ef2218c0 1 < suggested update operation key=UNOVYXDS5JEB tag=NQXZ5YJR90ED type=del size=166665 2024-04-17T04:45:08.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.496+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~RE0N1IW9QXH4.0 size=1048576 2024-04-17T04:45:08.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~RE0N1IW9QXH4.1 size=960839 2024-04-17T04:45:08.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.500+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=RE0N1IW9QXH4 tag=PFMCQ5WX34WB type=write size=3057991 2024-04-17T04:45:08.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~N0X1ZSCIIY89.0 size=1048576 2024-04-17T04:45:08.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~N0X1ZSCIIY89.1 size=43914 2024-04-17T04:45:08.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.508+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=N0X1ZSCIIY89 tag=ICT9CKE890CI type=write size=2141066 2024-04-17T04:45:08.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.512+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=HNMI18GJ1ZCS type=del size=113735 2024-04-17T04:45:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.516+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=OLMJCS7M4RT5 type=del 2024-04-17T04:45:08.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.520+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~PYD5CYDBYA9R.0 size=1048576 2024-04-17T04:45:08.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.524+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~PYD5CYDBYA9R.1 size=1048576 2024-04-17T04:45:08.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.528+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~PYD5CYDBYA9R.2 size=927777 2024-04-17T04:45:08.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.528+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=PYD5CYDBYA9R tag=IDCR1DQH3PSR type=write size=4073505 2024-04-17T04:45:08.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.528+0000 7fd8ef2218c0 1 < canceled operation key=1ZTJP8DRITEK tag=0KMI0E2YTI8D type=write size=321641 2024-04-17T04:45:08.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.532+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=TIXQPV8YSNW9 type=del size=1168777 2024-04-17T04:45:08.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.536+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~2CK6GIGQ5ETB.0 size=1048576 2024-04-17T04:45:08.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~2CK6GIGQ5ETB.1 size=653469 2024-04-17T04:45:08.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.544+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=2CK6GIGQ5ETB tag=APO09SPW0ARS type=write size=2750621 2024-04-17T04:45:08.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.548+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=7XVLES6IV5AA type=del size=2940664 2024-04-17T04:45:08.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.548+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=3UVAUL7SC15J type=del 2024-04-17T04:45:08.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.552+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=J7UPPBAPZSDC type=write size=392236 2024-04-17T04:45:08.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.556+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=K3GU5O22EJDS type=del 2024-04-17T04:45:08.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.560+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=PWMXHC7YB3DS type=del size=2535628 2024-04-17T04:45:08.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~26GGGYHNLTCF.0 size=169157 2024-04-17T04:45:08.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.568+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=26GGGYHNLTCF tag=30XSOU6N2LCS type=write size=1217733 2024-04-17T04:45:08.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.568+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=Q7G265U1LGVO type=del size=4036317 2024-04-17T04:45:08.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.572+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=1RLSYMLE50EK type=del 2024-04-17T04:45:08.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.576+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=R9RL56G1JIQ5 type=write size=325033 2024-04-17T04:45:08.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.580+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~JGE34PIPO0YB.0 size=1048576 2024-04-17T04:45:08.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.584+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~JGE34PIPO0YB.1 size=918637 2024-04-17T04:45:08.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.588+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=JGE34PIPO0YB tag=F476MQZPO5AN type=write size=3015789 2024-04-17T04:45:08.589 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~V4LLPSMQPIYY.0 size=1048576 2024-04-17T04:45:08.591 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~V4LLPSMQPIYY.1 size=732752 2024-04-17T04:45:08.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=V4LLPSMQPIYY tag=1J2U6JA3SSIG type=write size=2829904 2024-04-17T04:45:08.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~S71HON021WFQ.0 size=1048576 2024-04-17T04:45:08.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~S71HON021WFQ.1 size=75564 2024-04-17T04:45:08.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.612+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=S71HON021WFQ tag=EY822M1872HW type=write size=2172716 2024-04-17T04:45:08.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.616+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=0PHDWFUTXGNB type=write size=254708 2024-04-17T04:45:08.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~YV59272R21BK.0 size=221346 2024-04-17T04:45:08.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.624+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=YV59272R21BK tag=SN7QYMRWMQZA type=write size=1269922 2024-04-17T04:45:08.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.628+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=5M9ZOCIGDGMC type=write size=842230 2024-04-17T04:45:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.632+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=WR6RZ81SNQCJ type=del 2024-04-17T04:45:08.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.636+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=3ZY7HAJ370EM type=write size=793614 2024-04-17T04:45:08.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.636+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=R7710U5AG8R0 type=write size=623132 2024-04-17T04:45:08.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~IF5QNKCJIPTU.0 size=1048576 2024-04-17T04:45:08.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~IF5QNKCJIPTU.1 size=1048576 2024-04-17T04:45:08.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.648+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~IF5QNKCJIPTU.2 size=641200 2024-04-17T04:45:08.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.648+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=IF5QNKCJIPTU tag=Q05OSE55CS7Q type=write size=3786928 2024-04-17T04:45:08.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.652+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~5BOY8AFPN096.0 size=974466 2024-04-17T04:45:08.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.656+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=5BOY8AFPN096 tag=KOVS3VJ7UERE type=write size=2023042 2024-04-17T04:45:08.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.660+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~ASE28I902GHL.0 size=1048576 2024-04-17T04:45:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.664+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~ASE28I902GHL.1 size=789691 2024-04-17T04:45:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.664+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=ASE28I902GHL tag=3IPOIBD4W9TY type=write size=2886843 2024-04-17T04:45:08.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~NKPGAUBPFNSB.0 size=1048576 2024-04-17T04:45:08.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.668+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~NKPGAUBPFNSB.1 size=838675 2024-04-17T04:45:08.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.676+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=NKPGAUBPFNSB tag=H00ZIJL0C9VV type=write size=2935827 2024-04-17T04:45:08.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.676+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=2PQB8GEXPRWP type=write size=656422 2024-04-17T04:45:08.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~RE0N1IW9QXH4.0 size=1048576 2024-04-17T04:45:08.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.684+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~RE0N1IW9QXH4.1 size=960839 2024-04-17T04:45:08.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.688+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=RE0N1IW9QXH4 tag=PFMCQ5WX34WB type=write size=3057991 2024-04-17T04:45:08.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.696+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~S7WHN1KAUYRL.0 size=1048576 2024-04-17T04:45:08.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.696+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~S7WHN1KAUYRL.1 size=907225 2024-04-17T04:45:08.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.696+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=S7WHN1KAUYRL tag=9DWBLWHSZWIS type=write size=3004377 2024-04-17T04:45:08.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.700+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=HNMI18GJ1ZCS type=del 2024-04-17T04:45:08.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.704+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=UFL8WQMYJCGS type=del size=2950745 2024-04-17T04:45:08.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~PYD5CYDBYA9R.0 size=1048576 2024-04-17T04:45:08.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.712+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~PYD5CYDBYA9R.1 size=1048576 2024-04-17T04:45:08.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.712+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~PYD5CYDBYA9R.2 size=927777 2024-04-17T04:45:08.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.720+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=PYD5CYDBYA9R tag=IDCR1DQH3PSR type=write size=4073505 2024-04-17T04:45:08.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.720+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=66B6OQJOP719 type=del size=876200 2024-04-17T04:45:08.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.724+0000 7fd8ef2218c0 1 < suggested update operation key=UYCKMXE6LBUH tag=TIXQPV8YSNW9 type=del size=1168777 2024-04-17T04:45:08.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.728+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=DIPAZRQ7S1VK type=del size=1424248 2024-04-17T04:45:08.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.732+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=7XVLES6IV5AA type=del 2024-04-17T04:45:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.732+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=EGG6BD147QOG type=del size=392244 2024-04-17T04:45:08.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.736+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=J7UPPBAPZSDC type=write size=392236 2024-04-17T04:45:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.740+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=P713660N9W9V type=del size=3956738 2024-04-17T04:45:08.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.744+0000 7fd8ef2218c0 1 < canceled operation key=5K50O96P7OGD tag=PWMXHC7YB3DS type=del size=2535628 2024-04-17T04:45:08.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.744+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=TOUT2UFDM68F type=del size=2038292 2024-04-17T04:45:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.748+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=Q7G265U1LGVO type=del 2024-04-17T04:45:08.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.752+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=TOVXQNH4ALZ9 type=write size=387044 2024-04-17T04:45:08.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.756+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=R9RL56G1JIQ5 type=write size=325033 2024-04-17T04:45:08.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.760+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~BL0MUDDJJGUI.0 size=1048576 2024-04-17T04:45:08.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~BL0MUDDJJGUI.1 size=709760 2024-04-17T04:45:08.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.764+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=BL0MUDDJJGUI tag=LN2HLH5PE5W7 type=write size=2806912 2024-04-17T04:45:08.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.764+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~V4LLPSMQPIYY.0 size=1048576 2024-04-17T04:45:08.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.764+0000 7fd8ef2218c0 1 < canceled part key=_multipart_OZ1IST8DOGXW.2~V4LLPSMQPIYY.1 size=732752 2024-04-17T04:45:08.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.772+0000 7fd8ef2218c0 1 < canceled multipart upload key=OZ1IST8DOGXW upload=V4LLPSMQPIYY tag=1J2U6JA3SSIG type=write size=2829904 2024-04-17T04:45:08.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.776+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~2NSFJAH7CTXS.0 size=1048576 2024-04-17T04:45:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.780+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~2NSFJAH7CTXS.1 size=246521 2024-04-17T04:45:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.780+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=2NSFJAH7CTXS tag=R1OR13747CYK type=write size=2343673 2024-04-17T04:45:08.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.784+0000 7fd8ef2218c0 1 < completed write operation key=EWAW6EBW6W6U tag=0PHDWFUTXGNB type=write size=254708 2024-04-17T04:45:08.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~E7OCW0IQK5KP.0 size=675474 2024-04-17T04:45:08.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.788+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=E7OCW0IQK5KP tag=QNBL8DZPZMJI type=write size=1724050 2024-04-17T04:45:08.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.792+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=5M9ZOCIGDGMC type=write size=842230 2024-04-17T04:45:08.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.796+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=EFKUCB7L7WF0 type=write size=880415 2024-04-17T04:45:08.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.796+0000 7fd8ef2218c0 1 < canceled operation key=R3C8PAVPVGBY tag=3ZY7HAJ370EM type=write size=793614 2024-04-17T04:45:08.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.800+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=ZE0DTMJS8630 type=write size=124621 2024-04-17T04:45:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.804+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~IF5QNKCJIPTU.0 size=1048576 2024-04-17T04:45:08.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~IF5QNKCJIPTU.1 size=1048576 2024-04-17T04:45:08.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.808+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~IF5QNKCJIPTU.2 size=641200 2024-04-17T04:45:08.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.812+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=IF5QNKCJIPTU tag=Q05OSE55CS7Q type=write size=3786928 2024-04-17T04:45:08.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.816+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=HNWNPKK9PTHI type=del size=1758915 2024-04-17T04:45:08.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.820+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~ASE28I902GHL.0 size=1048576 2024-04-17T04:45:08.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.820+0000 7fd8ef2218c0 1 < canceled part key=_multipart_R3C8PAVPVGBY.2~ASE28I902GHL.1 size=789691 2024-04-17T04:45:08.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.824+0000 7fd8ef2218c0 1 < canceled multipart upload key=R3C8PAVPVGBY upload=ASE28I902GHL tag=3IPOIBD4W9TY type=write size=2886843 2024-04-17T04:45:08.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.828+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=R9M4M4J4DQXY type=del size=1614537 2024-04-17T04:45:08.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.832+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=2PQB8GEXPRWP type=write size=656422 2024-04-17T04:45:08.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~ST95PNJFN8AC.0 size=1048576 2024-04-17T04:45:08.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~ST95PNJFN8AC.1 size=919459 2024-04-17T04:45:08.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.836+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=ST95PNJFN8AC tag=X3M7B06OHYZL type=write size=3016611 2024-04-17T04:45:08.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.840+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~S7WHN1KAUYRL.0 size=1048576 2024-04-17T04:45:08.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.844+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~S7WHN1KAUYRL.1 size=907225 2024-04-17T04:45:08.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.848+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=S7WHN1KAUYRL tag=9DWBLWHSZWIS type=write size=3004377 2024-04-17T04:45:08.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.852+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=R43UXR36E654 type=del size=834096 2024-04-17T04:45:08.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.856+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=UFL8WQMYJCGS type=del 2024-04-17T04:45:08.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~SVUUEDHRN774.0 size=1048576 2024-04-17T04:45:08.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~SVUUEDHRN774.1 size=767595 2024-04-17T04:45:08.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.864+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=SVUUEDHRN774 tag=DZ18C5RVLIDI type=write size=2864747 2024-04-17T04:45:08.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.868+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=66B6OQJOP719 type=del 2024-04-17T04:45:08.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.868+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=8EVG00S6SS7T type=del size=3303710 2024-04-17T04:45:08.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.872+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=DIPAZRQ7S1VK type=del 2024-04-17T04:45:08.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.876+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=K7GAMRQPUEF8 type=del size=1765733 2024-04-17T04:45:08.875 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.880+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=EGG6BD147QOG type=del 2024-04-17T04:45:08.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.884+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=E7Q6NWDVZ9TG type=del size=4034596 2024-04-17T04:45:08.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.884+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=P713660N9W9V type=del 2024-04-17T04:45:08.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.888+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=YX56FT4C4NCA type=del size=100631 2024-04-17T04:45:08.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.896+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=TOUT2UFDM68F type=del 2024-04-17T04:45:08.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~ARXL41A2GBSW.0 size=1048576 2024-04-17T04:45:08.897 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~ARXL41A2GBSW.1 size=1048576 2024-04-17T04:45:08.900 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~ARXL41A2GBSW.2 size=54515 2024-04-17T04:45:08.900 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.904+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=ARXL41A2GBSW tag=7MPDIFI36LN7 type=write size=3200243 2024-04-17T04:45:08.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.908+0000 7fd8ef2218c0 1 < completed write operation key=VUYM3JC57QRP tag=TOVXQNH4ALZ9 type=write size=387044 2024-04-17T04:45:08.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~0LMFFKYG9O4B.0 size=1048576 2024-04-17T04:45:08.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~0LMFFKYG9O4B.1 size=1048576 2024-04-17T04:45:08.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~0LMFFKYG9O4B.2 size=100664 2024-04-17T04:45:08.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.920+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=0LMFFKYG9O4B tag=932I4OI3PE0O type=write size=3246392 2024-04-17T04:45:08.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.924+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~BL0MUDDJJGUI.0 size=1048576 2024-04-17T04:45:08.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.924+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~BL0MUDDJJGUI.1 size=709760 2024-04-17T04:45:08.925 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.928+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=BL0MUDDJJGUI tag=LN2HLH5PE5W7 type=write size=2806912 2024-04-17T04:45:08.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~40QXSGNF118A.0 size=1048576 2024-04-17T04:45:08.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~40QXSGNF118A.1 size=657440 2024-04-17T04:45:08.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.936+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=40QXSGNF118A tag=CY84XU159K48 type=write size=2754592 2024-04-17T04:45:08.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~2NSFJAH7CTXS.0 size=1048576 2024-04-17T04:45:08.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~2NSFJAH7CTXS.1 size=246521 2024-04-17T04:45:08.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.948+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=2NSFJAH7CTXS tag=R1OR13747CYK type=write size=2343673 2024-04-17T04:45:08.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.948+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=ITVMJETUE3PU type=del size=1787787 2024-04-17T04:45:08.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~E7OCW0IQK5KP.0 size=675474 2024-04-17T04:45:08.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.956+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=E7OCW0IQK5KP tag=QNBL8DZPZMJI type=write size=1724050 2024-04-17T04:45:08.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.960+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=9F65EJ3S2S0S type=write size=307917 2024-04-17T04:45:08.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.960+0000 7fd8ef2218c0 1 < completed write operation key=509DZ9DBJWC5 tag=EFKUCB7L7WF0 type=write size=880415 2024-04-17T04:45:08.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~31QONDAMCZC2.0 size=1048576 2024-04-17T04:45:08.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~31QONDAMCZC2.1 size=532830 2024-04-17T04:45:08.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.968+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=31QONDAMCZC2 tag=K3H1NZ5TR14M type=write size=2629982 2024-04-17T04:45:08.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.972+0000 7fd8ef2218c0 1 < canceled operation key=N11Y8FPN9XES tag=ZE0DTMJS8630 type=write size=124621 2024-04-17T04:45:08.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.972+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=NX34QRH9I7F5 type=del size=2951541 2024-04-17T04:45:08.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.976+0000 7fd8ef2218c0 1 < canceled operation key=BCG05NIQFA53 tag=HNWNPKK9PTHI type=del size=1758915 2024-04-17T04:45:08.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.980+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~TXA23E5KE2SX.0 size=499211 2024-04-17T04:45:08.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.980+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=TXA23E5KE2SX tag=G4FLKMC8I7TP type=write size=1547787 2024-04-17T04:45:08.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.984+0000 7fd8ef2218c0 1 < canceled operation key=GX9X0D45GC2I tag=R9M4M4J4DQXY type=del size=1614537 2024-04-17T04:45:08.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.988+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=04ICWT7EYN0L type=del size=1760512 2024-04-17T04:45:08.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.988+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~ST95PNJFN8AC.0 size=1048576 2024-04-17T04:45:08.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~ST95PNJFN8AC.1 size=919459 2024-04-17T04:45:08.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.996+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=ST95PNJFN8AC tag=X3M7B06OHYZL type=write size=3016611 2024-04-17T04:45:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:08.996+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=E43BYCH8XMZG type=del size=2023429 2024-04-17T04:45:08.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.004+0000 7fd8ef2218c0 1 < suggested remove operation key=GX9X0D45GC2I tag=R43UXR36E654 type=del 2024-04-17T04:45:09.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~USHPBNAC09J6.0 size=1048576 2024-04-17T04:45:09.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~USHPBNAC09J6.1 size=850285 2024-04-17T04:45:09.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=USHPBNAC09J6 tag=S4MEPE99URP4 type=write size=2947437 2024-04-17T04:45:09.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~SVUUEDHRN774.0 size=1048576 2024-04-17T04:45:09.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.016+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~SVUUEDHRN774.1 size=767595 2024-04-17T04:45:09.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.020+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=SVUUEDHRN774 tag=DZ18C5RVLIDI type=write size=2864747 2024-04-17T04:45:09.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.020+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=I3L0SOA65O40 type=del size=773837 2024-04-17T04:45:09.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.024+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=8EVG00S6SS7T type=del 2024-04-17T04:45:09.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.024+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=ZL1HW1E8FNPM type=del size=3693633 2024-04-17T04:45:09.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.028+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=K7GAMRQPUEF8 type=del 2024-04-17T04:45:09.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.032+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=7FYN9ZEKKOO8 type=del size=3779411 2024-04-17T04:45:09.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.032+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=E7Q6NWDVZ9TG type=del 2024-04-17T04:45:09.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.036+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~XKZJAA9A2BXX.0 size=1048576 2024-04-17T04:45:09.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.040+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~XKZJAA9A2BXX.1 size=1048576 2024-04-17T04:45:09.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.040+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~XKZJAA9A2BXX.2 size=940189 2024-04-17T04:45:09.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.040+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=XKZJAA9A2BXX tag=Y3U630IPVH7S type=write size=4085917 2024-04-17T04:45:09.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.044+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=YX56FT4C4NCA type=del 2024-04-17T04:45:09.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.048+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~RA3K1OPQZVA5.0 size=1048576 2024-04-17T04:45:09.044 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.048+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~RA3K1OPQZVA5.1 size=1048576 2024-04-17T04:45:09.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.052+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~RA3K1OPQZVA5.2 size=884177 2024-04-17T04:45:09.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.052+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=RA3K1OPQZVA5 tag=F78WP2ILDGUM type=write size=4029905 2024-04-17T04:45:09.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.052+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~ARXL41A2GBSW.0 size=1048576 2024-04-17T04:45:09.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~ARXL41A2GBSW.1 size=1048576 2024-04-17T04:45:09.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~ARXL41A2GBSW.2 size=54515 2024-04-17T04:45:09.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.060+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=ARXL41A2GBSW tag=7MPDIFI36LN7 type=write size=3200243 2024-04-17T04:45:09.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.064+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=E5NUKNJAKQU4 type=del size=3442711 2024-04-17T04:45:09.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~0LMFFKYG9O4B.0 size=1048576 2024-04-17T04:45:09.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~0LMFFKYG9O4B.1 size=1048576 2024-04-17T04:45:09.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~0LMFFKYG9O4B.2 size=100664 2024-04-17T04:45:09.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.076+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=0LMFFKYG9O4B tag=932I4OI3PE0O type=write size=3246392 2024-04-17T04:45:09.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~LL9818VLMRD1.0 size=1048576 2024-04-17T04:45:09.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~LL9818VLMRD1.1 size=946126 2024-04-17T04:45:09.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.084+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=LL9818VLMRD1 tag=K58K09MF50ZQ type=write size=3043278 2024-04-17T04:45:09.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~40QXSGNF118A.0 size=1048576 2024-04-17T04:45:09.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~40QXSGNF118A.1 size=657440 2024-04-17T04:45:09.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.092+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=40QXSGNF118A tag=CY84XU159K48 type=write size=2754592 2024-04-17T04:45:09.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.096+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=QRTX7NRX0L70 type=del size=3138379 2024-04-17T04:45:09.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.100+0000 7fd8ef2218c0 1 < suggested remove operation key=5ZULD0PHLS4V tag=ITVMJETUE3PU type=del 2024-04-17T04:45:09.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.104+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=1NZKGN4OR36D type=del size=3748471 2024-04-17T04:45:09.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.108+0000 7fd8ef2218c0 1 < suggested update operation key=1JOWGWFR7R6P tag=9F65EJ3S2S0S type=write size=307917 2024-04-17T04:45:09.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~8RDD3NS906XJ.0 size=1048576 2024-04-17T04:45:09.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~8RDD3NS906XJ.1 size=789690 2024-04-17T04:45:09.111 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=8RDD3NS906XJ tag=P74JG3LNFYT2 type=write size=2886842 2024-04-17T04:45:09.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~31QONDAMCZC2.0 size=1048576 2024-04-17T04:45:09.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~31QONDAMCZC2.1 size=532830 2024-04-17T04:45:09.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.124+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=31QONDAMCZC2 tag=K3H1NZ5TR14M type=write size=2629982 2024-04-17T04:45:09.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5U7HK5TZ8YLL.0 size=1048576 2024-04-17T04:45:09.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5U7HK5TZ8YLL.1 size=1048576 2024-04-17T04:45:09.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.136+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~5U7HK5TZ8YLL.2 size=519195 2024-04-17T04:45:09.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.136+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=5U7HK5TZ8YLL tag=H6LT9I5JYA9H type=write size=3664923 2024-04-17T04:45:09.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.140+0000 7fd8ef2218c0 1 < canceled operation key=G03VLS09F3BT tag=NX34QRH9I7F5 type=del size=2951541 2024-04-17T04:45:09.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~QOPOQKWTICH1.0 size=474240 2024-04-17T04:45:09.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.144+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=QOPOQKWTICH1 tag=6EOV8VU24AEL type=write size=1522816 2024-04-17T04:45:09.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~TXA23E5KE2SX.0 size=499211 2024-04-17T04:45:09.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.152+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=TXA23E5KE2SX tag=G4FLKMC8I7TP type=write size=1547787 2024-04-17T04:45:09.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.156+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=535BLQ3QA58I type=write size=151131 2024-04-17T04:45:09.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.160+0000 7fd8ef2218c0 -1 < failed to complete operation key=5ZULD0PHLS4V tag=04ICWT7EYN0L type=del size=1760512: (22) Invalid argument 2024-04-17T04:45:09.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.160+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=ERMTOED51HFJ type=del size=960349 2024-04-17T04:45:09.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.164+0000 7fd8ef2218c0 -1 < failed to complete operation key=5ZULD0PHLS4V tag=E43BYCH8XMZG type=del size=2023429: (22) Invalid argument 2024-04-17T04:45:09.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.168+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=7A3LZZA0QUNJ type=del size=867774 2024-04-17T04:45:09.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.172+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~USHPBNAC09J6.0 size=1048576 2024-04-17T04:45:09.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.172+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~USHPBNAC09J6.1 size=850285 2024-04-17T04:45:09.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.176+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=USHPBNAC09J6 tag=S4MEPE99URP4 type=write size=2947437 2024-04-17T04:45:09.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.180+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=VQKDSEEXENGJ type=del size=3963101 2024-04-17T04:45:09.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.184+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=I3L0SOA65O40 type=del 2024-04-17T04:45:09.182 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.188+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=YSF8JVLPW6ZA type=del size=1572697 2024-04-17T04:45:09.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.188+0000 7fd8ef2218c0 -1 < failed to complete operation key=1JOWGWFR7R6P tag=ZL1HW1E8FNPM type=del size=3693633: (22) Invalid argument 2024-04-17T04:45:09.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.192+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=A0LU7TK1JRGU type=del size=1775547 2024-04-17T04:45:09.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.196+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=7FYN9ZEKKOO8 type=del 2024-04-17T04:45:09.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.200+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=B5KKOQ5KH6RB type=del size=639132 2024-04-17T04:45:09.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.200+0000 7fd8ef2218c0 1 < canceled part key=_multipart_A5ZUB3YK06J3.2~XKZJAA9A2BXX.0 size=1048576 2024-04-17T04:45:09.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~XKZJAA9A2BXX.1 size=1048576 2024-04-17T04:45:09.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.200+0000 7fd8ef2218c0 1 < canceled part key=_multipart_A5ZUB3YK06J3.2~XKZJAA9A2BXX.2 size=940189 2024-04-17T04:45:09.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.208+0000 7fd8ef2218c0 1 < canceled multipart upload key=A5ZUB3YK06J3 upload=XKZJAA9A2BXX tag=Y3U630IPVH7S type=write size=4085917 2024-04-17T04:45:09.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.208+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=2KRXYDZC4P53 type=del size=1241241 2024-04-17T04:45:09.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.208+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~RA3K1OPQZVA5.0 size=1048576 2024-04-17T04:45:09.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.212+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~RA3K1OPQZVA5.1 size=1048576 2024-04-17T04:45:09.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.216+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~RA3K1OPQZVA5.2 size=884177 2024-04-17T04:45:09.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.220+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=RA3K1OPQZVA5 tag=F78WP2ILDGUM type=write size=4029905 2024-04-17T04:45:09.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.224+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~ZYVYSE13Y80H.0 size=930728 2024-04-17T04:45:09.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.224+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=ZYVYSE13Y80H tag=FKYZ2ZE1419U type=write size=1979304 2024-04-17T04:45:09.223 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.228+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=E5NUKNJAKQU4 type=del 2024-04-17T04:45:09.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.232+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=J05OW51SE48C type=del size=2309494 2024-04-17T04:45:09.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~LL9818VLMRD1.0 size=1048576 2024-04-17T04:45:09.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.236+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~LL9818VLMRD1.1 size=946126 2024-04-17T04:45:09.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.240+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=LL9818VLMRD1 tag=K58K09MF50ZQ type=write size=3043278 2024-04-17T04:45:09.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.248+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~Q57062KNSXPC.0 size=1048576 2024-04-17T04:45:09.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.248+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~Q57062KNSXPC.1 size=990325 2024-04-17T04:45:09.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.248+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=Q57062KNSXPC tag=SC6D2OT26HAS type=write size=3087477 2024-04-17T04:45:09.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.252+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=QRTX7NRX0L70 type=del 2024-04-17T04:45:09.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.256+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=ANXJUTAAEZZ8 type=del size=354163 2024-04-17T04:45:09.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.260+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=1NZKGN4OR36D type=del 2024-04-17T04:45:09.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.264+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GCEHN92ZQW1F.0 size=1048576 2024-04-17T04:45:09.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.268+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GCEHN92ZQW1F.1 size=1048576 2024-04-17T04:45:09.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.272+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~GCEHN92ZQW1F.2 size=678154 2024-04-17T04:45:09.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.272+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=GCEHN92ZQW1F tag=UYCP2V29TC18 type=write size=3823882 2024-04-17T04:45:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~8RDD3NS906XJ.0 size=1048576 2024-04-17T04:45:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~8RDD3NS906XJ.1 size=789690 2024-04-17T04:45:09.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.280+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=8RDD3NS906XJ tag=P74JG3LNFYT2 type=write size=2886842 2024-04-17T04:45:09.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.288+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=K9K4ZU93ASAR type=write size=138116 2024-04-17T04:45:09.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.288+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5U7HK5TZ8YLL.0 size=1048576 2024-04-17T04:45:09.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.288+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~5U7HK5TZ8YLL.1 size=1048576 2024-04-17T04:45:09.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.292+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~5U7HK5TZ8YLL.2 size=519195 2024-04-17T04:45:09.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.296+0000 7fd8ef2218c0 1 < canceled multipart upload key=6219DPY72LWC upload=5U7HK5TZ8YLL tag=H6LT9I5JYA9H type=write size=3664923 2024-04-17T04:45:09.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.304+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~HGCMSNVOQMF0.0 size=1048576 2024-04-17T04:45:09.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~HGCMSNVOQMF0.1 size=1048576 2024-04-17T04:45:09.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~HGCMSNVOQMF0.2 size=45217 2024-04-17T04:45:09.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.308+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=HGCMSNVOQMF0 tag=GAT11MYA91BT type=write size=3190945 2024-04-17T04:45:09.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~QOPOQKWTICH1.0 size=474240 2024-04-17T04:45:09.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.320+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=QOPOQKWTICH1 tag=6EOV8VU24AEL type=write size=1522816 2024-04-17T04:45:09.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~YIKSFC970F71.0 size=1048576 2024-04-17T04:45:09.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~YIKSFC970F71.1 size=1048576 2024-04-17T04:45:09.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~YIKSFC970F71.2 size=311752 2024-04-17T04:45:09.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.328+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=YIKSFC970F71 tag=PMP56FRIFM0G type=write size=3457480 2024-04-17T04:45:09.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.332+0000 7fd8ef2218c0 1 < completed write operation key=6WSZ5UL0V1Y1 tag=535BLQ3QA58I type=write size=151131 2024-04-17T04:45:09.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~SHC8D5G2JO4F.0 size=1048576 2024-04-17T04:45:09.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~SHC8D5G2JO4F.1 size=522785 2024-04-17T04:45:09.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.340+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=SHC8D5G2JO4F tag=CT35PRIV7ULY type=write size=2619937 2024-04-17T04:45:09.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.344+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=ERMTOED51HFJ type=del 2024-04-17T04:45:09.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~HAVLJ6VAJSRS.0 size=1048576 2024-04-17T04:45:09.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~HAVLJ6VAJSRS.1 size=509734 2024-04-17T04:45:09.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.352+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=HAVLJ6VAJSRS tag=6TP8663HSGI8 type=write size=2606886 2024-04-17T04:45:09.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.356+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=7A3LZZA0QUNJ type=del 2024-04-17T04:45:09.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.360+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=9I0WNI2LIZQE type=del size=1025947 2024-04-17T04:45:09.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.364+0000 7fd8ef2218c0 1 < canceled operation key=7BV3V2XMLSPH tag=VQKDSEEXENGJ type=del size=3963101 2024-04-17T04:45:09.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.368+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~PS8UA231EU6L.0 size=1048576 2024-04-17T04:45:09.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~PS8UA231EU6L.1 size=1048576 2024-04-17T04:45:09.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.376+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~PS8UA231EU6L.2 size=614151 2024-04-17T04:45:09.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.376+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=PS8UA231EU6L tag=7OI5IX5MOG1D type=write size=3759879 2024-04-17T04:45:09.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.380+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=YSF8JVLPW6ZA type=del 2024-04-17T04:45:09.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~8WMBPT098SN8.0 size=430712 2024-04-17T04:45:09.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.384+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=8WMBPT098SN8 tag=3GC09ATVNMQ9 type=write size=1479288 2024-04-17T04:45:09.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.388+0000 7fd8ef2218c0 1 < canceled operation key=A5ZUB3YK06J3 tag=A0LU7TK1JRGU type=del size=1775547 2024-04-17T04:45:09.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.388+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=QSKIQVOK5VF1 type=del size=2788203 2024-04-17T04:45:09.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.392+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=B5KKOQ5KH6RB type=del 2024-04-17T04:45:09.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.396+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=VWHZNTEP66VS type=del size=249431 2024-04-17T04:45:09.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.400+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=2KRXYDZC4P53 type=del 2024-04-17T04:45:09.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~2DIACE522OPA.0 size=1048576 2024-04-17T04:45:09.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~2DIACE522OPA.1 size=525000 2024-04-17T04:45:09.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.408+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=2DIACE522OPA tag=W6V4A32H939D type=write size=2622152 2024-04-17T04:45:09.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~ZYVYSE13Y80H.0 size=930728 2024-04-17T04:45:09.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.420+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=ZYVYSE13Y80H tag=FKYZ2ZE1419U type=write size=1979304 2024-04-17T04:45:09.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.424+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~5TW52QR8F4HU.0 size=1048576 2024-04-17T04:45:09.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.432+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~5TW52QR8F4HU.1 size=1048576 2024-04-17T04:45:09.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~5TW52QR8F4HU.2 size=700374 2024-04-17T04:45:09.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.436+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=5TW52QR8F4HU tag=1JLJBASU4AJ4 type=write size=3846102 2024-04-17T04:45:09.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.436+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=J05OW51SE48C type=del 2024-04-17T04:45:09.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.440+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=UJ81VE1MJ4H0 type=del size=4073089 2024-04-17T04:45:09.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.444+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~Q57062KNSXPC.0 size=1048576 2024-04-17T04:45:09.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.448+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~Q57062KNSXPC.1 size=990325 2024-04-17T04:45:09.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.452+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=Q57062KNSXPC tag=SC6D2OT26HAS type=write size=3087477 2024-04-17T04:45:09.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.456+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=R202P2MWB0NJ type=del size=1173000 2024-04-17T04:45:09.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.464+0000 7fd8ef2218c0 1 < suggested update operation key=VUYM3JC57QRP tag=ANXJUTAAEZZ8 type=del size=354163 2024-04-17T04:45:09.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.468+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=PX73SY2CFAPJ type=del size=3200374 2024-04-17T04:45:09.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.468+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GCEHN92ZQW1F.0 size=1048576 2024-04-17T04:45:09.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GCEHN92ZQW1F.1 size=1048576 2024-04-17T04:45:09.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.476+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~GCEHN92ZQW1F.2 size=678154 2024-04-17T04:45:09.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.480+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=GCEHN92ZQW1F tag=UYCP2V29TC18 type=write size=3823882 2024-04-17T04:45:09.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.484+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=DN52880L2PO0 type=del size=1405165 2024-04-17T04:45:09.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.488+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=K9K4ZU93ASAR type=write size=138116 2024-04-17T04:45:09.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.492+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=KIMZK4YS623U type=del size=3010565 2024-04-17T04:45:09.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.492+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~HGCMSNVOQMF0.0 size=1048576 2024-04-17T04:45:09.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.492+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~HGCMSNVOQMF0.1 size=1048576 2024-04-17T04:45:09.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.496+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~HGCMSNVOQMF0.2 size=45217 2024-04-17T04:45:09.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.500+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=HGCMSNVOQMF0 tag=GAT11MYA91BT type=write size=3190945 2024-04-17T04:45:09.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~KRUZQ9JI2EXS.0 size=1048576 2024-04-17T04:45:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~KRUZQ9JI2EXS.1 size=1048576 2024-04-17T04:45:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~KRUZQ9JI2EXS.2 size=1048155 2024-04-17T04:45:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=KRUZQ9JI2EXS tag=0JVOFGAT811Y type=write size=4193883 2024-04-17T04:45:09.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~YIKSFC970F71.0 size=1048576 2024-04-17T04:45:09.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.520+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~YIKSFC970F71.1 size=1048576 2024-04-17T04:45:09.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.520+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~YIKSFC970F71.2 size=311752 2024-04-17T04:45:09.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.528+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=YIKSFC970F71 tag=PMP56FRIFM0G type=write size=3457480 2024-04-17T04:45:09.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.532+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=UCJ93JEZIJN5 type=del size=3609735 2024-04-17T04:45:09.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.532+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~SHC8D5G2JO4F.0 size=1048576 2024-04-17T04:45:09.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.536+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~SHC8D5G2JO4F.1 size=522785 2024-04-17T04:45:09.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.540+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=SHC8D5G2JO4F tag=CT35PRIV7ULY type=write size=2619937 2024-04-17T04:45:09.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.548+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~9INNJK5U3KI2.0 size=850318 2024-04-17T04:45:09.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.548+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=9INNJK5U3KI2 tag=LVJQM3O3S5PG type=write size=1898894 2024-04-17T04:45:09.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~HAVLJ6VAJSRS.0 size=1048576 2024-04-17T04:45:09.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~HAVLJ6VAJSRS.1 size=509734 2024-04-17T04:45:09.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.560+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=HAVLJ6VAJSRS tag=6TP8663HSGI8 type=write size=2606886 2024-04-17T04:45:09.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~E92PB1VW0QRQ.0 size=649431 2024-04-17T04:45:09.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.564+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=E92PB1VW0QRQ tag=DTZ23HHKPYCP type=write size=1698007 2024-04-17T04:45:09.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.568+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=9I0WNI2LIZQE type=del 2024-04-17T04:45:09.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.572+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=3TVH9GXKEP2I type=del size=1322469 2024-04-17T04:45:09.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~PS8UA231EU6L.0 size=1048576 2024-04-17T04:45:09.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.580+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~PS8UA231EU6L.1 size=1048576 2024-04-17T04:45:09.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.580+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~PS8UA231EU6L.2 size=614151 2024-04-17T04:45:09.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.588+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=PS8UA231EU6L tag=7OI5IX5MOG1D type=write size=3759879 2024-04-17T04:45:09.589 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~OCQTUCDRFMHO.0 size=1048576 2024-04-17T04:45:09.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~OCQTUCDRFMHO.1 size=327888 2024-04-17T04:45:09.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=OCQTUCDRFMHO tag=X63A70CGV6LJ type=write size=2425040 2024-04-17T04:45:09.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~8WMBPT098SN8.0 size=430712 2024-04-17T04:45:09.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.604+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=8WMBPT098SN8 tag=3GC09ATVNMQ9 type=write size=1479288 2024-04-17T04:45:09.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~8SAAS6BUPCV7.0 size=672288 2024-04-17T04:45:09.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.612+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=8SAAS6BUPCV7 tag=V9ZJZ2BDHL6R type=write size=1720864 2024-04-17T04:45:09.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.616+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=QSKIQVOK5VF1 type=del 2024-04-17T04:45:09.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~WKM8T1LHT3EH.0 size=1048576 2024-04-17T04:45:09.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.624+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~WKM8T1LHT3EH.1 size=801254 2024-04-17T04:45:09.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.624+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=WKM8T1LHT3EH tag=A44QFP78069P type=write size=2898406 2024-04-17T04:45:09.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.628+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=VWHZNTEP66VS type=del 2024-04-17T04:45:09.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.632+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=UZA0WLTBZZE8 type=del size=164190 2024-04-17T04:45:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~2DIACE522OPA.0 size=1048576 2024-04-17T04:45:09.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.636+0000 7fd8ef2218c0 1 < canceled part key=_multipart_MS3T476JOEON.2~2DIACE522OPA.1 size=525000 2024-04-17T04:45:09.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.640+0000 7fd8ef2218c0 1 < canceled multipart upload key=MS3T476JOEON upload=2DIACE522OPA tag=W6V4A32H939D type=write size=2622152 2024-04-17T04:45:09.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.648+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~B4QGM84U357Y.0 size=1048576 2024-04-17T04:45:09.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.652+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~B4QGM84U357Y.1 size=139304 2024-04-17T04:45:09.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.652+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=B4QGM84U357Y tag=MHK5JXYY6TP1 type=write size=2236456 2024-04-17T04:45:09.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.652+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~5TW52QR8F4HU.0 size=1048576 2024-04-17T04:45:09.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.656+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~5TW52QR8F4HU.1 size=1048576 2024-04-17T04:45:09.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.660+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~5TW52QR8F4HU.2 size=700374 2024-04-17T04:45:09.660 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.664+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=5TW52QR8F4HU tag=1JLJBASU4AJ4 type=write size=3846102 2024-04-17T04:45:09.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.668+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=AGAVWI360A41 type=del size=3613912 2024-04-17T04:45:09.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.672+0000 7fd8ef2218c0 1 < canceled operation key=N11Y8FPN9XES tag=UJ81VE1MJ4H0 type=del size=4073089 2024-04-17T04:45:09.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.676+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=3EP7FU3ZF5G3 type=del size=3566962 2024-04-17T04:45:09.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.680+0000 7fd8ef2218c0 1 < suggested remove operation key=JVWRJBMFIRLL tag=R202P2MWB0NJ type=del 2024-04-17T04:45:09.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.684+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=3CIE94TEWCM0 type=del size=2491033 2024-04-17T04:45:09.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.688+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=PX73SY2CFAPJ type=del 2024-04-17T04:45:09.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.696+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~LDZQQ9V9VZZ2.0 size=449019 2024-04-17T04:45:09.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.696+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=LDZQQ9V9VZZ2 tag=RD0HRY1SJ92U type=write size=1497595 2024-04-17T04:45:09.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.700+0000 7fd8ef2218c0 1 < suggested remove operation key=N11Y8FPN9XES tag=DN52880L2PO0 type=del 2024-04-17T04:45:09.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.708+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~MO8Y8ESKU1UR.0 size=1048576 2024-04-17T04:45:09.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~MO8Y8ESKU1UR.1 size=308821 2024-04-17T04:45:09.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.712+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=MO8Y8ESKU1UR tag=3YNJK9OV8IQA type=write size=2405973 2024-04-17T04:45:09.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.716+0000 7fd8ef2218c0 1 < suggested update operation key=509DZ9DBJWC5 tag=KIMZK4YS623U type=del size=3010565 2024-04-17T04:45:09.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.720+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=Z2U16GIV65FY type=write size=572124 2024-04-17T04:45:09.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.720+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~KRUZQ9JI2EXS.0 size=1048576 2024-04-17T04:45:09.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.724+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~KRUZQ9JI2EXS.1 size=1048576 2024-04-17T04:45:09.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.728+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~KRUZQ9JI2EXS.2 size=1048155 2024-04-17T04:45:09.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.732+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=KRUZQ9JI2EXS tag=0JVOFGAT811Y type=write size=4193883 2024-04-17T04:45:09.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~KYJZ9OT1O7ZR.0 size=59132 2024-04-17T04:45:09.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=KYJZ9OT1O7ZR tag=1YVFXJ3TFYKN type=write size=1107708 2024-04-17T04:45:09.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.748+0000 7fd8ef2218c0 1 < canceled operation key=AE4CYL334ONO tag=UCJ93JEZIJN5 type=del size=3609735 2024-04-17T04:45:09.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.752+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=YA6ZZV2JLBA8 type=del size=3381880 2024-04-17T04:45:09.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.752+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~9INNJK5U3KI2.0 size=850318 2024-04-17T04:45:09.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.756+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=9INNJK5U3KI2 tag=LVJQM3O3S5PG type=write size=1898894 2024-04-17T04:45:09.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.760+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=DIS37P6IGKTI type=del size=3566275 2024-04-17T04:45:09.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.764+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~E92PB1VW0QRQ.0 size=649431 2024-04-17T04:45:09.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.768+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=E92PB1VW0QRQ tag=DTZ23HHKPYCP type=write size=1698007 2024-04-17T04:45:09.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.772+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=HWE6XF4TTQ9C type=del size=2363117 2024-04-17T04:45:09.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.776+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=3TVH9GXKEP2I type=del 2024-04-17T04:45:09.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.780+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=2E7778A9AF7E type=del size=1741037 2024-04-17T04:45:09.779 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.784+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~OCQTUCDRFMHO.0 size=1048576 2024-04-17T04:45:09.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.784+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~OCQTUCDRFMHO.1 size=327888 2024-04-17T04:45:09.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.792+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=OCQTUCDRFMHO tag=X63A70CGV6LJ type=write size=2425040 2024-04-17T04:45:09.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.796+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~8ZTPUHH4JOK0.0 size=1048576 2024-04-17T04:45:09.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~8ZTPUHH4JOK0.1 size=15822 2024-04-17T04:45:09.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.800+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=8ZTPUHH4JOK0 tag=8VRLO594VJW8 type=write size=2112974 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.804+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~8SAAS6BUPCV7.0 size=672288 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.808+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=8SAAS6BUPCV7 tag=V9ZJZ2BDHL6R type=write size=1720864 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.812+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=11436VVQ05FX type=del size=945632 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.816+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~WKM8T1LHT3EH.0 size=1048576 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.820+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~WKM8T1LHT3EH.1 size=801254 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.824+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=WKM8T1LHT3EH tag=A44QFP78069P type=write size=2898406 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~RD892RZWVBLX.0 size=1048576 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~RD892RZWVBLX.1 size=572384 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=RD892RZWVBLX tag=V71I2EX8C5R3 type=write size=2669536 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.840+0000 7fd8ef2218c0 1 < suggested remove operation key=2IY8DB37FI9H tag=UZA0WLTBZZE8 type=del 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.848+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4LQDX38H3N93.0 size=1048576 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.848+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4LQDX38H3N93.1 size=1048576 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.852+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4LQDX38H3N93.2 size=122178 2024-04-17T04:45:09.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.852+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=4LQDX38H3N93 tag=EQVCN6Y8X32E type=write size=3267906 2024-04-17T04:45:09.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.856+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~B4QGM84U357Y.0 size=1048576 2024-04-17T04:45:09.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.860+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~B4QGM84U357Y.1 size=139304 2024-04-17T04:45:09.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.864+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=B4QGM84U357Y tag=MHK5JXYY6TP1 type=write size=2236456 2024-04-17T04:45:09.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~CL220YD3I1XF.0 size=347511 2024-04-17T04:45:09.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.872+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=CL220YD3I1XF tag=01K65REQE3G4 type=write size=1396087 2024-04-17T04:45:09.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.876+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=AGAVWI360A41 type=del 2024-04-17T04:45:09.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.880+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=1YUJCJOZI9ZY type=del size=4053168 2024-04-17T04:45:09.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.884+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=3EP7FU3ZF5G3 type=del 2024-04-17T04:45:09.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.884+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=BV68UX7U4FB1 type=del size=2537609 2024-04-17T04:45:09.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.888+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=3CIE94TEWCM0 type=del 2024-04-17T04:45:09.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~B0YEBW9V9VRA.0 size=1048576 2024-04-17T04:45:09.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~B0YEBW9V9VRA.1 size=1048576 2024-04-17T04:45:09.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~B0YEBW9V9VRA.2 size=742889 2024-04-17T04:45:09.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.900+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=B0YEBW9V9VRA tag=JIM424LPSG35 type=write size=3888617 2024-04-17T04:45:09.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.904+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~LDZQQ9V9VZZ2.0 size=449019 2024-04-17T04:45:09.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.908+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=LDZQQ9V9VZZ2 tag=RD0HRY1SJ92U type=write size=1497595 2024-04-17T04:45:09.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.916+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~08OI6LZUYNXJ.0 size=1048576 2024-04-17T04:45:09.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~08OI6LZUYNXJ.1 size=1048576 2024-04-17T04:45:09.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.920+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~08OI6LZUYNXJ.2 size=719914 2024-04-17T04:45:09.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.920+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=08OI6LZUYNXJ tag=FLCEN5G0IF6N type=write size=3865642 2024-04-17T04:45:09.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.924+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~MO8Y8ESKU1UR.0 size=1048576 2024-04-17T04:45:09.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.928+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~MO8Y8ESKU1UR.1 size=308821 2024-04-17T04:45:09.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.936+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=MO8Y8ESKU1UR tag=3YNJK9OV8IQA type=write size=2405973 2024-04-17T04:45:09.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.940+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=HJ18ZZ2MGQMV type=del size=1736078 2024-04-17T04:45:09.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.940+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=Z2U16GIV65FY type=write size=572124 2024-04-17T04:45:09.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.944+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=QLJ6OAHRNTL9 type=write size=965278 2024-04-17T04:45:09.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~KYJZ9OT1O7ZR.0 size=59132 2024-04-17T04:45:09.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.956+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=KYJZ9OT1O7ZR tag=1YVFXJ3TFYKN type=write size=1107708 2024-04-17T04:45:09.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.956+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=UWEU87MFMPUJ type=del size=577304 2024-04-17T04:45:09.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.960+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=YA6ZZV2JLBA8 type=del 2024-04-17T04:45:09.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.964+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=U6PY269UP20J type=del size=982644 2024-04-17T04:45:09.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.968+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=DIS37P6IGKTI type=del 2024-04-17T04:45:09.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.972+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=VV4N11JY0NB1 type=del size=1551185 2024-04-17T04:45:09.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.976+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=HWE6XF4TTQ9C type=del 2024-04-17T04:45:09.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~2PN3NQ5M332F.0 size=1048576 2024-04-17T04:45:09.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~2PN3NQ5M332F.1 size=800613 2024-04-17T04:45:09.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.984+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=2PN3NQ5M332F tag=QRQKTG9MI2EQ type=write size=2897765 2024-04-17T04:45:09.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.988+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=2E7778A9AF7E type=del 2024-04-17T04:45:09.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:09.996+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~8SZDOGEMVZ0W.0 size=1048576 2024-04-17T04:45:09.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~8SZDOGEMVZ0W.1 size=1048576 2024-04-17T04:45:09.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.000+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~8SZDOGEMVZ0W.2 size=48687 2024-04-17T04:45:09.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.000+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=8SZDOGEMVZ0W tag=R8H9MYK6J1NH type=write size=3194415 2024-04-17T04:45:10.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.004+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~8ZTPUHH4JOK0.0 size=1048576 2024-04-17T04:45:10.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.008+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~8ZTPUHH4JOK0.1 size=15822 2024-04-17T04:45:10.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.012+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=8ZTPUHH4JOK0 tag=8VRLO594VJW8 type=write size=2112974 2024-04-17T04:45:10.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.016+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=7BEEUSSPZKHH type=del size=2217237 2024-04-17T04:45:10.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.020+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=11436VVQ05FX type=del 2024-04-17T04:45:10.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.024+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=BACRUAEENVDY type=del size=3068732 2024-04-17T04:45:10.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~RD892RZWVBLX.0 size=1048576 2024-04-17T04:45:10.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~RD892RZWVBLX.1 size=572384 2024-04-17T04:45:10.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.036+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=RD892RZWVBLX tag=V71I2EX8C5R3 type=write size=2669536 2024-04-17T04:45:10.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.040+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=UMAJM6650XA4 type=del size=3850318 2024-04-17T04:45:10.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4LQDX38H3N93.0 size=1048576 2024-04-17T04:45:10.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.044+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4LQDX38H3N93.1 size=1048576 2024-04-17T04:45:10.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.048+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4LQDX38H3N93.2 size=122178 2024-04-17T04:45:10.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.052+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=4LQDX38H3N93 tag=EQVCN6Y8X32E type=write size=3267906 2024-04-17T04:45:10.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.056+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=J4S3IUP3AYJ8 type=write size=1018548 2024-04-17T04:45:10.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~CL220YD3I1XF.0 size=347511 2024-04-17T04:45:10.061 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.064+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=CL220YD3I1XF tag=01K65REQE3G4 type=write size=1396087 2024-04-17T04:45:10.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.068+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=UIZTRNI1WVCW type=del size=1083172 2024-04-17T04:45:10.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.072+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=1YUJCJOZI9ZY type=del 2024-04-17T04:45:10.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~J180R6LUB8UJ.0 size=1048576 2024-04-17T04:45:10.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.080+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~J180R6LUB8UJ.1 size=364086 2024-04-17T04:45:10.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.080+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=J180R6LUB8UJ tag=XUCE489ZMZOV type=write size=2461238 2024-04-17T04:45:10.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.088+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=BV68UX7U4FB1 type=del size=2537609 2024-04-17T04:45:10.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.088+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=V8PSENS43EDB type=write size=596920 2024-04-17T04:45:10.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.092+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~B0YEBW9V9VRA.0 size=1048576 2024-04-17T04:45:10.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.096+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~B0YEBW9V9VRA.1 size=1048576 2024-04-17T04:45:10.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.100+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~B0YEBW9V9VRA.2 size=742889 2024-04-17T04:45:10.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.104+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=B0YEBW9V9VRA tag=JIM424LPSG35 type=write size=3888617 2024-04-17T04:45:10.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.108+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=QWE88J004AIC type=write size=100577 2024-04-17T04:45:10.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.112+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~08OI6LZUYNXJ.0 size=1048576 2024-04-17T04:45:10.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.112+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~08OI6LZUYNXJ.1 size=1048576 2024-04-17T04:45:10.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~08OI6LZUYNXJ.2 size=719914 2024-04-17T04:45:10.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.124+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=08OI6LZUYNXJ tag=FLCEN5G0IF6N type=write size=3865642 2024-04-17T04:45:10.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.124+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=NRCUBOPA2BUX type=del size=3245991 2024-04-17T04:45:10.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.128+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=HJ18ZZ2MGQMV type=del 2024-04-17T04:45:10.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.132+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=5WJYU6PZU95L type=del size=3719459 2024-04-17T04:45:10.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.136+0000 7fd8ef2218c0 1 < suggested update operation key=1ZTJP8DRITEK tag=QLJ6OAHRNTL9 type=write size=965278 2024-04-17T04:45:10.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~IUVUKV5SOTY1.0 size=1048576 2024-04-17T04:45:10.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~IUVUKV5SOTY1.1 size=1048576 2024-04-17T04:45:10.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~IUVUKV5SOTY1.2 size=827959 2024-04-17T04:45:10.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.148+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=IUVUKV5SOTY1 tag=VKT0RZVHVIP3 type=write size=3973687 2024-04-17T04:45:10.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.152+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=UWEU87MFMPUJ type=del 2024-04-17T04:45:10.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.156+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=T2YJ1IVR2S5O type=del size=2400999 2024-04-17T04:45:10.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.164+0000 7fd8ef2218c0 1 < suggested remove operation key=6219DPY72LWC tag=U6PY269UP20J type=del 2024-04-17T04:45:10.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.168+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=SNZWGQXCM9OV type=del size=370769 2024-04-17T04:45:10.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.172+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=VV4N11JY0NB1 type=del 2024-04-17T04:45:10.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.180+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~C4ZSA7MTNSPP.0 size=1048576 2024-04-17T04:45:10.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.180+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~C4ZSA7MTNSPP.1 size=196801 2024-04-17T04:45:10.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.180+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=C4ZSA7MTNSPP tag=VF5W0XZM18S1 type=write size=2293953 2024-04-17T04:45:10.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.184+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~2PN3NQ5M332F.0 size=1048576 2024-04-17T04:45:10.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.188+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~2PN3NQ5M332F.1 size=800613 2024-04-17T04:45:10.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.196+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=2PN3NQ5M332F tag=QRQKTG9MI2EQ type=write size=2897765 2024-04-17T04:45:10.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.200+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=NOAAJ34PWL07 type=del size=4168017 2024-04-17T04:45:10.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.200+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~8SZDOGEMVZ0W.0 size=1048576 2024-04-17T04:45:10.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~8SZDOGEMVZ0W.1 size=1048576 2024-04-17T04:45:10.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.208+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~8SZDOGEMVZ0W.2 size=48687 2024-04-17T04:45:10.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.212+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=8SZDOGEMVZ0W tag=R8H9MYK6J1NH type=write size=3194415 2024-04-17T04:45:10.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~UIA8BI5IX1HL.0 size=1048576 2024-04-17T04:45:10.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~UIA8BI5IX1HL.1 size=1048576 2024-04-17T04:45:10.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.224+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~UIA8BI5IX1HL.2 size=9595 2024-04-17T04:45:10.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.224+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=UIA8BI5IX1HL tag=P576R60HTQOH type=write size=3155323 2024-04-17T04:45:10.223 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.228+0000 7fd8ef2218c0 1 < canceled operation key=5ZULD0PHLS4V tag=7BEEUSSPZKHH type=del size=2217237 2024-04-17T04:45:10.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.232+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=I597Z2TICOEX type=del size=2869479 2024-04-17T04:45:10.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.236+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=BACRUAEENVDY type=del 2024-04-17T04:45:10.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.236+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=R7KYE36Z10YI type=del size=2153095 2024-04-17T04:45:10.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.240+0000 7fd8ef2218c0 -1 < failed to complete operation key=5FR1NEWE0LD1 tag=UMAJM6650XA4 type=del size=3850318: (22) Invalid argument 2024-04-17T04:45:10.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.244+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=CW5KF00008BX type=write size=841481 2024-04-17T04:45:10.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.248+0000 7fd8ef2218c0 1 < completed write operation key=GX9X0D45GC2I tag=J4S3IUP3AYJ8 type=write size=1018548 2024-04-17T04:45:10.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.252+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=0UU3G0TTYX38 type=write size=979185 2024-04-17T04:45:10.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.256+0000 7fd8ef2218c0 1 < canceled operation key=JVWRJBMFIRLL tag=UIZTRNI1WVCW type=del size=1083172 2024-04-17T04:45:10.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.256+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=ZY4TM1OJCN5M type=del size=3753073 2024-04-17T04:45:10.256 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.260+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~J180R6LUB8UJ.0 size=1048576 2024-04-17T04:45:10.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~J180R6LUB8UJ.1 size=364086 2024-04-17T04:45:10.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.268+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=J180R6LUB8UJ tag=XUCE489ZMZOV type=write size=2461238 2024-04-17T04:45:10.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.272+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=3GGNCJU8G3FX type=del size=719845 2024-04-17T04:45:10.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.276+0000 7fd8ef2218c0 1 < completed write operation key=5EFSN2FPUQB6 tag=V8PSENS43EDB type=write size=596920 2024-04-17T04:45:10.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.280+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=S1ETPMGB9E8U type=write size=260014 2024-04-17T04:45:10.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.280+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=QWE88J004AIC type=write size=100577 2024-04-17T04:45:10.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~AKAME3E3UDSO.0 size=414935 2024-04-17T04:45:10.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.288+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=AKAME3E3UDSO tag=IPL31586MKB9 type=write size=1463511 2024-04-17T04:45:10.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.292+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=NRCUBOPA2BUX type=del 2024-04-17T04:45:10.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.292+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=H6FN8BCSU76N type=del size=2366853 2024-04-17T04:45:10.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.296+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=5WJYU6PZU95L type=del 2024-04-17T04:45:10.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.300+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=HTEEFJ94NNFH type=del size=942534 2024-04-17T04:45:10.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~IUVUKV5SOTY1.0 size=1048576 2024-04-17T04:45:10.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.304+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~IUVUKV5SOTY1.1 size=1048576 2024-04-17T04:45:10.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.308+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~IUVUKV5SOTY1.2 size=827959 2024-04-17T04:45:10.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.312+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=IUVUKV5SOTY1 tag=VKT0RZVHVIP3 type=write size=3973687 2024-04-17T04:45:10.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.316+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=K16HR2E39YBD type=del size=1545177 2024-04-17T04:45:10.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.320+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=T2YJ1IVR2S5O type=del 2024-04-17T04:45:10.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.324+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~I4A2ES7NKOAH.0 size=1048576 2024-04-17T04:45:10.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~I4A2ES7NKOAH.1 size=502773 2024-04-17T04:45:10.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.328+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=I4A2ES7NKOAH tag=VHTP2VES9VVU type=write size=2599925 2024-04-17T04:45:10.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.332+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=SNZWGQXCM9OV type=del 2024-04-17T04:45:10.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.336+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~XDAKAJ65D9RB.0 size=1048576 2024-04-17T04:45:10.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~XDAKAJ65D9RB.1 size=28421 2024-04-17T04:45:10.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.340+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=XDAKAJ65D9RB tag=22KH80BW5WEM type=write size=2125573 2024-04-17T04:45:10.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.344+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~C4ZSA7MTNSPP.0 size=1048576 2024-04-17T04:45:10.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.344+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~C4ZSA7MTNSPP.1 size=196801 2024-04-17T04:45:10.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.352+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=C4ZSA7MTNSPP tag=VF5W0XZM18S1 type=write size=2293953 2024-04-17T04:45:10.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~5CI2TC3ROUA9.0 size=1048576 2024-04-17T04:45:10.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.360+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~5CI2TC3ROUA9.1 size=930772 2024-04-17T04:45:10.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.360+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=5CI2TC3ROUA9 tag=0GT5HC04ESVJ type=write size=3027924 2024-04-17T04:45:10.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.360+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=NOAAJ34PWL07 type=del 2024-04-17T04:45:10.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~717VE4HYW7E4.0 size=1048576 2024-04-17T04:45:10.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~717VE4HYW7E4.1 size=184081 2024-04-17T04:45:10.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.372+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=717VE4HYW7E4 tag=QGH510T7HD94 type=write size=2281233 2024-04-17T04:45:10.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.376+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~UIA8BI5IX1HL.0 size=1048576 2024-04-17T04:45:10.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.376+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5FR1NEWE0LD1.2~UIA8BI5IX1HL.1 size=1048576 2024-04-17T04:45:10.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~UIA8BI5IX1HL.2 size=9595 2024-04-17T04:45:10.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.384+0000 7fd8ef2218c0 1 < canceled multipart upload key=5FR1NEWE0LD1 upload=UIA8BI5IX1HL tag=P576R60HTQOH type=write size=3155323 2024-04-17T04:45:10.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.388+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=AOKNZJI4ZE8L type=write size=862749 2024-04-17T04:45:10.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.392+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=I597Z2TICOEX type=del 2024-04-17T04:45:10.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.396+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=YU5IDKX7PPFX type=write size=706297 2024-04-17T04:45:10.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.400+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=R7KYE36Z10YI type=del 2024-04-17T04:45:10.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.404+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~K8KK0N282BJ0.0 size=1048576 2024-04-17T04:45:10.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~K8KK0N282BJ0.1 size=475053 2024-04-17T04:45:10.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.408+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=K8KK0N282BJ0 tag=TTHOYCCEVALO type=write size=2572205 2024-04-17T04:45:10.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.412+0000 7fd8ef2218c0 1 < completed write operation key=EWAW6EBW6W6U tag=CW5KF00008BX type=write size=841481 2024-04-17T04:45:10.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.416+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=LBV97NMSBHJC type=del size=524138 2024-04-17T04:45:10.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.420+0000 7fd8ef2218c0 1 < suggested update operation key=R3C8PAVPVGBY tag=0UU3G0TTYX38 type=write size=979185 2024-04-17T04:45:10.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.424+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=DBA4CWHWQ86L type=del size=3467953 2024-04-17T04:45:10.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.428+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=ZY4TM1OJCN5M type=del 2024-04-17T04:45:10.431 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~06JAGFIUXKR0.0 size=1005685 2024-04-17T04:45:10.431 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.436+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=06JAGFIUXKR0 tag=45Z0GOUNQ15C type=write size=2054261 2024-04-17T04:45:10.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.440+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=3GGNCJU8G3FX type=del 2024-04-17T04:45:10.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.440+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=QS3OQS43NTU3 type=del size=710481 2024-04-17T04:45:10.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.444+0000 7fd8ef2218c0 1 < completed write operation key=UNOVYXDS5JEB tag=S1ETPMGB9E8U type=write size=260014 2024-04-17T04:45:10.443 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.448+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=U3CDXUI7N4GJ type=write size=195693 2024-04-17T04:45:10.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.452+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~AKAME3E3UDSO.0 size=414935 2024-04-17T04:45:10.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.456+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=AKAME3E3UDSO tag=IPL31586MKB9 type=write size=1463511 2024-04-17T04:45:10.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.460+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=2C2Q1OF9WS7C type=del size=598934 2024-04-17T04:45:10.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.464+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=H6FN8BCSU76N type=del 2024-04-17T04:45:10.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.464+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=L86WIKK30V9F type=del size=1383663 2024-04-17T04:45:10.464 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.468+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=HTEEFJ94NNFH type=del 2024-04-17T04:45:10.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.476+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~2C09U2G0LRJ7.0 size=1048576 2024-04-17T04:45:10.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.476+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~2C09U2G0LRJ7.1 size=716180 2024-04-17T04:45:10.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.476+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=2C09U2G0LRJ7 tag=PDVWR2RBIUKW type=write size=2813332 2024-04-17T04:45:10.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.484+0000 7fd8ef2218c0 1 < suggested remove operation key=A5ZUB3YK06J3 tag=K16HR2E39YBD type=del 2024-04-17T04:45:10.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.488+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=GQCBGADZV5SH type=del size=23354 2024-04-17T04:45:10.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.492+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~I4A2ES7NKOAH.0 size=1048576 2024-04-17T04:45:10.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.492+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~I4A2ES7NKOAH.1 size=502773 2024-04-17T04:45:10.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.500+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=I4A2ES7NKOAH tag=VHTP2VES9VVU type=write size=2599925 2024-04-17T04:45:10.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.504+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~JK8CBZM529RK.0 size=1048576 2024-04-17T04:45:10.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~JK8CBZM529RK.1 size=412122 2024-04-17T04:45:10.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.508+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=JK8CBZM529RK tag=PWJPGT7Z63XE type=write size=2509274 2024-04-17T04:45:10.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.512+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~XDAKAJ65D9RB.0 size=1048576 2024-04-17T04:45:10.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.512+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~XDAKAJ65D9RB.1 size=28421 2024-04-17T04:45:10.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.520+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=XDAKAJ65D9RB tag=22KH80BW5WEM type=write size=2125573 2024-04-17T04:45:10.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.520+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=XA2F5P543SDT type=write size=618300 2024-04-17T04:45:10.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.524+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~5CI2TC3ROUA9.0 size=1048576 2024-04-17T04:45:10.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~5CI2TC3ROUA9.1 size=930772 2024-04-17T04:45:10.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.536+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=5CI2TC3ROUA9 tag=0GT5HC04ESVJ type=write size=3027924 2024-04-17T04:45:10.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.540+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~HTK0C0KXGOLI.0 size=248875 2024-04-17T04:45:10.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.540+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=HTK0C0KXGOLI tag=V1R9TU5FUBRG type=write size=1297451 2024-04-17T04:45:10.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~717VE4HYW7E4.0 size=1048576 2024-04-17T04:45:10.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.544+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~717VE4HYW7E4.1 size=184081 2024-04-17T04:45:10.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.552+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=717VE4HYW7E4 tag=QGH510T7HD94 type=write size=2281233 2024-04-17T04:45:10.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.556+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~XQV0TIHWJBPH.0 size=846373 2024-04-17T04:45:10.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.556+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=XQV0TIHWJBPH tag=KBK47HEJ42G3 type=write size=1894949 2024-04-17T04:45:10.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.560+0000 7fd8ef2218c0 1 < completed write operation key=EWAW6EBW6W6U tag=AOKNZJI4ZE8L type=write size=862749 2024-04-17T04:45:10.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.564+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=5150T4ANU9DF type=del size=644688 2024-04-17T04:45:10.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.568+0000 7fd8ef2218c0 1 < suggested remove operation key=R3C8PAVPVGBY tag=YU5IDKX7PPFX type=write 2024-04-17T04:45:10.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.572+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=TJGG8HC6T8FK type=del size=1043137 2024-04-17T04:45:10.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.572+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~K8KK0N282BJ0.0 size=1048576 2024-04-17T04:45:10.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~K8KK0N282BJ0.1 size=475053 2024-04-17T04:45:10.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.580+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=K8KK0N282BJ0 tag=TTHOYCCEVALO type=write size=2572205 2024-04-17T04:45:10.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.588+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~W6PDHPOLCGFH.0 size=1048576 2024-04-17T04:45:10.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.588+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~W6PDHPOLCGFH.1 size=1048576 2024-04-17T04:45:10.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~W6PDHPOLCGFH.2 size=940598 2024-04-17T04:45:10.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.592+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=W6PDHPOLCGFH tag=RM8VA2825YUN type=write size=4086326 2024-04-17T04:45:10.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.596+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=LBV97NMSBHJC type=del 2024-04-17T04:45:10.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.596+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=EUF0LG3KT8VI type=del size=2153236 2024-04-17T04:45:10.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.600+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=DBA4CWHWQ86L type=del 2024-04-17T04:45:10.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.604+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=IVHHBI1NIYL1 type=del size=3704147 2024-04-17T04:45:10.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.608+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~06JAGFIUXKR0.0 size=1005685 2024-04-17T04:45:10.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.612+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=06JAGFIUXKR0 tag=45Z0GOUNQ15C type=write size=2054261 2024-04-17T04:45:10.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.616+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=UX2WTFT9OMIZ type=del size=394299 2024-04-17T04:45:10.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.620+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=QS3OQS43NTU3 type=del 2024-04-17T04:45:10.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.620+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=ZMOK4VCG21YA type=write size=482107 2024-04-17T04:45:10.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.624+0000 7fd8ef2218c0 1 < canceled operation key=5K50O96P7OGD tag=U3CDXUI7N4GJ type=write size=195693 2024-04-17T04:45:10.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.628+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=1YWHSKZ0G2MT type=del size=185440 2024-04-17T04:45:10.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.632+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=2C2Q1OF9WS7C type=del 2024-04-17T04:45:10.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.632+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=LKJUKMN9GFCX type=del size=1942045 2024-04-17T04:45:10.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.636+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=L86WIKK30V9F type=del 2024-04-17T04:45:10.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.640+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=O23TWPABXI1G type=del size=2221706 2024-04-17T04:45:10.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.640+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~2C09U2G0LRJ7.0 size=1048576 2024-04-17T04:45:10.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.640+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UNOVYXDS5JEB.2~2C09U2G0LRJ7.1 size=716180 2024-04-17T04:45:10.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.648+0000 7fd8ef2218c0 1 < canceled multipart upload key=UNOVYXDS5JEB upload=2C09U2G0LRJ7 tag=PDVWR2RBIUKW type=write size=2813332 2024-04-17T04:45:10.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.648+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=RGZ83ZIU5Z7D type=del size=1870052 2024-04-17T04:45:10.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.652+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=GQCBGADZV5SH type=del 2024-04-17T04:45:10.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.656+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=WU0PW5QL2KZ7 type=del size=2401724 2024-04-17T04:45:10.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.660+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~JK8CBZM529RK.0 size=1048576 2024-04-17T04:45:10.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.660+0000 7fd8ef2218c0 1 < canceled part key=_multipart_3OFQH4KVYN18.2~JK8CBZM529RK.1 size=412122 2024-04-17T04:45:10.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.668+0000 7fd8ef2218c0 1 < canceled multipart upload key=3OFQH4KVYN18 upload=JK8CBZM529RK tag=PWJPGT7Z63XE type=write size=2509274 2024-04-17T04:45:10.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.668+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=6L2P8PRE9EW0 type=del size=542182 2024-04-17T04:45:10.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.672+0000 7fd8ef2218c0 1 < completed write operation key=509DZ9DBJWC5 tag=XA2F5P543SDT type=write size=618300 2024-04-17T04:45:10.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.676+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=AFQ4SWPIKM6A type=del size=2393572 2024-04-17T04:45:10.674 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~HTK0C0KXGOLI.0 size=248875 2024-04-17T04:45:10.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.684+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=HTK0C0KXGOLI tag=V1R9TU5FUBRG type=write size=1297451 2024-04-17T04:45:10.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.688+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=7398027827WX type=write size=294182 2024-04-17T04:45:10.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.688+0000 7fd8ef2218c0 1 < completed part key=_multipart_G03VLS09F3BT.2~XQV0TIHWJBPH.0 size=846373 2024-04-17T04:45:10.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.696+0000 7fd8ef2218c0 1 < completed multipart upload key=G03VLS09F3BT upload=XQV0TIHWJBPH tag=KBK47HEJ42G3 type=write size=1894949 2024-04-17T04:45:10.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~DAFLYG5W68KB.0 size=1048576 2024-04-17T04:45:10.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~DAFLYG5W68KB.1 size=1048576 2024-04-17T04:45:10.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.704+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~DAFLYG5W68KB.2 size=1046847 2024-04-17T04:45:10.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.704+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=DAFLYG5W68KB tag=MG8F02ER5LB0 type=write size=4192575 2024-04-17T04:45:10.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.712+0000 7fd8ef2218c0 1 < suggested remove operation key=VUYM3JC57QRP tag=5150T4ANU9DF type=del 2024-04-17T04:45:10.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~NOY9QNOWS5JP.0 size=1048576 2024-04-17T04:45:10.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~NOY9QNOWS5JP.1 size=736641 2024-04-17T04:45:10.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.720+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=NOY9QNOWS5JP tag=ISC28SBTIH9S type=write size=2833793 2024-04-17T04:45:10.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.728+0000 7fd8ef2218c0 1 < suggested update operation key=5K50O96P7OGD tag=TJGG8HC6T8FK type=del size=1043137 2024-04-17T04:45:10.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.732+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~7M2FJ0OJPRE2.0 size=1048576 2024-04-17T04:45:10.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~7M2FJ0OJPRE2.1 size=1048576 2024-04-17T04:45:10.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~7M2FJ0OJPRE2.2 size=456833 2024-04-17T04:45:10.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.736+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=7M2FJ0OJPRE2 tag=01KZ6S1HBCKX type=write size=3602561 2024-04-17T04:45:10.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~W6PDHPOLCGFH.0 size=1048576 2024-04-17T04:45:10.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~W6PDHPOLCGFH.1 size=1048576 2024-04-17T04:45:10.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.744+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~W6PDHPOLCGFH.2 size=940598 2024-04-17T04:45:10.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.748+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=W6PDHPOLCGFH tag=RM8VA2825YUN type=write size=4086326 2024-04-17T04:45:10.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.752+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=GBIV5LUIBTJV type=write size=28802 2024-04-17T04:45:10.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.760+0000 7fd8ef2218c0 1 < suggested update operation key=3OFQH4KVYN18 tag=EUF0LG3KT8VI type=del size=2153236 2024-04-17T04:45:10.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~0BE4R3VYZ2KM.0 size=1048576 2024-04-17T04:45:10.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.768+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~0BE4R3VYZ2KM.1 size=623480 2024-04-17T04:45:10.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.768+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=0BE4R3VYZ2KM tag=4QWV6MV5VAKM type=write size=2720632 2024-04-17T04:45:10.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.772+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=IVHHBI1NIYL1 type=del 2024-04-17T04:45:10.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.776+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=U05V05SMW3BW type=del size=166385 2024-04-17T04:45:10.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.780+0000 7fd8ef2218c0 -1 < failed to complete operation key=3OFQH4KVYN18 tag=UX2WTFT9OMIZ type=del size=394299: (22) Invalid argument 2024-04-17T04:45:10.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.784+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=RCIZN5HJ024J type=del size=1263188 2024-04-17T04:45:10.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.784+0000 7fd8ef2218c0 -1 < failed to complete operation key=5K50O96P7OGD tag=ZMOK4VCG21YA type=write size=482107: (22) Invalid argument 2024-04-17T04:45:10.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~HTBGECY8J5CP.0 size=455120 2024-04-17T04:45:10.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=HTBGECY8J5CP tag=6WOVZ06V0JD2 type=write size=1503696 2024-04-17T04:45:10.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.796+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=1YWHSKZ0G2MT type=del 2024-04-17T04:45:10.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.800+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=WLVTOKP77NLL type=del size=403196 2024-04-17T04:45:10.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.800+0000 7fd8ef2218c0 -1 < failed to complete operation key=5K50O96P7OGD tag=LKJUKMN9GFCX type=del size=1942045: (22) Invalid argument 2024-04-17T04:45:10.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.808+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~EGJMQTIWJLGL.0 size=1048576 2024-04-17T04:45:10.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.812+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~EGJMQTIWJLGL.1 size=708718 2024-04-17T04:45:10.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.812+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=EGJMQTIWJLGL tag=H1V510ICII43 type=write size=2805870 2024-04-17T04:45:10.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.816+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=O23TWPABXI1G type=del 2024-04-17T04:45:10.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.820+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=VV1B7RSO274E type=del size=404625 2024-04-17T04:45:10.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.824+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=RGZ83ZIU5Z7D type=del 2024-04-17T04:45:10.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.828+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=RX6CT26O5VMA type=del size=2317629 2024-04-17T04:45:10.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.832+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=WU0PW5QL2KZ7 type=del 2024-04-17T04:45:10.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~OXIW0A3BU1QM.0 size=1048576 2024-04-17T04:45:10.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~OXIW0A3BU1QM.1 size=412569 2024-04-17T04:45:10.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=OXIW0A3BU1QM tag=30G5B64MOWTJ type=write size=2509721 2024-04-17T04:45:10.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.844+0000 7fd8ef2218c0 1 < canceled operation key=A5ZUB3YK06J3 tag=6L2P8PRE9EW0 type=del size=542182 2024-04-17T04:45:10.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.848+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=SLJK8PIMDRXX type=del size=4049940 2024-04-17T04:45:10.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.852+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=AFQ4SWPIKM6A type=del 2024-04-17T04:45:10.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.856+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=N14QYUQLY3J6 type=del size=1994215 2024-04-17T04:45:10.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.860+0000 7fd8ef2218c0 1 < completed write operation key=1JOWGWFR7R6P tag=7398027827WX type=write size=294182 2024-04-17T04:45:10.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.864+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=22YW9HWA5CZK type=del size=366255 2024-04-17T04:45:10.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.868+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~DAFLYG5W68KB.0 size=1048576 2024-04-17T04:45:10.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.872+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~DAFLYG5W68KB.1 size=1048576 2024-04-17T04:45:10.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.872+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~DAFLYG5W68KB.2 size=1046847 2024-04-17T04:45:10.875 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.880+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=DAFLYG5W68KB tag=MG8F02ER5LB0 type=write size=4192575 2024-04-17T04:45:10.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.884+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=QBOLQTQ6D2T2 type=write size=936785 2024-04-17T04:45:10.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~NOY9QNOWS5JP.0 size=1048576 2024-04-17T04:45:10.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~NOY9QNOWS5JP.1 size=736641 2024-04-17T04:45:10.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.896+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=NOY9QNOWS5JP tag=ISC28SBTIH9S type=write size=2833793 2024-04-17T04:45:10.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~GAF1JFGAJ1XD.0 size=1048576 2024-04-17T04:45:10.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~GAF1JFGAJ1XD.1 size=1048576 2024-04-17T04:45:10.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~GAF1JFGAJ1XD.2 size=898257 2024-04-17T04:45:10.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.908+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=GAF1JFGAJ1XD tag=RJ887HSVH14N type=write size=4043985 2024-04-17T04:45:10.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.908+0000 7fd8ef2218c0 1 < canceled part key=_multipart_AE4CYL334ONO.2~7M2FJ0OJPRE2.0 size=1048576 2024-04-17T04:45:10.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.912+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~7M2FJ0OJPRE2.1 size=1048576 2024-04-17T04:45:10.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~7M2FJ0OJPRE2.2 size=456833 2024-04-17T04:45:10.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.920+0000 7fd8ef2218c0 1 < canceled multipart upload key=AE4CYL334ONO upload=7M2FJ0OJPRE2 tag=01KZ6S1HBCKX type=write size=3602561 2024-04-17T04:45:10.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.924+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=HCGUXXI3M0XN type=del size=1641025 2024-04-17T04:45:10.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.928+0000 7fd8ef2218c0 1 < completed write operation key=5ZULD0PHLS4V tag=GBIV5LUIBTJV type=write size=28802 2024-04-17T04:45:10.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.932+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=L0IYHZE6VUIK type=del size=3221941 2024-04-17T04:45:10.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~0BE4R3VYZ2KM.0 size=1048576 2024-04-17T04:45:10.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~0BE4R3VYZ2KM.1 size=623480 2024-04-17T04:45:10.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.940+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=0BE4R3VYZ2KM tag=4QWV6MV5VAKM type=write size=2720632 2024-04-17T04:45:10.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.944+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~068OZ09N14SE.0 size=1048576 2024-04-17T04:45:10.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.948+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~068OZ09N14SE.1 size=1048576 2024-04-17T04:45:10.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~068OZ09N14SE.2 size=602209 2024-04-17T04:45:10.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.952+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=068OZ09N14SE tag=SFDB94L7R0Y3 type=write size=3747937 2024-04-17T04:45:10.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.952+0000 7fd8ef2218c0 1 < canceled operation key=N11Y8FPN9XES tag=U05V05SMW3BW type=del size=166385 2024-04-17T04:45:10.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.956+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=B31CGUXWNX53 type=del size=2645996 2024-04-17T04:45:10.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.960+0000 7fd8ef2218c0 1 < canceled operation key=5ZULD0PHLS4V tag=RCIZN5HJ024J type=del size=1263188 2024-04-17T04:45:10.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.964+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~U63XDWZAKTSO.0 size=1048576 2024-04-17T04:45:10.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.968+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~U63XDWZAKTSO.1 size=1048576 2024-04-17T04:45:10.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~U63XDWZAKTSO.2 size=688909 2024-04-17T04:45:10.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.972+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=U63XDWZAKTSO tag=DZI81A5CK0ZC type=write size=3834637 2024-04-17T04:45:10.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.972+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~HTBGECY8J5CP.0 size=455120 2024-04-17T04:45:10.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.976+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=HTBGECY8J5CP tag=6WOVZ06V0JD2 type=write size=1503696 2024-04-17T04:45:10.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.980+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=AQ207MUKBV9O type=write size=909765 2024-04-17T04:45:10.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.984+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=WLVTOKP77NLL type=del 2024-04-17T04:45:10.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~8L9ZICQI0GGJ.0 size=825204 2024-04-17T04:45:10.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.988+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=8L9ZICQI0GGJ tag=1P8EP3NAOBHZ type=write size=1873780 2024-04-17T04:45:10.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.992+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~EGJMQTIWJLGL.0 size=1048576 2024-04-17T04:45:10.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.992+0000 7fd8ef2218c0 1 < canceled part key=_multipart_6219DPY72LWC.2~EGJMQTIWJLGL.1 size=708718 2024-04-17T04:45:10.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.996+0000 7fd8ef2218c0 1 < canceled multipart upload key=6219DPY72LWC upload=EGJMQTIWJLGL tag=H1V510ICII43 type=write size=2805870 2024-04-17T04:45:10.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:10.996+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=LFKWECT6GZJ0 type=del size=1773283 2024-04-17T04:45:10.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.000+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=VV1B7RSO274E type=del 2024-04-17T04:45:11.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~NOLRWRV0TR24.0 size=1048576 2024-04-17T04:45:11.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~NOLRWRV0TR24.1 size=1048576 2024-04-17T04:45:11.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~NOLRWRV0TR24.2 size=215115 2024-04-17T04:45:11.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=NOLRWRV0TR24 tag=V6JI70OPQ0L3 type=write size=3360843 2024-04-17T04:45:11.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.012+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=RX6CT26O5VMA type=del 2024-04-17T04:45:11.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~H9X2LOXZ8CCD.0 size=1048576 2024-04-17T04:45:11.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.020+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~H9X2LOXZ8CCD.1 size=1048576 2024-04-17T04:45:11.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.020+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~H9X2LOXZ8CCD.2 size=131256 2024-04-17T04:45:11.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.020+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=H9X2LOXZ8CCD tag=3Y8RHLQBNPF4 type=write size=3276984 2024-04-17T04:45:11.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.024+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~OXIW0A3BU1QM.0 size=1048576 2024-04-17T04:45:11.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.024+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~OXIW0A3BU1QM.1 size=412569 2024-04-17T04:45:11.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.032+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=OXIW0A3BU1QM tag=30G5B64MOWTJ type=write size=2509721 2024-04-17T04:45:11.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.032+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=H08FFEOZEACZ type=del size=1536594 2024-04-17T04:45:11.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.036+0000 7fd8ef2218c0 1 < suggested remove operation key=OZ1IST8DOGXW tag=SLJK8PIMDRXX type=del 2024-04-17T04:45:11.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.040+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=E3S01N9RIASJ type=write size=434667 2024-04-17T04:45:11.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.044+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=N14QYUQLY3J6 type=del 2024-04-17T04:45:11.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.048+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~PMWWKUS5F4CN.0 size=47682 2024-04-17T04:45:11.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.048+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=PMWWKUS5F4CN tag=ESTJPUUHRO1O type=write size=1096258 2024-04-17T04:45:11.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.052+0000 7fd8ef2218c0 1 < suggested remove operation key=6219DPY72LWC tag=22YW9HWA5CZK type=del 2024-04-17T04:45:11.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.056+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=OM2K0K2QOY10 type=write size=770028 2024-04-17T04:45:11.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.056+0000 7fd8ef2218c0 1 < completed write operation key=AE4CYL334ONO tag=QBOLQTQ6D2T2 type=write size=936785 2024-04-17T04:45:11.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.060+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=49SRM2N0E0VS type=del size=3312750 2024-04-17T04:45:11.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~GAF1JFGAJ1XD.0 size=1048576 2024-04-17T04:45:11.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.064+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~GAF1JFGAJ1XD.1 size=1048576 2024-04-17T04:45:11.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.064+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~GAF1JFGAJ1XD.2 size=898257 2024-04-17T04:45:11.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.068+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=GAF1JFGAJ1XD tag=RJ887HSVH14N type=write size=4043985 2024-04-17T04:45:11.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~AJDQD5H3T0O5.0 size=1048576 2024-04-17T04:45:11.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~AJDQD5H3T0O5.1 size=377041 2024-04-17T04:45:11.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.076+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=AJDQD5H3T0O5 tag=1MNVTEG61DB1 type=write size=2474193 2024-04-17T04:45:11.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.076+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=HCGUXXI3M0XN type=del 2024-04-17T04:45:11.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.080+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~N3J04R2APL5Q.0 size=843721 2024-04-17T04:45:11.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.080+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=N3J04R2APL5Q tag=WJ8BPY3EXZA6 type=write size=1892297 2024-04-17T04:45:11.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.084+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=L0IYHZE6VUIK type=del 2024-04-17T04:45:11.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.088+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=N8ED201BCZZY type=del size=3344964 2024-04-17T04:45:11.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~068OZ09N14SE.0 size=1048576 2024-04-17T04:45:11.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.088+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1ZTJP8DRITEK.2~068OZ09N14SE.1 size=1048576 2024-04-17T04:45:11.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.092+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~068OZ09N14SE.2 size=602209 2024-04-17T04:45:11.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.096+0000 7fd8ef2218c0 1 < canceled multipart upload key=1ZTJP8DRITEK upload=068OZ09N14SE tag=SFDB94L7R0Y3 type=write size=3747937 2024-04-17T04:45:11.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~AZQTKFPVZ350.0 size=761728 2024-04-17T04:45:11.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.100+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=AZQTKFPVZ350 tag=UX7XVHI3G43F type=write size=1810304 2024-04-17T04:45:11.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.104+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=B31CGUXWNX53 type=del 2024-04-17T04:45:11.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.108+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~789TAT9BL1UD.0 size=1048576 2024-04-17T04:45:11.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.108+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~789TAT9BL1UD.1 size=1048576 2024-04-17T04:45:11.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~789TAT9BL1UD.2 size=274735 2024-04-17T04:45:11.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.112+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=789TAT9BL1UD tag=R1JX396U8RO6 type=write size=3420463 2024-04-17T04:45:11.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.112+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~U63XDWZAKTSO.0 size=1048576 2024-04-17T04:45:11.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.116+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~U63XDWZAKTSO.1 size=1048576 2024-04-17T04:45:11.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.120+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~U63XDWZAKTSO.2 size=688909 2024-04-17T04:45:11.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.124+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=U63XDWZAKTSO tag=DZI81A5CK0ZC type=write size=3834637 2024-04-17T04:45:11.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.128+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~24302JO31BYR.0 size=1048576 2024-04-17T04:45:11.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~24302JO31BYR.1 size=1048576 2024-04-17T04:45:11.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~24302JO31BYR.2 size=987652 2024-04-17T04:45:11.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.132+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=24302JO31BYR tag=WPWCIRUL9ISD type=write size=4133380 2024-04-17T04:45:11.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.136+0000 7fd8ef2218c0 1 < completed write operation key=GX9X0D45GC2I tag=AQ207MUKBV9O type=write size=909765 2024-04-17T04:45:11.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~76P0YPW1PIRU.0 size=1048576 2024-04-17T04:45:11.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.144+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~76P0YPW1PIRU.1 size=975106 2024-04-17T04:45:11.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.144+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=76P0YPW1PIRU tag=Y7T6BAHD65GN type=write size=3072258 2024-04-17T04:45:11.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~8L9ZICQI0GGJ.0 size=825204 2024-04-17T04:45:11.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.152+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=8L9ZICQI0GGJ tag=1P8EP3NAOBHZ type=write size=1873780 2024-04-17T04:45:11.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.160+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~M3BGQCYXBNM3.0 size=2910 2024-04-17T04:45:11.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.160+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=M3BGQCYXBNM3 tag=JMYLZ2WBMBZM type=write size=1051486 2024-04-17T04:45:11.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.168+0000 7fd8ef2218c0 1 < suggested update operation key=EWAW6EBW6W6U tag=LFKWECT6GZJ0 type=del size=1773283 2024-04-17T04:45:11.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.172+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=881YT9A3M89S type=del size=2134454 2024-04-17T04:45:11.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~NOLRWRV0TR24.0 size=1048576 2024-04-17T04:45:11.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~NOLRWRV0TR24.1 size=1048576 2024-04-17T04:45:11.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.180+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~NOLRWRV0TR24.2 size=215115 2024-04-17T04:45:11.181 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.184+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=NOLRWRV0TR24 tag=V6JI70OPQ0L3 type=write size=3360843 2024-04-17T04:45:11.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~59WIDES6KQST.0 size=1048576 2024-04-17T04:45:11.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~59WIDES6KQST.1 size=1048576 2024-04-17T04:45:11.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.196+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~59WIDES6KQST.2 size=287560 2024-04-17T04:45:11.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.196+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=59WIDES6KQST tag=EITBPKWTEIU6 type=write size=3433288 2024-04-17T04:45:11.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~H9X2LOXZ8CCD.0 size=1048576 2024-04-17T04:45:11.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~H9X2LOXZ8CCD.1 size=1048576 2024-04-17T04:45:11.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~H9X2LOXZ8CCD.2 size=131256 2024-04-17T04:45:11.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.208+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=H9X2LOXZ8CCD tag=3Y8RHLQBNPF4 type=write size=3276984 2024-04-17T04:45:11.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.212+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=DN8AEZ9UD3M5 type=del size=198410 2024-04-17T04:45:11.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.216+0000 7fd8ef2218c0 -1 < failed to complete operation key=EWAW6EBW6W6U tag=H08FFEOZEACZ type=del size=1536594: (22) Invalid argument 2024-04-17T04:45:11.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.220+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=52CY00B4VLVU type=del size=514966 2024-04-17T04:45:11.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.224+0000 7fd8ef2218c0 1 < completed write operation key=UNOVYXDS5JEB tag=E3S01N9RIASJ type=write size=434667 2024-04-17T04:45:11.223 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.228+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=2N5I1YCJ0AQ1 type=del size=1562092 2024-04-17T04:45:11.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~PMWWKUS5F4CN.0 size=47682 2024-04-17T04:45:11.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.236+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=PMWWKUS5F4CN tag=ESTJPUUHRO1O type=write size=1096258 2024-04-17T04:45:11.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.240+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~ST0GO70T47S8.0 size=1048576 2024-04-17T04:45:11.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~ST0GO70T47S8.1 size=1047556 2024-04-17T04:45:11.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.244+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=ST0GO70T47S8 tag=CBX5PE3XGEGM type=write size=3144708 2024-04-17T04:45:11.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.244+0000 7fd8ef2218c0 1 < completed write operation key=JVWRJBMFIRLL tag=OM2K0K2QOY10 type=write size=770028 2024-04-17T04:45:11.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~7PH9RFA8FQZ6.0 size=1048576 2024-04-17T04:45:11.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~7PH9RFA8FQZ6.1 size=359078 2024-04-17T04:45:11.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.252+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=7PH9RFA8FQZ6 tag=8WH4F6PD09U0 type=write size=2456230 2024-04-17T04:45:11.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.256+0000 7fd8ef2218c0 1 < suggested update operation key=M3U8L7M8JKG9 tag=49SRM2N0E0VS type=del size=3312750 2024-04-17T04:45:11.256 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.260+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=EI9VL75JSI44 type=del size=3172195 2024-04-17T04:45:11.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~AJDQD5H3T0O5.0 size=1048576 2024-04-17T04:45:11.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.268+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~AJDQD5H3T0O5.1 size=377041 2024-04-17T04:45:11.268 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.272+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=AJDQD5H3T0O5 tag=1MNVTEG61DB1 type=write size=2474193 2024-04-17T04:45:11.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.276+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=HA80PGMKE1M0 type=del size=4185433 2024-04-17T04:45:11.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.280+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~N3J04R2APL5Q.0 size=843721 2024-04-17T04:45:11.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.284+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=N3J04R2APL5Q tag=WJ8BPY3EXZA6 type=write size=1892297 2024-04-17T04:45:11.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.288+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=KK5N19QR0QLT type=del size=204809 2024-04-17T04:45:11.288 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.292+0000 7fd8ef2218c0 1 < canceled operation key=UEWLJ3E9ODUL tag=N8ED201BCZZY type=del size=3344964 2024-04-17T04:45:11.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.296+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=3SJDF1WKWX6I type=del size=2240146 2024-04-17T04:45:11.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.300+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~AZQTKFPVZ350.0 size=761728 2024-04-17T04:45:11.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.304+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=AZQTKFPVZ350 tag=UX7XVHI3G43F type=write size=1810304 2024-04-17T04:45:11.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.308+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=BRBTTHDM0ZOX type=del size=453667 2024-04-17T04:45:11.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~789TAT9BL1UD.0 size=1048576 2024-04-17T04:45:11.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.312+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~789TAT9BL1UD.1 size=1048576 2024-04-17T04:45:11.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~789TAT9BL1UD.2 size=274735 2024-04-17T04:45:11.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.320+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=789TAT9BL1UD tag=R1JX396U8RO6 type=write size=3420463 2024-04-17T04:45:11.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~55JZTD9B551I.0 size=1048576 2024-04-17T04:45:11.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.328+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~55JZTD9B551I.1 size=1048576 2024-04-17T04:45:11.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~55JZTD9B551I.2 size=551178 2024-04-17T04:45:11.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.332+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=55JZTD9B551I tag=E2JD6QHWGUFI type=write size=3696906 2024-04-17T04:45:11.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.332+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~24302JO31BYR.0 size=1048576 2024-04-17T04:45:11.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.336+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~24302JO31BYR.1 size=1048576 2024-04-17T04:45:11.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.340+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~24302JO31BYR.2 size=987652 2024-04-17T04:45:11.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.344+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=24302JO31BYR tag=WPWCIRUL9ISD type=write size=4133380 2024-04-17T04:45:11.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~CUSUQUVMPFZR.0 size=216019 2024-04-17T04:45:11.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.352+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=CUSUQUVMPFZR tag=TWRAQLU3J3KS type=write size=1264595 2024-04-17T04:45:11.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.352+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~76P0YPW1PIRU.0 size=1048576 2024-04-17T04:45:11.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.356+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~76P0YPW1PIRU.1 size=975106 2024-04-17T04:45:11.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.364+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=76P0YPW1PIRU tag=Y7T6BAHD65GN type=write size=3072258 2024-04-17T04:45:11.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.368+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~GWH080E4AUL8.0 size=1048576 2024-04-17T04:45:11.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~GWH080E4AUL8.1 size=1048576 2024-04-17T04:45:11.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~GWH080E4AUL8.2 size=204094 2024-04-17T04:45:11.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.372+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=GWH080E4AUL8 tag=T347Z2XIM3W6 type=write size=3349822 2024-04-17T04:45:11.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.376+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~M3BGQCYXBNM3.0 size=2910 2024-04-17T04:45:11.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.380+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=M3BGQCYXBNM3 tag=JMYLZ2WBMBZM type=write size=1051486 2024-04-17T04:45:11.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.384+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=0FIVVPAOO4NC type=del size=2832570 2024-04-17T04:45:11.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.392+0000 7fd8ef2218c0 1 < suggested remove operation key=UNOVYXDS5JEB tag=881YT9A3M89S type=del 2024-04-17T04:45:11.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~OESDFCBO0BKW.0 size=843222 2024-04-17T04:45:11.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.396+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=OESDFCBO0BKW tag=RI981BW82K14 type=write size=1891798 2024-04-17T04:45:11.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.400+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~59WIDES6KQST.0 size=1048576 2024-04-17T04:45:11.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.404+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~59WIDES6KQST.1 size=1048576 2024-04-17T04:45:11.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.404+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~59WIDES6KQST.2 size=287560 2024-04-17T04:45:11.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.412+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=59WIDES6KQST tag=EITBPKWTEIU6 type=write size=3433288 2024-04-17T04:45:11.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.412+0000 7fd8ef2218c0 1 > prepared operation key=JVWRJBMFIRLL tag=YGX72XRXH9HY type=del size=2366270 2024-04-17T04:45:11.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.416+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=DN8AEZ9UD3M5 type=del 2024-04-17T04:45:11.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.424+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~1H30GZM1P7RP.0 size=1048576 2024-04-17T04:45:11.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.424+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~1H30GZM1P7RP.1 size=320420 2024-04-17T04:45:11.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.424+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=1H30GZM1P7RP tag=S5A2K70RZFED type=write size=2417572 2024-04-17T04:45:11.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.428+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=52CY00B4VLVU type=del 2024-04-17T04:45:11.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~D64VB831NP2T.0 size=1048576 2024-04-17T04:45:11.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~D64VB831NP2T.1 size=1048576 2024-04-17T04:45:11.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~D64VB831NP2T.2 size=624856 2024-04-17T04:45:11.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.440+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=D64VB831NP2T tag=8HT21J6AJ67F type=write size=3770584 2024-04-17T04:45:11.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.444+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=2N5I1YCJ0AQ1 type=del 2024-04-17T04:45:11.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.448+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~Q0RRBQC9009Q.0 size=1048576 2024-04-17T04:45:11.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~Q0RRBQC9009Q.1 size=223539 2024-04-17T04:45:11.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.452+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=Q0RRBQC9009Q tag=TVU3A6BTYHI8 type=write size=2320691 2024-04-17T04:45:11.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.452+0000 7fd8ef2218c0 1 < canceled part key=_multipart_MS3T476JOEON.2~ST0GO70T47S8.0 size=1048576 2024-04-17T04:45:11.450 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~ST0GO70T47S8.1 size=1047556 2024-04-17T04:45:11.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.460+0000 7fd8ef2218c0 1 < canceled multipart upload key=MS3T476JOEON upload=ST0GO70T47S8 tag=CBX5PE3XGEGM type=write size=3144708 2024-04-17T04:45:11.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.464+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=T7R1XVIUWB22 type=del size=1078600 2024-04-17T04:45:11.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.468+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~7PH9RFA8FQZ6.0 size=1048576 2024-04-17T04:45:11.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~7PH9RFA8FQZ6.1 size=359078 2024-04-17T04:45:11.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.476+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=7PH9RFA8FQZ6 tag=8WH4F6PD09U0 type=write size=2456230 2024-04-17T04:45:11.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.480+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=J80FPWBPNXVK type=del size=1322400 2024-04-17T04:45:11.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.484+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=EI9VL75JSI44 type=del 2024-04-17T04:45:11.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~VIV07L0HK2WT.0 size=1048576 2024-04-17T04:45:11.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.492+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~VIV07L0HK2WT.1 size=1048576 2024-04-17T04:45:11.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.496+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~VIV07L0HK2WT.2 size=496190 2024-04-17T04:45:11.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.496+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=VIV07L0HK2WT tag=NCZ9SLAA8H5E type=write size=3641918 2024-04-17T04:45:11.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.496+0000 7fd8ef2218c0 1 < canceled operation key=N11Y8FPN9XES tag=HA80PGMKE1M0 type=del size=4185433 2024-04-17T04:45:11.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.504+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~042DEZ5H8IWE.0 size=564596 2024-04-17T04:45:11.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.504+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=042DEZ5H8IWE tag=DMCK19BXO8ZE type=write size=1613172 2024-04-17T04:45:11.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.508+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=KK5N19QR0QLT type=del 2024-04-17T04:45:11.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.512+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=CPZF02EYVPDM type=del size=4149935 2024-04-17T04:45:11.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.516+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=3SJDF1WKWX6I type=del 2024-04-17T04:45:11.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.520+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=GF3KQIBADKKW type=del size=4155360 2024-04-17T04:45:11.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.524+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=BRBTTHDM0ZOX type=del 2024-04-17T04:45:11.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~GNKLUF11K1E2.0 size=1048576 2024-04-17T04:45:11.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~GNKLUF11K1E2.1 size=353656 2024-04-17T04:45:11.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.532+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=GNKLUF11K1E2 tag=GOEYT96T9788 type=write size=2450808 2024-04-17T04:45:11.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.536+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~55JZTD9B551I.0 size=1048576 2024-04-17T04:45:11.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~55JZTD9B551I.1 size=1048576 2024-04-17T04:45:11.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~55JZTD9B551I.2 size=551178 2024-04-17T04:45:11.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.544+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=55JZTD9B551I tag=E2JD6QHWGUFI type=write size=3696906 2024-04-17T04:45:11.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.548+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=R7B03K1NM916 type=del size=2670298 2024-04-17T04:45:11.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.552+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~CUSUQUVMPFZR.0 size=216019 2024-04-17T04:45:11.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.556+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=CUSUQUVMPFZR tag=TWRAQLU3J3KS type=write size=1264595 2024-04-17T04:45:11.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~MYM6DFFP85TU.0 size=1048576 2024-04-17T04:45:11.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~MYM6DFFP85TU.1 size=1048576 2024-04-17T04:45:11.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.568+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~MYM6DFFP85TU.2 size=382397 2024-04-17T04:45:11.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.568+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=MYM6DFFP85TU tag=05Z1P8NCIJXB type=write size=3528125 2024-04-17T04:45:11.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.572+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~GWH080E4AUL8.0 size=1048576 2024-04-17T04:45:11.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~GWH080E4AUL8.1 size=1048576 2024-04-17T04:45:11.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.576+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5K50O96P7OGD.2~GWH080E4AUL8.2 size=204094 2024-04-17T04:45:11.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.580+0000 7fd8ef2218c0 1 < canceled multipart upload key=5K50O96P7OGD upload=GWH080E4AUL8 tag=T347Z2XIM3W6 type=write size=3349822 2024-04-17T04:45:11.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.584+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=1LM402C26SZS type=del size=2451445 2024-04-17T04:45:11.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.584+0000 7fd8ef2218c0 1 < canceled operation key=2IY8DB37FI9H tag=0FIVVPAOO4NC type=del size=2832570 2024-04-17T04:45:11.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~OZOKJOHU5N23.0 size=1048576 2024-04-17T04:45:11.589 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~OZOKJOHU5N23.1 size=1048576 2024-04-17T04:45:11.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~OZOKJOHU5N23.2 size=650405 2024-04-17T04:45:11.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=OZOKJOHU5N23 tag=YP9KCQIMWS77 type=write size=3796133 2024-04-17T04:45:11.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.600+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~OESDFCBO0BKW.0 size=843222 2024-04-17T04:45:11.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.604+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=OESDFCBO0BKW tag=RI981BW82K14 type=write size=1891798 2024-04-17T04:45:11.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.608+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=GAA8ZPNXH6W0 type=write size=486289 2024-04-17T04:45:11.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.612+0000 7fd8ef2218c0 1 < completed delete operation key=JVWRJBMFIRLL tag=YGX72XRXH9HY type=del 2024-04-17T04:45:11.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.616+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=YXIPZC3C8IH6 type=write size=190153 2024-04-17T04:45:11.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.616+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~1H30GZM1P7RP.0 size=1048576 2024-04-17T04:45:11.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~1H30GZM1P7RP.1 size=320420 2024-04-17T04:45:11.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.624+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=1H30GZM1P7RP tag=S5A2K70RZFED type=write size=2417572 2024-04-17T04:45:11.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.628+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=CZQRE482PXCV type=del size=2616769 2024-04-17T04:45:11.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~D64VB831NP2T.0 size=1048576 2024-04-17T04:45:11.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~D64VB831NP2T.1 size=1048576 2024-04-17T04:45:11.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~D64VB831NP2T.2 size=624856 2024-04-17T04:45:11.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.640+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=D64VB831NP2T tag=8HT21J6AJ67F type=write size=3770584 2024-04-17T04:45:11.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.644+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=H388SYSSBM6Q type=del size=3248208 2024-04-17T04:45:11.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.648+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~Q0RRBQC9009Q.0 size=1048576 2024-04-17T04:45:11.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.648+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1JOWGWFR7R6P.2~Q0RRBQC9009Q.1 size=223539 2024-04-17T04:45:11.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.652+0000 7fd8ef2218c0 1 < canceled multipart upload key=1JOWGWFR7R6P upload=Q0RRBQC9009Q tag=TVU3A6BTYHI8 type=write size=2320691 2024-04-17T04:45:11.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~46PX50MB1GDQ.0 size=12477 2024-04-17T04:45:11.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.656+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=46PX50MB1GDQ tag=JJJ9ZFSNFC8G type=write size=1061053 2024-04-17T04:45:11.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.660+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=T7R1XVIUWB22 type=del 2024-04-17T04:45:11.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.664+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=8EJSRH7TBK93 type=del size=3609456 2024-04-17T04:45:11.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.668+0000 7fd8ef2218c0 1 < suggested remove operation key=1JOWGWFR7R6P tag=J80FPWBPNXVK type=del 2024-04-17T04:45:11.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.672+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=EW6FDNNJ9XRX type=write size=612331 2024-04-17T04:45:11.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~VIV07L0HK2WT.0 size=1048576 2024-04-17T04:45:11.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~VIV07L0HK2WT.1 size=1048576 2024-04-17T04:45:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~VIV07L0HK2WT.2 size=496190 2024-04-17T04:45:11.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.684+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=VIV07L0HK2WT tag=NCZ9SLAA8H5E type=write size=3641918 2024-04-17T04:45:11.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.688+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=XSU8PSEMFCU5 type=del size=485436 2024-04-17T04:45:11.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.692+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~042DEZ5H8IWE.0 size=564596 2024-04-17T04:45:11.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.696+0000 7fd8ef2218c0 1 < completed multipart upload key=UYCKMXE6LBUH upload=042DEZ5H8IWE tag=DMCK19BXO8ZE type=write size=1613172 2024-04-17T04:45:11.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~TRHAQDK8FYEY.0 size=33140 2024-04-17T04:45:11.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.700+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=TRHAQDK8FYEY tag=XIWYTQW1VML1 type=write size=1081716 2024-04-17T04:45:11.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.704+0000 7fd8ef2218c0 1 < canceled operation key=M3U8L7M8JKG9 tag=CPZF02EYVPDM type=del size=4149935 2024-04-17T04:45:11.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.708+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=V5NLEJNSOME2 type=del size=2085921 2024-04-17T04:45:11.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.712+0000 7fd8ef2218c0 1 < suggested update operation key=2S1IT3TO1BPU tag=GF3KQIBADKKW type=del size=4155360 2024-04-17T04:45:11.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.716+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=HMDD3KI0POHB type=del size=716900 2024-04-17T04:45:11.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.716+0000 7fd8ef2218c0 1 < canceled part key=_multipart_VUYM3JC57QRP.2~GNKLUF11K1E2.0 size=1048576 2024-04-17T04:45:11.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.720+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~GNKLUF11K1E2.1 size=353656 2024-04-17T04:45:11.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.724+0000 7fd8ef2218c0 1 < canceled multipart upload key=VUYM3JC57QRP upload=GNKLUF11K1E2 tag=GOEYT96T9788 type=write size=2450808 2024-04-17T04:45:11.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.728+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=WQRY16TOO4L6 type=del size=2305321 2024-04-17T04:45:11.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.728+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=R7B03K1NM916 type=del 2024-04-17T04:45:11.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~FQAKCT56K548.0 size=714211 2024-04-17T04:45:11.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.736+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=FQAKCT56K548 tag=HJP8JN3Y3QG1 type=write size=1762787 2024-04-17T04:45:11.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~MYM6DFFP85TU.0 size=1048576 2024-04-17T04:45:11.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.740+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~MYM6DFFP85TU.1 size=1048576 2024-04-17T04:45:11.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.740+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~MYM6DFFP85TU.2 size=382397 2024-04-17T04:45:11.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.748+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=MYM6DFFP85TU tag=05Z1P8NCIJXB type=write size=3528125 2024-04-17T04:45:11.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.752+0000 7fd8ef2218c0 1 > prepared part key=_multipart_G03VLS09F3BT.2~O6OBFGASVNMG.0 size=271334 2024-04-17T04:45:11.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.752+0000 7fd8ef2218c0 1 > prepared multipart upload key=G03VLS09F3BT upload=O6OBFGASVNMG tag=KHFFERLF1AUH type=write size=1319910 2024-04-17T04:45:11.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.756+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=1LM402C26SZS type=del 2024-04-17T04:45:11.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.760+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~63CEX3ZN96MX.0 size=1048576 2024-04-17T04:45:11.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~63CEX3ZN96MX.1 size=1048576 2024-04-17T04:45:11.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.764+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~63CEX3ZN96MX.2 size=721446 2024-04-17T04:45:11.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.764+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=63CEX3ZN96MX tag=A3J016CHKI27 type=write size=3867174 2024-04-17T04:45:11.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~OZOKJOHU5N23.0 size=1048576 2024-04-17T04:45:11.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.772+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~OZOKJOHU5N23.1 size=1048576 2024-04-17T04:45:11.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.772+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~OZOKJOHU5N23.2 size=650405 2024-04-17T04:45:11.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.780+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=OZOKJOHU5N23 tag=YP9KCQIMWS77 type=write size=3796133 2024-04-17T04:45:11.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.784+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~P4TCOJI3SYM0.0 size=1048576 2024-04-17T04:45:11.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~P4TCOJI3SYM0.1 size=1048576 2024-04-17T04:45:11.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~P4TCOJI3SYM0.2 size=383897 2024-04-17T04:45:11.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.788+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=P4TCOJI3SYM0 tag=8QBDH0VDQOGQ type=write size=3529625 2024-04-17T04:45:11.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.792+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=GAA8ZPNXH6W0 type=write size=486289 2024-04-17T04:45:11.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.800+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~U48F3W9XBAPB.0 size=833043 2024-04-17T04:45:11.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.800+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=U48F3W9XBAPB tag=ONGWF4GP2M54 type=write size=1881619 2024-04-17T04:45:11.803 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.808+0000 7fd8ef2218c0 1 < suggested update operation key=7BV3V2XMLSPH tag=YXIPZC3C8IH6 type=write size=190153 2024-04-17T04:45:11.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.812+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=SBH546M4MW3X type=del size=2582062 2024-04-17T04:45:11.808 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.812+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=CZQRE482PXCV type=del 2024-04-17T04:45:11.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.816+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=2RS93F41V51N type=del size=3970230 2024-04-17T04:45:11.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.820+0000 7fd8ef2218c0 1 < canceled operation key=UYCKMXE6LBUH tag=H388SYSSBM6Q type=del size=3248208 2024-04-17T04:45:11.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.820+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=EAGA5FDTHR2P type=del size=3885665 2024-04-17T04:45:11.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~46PX50MB1GDQ.0 size=12477 2024-04-17T04:45:11.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.828+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=46PX50MB1GDQ tag=JJJ9ZFSNFC8G type=write size=1061053 2024-04-17T04:45:11.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.832+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=CUCD3307XLDS type=write size=479379 2024-04-17T04:45:11.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.836+0000 7fd8ef2218c0 1 < suggested update operation key=3OFQH4KVYN18 tag=8EJSRH7TBK93 type=del size=3609456 2024-04-17T04:45:11.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.840+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=TXFQZQ0L7MPN type=del size=512302 2024-04-17T04:45:11.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.844+0000 7fd8ef2218c0 -1 < failed to complete operation key=7BV3V2XMLSPH tag=EW6FDNNJ9XRX type=write size=612331: (22) Invalid argument 2024-04-17T04:45:11.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.848+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=MR1T0L0M7WRM type=del size=318984 2024-04-17T04:45:11.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.848+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=XSU8PSEMFCU5 type=del 2024-04-17T04:45:11.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.852+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=NVS4SJT3L577 type=del size=3769724 2024-04-17T04:45:11.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.856+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~TRHAQDK8FYEY.0 size=33140 2024-04-17T04:45:11.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.860+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=TRHAQDK8FYEY tag=XIWYTQW1VML1 type=write size=1081716 2024-04-17T04:45:11.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~ZZLH653B2QW3.0 size=1048576 2024-04-17T04:45:11.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~ZZLH653B2QW3.1 size=1048576 2024-04-17T04:45:11.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~ZZLH653B2QW3.2 size=706603 2024-04-17T04:45:11.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.868+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=ZZLH653B2QW3 tag=G4PXZZFCF1YG type=write size=3852331 2024-04-17T04:45:11.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.872+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=V5NLEJNSOME2 type=del 2024-04-17T04:45:11.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.876+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~DCC4F6YU1546.0 size=1048576 2024-04-17T04:45:11.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.880+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~DCC4F6YU1546.1 size=210864 2024-04-17T04:45:11.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.880+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=DCC4F6YU1546 tag=XDCCO9SECUHF type=write size=2308016 2024-04-17T04:45:11.876 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.880+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=HMDD3KI0POHB type=del 2024-04-17T04:45:11.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.884+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=LOBRY19EAJG8 type=del size=3273631 2024-04-17T04:45:11.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.888+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=WQRY16TOO4L6 type=del size=2305321 2024-04-17T04:45:11.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.892+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=XJK2BUJKL8V9 type=del size=2414627 2024-04-17T04:45:11.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~FQAKCT56K548.0 size=714211 2024-04-17T04:45:11.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.900+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=FQAKCT56K548 tag=HJP8JN3Y3QG1 type=write size=1762787 2024-04-17T04:45:11.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~3KFME9KDB3MD.0 size=1048576 2024-04-17T04:45:11.900 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~3KFME9KDB3MD.1 size=1048576 2024-04-17T04:45:11.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.908+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~3KFME9KDB3MD.2 size=355659 2024-04-17T04:45:11.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.908+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=3KFME9KDB3MD tag=OVWIT1G60GF2 type=write size=3501387 2024-04-17T04:45:11.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.908+0000 7fd8ef2218c0 1 < canceled part key=_multipart_G03VLS09F3BT.2~O6OBFGASVNMG.0 size=271334 2024-04-17T04:45:11.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.912+0000 7fd8ef2218c0 1 < canceled multipart upload key=G03VLS09F3BT upload=O6OBFGASVNMG tag=KHFFERLF1AUH type=write size=1319910 2024-04-17T04:45:11.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.916+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=WNSLE0WNUB6J type=write size=255052 2024-04-17T04:45:11.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.916+0000 7fd8ef2218c0 1 < canceled part key=_multipart_82D6YWKJ1TFQ.2~63CEX3ZN96MX.0 size=1048576 2024-04-17T04:45:11.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~63CEX3ZN96MX.1 size=1048576 2024-04-17T04:45:11.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.920+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~63CEX3ZN96MX.2 size=721446 2024-04-17T04:45:11.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.924+0000 7fd8ef2218c0 1 < canceled multipart upload key=82D6YWKJ1TFQ upload=63CEX3ZN96MX tag=A3J016CHKI27 type=write size=3867174 2024-04-17T04:45:11.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.928+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=SAELYILNCOOD type=del size=3021354 2024-04-17T04:45:11.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.932+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~P4TCOJI3SYM0.0 size=1048576 2024-04-17T04:45:11.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~P4TCOJI3SYM0.1 size=1048576 2024-04-17T04:45:11.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.936+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~P4TCOJI3SYM0.2 size=383897 2024-04-17T04:45:11.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.940+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=P4TCOJI3SYM0 tag=8QBDH0VDQOGQ type=write size=3529625 2024-04-17T04:45:11.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.948+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~4MHJPRFQ1UX6.0 size=1048576 2024-04-17T04:45:11.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.948+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~4MHJPRFQ1UX6.1 size=1048576 2024-04-17T04:45:11.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.952+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UYCKMXE6LBUH.2~4MHJPRFQ1UX6.2 size=681775 2024-04-17T04:45:11.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.952+0000 7fd8ef2218c0 1 > prepared multipart upload key=UYCKMXE6LBUH upload=4MHJPRFQ1UX6 tag=PCRJWJJK3FB3 type=write size=3827503 2024-04-17T04:45:11.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.956+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~U48F3W9XBAPB.0 size=833043 2024-04-17T04:45:11.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.960+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=U48F3W9XBAPB tag=ONGWF4GP2M54 type=write size=1881619 2024-04-17T04:45:11.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.960+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=PUI9PPYIJXRB type=del size=2122928 2024-04-17T04:45:11.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.968+0000 7fd8ef2218c0 1 < suggested update operation key=7BV3V2XMLSPH tag=SBH546M4MW3X type=del size=2582062 2024-04-17T04:45:11.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.972+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~683DR2HOU1IW.0 size=666446 2024-04-17T04:45:11.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.972+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=683DR2HOU1IW tag=Y6RPA11GM1D3 type=write size=1715022 2024-04-17T04:45:11.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.976+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=2RS93F41V51N type=del 2024-04-17T04:45:11.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.980+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=4OAHWXE6UK3L type=del size=3159057 2024-04-17T04:45:11.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.984+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=EAGA5FDTHR2P type=del 2024-04-17T04:45:11.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.984+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=Q2QHI78X7Q0K type=del size=3727121 2024-04-17T04:45:11.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.988+0000 7fd8ef2218c0 1 < completed write operation key=UEWLJ3E9ODUL tag=CUCD3307XLDS type=write size=479379 2024-04-17T04:45:11.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.992+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=SQY2R2D7AJD3 type=del size=3170021 2024-04-17T04:45:11.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.996+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=TXFQZQ0L7MPN type=del 2024-04-17T04:45:11.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:11.996+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=MZ4AX02Q62J4 type=del size=3517562 2024-04-17T04:45:11.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.000+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=MR1T0L0M7WRM type=del 2024-04-17T04:45:12.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~O6TOFD2T0JZ9.0 size=1048576 2024-04-17T04:45:12.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~O6TOFD2T0JZ9.1 size=1048576 2024-04-17T04:45:12.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.012+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~O6TOFD2T0JZ9.2 size=343241 2024-04-17T04:45:12.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.012+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=O6TOFD2T0JZ9 tag=3AZZALLBS1IH type=write size=3488969 2024-04-17T04:45:12.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.016+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=NVS4SJT3L577 type=del 2024-04-17T04:45:12.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.016+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=Q665X45ZEI6D type=del size=728342 2024-04-17T04:45:12.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.020+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~ZZLH653B2QW3.0 size=1048576 2024-04-17T04:45:12.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.024+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~ZZLH653B2QW3.1 size=1048576 2024-04-17T04:45:12.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.024+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~ZZLH653B2QW3.2 size=706603 2024-04-17T04:45:12.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.032+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=ZZLH653B2QW3 tag=G4PXZZFCF1YG type=write size=3852331 2024-04-17T04:45:12.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.032+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=78M2YXPV4A2C type=write size=862902 2024-04-17T04:45:12.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~DCC4F6YU1546.0 size=1048576 2024-04-17T04:45:12.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~DCC4F6YU1546.1 size=210864 2024-04-17T04:45:12.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.044+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=DCC4F6YU1546 tag=XDCCO9SECUHF type=write size=2308016 2024-04-17T04:45:12.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.048+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=KL9R0TR9I34D type=del size=2810767 2024-04-17T04:45:12.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.052+0000 7fd8ef2218c0 1 < canceled operation key=509DZ9DBJWC5 tag=LOBRY19EAJG8 type=del size=3273631 2024-04-17T04:45:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.056+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=VF341V85KK8J type=del size=4073024 2024-04-17T04:45:12.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.060+0000 7fd8ef2218c0 1 < suggested update operation key=M3U8L7M8JKG9 tag=XJK2BUJKL8V9 type=del size=2414627 2024-04-17T04:45:12.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.064+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=KZ8HGJ84K9MW type=write size=639565 2024-04-17T04:45:12.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~3KFME9KDB3MD.0 size=1048576 2024-04-17T04:45:12.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.072+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~3KFME9KDB3MD.1 size=1048576 2024-04-17T04:45:12.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.076+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~3KFME9KDB3MD.2 size=355659 2024-04-17T04:45:12.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.080+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=3KFME9KDB3MD tag=OVWIT1G60GF2 type=write size=3501387 2024-04-17T04:45:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.088+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~YC8Q3J9XCEWC.0 size=1048576 2024-04-17T04:45:12.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.088+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~YC8Q3J9XCEWC.1 size=632532 2024-04-17T04:45:12.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.092+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=YC8Q3J9XCEWC tag=K4U0RS7G1Z8T type=write size=2729684 2024-04-17T04:45:12.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.092+0000 7fd8ef2218c0 1 < completed write operation key=5K50O96P7OGD tag=WNSLE0WNUB6J type=write size=255052 2024-04-17T04:45:12.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.096+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=HVEDTPV5GA8E type=del size=3056078 2024-04-17T04:45:12.095 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.100+0000 7fd8ef2218c0 1 < canceled operation key=5ZULD0PHLS4V tag=SAELYILNCOOD type=del size=3021354 2024-04-17T04:45:12.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~LBXOE26QAM0B.0 size=329042 2024-04-17T04:45:12.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.104+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=LBXOE26QAM0B tag=YNAN3LA1GJ2G type=write size=1377618 2024-04-17T04:45:12.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~4MHJPRFQ1UX6.0 size=1048576 2024-04-17T04:45:12.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.108+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UYCKMXE6LBUH.2~4MHJPRFQ1UX6.1 size=1048576 2024-04-17T04:45:12.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.108+0000 7fd8ef2218c0 1 < completed part key=_multipart_UYCKMXE6LBUH.2~4MHJPRFQ1UX6.2 size=681775 2024-04-17T04:45:12.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.116+0000 7fd8ef2218c0 1 < canceled multipart upload key=UYCKMXE6LBUH upload=4MHJPRFQ1UX6 tag=PCRJWJJK3FB3 type=write size=3827503 2024-04-17T04:45:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.120+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~QQYNP72QB34F.0 size=1048576 2024-04-17T04:45:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~QQYNP72QB34F.1 size=1048576 2024-04-17T04:45:12.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~QQYNP72QB34F.2 size=686186 2024-04-17T04:45:12.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.124+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=QQYNP72QB34F tag=X7M7B3U22T80 type=write size=3831914 2024-04-17T04:45:12.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.128+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=PUI9PPYIJXRB type=del 2024-04-17T04:45:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.132+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~U3V3IYNDUW23.0 size=1048576 2024-04-17T04:45:12.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.136+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~U3V3IYNDUW23.1 size=1048576 2024-04-17T04:45:12.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.136+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~U3V3IYNDUW23.2 size=557129 2024-04-17T04:45:12.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.136+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=U3V3IYNDUW23 tag=P44ZFFISFEYQ type=write size=3702857 2024-04-17T04:45:12.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.140+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~683DR2HOU1IW.0 size=666446 2024-04-17T04:45:12.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.144+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=683DR2HOU1IW tag=Y6RPA11GM1D3 type=write size=1715022 2024-04-17T04:45:12.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.152+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~FCF1AI0CA895.0 size=921050 2024-04-17T04:45:12.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.152+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=FCF1AI0CA895 tag=DEYR4Z0STCZV type=write size=1969626 2024-04-17T04:45:12.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.152+0000 7fd8ef2218c0 1 < canceled operation key=MS3T476JOEON tag=4OAHWXE6UK3L type=del size=3159057 2024-04-17T04:45:12.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.156+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=XHS59RJ53HWO type=del size=434967 2024-04-17T04:45:12.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.160+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=Q2QHI78X7Q0K type=del 2024-04-17T04:45:12.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.164+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=0KG3LV9VBQW7 type=del size=1406244 2024-04-17T04:45:12.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.164+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=SQY2R2D7AJD3 type=del 2024-04-17T04:45:12.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.168+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=1SW9SAVMFYHI type=del size=63434 2024-04-17T04:45:12.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.172+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=MZ4AX02Q62J4 type=del 2024-04-17T04:45:12.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.176+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=59K6OE5GRFEM type=del size=3987614 2024-04-17T04:45:12.174 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.180+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~O6TOFD2T0JZ9.0 size=1048576 2024-04-17T04:45:12.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.180+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~O6TOFD2T0JZ9.1 size=1048576 2024-04-17T04:45:12.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.184+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~O6TOFD2T0JZ9.2 size=343241 2024-04-17T04:45:12.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.188+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=O6TOFD2T0JZ9 tag=3AZZALLBS1IH type=write size=3488969 2024-04-17T04:45:12.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.192+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=5IO0991MBFE5 type=del size=4005673 2024-04-17T04:45:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.196+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=Q665X45ZEI6D type=del 2024-04-17T04:45:12.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.196+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=XX16TFDH60ZA type=del size=3661099 2024-04-17T04:45:12.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.200+0000 7fd8ef2218c0 1 < completed write operation key=5K50O96P7OGD tag=78M2YXPV4A2C type=write size=862902 2024-04-17T04:45:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~U0X749JY92XZ.0 size=1048576 2024-04-17T04:45:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~U0X749JY92XZ.1 size=1048576 2024-04-17T04:45:12.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.212+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~U0X749JY92XZ.2 size=161201 2024-04-17T04:45:12.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.212+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=U0X749JY92XZ tag=8Y8CULV6PKUF type=write size=3306929 2024-04-17T04:45:12.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.216+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=KL9R0TR9I34D type=del 2024-04-17T04:45:12.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~OSAEYYG7IA6L.0 size=1048576 2024-04-17T04:45:12.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~OSAEYYG7IA6L.1 size=522704 2024-04-17T04:45:12.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.220+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=OSAEYYG7IA6L tag=0T9Q0D842YY2 type=write size=2619856 2024-04-17T04:45:12.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.224+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=VF341V85KK8J type=del 2024-04-17T04:45:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.228+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~ATS2BKR1NLRA.0 size=1048576 2024-04-17T04:45:12.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.232+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~ATS2BKR1NLRA.1 size=499861 2024-04-17T04:45:12.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.232+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=ATS2BKR1NLRA tag=A40QZK30NO7J type=write size=2597013 2024-04-17T04:45:12.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.236+0000 7fd8ef2218c0 1 < suggested update operation key=6219DPY72LWC tag=KZ8HGJ84K9MW type=write size=639565 2024-04-17T04:45:12.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.240+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=QNYRW537FI6Y type=write size=244301 2024-04-17T04:45:12.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.240+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5ZULD0PHLS4V.2~YC8Q3J9XCEWC.0 size=1048576 2024-04-17T04:45:12.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.244+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~YC8Q3J9XCEWC.1 size=632532 2024-04-17T04:45:12.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.248+0000 7fd8ef2218c0 1 < canceled multipart upload key=5ZULD0PHLS4V upload=YC8Q3J9XCEWC tag=K4U0RS7G1Z8T type=write size=2729684 2024-04-17T04:45:12.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.252+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=LE1RJXUNQ5VU type=del size=3918886 2024-04-17T04:45:12.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.256+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=HVEDTPV5GA8E type=del 2024-04-17T04:45:12.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.260+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=W9LT6PDX7XG2 type=del size=2434082 2024-04-17T04:45:12.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~LBXOE26QAM0B.0 size=329042 2024-04-17T04:45:12.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.268+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=LBXOE26QAM0B tag=YNAN3LA1GJ2G type=write size=1377618 2024-04-17T04:45:12.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.272+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=5OM2ISHNG9YH type=del size=467084 2024-04-17T04:45:12.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~QQYNP72QB34F.0 size=1048576 2024-04-17T04:45:12.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.276+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~QQYNP72QB34F.1 size=1048576 2024-04-17T04:45:12.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.280+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~QQYNP72QB34F.2 size=686186 2024-04-17T04:45:12.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.284+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=QQYNP72QB34F tag=X7M7B3U22T80 type=write size=3831914 2024-04-17T04:45:12.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_509DZ9DBJWC5.2~39B0XJKPXB7P.0 size=258373 2024-04-17T04:45:12.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.288+0000 7fd8ef2218c0 1 > prepared multipart upload key=509DZ9DBJWC5 upload=39B0XJKPXB7P tag=G22OG7XXLT0Z type=write size=1306949 2024-04-17T04:45:12.288 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.292+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~U3V3IYNDUW23.0 size=1048576 2024-04-17T04:45:12.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~U3V3IYNDUW23.1 size=1048576 2024-04-17T04:45:12.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.296+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~U3V3IYNDUW23.2 size=557129 2024-04-17T04:45:12.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.304+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=U3V3IYNDUW23 tag=P44ZFFISFEYQ type=write size=3702857 2024-04-17T04:45:12.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.308+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=LZPZFON8TFEH type=del size=3084186 2024-04-17T04:45:12.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~FCF1AI0CA895.0 size=921050 2024-04-17T04:45:12.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.316+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=FCF1AI0CA895 tag=DEYR4Z0STCZV type=write size=1969626 2024-04-17T04:45:12.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.320+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=IWLJF38VKD33 type=del size=2650975 2024-04-17T04:45:12.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.324+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=XHS59RJ53HWO type=del 2024-04-17T04:45:12.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.324+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=IPXW0AROOY2B type=write size=382051 2024-04-17T04:45:12.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.328+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=0KG3LV9VBQW7 type=del 2024-04-17T04:45:12.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.332+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=AHXR4XHQNRAQ type=del size=568556 2024-04-17T04:45:12.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.336+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=1SW9SAVMFYHI type=del 2024-04-17T04:45:12.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.340+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~IJAPXXKJ3DJ9.0 size=1048576 2024-04-17T04:45:12.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.344+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~IJAPXXKJ3DJ9.1 size=112565 2024-04-17T04:45:12.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.344+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=IJAPXXKJ3DJ9 tag=J6OP3L1XBP7C type=write size=2209717 2024-04-17T04:45:12.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.352+0000 7fd8ef2218c0 1 < suggested remove operation key=WCIJICR0FT2C tag=59K6OE5GRFEM type=del 2024-04-17T04:45:12.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~4DXAFNO7Y1PR.0 size=35780 2024-04-17T04:45:12.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.356+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=4DXAFNO7Y1PR tag=LDDV0E1VACOU type=write size=1084356 2024-04-17T04:45:12.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.364+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=5IO0991MBFE5 type=del 2024-04-17T04:45:12.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.368+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=B2571X3CS4MI type=write size=845209 2024-04-17T04:45:12.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.372+0000 7fd8ef2218c0 1 < suggested remove operation key=UYCKMXE6LBUH tag=XX16TFDH60ZA type=del 2024-04-17T04:45:12.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.376+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=PWKI9Q4QFWW2 type=del size=1669720 2024-04-17T04:45:12.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~U0X749JY92XZ.0 size=1048576 2024-04-17T04:45:12.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~U0X749JY92XZ.1 size=1048576 2024-04-17T04:45:12.379 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.384+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~U0X749JY92XZ.2 size=161201 2024-04-17T04:45:12.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.388+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=U0X749JY92XZ tag=8Y8CULV6PKUF type=write size=3306929 2024-04-17T04:45:12.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~Q24Z9EPJGA1Q.0 size=1027178 2024-04-17T04:45:12.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.396+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=Q24Z9EPJGA1Q tag=UFG87B2A73U3 type=write size=2075754 2024-04-17T04:45:12.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.400+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~OSAEYYG7IA6L.0 size=1048576 2024-04-17T04:45:12.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.400+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~OSAEYYG7IA6L.1 size=522704 2024-04-17T04:45:12.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.404+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=OSAEYYG7IA6L tag=0T9Q0D842YY2 type=write size=2619856 2024-04-17T04:45:12.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.408+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=ORA4E6W0FYPV type=del size=320573 2024-04-17T04:45:12.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.408+0000 7fd8ef2218c0 1 < canceled part key=_multipart_AE4CYL334ONO.2~ATS2BKR1NLRA.0 size=1048576 2024-04-17T04:45:12.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.412+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~ATS2BKR1NLRA.1 size=499861 2024-04-17T04:45:12.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.416+0000 7fd8ef2218c0 1 < canceled multipart upload key=AE4CYL334ONO upload=ATS2BKR1NLRA tag=A40QZK30NO7J type=write size=2597013 2024-04-17T04:45:12.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.420+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=G1QWZ3DGX1LT type=del size=1202225 2024-04-17T04:45:12.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.424+0000 7fd8ef2218c0 1 < canceled operation key=2IY8DB37FI9H tag=QNYRW537FI6Y type=write size=244301 2024-04-17T04:45:12.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.428+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=VL7RPXYXXIFL type=del size=2073019 2024-04-17T04:45:12.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.432+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=LE1RJXUNQ5VU type=del 2024-04-17T04:45:12.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~U7A3AQQ31ETX.0 size=1048576 2024-04-17T04:45:12.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~U7A3AQQ31ETX.1 size=1048576 2024-04-17T04:45:12.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.444+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~U7A3AQQ31ETX.2 size=651543 2024-04-17T04:45:12.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.444+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=U7A3AQQ31ETX tag=NTWVFQ6T34J0 type=write size=3797271 2024-04-17T04:45:12.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.448+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=W9LT6PDX7XG2 type=del 2024-04-17T04:45:12.445 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.448+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=Y0J3HVZLXH7E type=del size=2347456 2024-04-17T04:45:12.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.456+0000 7fd8ef2218c0 1 < suggested update operation key=B63EZS0LAC30 tag=5OM2ISHNG9YH type=del size=467084 2024-04-17T04:45:12.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.464+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~CXSM9DU1F21B.0 size=1048576 2024-04-17T04:45:12.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.464+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~CXSM9DU1F21B.1 size=4974 2024-04-17T04:45:12.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.464+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=CXSM9DU1F21B tag=F0PXDU228U59 type=write size=2102126 2024-04-17T04:45:12.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.468+0000 7fd8ef2218c0 1 < completed part key=_multipart_509DZ9DBJWC5.2~39B0XJKPXB7P.0 size=258373 2024-04-17T04:45:12.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.476+0000 7fd8ef2218c0 1 < completed multipart upload key=509DZ9DBJWC5 upload=39B0XJKPXB7P tag=G22OG7XXLT0Z type=write size=1306949 2024-04-17T04:45:12.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.480+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~VSY1DJIY63FX.0 size=1048576 2024-04-17T04:45:12.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.484+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~VSY1DJIY63FX.1 size=1048576 2024-04-17T04:45:12.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.488+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~VSY1DJIY63FX.2 size=651549 2024-04-17T04:45:12.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.488+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=VSY1DJIY63FX tag=GMHWKAXHQDGH type=write size=3797277 2024-04-17T04:45:12.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.488+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=LZPZFON8TFEH type=del 2024-04-17T04:45:12.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.492+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=Z3Y9KZDRWNDD type=write size=622963 2024-04-17T04:45:12.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.496+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=IWLJF38VKD33 type=del 2024-04-17T04:45:12.495 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.500+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=PEG9PZTF4HM0 type=del size=2703349 2024-04-17T04:45:12.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.504+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=IPXW0AROOY2B type=write size=382051 2024-04-17T04:45:12.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.508+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~C35ARS025802.0 size=1048576 2024-04-17T04:45:12.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.512+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~C35ARS025802.1 size=115160 2024-04-17T04:45:12.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.512+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=C35ARS025802 tag=S87PGFD8H6M5 type=write size=2212312 2024-04-17T04:45:12.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.516+0000 7fd8ef2218c0 -1 < failed to complete operation key=B63EZS0LAC30 tag=AHXR4XHQNRAQ type=del size=568556: (22) Invalid argument 2024-04-17T04:45:12.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.520+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=K2C7WCQLH3UV type=write size=358577 2024-04-17T04:45:12.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.520+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UEWLJ3E9ODUL.2~IJAPXXKJ3DJ9.0 size=1048576 2024-04-17T04:45:12.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.520+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~IJAPXXKJ3DJ9.1 size=112565 2024-04-17T04:45:12.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.528+0000 7fd8ef2218c0 1 < canceled multipart upload key=UEWLJ3E9ODUL upload=IJAPXXKJ3DJ9 tag=J6OP3L1XBP7C type=write size=2209717 2024-04-17T04:45:12.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.532+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~FW3R7OS9L66R.0 size=1048576 2024-04-17T04:45:12.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~FW3R7OS9L66R.1 size=754994 2024-04-17T04:45:12.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.536+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=FW3R7OS9L66R tag=EZY0XGPF4PYV type=write size=2852146 2024-04-17T04:45:12.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~4DXAFNO7Y1PR.0 size=35780 2024-04-17T04:45:12.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.544+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=4DXAFNO7Y1PR tag=LDDV0E1VACOU type=write size=1084356 2024-04-17T04:45:12.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.548+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=65LNMPJPOSLP type=del size=917514 2024-04-17T04:45:12.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.552+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=B2571X3CS4MI type=write size=845209 2024-04-17T04:45:12.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~W3CPHO8HZ977.0 size=1048576 2024-04-17T04:45:12.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.560+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~W3CPHO8HZ977.1 size=1048576 2024-04-17T04:45:12.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.564+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~W3CPHO8HZ977.2 size=66695 2024-04-17T04:45:12.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.564+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=W3CPHO8HZ977 tag=JCNVQ8XX5OSR type=write size=3212423 2024-04-17T04:45:12.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.568+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=PWKI9Q4QFWW2 type=del 2024-04-17T04:45:12.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~RT2A9I7KLDM3.0 size=1048576 2024-04-17T04:45:12.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.576+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~RT2A9I7KLDM3.1 size=1048576 2024-04-17T04:45:12.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~RT2A9I7KLDM3.2 size=418180 2024-04-17T04:45:12.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.580+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=RT2A9I7KLDM3 tag=DBODOC1BLDLZ type=write size=3563908 2024-04-17T04:45:12.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.580+0000 7fd8ef2218c0 1 < canceled part key=_multipart_UNOVYXDS5JEB.2~Q24Z9EPJGA1Q.0 size=1027178 2024-04-17T04:45:12.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.588+0000 7fd8ef2218c0 1 < canceled multipart upload key=UNOVYXDS5JEB upload=Q24Z9EPJGA1Q tag=UFG87B2A73U3 type=write size=2075754 2024-04-17T04:45:12.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.592+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~PJ39JUWMEBW0.0 size=1048576 2024-04-17T04:45:12.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.596+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~PJ39JUWMEBW0.1 size=435973 2024-04-17T04:45:12.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.596+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=PJ39JUWMEBW0 tag=ROPR0PLBJCMZ type=write size=2533125 2024-04-17T04:45:12.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.600+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=ORA4E6W0FYPV type=del 2024-04-17T04:45:12.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.600+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=7CMOW7NR2XJD type=del size=4193317 2024-04-17T04:45:12.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.608+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=G1QWZ3DGX1LT type=del 2024-04-17T04:45:12.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~1J2PBM8J9ZKQ.0 size=1048576 2024-04-17T04:45:12.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.616+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~1J2PBM8J9ZKQ.1 size=937492 2024-04-17T04:45:12.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.616+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=1J2PBM8J9ZKQ tag=HM4NQVB11KBT type=write size=3034644 2024-04-17T04:45:12.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.620+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=VL7RPXYXXIFL type=del 2024-04-17T04:45:12.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.624+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=E220EM8O5KQH type=write size=19387 2024-04-17T04:45:12.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.628+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~U7A3AQQ31ETX.0 size=1048576 2024-04-17T04:45:12.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.632+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~U7A3AQQ31ETX.1 size=1048576 2024-04-17T04:45:12.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.636+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~U7A3AQQ31ETX.2 size=651543 2024-04-17T04:45:12.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.640+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=U7A3AQQ31ETX tag=NTWVFQ6T34J0 type=write size=3797271 2024-04-17T04:45:12.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~28NOVU4MKP0K.0 size=709757 2024-04-17T04:45:12.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.644+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=28NOVU4MKP0K tag=TP764Y0WPUJQ type=write size=1758333 2024-04-17T04:45:12.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.648+0000 7fd8ef2218c0 1 < canceled operation key=N11Y8FPN9XES tag=Y0J3HVZLXH7E type=del size=2347456 2024-04-17T04:45:12.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.652+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=OCEEXPA4MTGI type=del size=3525272 2024-04-17T04:45:12.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.656+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~CXSM9DU1F21B.0 size=1048576 2024-04-17T04:45:12.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.660+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~CXSM9DU1F21B.1 size=4974 2024-04-17T04:45:12.660 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.664+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=CXSM9DU1F21B tag=F0PXDU228U59 type=write size=2102126 2024-04-17T04:45:12.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.668+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=V9QOUSJALBFY type=del size=3670253 2024-04-17T04:45:12.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~VSY1DJIY63FX.0 size=1048576 2024-04-17T04:45:12.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~VSY1DJIY63FX.1 size=1048576 2024-04-17T04:45:12.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~VSY1DJIY63FX.2 size=651549 2024-04-17T04:45:12.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.684+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=VSY1DJIY63FX tag=GMHWKAXHQDGH type=write size=3797277 2024-04-17T04:45:12.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.692+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6WSZ5UL0V1Y1.2~DBA1PRSIRUCB.0 size=221028 2024-04-17T04:45:12.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.692+0000 7fd8ef2218c0 1 > prepared multipart upload key=6WSZ5UL0V1Y1 upload=DBA1PRSIRUCB tag=XEAK27DD2Z4C type=write size=1269604 2024-04-17T04:45:12.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.696+0000 7fd8ef2218c0 1 < completed write operation key=B63EZS0LAC30 tag=Z3Y9KZDRWNDD type=write size=622963 2024-04-17T04:45:12.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.700+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~3HHR4563HNOW.0 size=544225 2024-04-17T04:45:12.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.700+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=3HHR4563HNOW tag=BNGFZXJIGXBR type=write size=1592801 2024-04-17T04:45:12.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.704+0000 7fd8ef2218c0 1 < canceled operation key=UYCKMXE6LBUH tag=PEG9PZTF4HM0 type=del size=2703349 2024-04-17T04:45:12.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.704+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=OG9ZRWYZCMUH type=del size=4120908 2024-04-17T04:45:12.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.708+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~C35ARS025802.0 size=1048576 2024-04-17T04:45:12.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.712+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~C35ARS025802.1 size=115160 2024-04-17T04:45:12.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.716+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=C35ARS025802 tag=S87PGFD8H6M5 type=write size=2212312 2024-04-17T04:45:12.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.724+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4XN2TWPITM0P.0 size=1048576 2024-04-17T04:45:12.721 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.724+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4XN2TWPITM0P.1 size=1048576 2024-04-17T04:45:12.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.732+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~4XN2TWPITM0P.2 size=254536 2024-04-17T04:45:12.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.732+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=4XN2TWPITM0P tag=554EI90ECP7R type=write size=3400264 2024-04-17T04:45:12.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.732+0000 7fd8ef2218c0 1 < completed write operation key=6219DPY72LWC tag=K2C7WCQLH3UV type=write size=358577 2024-04-17T04:45:12.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.740+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~OEZL0BH0JXL3.0 size=1048576 2024-04-17T04:45:12.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~OEZL0BH0JXL3.1 size=1048576 2024-04-17T04:45:12.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.744+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~OEZL0BH0JXL3.2 size=430169 2024-04-17T04:45:12.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.744+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=OEZL0BH0JXL3 tag=OXI571GU38CU type=write size=3575897 2024-04-17T04:45:12.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.748+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~FW3R7OS9L66R.0 size=1048576 2024-04-17T04:45:12.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.752+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~FW3R7OS9L66R.1 size=754994 2024-04-17T04:45:12.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.756+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=FW3R7OS9L66R tag=EZY0XGPF4PYV type=write size=2852146 2024-04-17T04:45:12.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.760+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=ALO0L7ORKIBE type=del size=1505937 2024-04-17T04:45:12.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.764+0000 7fd8ef2218c0 1 < completed delete operation key=B63EZS0LAC30 tag=65LNMPJPOSLP type=del 2024-04-17T04:45:12.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.768+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=7QYA3OJ5Z04Y type=del size=2535957 2024-04-17T04:45:12.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.776+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~W3CPHO8HZ977.0 size=1048576 2024-04-17T04:45:12.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.776+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~W3CPHO8HZ977.1 size=1048576 2024-04-17T04:45:12.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.780+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~W3CPHO8HZ977.2 size=66695 2024-04-17T04:45:12.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.788+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=W3CPHO8HZ977 tag=JCNVQ8XX5OSR type=write size=3212423 2024-04-17T04:45:12.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.796+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~W8MDJ8CRSDXT.0 size=1016369 2024-04-17T04:45:12.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.796+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=W8MDJ8CRSDXT tag=XIXCGQ436M9U type=write size=2064945 2024-04-17T04:45:12.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~RT2A9I7KLDM3.0 size=1048576 2024-04-17T04:45:12.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.800+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~RT2A9I7KLDM3.1 size=1048576 2024-04-17T04:45:12.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.804+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~RT2A9I7KLDM3.2 size=418180 2024-04-17T04:45:12.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.812+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=RT2A9I7KLDM3 tag=DBODOC1BLDLZ type=write size=3563908 2024-04-17T04:45:12.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~NPWEP6W475ID.0 size=10963 2024-04-17T04:45:12.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.816+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=NPWEP6W475ID tag=B24SCUYL0RTP type=write size=1059539 2024-04-17T04:45:12.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.820+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~PJ39JUWMEBW0.0 size=1048576 2024-04-17T04:45:12.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~PJ39JUWMEBW0.1 size=435973 2024-04-17T04:45:12.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.832+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=PJ39JUWMEBW0 tag=ROPR0PLBJCMZ type=write size=2533125 2024-04-17T04:45:12.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.836+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~VAKCH7QFRZM4.0 size=1048576 2024-04-17T04:45:12.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.840+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~VAKCH7QFRZM4.1 size=1048576 2024-04-17T04:45:12.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~VAKCH7QFRZM4.2 size=123062 2024-04-17T04:45:12.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=VAKCH7QFRZM4 tag=3OGK62EKV360 type=write size=3268790 2024-04-17T04:45:12.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.844+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=7CMOW7NR2XJD type=del 2024-04-17T04:45:12.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.848+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=29PTJDL08JEG type=write size=942290 2024-04-17T04:45:12.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.852+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~1J2PBM8J9ZKQ.0 size=1048576 2024-04-17T04:45:12.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.856+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~1J2PBM8J9ZKQ.1 size=937492 2024-04-17T04:45:12.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.864+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=1J2PBM8J9ZKQ tag=HM4NQVB11KBT type=write size=3034644 2024-04-17T04:45:12.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~A0SEF3OJFHOI.0 size=1048576 2024-04-17T04:45:12.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~A0SEF3OJFHOI.1 size=976938 2024-04-17T04:45:12.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.872+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=A0SEF3OJFHOI tag=7O3PPZAXTNUG type=write size=3074090 2024-04-17T04:45:12.875 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.880+0000 7fd8ef2218c0 1 < suggested update operation key=2IY8DB37FI9H tag=E220EM8O5KQH type=write size=19387 2024-04-17T04:45:12.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.884+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=MEQ5K3TIF8A2 type=del size=788604 2024-04-17T04:45:12.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~28NOVU4MKP0K.0 size=709757 2024-04-17T04:45:12.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.892+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=28NOVU4MKP0K tag=TP764Y0WPUJQ type=write size=1758333 2024-04-17T04:45:12.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~24UUDS00R63S.0 size=562030 2024-04-17T04:45:12.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.900+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=24UUDS00R63S tag=KLS50XN8XB05 type=write size=1610606 2024-04-17T04:45:12.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.908+0000 7fd8ef2218c0 1 < suggested remove operation key=UNOVYXDS5JEB tag=OCEEXPA4MTGI type=del 2024-04-17T04:45:12.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.912+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~KEBIJ7N5VCWN.0 size=761486 2024-04-17T04:45:12.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.912+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=KEBIJ7N5VCWN tag=2EZ7M50MR70A type=write size=1810062 2024-04-17T04:45:12.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.916+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=V9QOUSJALBFY type=del 2024-04-17T04:45:12.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~K1SMZ8Y8WYBV.0 size=740205 2024-04-17T04:45:12.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.924+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=K1SMZ8Y8WYBV tag=4QDVGAPI6941 type=write size=1788781 2024-04-17T04:45:12.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.924+0000 7fd8ef2218c0 1 < completed part key=_multipart_6WSZ5UL0V1Y1.2~DBA1PRSIRUCB.0 size=221028 2024-04-17T04:45:12.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.932+0000 7fd8ef2218c0 1 < completed multipart upload key=6WSZ5UL0V1Y1 upload=DBA1PRSIRUCB tag=XEAK27DD2Z4C type=write size=1269604 2024-04-17T04:45:12.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.936+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=9Y90J3MDEJ7L type=del size=2534640 2024-04-17T04:45:12.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.940+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~3HHR4563HNOW.0 size=544225 2024-04-17T04:45:12.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.944+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=3HHR4563HNOW tag=BNGFZXJIGXBR type=write size=1592801 2024-04-17T04:45:12.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.948+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=F3U47J3J6SKD type=write size=526540 2024-04-17T04:45:12.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.952+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=OG9ZRWYZCMUH type=del 2024-04-17T04:45:12.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.956+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~7DEUCELYIKUI.0 size=1048576 2024-04-17T04:45:12.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~7DEUCELYIKUI.1 size=1048576 2024-04-17T04:45:12.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.960+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~7DEUCELYIKUI.2 size=856948 2024-04-17T04:45:12.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.960+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=7DEUCELYIKUI tag=I1IDWXIRL7G7 type=write size=4002676 2024-04-17T04:45:12.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.964+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4XN2TWPITM0P.0 size=1048576 2024-04-17T04:45:12.961 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.968+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4XN2TWPITM0P.1 size=1048576 2024-04-17T04:45:12.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.968+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~4XN2TWPITM0P.2 size=254536 2024-04-17T04:45:12.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.976+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=4XN2TWPITM0P tag=554EI90ECP7R type=write size=3400264 2024-04-17T04:45:12.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.976+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=N6H1C1B5TJ5U type=del size=705684 2024-04-17T04:45:12.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.980+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~OEZL0BH0JXL3.0 size=1048576 2024-04-17T04:45:12.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.984+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~OEZL0BH0JXL3.1 size=1048576 2024-04-17T04:45:12.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.984+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~OEZL0BH0JXL3.2 size=430169 2024-04-17T04:45:12.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.992+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=OEZL0BH0JXL3 tag=OXI571GU38CU type=write size=3575897 2024-04-17T04:45:12.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:12.996+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=4HCVK8QJWEKT type=del size=2888339 2024-04-17T04:45:12.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.004+0000 7fd8ef2218c0 1 < suggested remove operation key=5EFSN2FPUQB6 tag=ALO0L7ORKIBE type=del 2024-04-17T04:45:13.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~ZT2398BRYYPY.0 size=882904 2024-04-17T04:45:13.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=ZT2398BRYYPY tag=6SZMW4E7PMQF type=write size=1931480 2024-04-17T04:45:13.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.012+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=7QYA3OJ5Z04Y type=del 2024-04-17T04:45:13.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~CC5HZP7Q04UU.0 size=1048576 2024-04-17T04:45:13.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.016+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~CC5HZP7Q04UU.1 size=408083 2024-04-17T04:45:13.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.016+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=CC5HZP7Q04UU tag=8XYX9NEJAFCE type=write size=2505235 2024-04-17T04:45:13.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.020+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~W8MDJ8CRSDXT.0 size=1016369 2024-04-17T04:45:13.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.024+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=W8MDJ8CRSDXT tag=XIXCGQ436M9U type=write size=2064945 2024-04-17T04:45:13.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~Z1TG4WWN80WA.0 size=1048576 2024-04-17T04:45:13.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.032+0000 7fd8ef2218c0 1 > prepared part key=_multipart_82D6YWKJ1TFQ.2~Z1TG4WWN80WA.1 size=348204 2024-04-17T04:45:13.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.032+0000 7fd8ef2218c0 1 > prepared multipart upload key=82D6YWKJ1TFQ upload=Z1TG4WWN80WA tag=EJDOX05UEV9S type=write size=2445356 2024-04-17T04:45:13.029 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.032+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~NPWEP6W475ID.0 size=10963 2024-04-17T04:45:13.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.036+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=NPWEP6W475ID tag=B24SCUYL0RTP type=write size=1059539 2024-04-17T04:45:13.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.040+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=C78XBHYX8CHT type=del size=3646217 2024-04-17T04:45:13.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.044+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~VAKCH7QFRZM4.0 size=1048576 2024-04-17T04:45:13.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.044+0000 7fd8ef2218c0 1 < canceled part key=_multipart_1ZTJP8DRITEK.2~VAKCH7QFRZM4.1 size=1048576 2024-04-17T04:45:13.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.044+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~VAKCH7QFRZM4.2 size=123062 2024-04-17T04:45:13.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.048+0000 7fd8ef2218c0 1 < canceled multipart upload key=1ZTJP8DRITEK upload=VAKCH7QFRZM4 tag=3OGK62EKV360 type=write size=3268790 2024-04-17T04:45:13.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.056+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~4YJN37CCECYW.0 size=1048576 2024-04-17T04:45:13.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.056+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~4YJN37CCECYW.1 size=1048576 2024-04-17T04:45:13.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.060+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~4YJN37CCECYW.2 size=159037 2024-04-17T04:45:13.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.060+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=4YJN37CCECYW tag=JOGR2VR9O009 type=write size=3304765 2024-04-17T04:45:13.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.060+0000 7fd8ef2218c0 1 < completed write operation key=R3C8PAVPVGBY tag=29PTJDL08JEG type=write size=942290 2024-04-17T04:45:13.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.064+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=QG9Q3DRINX4C type=write size=1006756 2024-04-17T04:45:13.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~A0SEF3OJFHOI.0 size=1048576 2024-04-17T04:45:13.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.068+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~A0SEF3OJFHOI.1 size=976938 2024-04-17T04:45:13.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.076+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=A0SEF3OJFHOI tag=7O3PPZAXTNUG type=write size=3074090 2024-04-17T04:45:13.072 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.076+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=77FK1TYJQ4PL type=del size=3450957 2024-04-17T04:45:13.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.080+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=MEQ5K3TIF8A2 type=del 2024-04-17T04:45:13.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.084+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~Y2TSJYEFUXGN.0 size=1048576 2024-04-17T04:45:13.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.088+0000 7fd8ef2218c0 1 > prepared part key=_multipart_6219DPY72LWC.2~Y2TSJYEFUXGN.1 size=321213 2024-04-17T04:45:13.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.088+0000 7fd8ef2218c0 1 > prepared multipart upload key=6219DPY72LWC upload=Y2TSJYEFUXGN tag=GVSAOUWLZI3W type=write size=2418365 2024-04-17T04:45:13.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~24UUDS00R63S.0 size=562030 2024-04-17T04:45:13.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.096+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=24UUDS00R63S tag=KLS50XN8XB05 type=write size=1610606 2024-04-17T04:45:13.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.096+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=N2LYX25TZLI1 type=write size=1001033 2024-04-17T04:45:13.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.100+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~KEBIJ7N5VCWN.0 size=761486 2024-04-17T04:45:13.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.104+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=KEBIJ7N5VCWN tag=2EZ7M50MR70A type=write size=1810062 2024-04-17T04:45:13.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.108+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=D2MU8RGLRVCP type=del size=3939038 2024-04-17T04:45:13.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.112+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~K1SMZ8Y8WYBV.0 size=740205 2024-04-17T04:45:13.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.116+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=K1SMZ8Y8WYBV tag=4QDVGAPI6941 type=write size=1788781 2024-04-17T04:45:13.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.116+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=S7UAD1725ZEK type=del size=2234761 2024-04-17T04:45:13.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.120+0000 7fd8ef2218c0 -1 < failed to complete operation key=5EFSN2FPUQB6 tag=9Y90J3MDEJ7L type=del size=2534640: (22) Invalid argument 2024-04-17T04:45:13.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.124+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=6RXA9S4HNN3F type=del size=655241 2024-04-17T04:45:13.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.128+0000 7fd8ef2218c0 1 < completed write operation key=5FR1NEWE0LD1 tag=F3U47J3J6SKD type=write size=526540 2024-04-17T04:45:13.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.128+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=OTXZOBNF4K37 type=del size=549273 2024-04-17T04:45:13.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~7DEUCELYIKUI.0 size=1048576 2024-04-17T04:45:13.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~7DEUCELYIKUI.1 size=1048576 2024-04-17T04:45:13.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~7DEUCELYIKUI.2 size=856948 2024-04-17T04:45:13.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.140+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=7DEUCELYIKUI tag=I1IDWXIRL7G7 type=write size=4002676 2024-04-17T04:45:13.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~2KWDSQVTPIO3.0 size=1048576 2024-04-17T04:45:13.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.148+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~2KWDSQVTPIO3.1 size=797149 2024-04-17T04:45:13.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.148+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=2KWDSQVTPIO3 tag=AQQXOQIT6WMS type=write size=2894301 2024-04-17T04:45:13.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.152+0000 7fd8ef2218c0 1 < suggested update operation key=M3U8L7M8JKG9 tag=N6H1C1B5TJ5U type=del size=705684 2024-04-17T04:45:13.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.156+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=FQZ1QLLV66NL type=write size=263756 2024-04-17T04:45:13.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.160+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=4HCVK8QJWEKT type=del 2024-04-17T04:45:13.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.164+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=C8P3Q6BVRAG1 type=del size=2100386 2024-04-17T04:45:13.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.164+0000 7fd8ef2218c0 1 < canceled part key=_multipart_EWAW6EBW6W6U.2~ZT2398BRYYPY.0 size=882904 2024-04-17T04:45:13.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.172+0000 7fd8ef2218c0 1 < canceled multipart upload key=EWAW6EBW6W6U upload=ZT2398BRYYPY tag=6SZMW4E7PMQF type=write size=1931480 2024-04-17T04:45:13.169 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.172+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=8NDZ56EEDVU8 type=del size=4188437 2024-04-17T04:45:13.172 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~CC5HZP7Q04UU.0 size=1048576 2024-04-17T04:45:13.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.180+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~CC5HZP7Q04UU.1 size=408083 2024-04-17T04:45:13.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.184+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=CC5HZP7Q04UU tag=8XYX9NEJAFCE type=write size=2505235 2024-04-17T04:45:13.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.188+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~3K35H5BSM2J9.0 size=1048576 2024-04-17T04:45:13.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.192+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~3K35H5BSM2J9.1 size=1048576 2024-04-17T04:45:13.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.196+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~3K35H5BSM2J9.2 size=931501 2024-04-17T04:45:13.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.196+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=3K35H5BSM2J9 tag=M72YGJNZM6LC type=write size=4077229 2024-04-17T04:45:13.192 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.196+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~Z1TG4WWN80WA.0 size=1048576 2024-04-17T04:45:13.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.200+0000 7fd8ef2218c0 1 < completed part key=_multipart_82D6YWKJ1TFQ.2~Z1TG4WWN80WA.1 size=348204 2024-04-17T04:45:13.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.204+0000 7fd8ef2218c0 1 < completed multipart upload key=82D6YWKJ1TFQ upload=Z1TG4WWN80WA tag=EJDOX05UEV9S type=write size=2445356 2024-04-17T04:45:13.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.208+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~K62BMHFOY292.0 size=1048576 2024-04-17T04:45:13.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.212+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~K62BMHFOY292.1 size=951384 2024-04-17T04:45:13.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.212+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=K62BMHFOY292 tag=SGVZRU225AYZ type=write size=3048536 2024-04-17T04:45:13.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.216+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=C78XBHYX8CHT type=del 2024-04-17T04:45:13.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.220+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~C5XLXSZKHCP6.0 size=1048576 2024-04-17T04:45:13.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.224+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~C5XLXSZKHCP6.1 size=670412 2024-04-17T04:45:13.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.224+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=C5XLXSZKHCP6 tag=T91M8G7GNDKH type=write size=2767564 2024-04-17T04:45:13.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.224+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~4YJN37CCECYW.0 size=1048576 2024-04-17T04:45:13.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~4YJN37CCECYW.1 size=1048576 2024-04-17T04:45:13.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.232+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~4YJN37CCECYW.2 size=159037 2024-04-17T04:45:13.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.236+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=4YJN37CCECYW tag=JOGR2VR9O009 type=write size=3304765 2024-04-17T04:45:13.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.240+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=ADO94AUDH9WS type=del size=125332 2024-04-17T04:45:13.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.244+0000 7fd8ef2218c0 1 < suggested remove operation key=5K50O96P7OGD tag=QG9Q3DRINX4C type=write 2024-04-17T04:45:13.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.252+0000 7fd8ef2218c0 1 > prepared part key=_multipart_3OFQH4KVYN18.2~R2NZZLDW7NCD.0 size=335259 2024-04-17T04:45:13.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.252+0000 7fd8ef2218c0 1 > prepared multipart upload key=3OFQH4KVYN18 upload=R2NZZLDW7NCD tag=3E6Q99T8EJLT type=write size=1383835 2024-04-17T04:45:13.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.256+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=77FK1TYJQ4PL type=del 2024-04-17T04:45:13.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.260+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=9Z6Y6P6R20CE type=del size=760523 2024-04-17T04:45:13.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~Y2TSJYEFUXGN.0 size=1048576 2024-04-17T04:45:13.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_6219DPY72LWC.2~Y2TSJYEFUXGN.1 size=321213 2024-04-17T04:45:13.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.272+0000 7fd8ef2218c0 1 < completed multipart upload key=6219DPY72LWC upload=Y2TSJYEFUXGN tag=GVSAOUWLZI3W type=write size=2418365 2024-04-17T04:45:13.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.276+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~ESN2O26Q2LQX.0 size=997483 2024-04-17T04:45:13.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.276+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=ESN2O26Q2LQX tag=D2XO6BURA6QZ type=write size=2046059 2024-04-17T04:45:13.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.280+0000 7fd8ef2218c0 1 < completed write operation key=509DZ9DBJWC5 tag=N2LYX25TZLI1 type=write size=1001033 2024-04-17T04:45:13.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.280+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=HFLV5IQUPUTE type=del size=2558321 2024-04-17T04:45:13.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.284+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=D2MU8RGLRVCP type=del 2024-04-17T04:45:13.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.288+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=31BQ7M4T8FXP type=del size=1783765 2024-04-17T04:45:13.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.288+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=S7UAD1725ZEK type=del 2024-04-17T04:45:13.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.292+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=FCEEXPEXVOQC type=del size=2738092 2024-04-17T04:45:13.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.296+0000 7fd8ef2218c0 -1 < failed to complete operation key=M3U8L7M8JKG9 tag=6RXA9S4HNN3F type=del size=655241: (22) Invalid argument 2024-04-17T04:45:13.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.296+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=LF5LE53F6S1T type=del size=2360821 2024-04-17T04:45:13.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.304+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=OTXZOBNF4K37 type=del 2024-04-17T04:45:13.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~S1NKCLMVZCS5.0 size=1048576 2024-04-17T04:45:13.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.312+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~S1NKCLMVZCS5.1 size=1048576 2024-04-17T04:45:13.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.312+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~S1NKCLMVZCS5.2 size=98570 2024-04-17T04:45:13.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.312+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=S1NKCLMVZCS5 tag=1IN3GLUYF90E type=write size=3244298 2024-04-17T04:45:13.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~2KWDSQVTPIO3.0 size=1048576 2024-04-17T04:45:13.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.316+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2S1IT3TO1BPU.2~2KWDSQVTPIO3.1 size=797149 2024-04-17T04:45:13.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.320+0000 7fd8ef2218c0 1 < canceled multipart upload key=2S1IT3TO1BPU upload=2KWDSQVTPIO3 tag=AQQXOQIT6WMS type=write size=2894301 2024-04-17T04:45:13.320 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.324+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=DL6V959K4NKI type=del size=2729932 2024-04-17T04:45:13.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.328+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=FQZ1QLLV66NL type=write size=263756 2024-04-17T04:45:13.329 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~PT82OJSKY673.0 size=1048576 2024-04-17T04:45:13.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.336+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~PT82OJSKY673.1 size=746165 2024-04-17T04:45:13.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.336+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=PT82OJSKY673 tag=WIOWZMNBJ8U8 type=write size=2843317 2024-04-17T04:45:13.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.340+0000 7fd8ef2218c0 1 < suggested remove operation key=509DZ9DBJWC5 tag=C8P3Q6BVRAG1 type=del 2024-04-17T04:45:13.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.344+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=V1HOYVZB8NFV type=del size=3774626 2024-04-17T04:45:13.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.348+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=8NDZ56EEDVU8 type=del 2024-04-17T04:45:13.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.352+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=42YPD4YNLU2Y type=del size=2149219 2024-04-17T04:45:13.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.356+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~3K35H5BSM2J9.0 size=1048576 2024-04-17T04:45:13.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.356+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~3K35H5BSM2J9.1 size=1048576 2024-04-17T04:45:13.354 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.360+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~3K35H5BSM2J9.2 size=931501 2024-04-17T04:45:13.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.364+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=3K35H5BSM2J9 tag=M72YGJNZM6LC type=write size=4077229 2024-04-17T04:45:13.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.368+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=Z4KNK040OCYI type=del size=2987515 2024-04-17T04:45:13.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.372+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~K62BMHFOY292.0 size=1048576 2024-04-17T04:45:13.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.372+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~K62BMHFOY292.1 size=951384 2024-04-17T04:45:13.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.376+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=K62BMHFOY292 tag=SGVZRU225AYZ type=write size=3048536 2024-04-17T04:45:13.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.384+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~S9LRQEUCF0OS.0 size=379454 2024-04-17T04:45:13.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.384+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=S9LRQEUCF0OS tag=O0829WCFGQDG type=write size=1428030 2024-04-17T04:45:13.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.384+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~C5XLXSZKHCP6.0 size=1048576 2024-04-17T04:45:13.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.384+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~C5XLXSZKHCP6.1 size=670412 2024-04-17T04:45:13.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.388+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=C5XLXSZKHCP6 tag=T91M8G7GNDKH type=write size=2767564 2024-04-17T04:45:13.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.392+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=PSHHVNIIDZBJ type=write size=279642 2024-04-17T04:45:13.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.392+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=ADO94AUDH9WS type=del 2024-04-17T04:45:13.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.400+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~K6GZQ4IRK82W.0 size=1048576 2024-04-17T04:45:13.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.400+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~K6GZQ4IRK82W.1 size=612745 2024-04-17T04:45:13.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.400+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=K6GZQ4IRK82W tag=AH5JLOKR251P type=write size=2709897 2024-04-17T04:45:13.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.404+0000 7fd8ef2218c0 1 < completed part key=_multipart_3OFQH4KVYN18.2~R2NZZLDW7NCD.0 size=335259 2024-04-17T04:45:13.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.404+0000 7fd8ef2218c0 1 < completed multipart upload key=3OFQH4KVYN18 upload=R2NZZLDW7NCD tag=3E6Q99T8EJLT type=write size=1383835 2024-04-17T04:45:13.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.408+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~YD2T17R3HOJ6.0 size=1048576 2024-04-17T04:45:13.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.412+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~YD2T17R3HOJ6.1 size=629239 2024-04-17T04:45:13.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.412+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=YD2T17R3HOJ6 tag=A31HVOXVUZ98 type=write size=2726391 2024-04-17T04:45:13.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.412+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=9Z6Y6P6R20CE type=del 2024-04-17T04:45:13.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.416+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~SPZ7BQO564LZ.0 size=1048576 2024-04-17T04:45:13.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.420+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~SPZ7BQO564LZ.1 size=1048576 2024-04-17T04:45:13.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.420+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~SPZ7BQO564LZ.2 size=237903 2024-04-17T04:45:13.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.420+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=SPZ7BQO564LZ tag=VPPUE27B3ADI type=write size=3383631 2024-04-17T04:45:13.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.420+0000 7fd8ef2218c0 1 < canceled part key=_multipart_MS3T476JOEON.2~ESN2O26Q2LQX.0 size=997483 2024-04-17T04:45:13.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.424+0000 7fd8ef2218c0 1 < canceled multipart upload key=MS3T476JOEON upload=ESN2O26Q2LQX tag=D2XO6BURA6QZ type=write size=2046059 2024-04-17T04:45:13.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.428+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=0OGW27V8MS8I type=write size=387303 2024-04-17T04:45:13.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.428+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=HFLV5IQUPUTE type=del 2024-04-17T04:45:13.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.432+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=B92C1PH01IOG type=write size=972858 2024-04-17T04:45:13.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.436+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=31BQ7M4T8FXP type=del 2024-04-17T04:45:13.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.436+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~0UY59162BR4J.0 size=1048576 2024-04-17T04:45:13.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.440+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~0UY59162BR4J.1 size=185473 2024-04-17T04:45:13.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.440+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=0UY59162BR4J tag=A0C0WHQMCDZV type=write size=2282625 2024-04-17T04:45:13.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.440+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=FCEEXPEXVOQC type=del 2024-04-17T04:45:13.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.444+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=EL8CO223KXVA type=del size=1019673 2024-04-17T04:45:13.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.448+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=LF5LE53F6S1T type=del 2024-04-17T04:45:13.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~BF7QVUUKI1R4.0 size=1048576 2024-04-17T04:45:13.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~BF7QVUUKI1R4.1 size=1048576 2024-04-17T04:45:13.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.452+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~BF7QVUUKI1R4.2 size=269617 2024-04-17T04:45:13.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.452+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=BF7QVUUKI1R4 tag=BVEJT7CLJXT9 type=write size=3415345 2024-04-17T04:45:13.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~S1NKCLMVZCS5.0 size=1048576 2024-04-17T04:45:13.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.456+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~S1NKCLMVZCS5.1 size=1048576 2024-04-17T04:45:13.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.456+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2S1IT3TO1BPU.2~S1NKCLMVZCS5.2 size=98570 2024-04-17T04:45:13.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.460+0000 7fd8ef2218c0 1 < canceled multipart upload key=2S1IT3TO1BPU upload=S1NKCLMVZCS5 tag=1IN3GLUYF90E type=write size=3244298 2024-04-17T04:45:13.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.464+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=72P1R67S4LZ6 type=del size=1640755 2024-04-17T04:45:13.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.464+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=DL6V959K4NKI type=del 2024-04-17T04:45:13.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.468+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=C7DZ4MYZ9XSO type=write size=860920 2024-04-17T04:45:13.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.468+0000 7fd8ef2218c0 1 < canceled part key=_multipart_N11Y8FPN9XES.2~PT82OJSKY673.0 size=1048576 2024-04-17T04:45:13.465 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.472+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~PT82OJSKY673.1 size=746165 2024-04-17T04:45:13.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.472+0000 7fd8ef2218c0 1 < canceled multipart upload key=N11Y8FPN9XES upload=PT82OJSKY673 tag=WIOWZMNBJ8U8 type=write size=2843317 2024-04-17T04:45:13.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.476+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=VWH7WU7MFXEM type=del size=3901032 2024-04-17T04:45:13.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.480+0000 7fd8ef2218c0 1 < suggested remove operation key=5EFSN2FPUQB6 tag=V1HOYVZB8NFV type=del 2024-04-17T04:45:13.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.484+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=8F2TWDVP9ZDS type=del size=2478037 2024-04-17T04:45:13.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.484+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=42YPD4YNLU2Y type=del 2024-04-17T04:45:13.482 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.488+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=UHQVSR7ZEB6K type=del size=3011829 2024-04-17T04:45:13.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.492+0000 7fd8ef2218c0 1 < suggested update operation key=5EFSN2FPUQB6 tag=Z4KNK040OCYI type=del size=2987515 2024-04-17T04:45:13.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.496+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~N0ETOPY0CKWP.0 size=1048576 2024-04-17T04:45:13.495 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~N0ETOPY0CKWP.1 size=1048576 2024-04-17T04:45:13.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.500+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~N0ETOPY0CKWP.2 size=575576 2024-04-17T04:45:13.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.500+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=N0ETOPY0CKWP tag=0EFRDF97Y74M type=write size=3721304 2024-04-17T04:45:13.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.504+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~S9LRQEUCF0OS.0 size=379454 2024-04-17T04:45:13.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.508+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=S9LRQEUCF0OS tag=O0829WCFGQDG type=write size=1428030 2024-04-17T04:45:13.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.508+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=E6J0D4MTVFJ6 type=del size=427475 2024-04-17T04:45:13.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.512+0000 7fd8ef2218c0 1 < completed write operation key=EWAW6EBW6W6U tag=PSHHVNIIDZBJ type=write size=279642 2024-04-17T04:45:13.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.512+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=RKRCAAUERJ7U type=write size=267227 2024-04-17T04:45:13.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~K6GZQ4IRK82W.0 size=1048576 2024-04-17T04:45:13.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.516+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~K6GZQ4IRK82W.1 size=612745 2024-04-17T04:45:13.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.520+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=K6GZQ4IRK82W tag=AH5JLOKR251P type=write size=2709897 2024-04-17T04:45:13.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.524+0000 7fd8ef2218c0 1 > prepared operation key=A5ZUB3YK06J3 tag=VVPOUIVEGD2O type=del size=1804909 2024-04-17T04:45:13.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.524+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~YD2T17R3HOJ6.0 size=1048576 2024-04-17T04:45:13.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.528+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~YD2T17R3HOJ6.1 size=629239 2024-04-17T04:45:13.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.532+0000 7fd8ef2218c0 1 < completed multipart upload key=5EFSN2FPUQB6 upload=YD2T17R3HOJ6 tag=A31HVOXVUZ98 type=write size=2726391 2024-04-17T04:45:13.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.532+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=NXB395BZ79L1 type=write size=940620 2024-04-17T04:45:13.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.536+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~SPZ7BQO564LZ.0 size=1048576 2024-04-17T04:45:13.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.536+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~SPZ7BQO564LZ.1 size=1048576 2024-04-17T04:45:13.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.540+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~SPZ7BQO564LZ.2 size=237903 2024-04-17T04:45:13.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.540+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=SPZ7BQO564LZ tag=VPPUE27B3ADI type=write size=3383631 2024-04-17T04:45:13.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.544+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~EXJ236CVP4RF.0 size=642644 2024-04-17T04:45:13.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.544+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=EXJ236CVP4RF tag=INS7VOWA6GAD type=write size=1691220 2024-04-17T04:45:13.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.548+0000 7fd8ef2218c0 1 < completed write operation key=5FR1NEWE0LD1 tag=0OGW27V8MS8I type=write size=387303 2024-04-17T04:45:13.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.548+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=QIE6OBDOOC8L type=del size=1401586 2024-04-17T04:45:13.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.552+0000 7fd8ef2218c0 1 < completed write operation key=BCG05NIQFA53 tag=B92C1PH01IOG type=write size=972858 2024-04-17T04:45:13.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.556+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~E09ERO2GGFY8.0 size=280091 2024-04-17T04:45:13.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.556+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=E09ERO2GGFY8 tag=492BO9T27ZRJ type=write size=1328667 2024-04-17T04:45:13.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.556+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~0UY59162BR4J.0 size=1048576 2024-04-17T04:45:13.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~0UY59162BR4J.1 size=185473 2024-04-17T04:45:13.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.564+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=0UY59162BR4J tag=A0C0WHQMCDZV type=write size=2282625 2024-04-17T04:45:13.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.564+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=GBBR7DBZXV6B type=del size=3094271 2024-04-17T04:45:13.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.568+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=EL8CO223KXVA type=del 2024-04-17T04:45:13.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.568+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=AFJ4WAWTZO1H type=del size=1722213 2024-04-17T04:45:13.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.572+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~BF7QVUUKI1R4.0 size=1048576 2024-04-17T04:45:13.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.572+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~BF7QVUUKI1R4.1 size=1048576 2024-04-17T04:45:13.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.576+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~BF7QVUUKI1R4.2 size=269617 2024-04-17T04:45:13.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.580+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=BF7QVUUKI1R4 tag=BVEJT7CLJXT9 type=write size=3415345 2024-04-17T04:45:13.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.580+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=D4QKOGXB78WW type=del size=647690 2024-04-17T04:45:13.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.584+0000 7fd8ef2218c0 1 < suggested remove operation key=1ZTJP8DRITEK tag=72P1R67S4LZ6 type=del 2024-04-17T04:45:13.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.588+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=ZTRED597J23F type=write size=10801 2024-04-17T04:45:13.586 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.592+0000 7fd8ef2218c0 1 < suggested update operation key=OZ1IST8DOGXW tag=C7DZ4MYZ9XSO type=write size=860920 2024-04-17T04:45:13.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.592+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=8ACPPT30YO67 type=del size=3803622 2024-04-17T04:45:13.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.596+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=VWH7WU7MFXEM type=del 2024-04-17T04:45:13.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.600+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~MMDLC61PLVLK.0 size=1048576 2024-04-17T04:45:13.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.600+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~MMDLC61PLVLK.1 size=1048576 2024-04-17T04:45:13.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.604+0000 7fd8ef2218c0 1 > prepared part key=_multipart_WCIJICR0FT2C.2~MMDLC61PLVLK.2 size=195698 2024-04-17T04:45:13.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.604+0000 7fd8ef2218c0 1 > prepared multipart upload key=WCIJICR0FT2C upload=MMDLC61PLVLK tag=WVF4IV0S61V6 type=write size=3341426 2024-04-17T04:45:13.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.608+0000 7fd8ef2218c0 1 < suggested remove operation key=5EFSN2FPUQB6 tag=8F2TWDVP9ZDS type=del 2024-04-17T04:45:13.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~SEGDNZ88ZIK3.0 size=1048576 2024-04-17T04:45:13.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.612+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~SEGDNZ88ZIK3.1 size=294564 2024-04-17T04:45:13.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.612+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=SEGDNZ88ZIK3 tag=Q8GA6QB68MVH type=write size=2391716 2024-04-17T04:45:13.613 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.616+0000 7fd8ef2218c0 1 < suggested update operation key=G03VLS09F3BT tag=UHQVSR7ZEB6K type=del size=3011829 2024-04-17T04:45:13.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.620+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=MPR73C69CV10 type=del size=3248596 2024-04-17T04:45:13.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.620+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2IY8DB37FI9H.2~N0ETOPY0CKWP.0 size=1048576 2024-04-17T04:45:13.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.624+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~N0ETOPY0CKWP.1 size=1048576 2024-04-17T04:45:13.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.624+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~N0ETOPY0CKWP.2 size=575576 2024-04-17T04:45:13.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.628+0000 7fd8ef2218c0 1 < canceled multipart upload key=2IY8DB37FI9H upload=N0ETOPY0CKWP tag=0EFRDF97Y74M type=write size=3721304 2024-04-17T04:45:13.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.632+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=FIG4MYOV4I9P type=del size=3631237 2024-04-17T04:45:13.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.632+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=E6J0D4MTVFJ6 type=del 2024-04-17T04:45:13.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.636+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~4LRWVH2IF6M4.0 size=569066 2024-04-17T04:45:13.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.636+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=4LRWVH2IF6M4 tag=9BEVU0YKZRX0 type=write size=1617642 2024-04-17T04:45:13.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.640+0000 7fd8ef2218c0 -1 < failed to complete operation key=5EFSN2FPUQB6 tag=RKRCAAUERJ7U type=write size=267227: (22) Invalid argument 2024-04-17T04:45:13.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4WQIEPJKAJ9F.0 size=1048576 2024-04-17T04:45:13.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4WQIEPJKAJ9F.1 size=1048576 2024-04-17T04:45:13.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.644+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5EFSN2FPUQB6.2~4WQIEPJKAJ9F.2 size=497547 2024-04-17T04:45:13.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.644+0000 7fd8ef2218c0 1 > prepared multipart upload key=5EFSN2FPUQB6 upload=4WQIEPJKAJ9F tag=YP33DV1CFO1I type=write size=3643275 2024-04-17T04:45:13.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.648+0000 7fd8ef2218c0 1 < completed delete operation key=A5ZUB3YK06J3 tag=VVPOUIVEGD2O type=del 2024-04-17T04:45:13.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.652+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=7452QQW2YOO5 type=del size=2949585 2024-04-17T04:45:13.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.656+0000 7fd8ef2218c0 1 < suggested update operation key=M3U8L7M8JKG9 tag=NXB395BZ79L1 type=write size=940620 2024-04-17T04:45:13.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.656+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=DLVVLWG7FZ2M type=del size=1444640 2024-04-17T04:45:13.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.660+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~EXJ236CVP4RF.0 size=642644 2024-04-17T04:45:13.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.664+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=EXJ236CVP4RF tag=INS7VOWA6GAD type=write size=1691220 2024-04-17T04:45:13.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.668+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=Z3N64MK0ZZTV type=del size=1591114 2024-04-17T04:45:13.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.672+0000 7fd8ef2218c0 1 < suggested remove operation key=UYCKMXE6LBUH tag=QIE6OBDOOC8L type=del 2024-04-17T04:45:13.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.676+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=K3SFUSHZ8LF7 type=del size=3926307 2024-04-17T04:45:13.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.676+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~E09ERO2GGFY8.0 size=280091 2024-04-17T04:45:13.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.684+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=E09ERO2GGFY8 tag=492BO9T27ZRJ type=write size=1328667 2024-04-17T04:45:13.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.684+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=GIP7UFKY0K0V type=del size=3444212 2024-04-17T04:45:13.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.688+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=GBBR7DBZXV6B type=del 2024-04-17T04:45:13.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.692+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~STQH6VQ3FQMA.0 size=437003 2024-04-17T04:45:13.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.692+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=STQH6VQ3FQMA tag=VO3WOWCUGU1D type=write size=1485579 2024-04-17T04:45:13.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.696+0000 7fd8ef2218c0 1 < suggested remove operation key=M3U8L7M8JKG9 tag=AFJ4WAWTZO1H type=del 2024-04-17T04:45:13.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.700+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=RS7XAFO1LBVZ type=del size=388153 2024-04-17T04:45:13.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.704+0000 7fd8ef2218c0 -1 < failed to complete operation key=OZ1IST8DOGXW tag=D4QKOGXB78WW type=del size=647690: (22) Invalid argument 2024-04-17T04:45:13.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~EVVZSN890TZ0.0 size=1048576 2024-04-17T04:45:13.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.712+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~EVVZSN890TZ0.1 size=1048576 2024-04-17T04:45:13.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2IY8DB37FI9H.2~EVVZSN890TZ0.2 size=71861 2024-04-17T04:45:13.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.716+0000 7fd8ef2218c0 1 > prepared multipart upload key=2IY8DB37FI9H upload=EVVZSN890TZ0 tag=3VKY4KQW3VQ3 type=write size=3217589 2024-04-17T04:45:13.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.716+0000 7fd8ef2218c0 1 < canceled operation key=R3C8PAVPVGBY tag=ZTRED597J23F type=write size=10801 2024-04-17T04:45:13.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.724+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~44SHKWPIYMWE.0 size=821067 2024-04-17T04:45:13.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.724+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=44SHKWPIYMWE tag=9PYX0HV3DBSP type=write size=1869643 2024-04-17T04:45:13.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.728+0000 7fd8ef2218c0 1 < suggested remove operation key=AE4CYL334ONO tag=8ACPPT30YO67 type=del 2024-04-17T04:45:13.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.728+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=LKRA4QQJAYS3 type=del size=2820372 2024-04-17T04:45:13.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.732+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~MMDLC61PLVLK.0 size=1048576 2024-04-17T04:45:13.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.732+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~MMDLC61PLVLK.1 size=1048576 2024-04-17T04:45:13.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.736+0000 7fd8ef2218c0 1 < completed part key=_multipart_WCIJICR0FT2C.2~MMDLC61PLVLK.2 size=195698 2024-04-17T04:45:13.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.740+0000 7fd8ef2218c0 1 < completed multipart upload key=WCIJICR0FT2C upload=MMDLC61PLVLK tag=WVF4IV0S61V6 type=write size=3341426 2024-04-17T04:45:13.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.740+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=19XY8JCV8QYU type=write size=920304 2024-04-17T04:45:13.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.744+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~SEGDNZ88ZIK3.0 size=1048576 2024-04-17T04:45:13.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.744+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~SEGDNZ88ZIK3.1 size=294564 2024-04-17T04:45:13.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.748+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=SEGDNZ88ZIK3 tag=Q8GA6QB68MVH type=write size=2391716 2024-04-17T04:45:13.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.752+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=ZK4LT091TR4M type=write size=520095 2024-04-17T04:45:13.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.752+0000 7fd8ef2218c0 -1 < failed to complete operation key=UYCKMXE6LBUH tag=MPR73C69CV10 type=del size=3248596: (22) Invalid argument 2024-04-17T04:45:13.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.756+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CS1K3LSK1213.0 size=1048576 2024-04-17T04:45:13.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.760+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~CS1K3LSK1213.1 size=236155 2024-04-17T04:45:13.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.760+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=CS1K3LSK1213 tag=DJU2ZPPVE0FA type=write size=2333307 2024-04-17T04:45:13.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.760+0000 7fd8ef2218c0 1 < completed delete operation key=1ZTJP8DRITEK tag=FIG4MYOV4I9P type=del 2024-04-17T04:45:13.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.764+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=87SEXPDBJ9GU type=write size=189104 2024-04-17T04:45:13.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.768+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~4LRWVH2IF6M4.0 size=569066 2024-04-17T04:45:13.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.772+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=4LRWVH2IF6M4 tag=9BEVU0YKZRX0 type=write size=1617642 2024-04-17T04:45:13.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.776+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=NQ0DQSYXK3N5 type=write size=561427 2024-04-17T04:45:13.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.780+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~4WQIEPJKAJ9F.0 size=1048576 2024-04-17T04:45:13.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.780+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5EFSN2FPUQB6.2~4WQIEPJKAJ9F.1 size=1048576 2024-04-17T04:45:13.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.780+0000 7fd8ef2218c0 1 < completed part key=_multipart_5EFSN2FPUQB6.2~4WQIEPJKAJ9F.2 size=497547 2024-04-17T04:45:13.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.784+0000 7fd8ef2218c0 1 < canceled multipart upload key=5EFSN2FPUQB6 upload=4WQIEPJKAJ9F tag=YP33DV1CFO1I type=write size=3643275 2024-04-17T04:45:13.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.788+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~TJPN6QPY8B7M.0 size=1048576 2024-04-17T04:45:13.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~TJPN6QPY8B7M.1 size=588979 2024-04-17T04:45:13.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=TJPN6QPY8B7M tag=UTNIDM606GQG type=write size=2686131 2024-04-17T04:45:13.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.792+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=7452QQW2YOO5 type=del 2024-04-17T04:45:13.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.796+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=2QKZ28ISYJGH type=del size=1403763 2024-04-17T04:45:13.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.800+0000 7fd8ef2218c0 1 < canceled operation key=7BV3V2XMLSPH tag=DLVVLWG7FZ2M type=del size=1444640 2024-04-17T04:45:13.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.800+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=33B6RK5RPHQK type=write size=669124 2024-04-17T04:45:13.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.804+0000 7fd8ef2218c0 1 < canceled operation key=82D6YWKJ1TFQ tag=Z3N64MK0ZZTV type=del size=1591114 2024-04-17T04:45:13.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.804+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=9BKM8IFHEGDP type=del size=3895751 2024-04-17T04:45:13.805 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.812+0000 7fd8ef2218c0 1 < suggested update operation key=UYCKMXE6LBUH tag=K3SFUSHZ8LF7 type=del size=3926307 2024-04-17T04:45:13.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.812+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~PT6EQORE57GM.0 size=1048576 2024-04-17T04:45:13.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.816+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~PT6EQORE57GM.1 size=1020859 2024-04-17T04:45:13.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.816+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=PT6EQORE57GM tag=LSFE63JQ4R3M type=write size=3118011 2024-04-17T04:45:13.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.820+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=GIP7UFKY0K0V type=del 2024-04-17T04:45:13.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.820+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=RFKMRMEDJM5A type=write size=73876 2024-04-17T04:45:13.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.824+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~STQH6VQ3FQMA.0 size=437003 2024-04-17T04:45:13.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.828+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=STQH6VQ3FQMA tag=VO3WOWCUGU1D type=write size=1485579 2024-04-17T04:45:13.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~QL3YY0XA6CE2.0 size=1048576 2024-04-17T04:45:13.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~QL3YY0XA6CE2.1 size=1002969 2024-04-17T04:45:13.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=QL3YY0XA6CE2 tag=9ANRAF09OX4N type=write size=3100121 2024-04-17T04:45:13.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.836+0000 7fd8ef2218c0 1 < suggested remove operation key=3OFQH4KVYN18 tag=RS7XAFO1LBVZ type=del 2024-04-17T04:45:13.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.844+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~FBV4X3O7A06P.0 size=789491 2024-04-17T04:45:13.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.844+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=FBV4X3O7A06P tag=QZ9R8TNUJMMC type=write size=1838067 2024-04-17T04:45:13.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.844+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~EVVZSN890TZ0.0 size=1048576 2024-04-17T04:45:13.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.844+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~EVVZSN890TZ0.1 size=1048576 2024-04-17T04:45:13.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.848+0000 7fd8ef2218c0 1 < completed part key=_multipart_2IY8DB37FI9H.2~EVVZSN890TZ0.2 size=71861 2024-04-17T04:45:13.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.852+0000 7fd8ef2218c0 1 < completed multipart upload key=2IY8DB37FI9H upload=EVVZSN890TZ0 tag=3VKY4KQW3VQ3 type=write size=3217589 2024-04-17T04:45:13.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.856+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=OAPU4G2J7GI7 type=del size=786175 2024-04-17T04:45:13.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.856+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~44SHKWPIYMWE.0 size=821067 2024-04-17T04:45:13.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.860+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=44SHKWPIYMWE tag=9PYX0HV3DBSP type=write size=1869643 2024-04-17T04:45:13.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~EPSXPO1Y3N3Z.0 size=1048576 2024-04-17T04:45:13.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~EPSXPO1Y3N3Z.1 size=1048576 2024-04-17T04:45:13.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.868+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~EPSXPO1Y3N3Z.2 size=62778 2024-04-17T04:45:13.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.868+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=EPSXPO1Y3N3Z tag=RZ1I5L78PM2Z type=write size=3208506 2024-04-17T04:45:13.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.868+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=LKRA4QQJAYS3 type=del 2024-04-17T04:45:13.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.872+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~863PZW17L0P1.0 size=1048576 2024-04-17T04:45:13.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.876+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~863PZW17L0P1.1 size=494120 2024-04-17T04:45:13.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.876+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=863PZW17L0P1 tag=ZAOO3QAVAP2J type=write size=2591272 2024-04-17T04:45:13.875 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.880+0000 7fd8ef2218c0 1 < suggested remove operation key=UNOVYXDS5JEB tag=19XY8JCV8QYU type=write 2024-04-17T04:45:13.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.880+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=283BNW7HN3L2 type=del size=692178 2024-04-17T04:45:13.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.884+0000 7fd8ef2218c0 1 < suggested remove operation key=1ZTJP8DRITEK tag=ZK4LT091TR4M type=write 2024-04-17T04:45:13.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.888+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=N19K3HZITX0H type=del size=2500961 2024-04-17T04:45:13.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.888+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~CS1K3LSK1213.0 size=1048576 2024-04-17T04:45:13.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.888+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~CS1K3LSK1213.1 size=236155 2024-04-17T04:45:13.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.892+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=CS1K3LSK1213 tag=DJU2ZPPVE0FA type=write size=2333307 2024-04-17T04:45:13.893 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.896+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~28VJ482PMFP5.0 size=1048576 2024-04-17T04:45:13.894 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~28VJ482PMFP5.1 size=1048576 2024-04-17T04:45:13.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.900+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~28VJ482PMFP5.2 size=1006917 2024-04-17T04:45:13.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.900+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=28VJ482PMFP5 tag=L3ID3E67JHC9 type=write size=4152645 2024-04-17T04:45:13.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.904+0000 7fd8ef2218c0 1 < completed write operation key=WCIJICR0FT2C tag=87SEXPDBJ9GU type=write size=189104 2024-04-17T04:45:13.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.904+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=E69RTLCU1T8M type=del size=1573725 2024-04-17T04:45:13.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.908+0000 7fd8ef2218c0 1 < completed write operation key=M3U8L7M8JKG9 tag=NQ0DQSYXK3N5 type=write size=561427 2024-04-17T04:45:13.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.912+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=UDBFSHD8LUCA type=del size=2489981 2024-04-17T04:45:13.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.912+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~TJPN6QPY8B7M.0 size=1048576 2024-04-17T04:45:13.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.916+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~TJPN6QPY8B7M.1 size=588979 2024-04-17T04:45:13.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.916+0000 7fd8ef2218c0 1 < completed multipart upload key=BCG05NIQFA53 upload=TJPN6QPY8B7M tag=UTNIDM606GQG type=write size=2686131 2024-04-17T04:45:13.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.924+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~E20F5D97OX0Y.0 size=1048576 2024-04-17T04:45:13.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.928+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~E20F5D97OX0Y.1 size=813685 2024-04-17T04:45:13.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.928+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=E20F5D97OX0Y tag=69JX079JZM0S type=write size=2910837 2024-04-17T04:45:13.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.932+0000 7fd8ef2218c0 1 < suggested remove operation key=6219DPY72LWC tag=2QKZ28ISYJGH type=del 2024-04-17T04:45:13.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~6MABYDIY1HWM.0 size=1048576 2024-04-17T04:45:13.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~6MABYDIY1HWM.1 size=1048576 2024-04-17T04:45:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.940+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~6MABYDIY1HWM.2 size=558356 2024-04-17T04:45:13.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.940+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=6MABYDIY1HWM tag=HW2J6T142QYA type=write size=3704084 2024-04-17T04:45:13.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.944+0000 7fd8ef2218c0 1 < completed write operation key=2S1IT3TO1BPU tag=33B6RK5RPHQK type=write size=669124 2024-04-17T04:45:13.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.944+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=00TWK7SEAVQ5 type=del size=1658500 2024-04-17T04:45:13.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.948+0000 7fd8ef2218c0 -1 < failed to complete operation key=1ZTJP8DRITEK tag=9BKM8IFHEGDP type=del size=3895751: (22) Invalid argument 2024-04-17T04:45:13.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.948+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=XM7UUKLX4MI6 type=del size=2837964 2024-04-17T04:45:13.947 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~PT6EQORE57GM.0 size=1048576 2024-04-17T04:45:13.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~PT6EQORE57GM.1 size=1020859 2024-04-17T04:45:13.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.956+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=PT6EQORE57GM tag=LSFE63JQ4R3M type=write size=3118011 2024-04-17T04:45:13.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.960+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=82KHHEHEOIO7 type=del size=1604678 2024-04-17T04:45:13.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.960+0000 7fd8ef2218c0 1 < completed write operation key=509DZ9DBJWC5 tag=RFKMRMEDJM5A type=write size=73876 2024-04-17T04:45:13.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.964+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=Y1SSMNZO784Z type=del size=2968327 2024-04-17T04:45:13.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.964+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~QL3YY0XA6CE2.0 size=1048576 2024-04-17T04:45:13.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.968+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~QL3YY0XA6CE2.1 size=1002969 2024-04-17T04:45:13.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.968+0000 7fd8ef2218c0 1 < completed multipart upload key=M3U8L7M8JKG9 upload=QL3YY0XA6CE2 tag=9ANRAF09OX4N type=write size=3100121 2024-04-17T04:45:13.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.972+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=DOHETC8A44FL type=write size=144645 2024-04-17T04:45:13.970 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.976+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~FBV4X3O7A06P.0 size=789491 2024-04-17T04:45:13.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.976+0000 7fd8ef2218c0 1 < completed multipart upload key=JVWRJBMFIRLL upload=FBV4X3O7A06P tag=QZ9R8TNUJMMC type=write size=1838067 2024-04-17T04:45:13.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.980+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=KEOZEH6GXNIU type=del size=3914018 2024-04-17T04:45:13.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.984+0000 7fd8ef2218c0 1 < suggested remove operation key=G03VLS09F3BT tag=OAPU4G2J7GI7 type=del 2024-04-17T04:45:13.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~5TUI399P3YAZ.0 size=1048576 2024-04-17T04:45:13.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.992+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~5TUI399P3YAZ.1 size=1048576 2024-04-17T04:45:13.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.992+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~5TUI399P3YAZ.2 size=551806 2024-04-17T04:45:13.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.992+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=5TUI399P3YAZ tag=N1R2SACBL8ZD type=write size=3697534 2024-04-17T04:45:13.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~EPSXPO1Y3N3Z.0 size=1048576 2024-04-17T04:45:13.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~EPSXPO1Y3N3Z.1 size=1048576 2024-04-17T04:45:13.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:13.996+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~EPSXPO1Y3N3Z.2 size=62778 2024-04-17T04:45:13.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.000+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=EPSXPO1Y3N3Z tag=RZ1I5L78PM2Z type=write size=3208506 2024-04-17T04:45:14.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.004+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~7SLWHOGMM66W.0 size=1048576 2024-04-17T04:45:14.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~7SLWHOGMM66W.1 size=1048576 2024-04-17T04:45:14.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.008+0000 7fd8ef2218c0 1 > prepared part key=_multipart_OZ1IST8DOGXW.2~7SLWHOGMM66W.2 size=711880 2024-04-17T04:45:14.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.008+0000 7fd8ef2218c0 1 > prepared multipart upload key=OZ1IST8DOGXW upload=7SLWHOGMM66W tag=WJ94QSFTP4RC type=write size=3857608 2024-04-17T04:45:14.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~863PZW17L0P1.0 size=1048576 2024-04-17T04:45:14.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.012+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~863PZW17L0P1.1 size=494120 2024-04-17T04:45:14.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.016+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=863PZW17L0P1 tag=ZAOO3QAVAP2J type=write size=2591272 2024-04-17T04:45:14.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.020+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=5OEFWLZTA60Q type=del size=2822652 2024-04-17T04:45:14.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.020+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=283BNW7HN3L2 type=del 2024-04-17T04:45:14.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.024+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~BN4T1E2TLK0H.0 size=1048576 2024-04-17T04:45:14.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.028+0000 7fd8ef2218c0 1 > prepared part key=_multipart_M3U8L7M8JKG9.2~BN4T1E2TLK0H.1 size=982024 2024-04-17T04:45:14.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.028+0000 7fd8ef2218c0 1 > prepared multipart upload key=M3U8L7M8JKG9 upload=BN4T1E2TLK0H tag=9ZKMYCVN6ZY1 type=write size=3079176 2024-04-17T04:45:14.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.028+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=N19K3HZITX0H type=del 2024-04-17T04:45:14.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.032+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~U8GRPIZ9H8AX.0 size=1048576 2024-04-17T04:45:14.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.036+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~U8GRPIZ9H8AX.1 size=402806 2024-04-17T04:45:14.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.036+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=U8GRPIZ9H8AX tag=5E5TOVIMMDNA type=write size=2499958 2024-04-17T04:45:14.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.036+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~28VJ482PMFP5.0 size=1048576 2024-04-17T04:45:14.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.036+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~28VJ482PMFP5.1 size=1048576 2024-04-17T04:45:14.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.040+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~28VJ482PMFP5.2 size=1006917 2024-04-17T04:45:14.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.044+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=28VJ482PMFP5 tag=L3ID3E67JHC9 type=write size=4152645 2024-04-17T04:45:14.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.044+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=LC4F5WF6VJFN type=del size=2174997 2024-04-17T04:45:14.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.048+0000 7fd8ef2218c0 -1 < failed to complete operation key=6219DPY72LWC tag=E69RTLCU1T8M type=del size=1573725: (22) Invalid argument 2024-04-17T04:45:14.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.048+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=47VY38TTGSH7 type=del size=575388 2024-04-17T04:45:14.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.052+0000 7fd8ef2218c0 1 < suggested update operation key=UYCKMXE6LBUH tag=UDBFSHD8LUCA type=del size=2489981 2024-04-17T04:45:14.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.056+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=U9DUDJQPV69Y type=del size=1745441 2024-04-17T04:45:14.054 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~E20F5D97OX0Y.0 size=1048576 2024-04-17T04:45:14.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.060+0000 7fd8ef2218c0 1 < completed part key=_multipart_5FR1NEWE0LD1.2~E20F5D97OX0Y.1 size=813685 2024-04-17T04:45:14.060 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.064+0000 7fd8ef2218c0 1 < completed multipart upload key=5FR1NEWE0LD1 upload=E20F5D97OX0Y tag=69JX079JZM0S type=write size=2910837 2024-04-17T04:45:14.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.068+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~2MNK54A7TC92.0 size=1048576 2024-04-17T04:45:14.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~2MNK54A7TC92.1 size=1048576 2024-04-17T04:45:14.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.072+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~2MNK54A7TC92.2 size=513749 2024-04-17T04:45:14.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.072+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=2MNK54A7TC92 tag=AZR2S78HFQ1K type=write size=3659477 2024-04-17T04:45:14.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.076+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~6MABYDIY1HWM.0 size=1048576 2024-04-17T04:45:14.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.076+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~6MABYDIY1HWM.1 size=1048576 2024-04-17T04:45:14.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.080+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~6MABYDIY1HWM.2 size=558356 2024-04-17T04:45:14.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.084+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=6MABYDIY1HWM tag=HW2J6T142QYA type=write size=3704084 2024-04-17T04:45:14.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.084+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=OJUZYW4YVGDE type=del size=132248 2024-04-17T04:45:14.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.088+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=00TWK7SEAVQ5 type=del 2024-04-17T04:45:14.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.092+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=D2BV3ZMEE0JZ type=write size=635451 2024-04-17T04:45:14.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.092+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=XM7UUKLX4MI6 type=del 2024-04-17T04:45:14.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~CX9WF84Y3435.0 size=253977 2024-04-17T04:45:14.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.100+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=CX9WF84Y3435 tag=DJ7JIFW3PVGT type=write size=1302553 2024-04-17T04:45:14.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.104+0000 7fd8ef2218c0 1 < suggested remove operation key=MS3T476JOEON tag=82KHHEHEOIO7 type=del 2024-04-17T04:45:14.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.108+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=CISRZIGAMEXL type=del size=76009 2024-04-17T04:45:14.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.112+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=Y1SSMNZO784Z type=del 2024-04-17T04:45:14.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.112+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=Q9A3Q1U8CGNA type=del size=2829667 2024-04-17T04:45:14.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.116+0000 7fd8ef2218c0 1 < completed write operation key=WCIJICR0FT2C tag=DOHETC8A44FL type=write size=144645 2024-04-17T04:45:14.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.120+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~0VN1KD6ZXYYU.0 size=1048576 2024-04-17T04:45:14.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.124+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~0VN1KD6ZXYYU.1 size=29577 2024-04-17T04:45:14.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.124+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=0VN1KD6ZXYYU tag=N0LA02ZRHOX3 type=write size=2126729 2024-04-17T04:45:14.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.124+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=KEOZEH6GXNIU type=del 2024-04-17T04:45:14.124 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.128+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=J7XJ3BRSCGFO type=del size=3866620 2024-04-17T04:45:14.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~5TUI399P3YAZ.0 size=1048576 2024-04-17T04:45:14.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.132+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~5TUI399P3YAZ.1 size=1048576 2024-04-17T04:45:14.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.136+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~5TUI399P3YAZ.2 size=551806 2024-04-17T04:45:14.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.140+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=5TUI399P3YAZ tag=N1R2SACBL8ZD type=write size=3697534 2024-04-17T04:45:14.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.144+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=N5V08UH6VPZT type=del size=3165997 2024-04-17T04:45:14.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~7SLWHOGMM66W.0 size=1048576 2024-04-17T04:45:14.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~7SLWHOGMM66W.1 size=1048576 2024-04-17T04:45:14.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.152+0000 7fd8ef2218c0 1 < completed part key=_multipart_OZ1IST8DOGXW.2~7SLWHOGMM66W.2 size=711880 2024-04-17T04:45:14.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.156+0000 7fd8ef2218c0 1 < completed multipart upload key=OZ1IST8DOGXW upload=7SLWHOGMM66W tag=WJ94QSFTP4RC type=write size=3857608 2024-04-17T04:45:14.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.156+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=4KQWC3OYOE94 type=del size=2535468 2024-04-17T04:45:14.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.160+0000 7fd8ef2218c0 1 < canceled operation key=3OFQH4KVYN18 tag=5OEFWLZTA60Q type=del size=2822652 2024-04-17T04:45:14.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.164+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=TF4V28WDAZXI type=del size=2130866 2024-04-17T04:45:14.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.164+0000 7fd8ef2218c0 1 < completed part key=_multipart_M3U8L7M8JKG9.2~BN4T1E2TLK0H.0 size=1048576 2024-04-17T04:45:14.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.164+0000 7fd8ef2218c0 1 < canceled part key=_multipart_M3U8L7M8JKG9.2~BN4T1E2TLK0H.1 size=982024 2024-04-17T04:45:14.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.168+0000 7fd8ef2218c0 1 < canceled multipart upload key=M3U8L7M8JKG9 upload=BN4T1E2TLK0H tag=9ZKMYCVN6ZY1 type=write size=3079176 2024-04-17T04:45:14.167 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.172+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=3KB3P1A2XQD2 type=del size=910195 2024-04-17T04:45:14.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~U8GRPIZ9H8AX.0 size=1048576 2024-04-17T04:45:14.172 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.176+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~U8GRPIZ9H8AX.1 size=402806 2024-04-17T04:45:14.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.180+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=U8GRPIZ9H8AX tag=5E5TOVIMMDNA type=write size=2499958 2024-04-17T04:45:14.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.184+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=6TS709XHTNY5 type=del size=1153124 2024-04-17T04:45:14.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.188+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=LC4F5WF6VJFN type=del 2024-04-17T04:45:14.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.192+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=AFLP1ALCD58X type=del size=1657476 2024-04-17T04:45:14.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.192+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=47VY38TTGSH7 type=del 2024-04-17T04:45:14.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.196+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=WIVW5KE5KKWF type=write size=137522 2024-04-17T04:45:14.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.200+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=U9DUDJQPV69Y type=del 2024-04-17T04:45:14.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.200+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=RQLGJGQNDDSN type=del size=317995 2024-04-17T04:45:14.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.204+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~2MNK54A7TC92.0 size=1048576 2024-04-17T04:45:14.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.208+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~2MNK54A7TC92.1 size=1048576 2024-04-17T04:45:14.202 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.208+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~2MNK54A7TC92.2 size=513749 2024-04-17T04:45:14.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.212+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=2MNK54A7TC92 tag=AZR2S78HFQ1K type=write size=3659477 2024-04-17T04:45:14.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.216+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=U9YNXYSJ45X1 type=del size=2686501 2024-04-17T04:45:14.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.216+0000 7fd8ef2218c0 1 < canceled operation key=UEWLJ3E9ODUL tag=OJUZYW4YVGDE type=del size=132248 2024-04-17T04:45:14.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.220+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=OWN2HUWBN2QT type=del size=3908267 2024-04-17T04:45:14.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.224+0000 7fd8ef2218c0 1 < suggested remove operation key=82D6YWKJ1TFQ tag=D2BV3ZMEE0JZ type=write 2024-04-17T04:45:14.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.228+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=APTA9FE561RS type=write size=856793 2024-04-17T04:45:14.225 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~CX9WF84Y3435.0 size=253977 2024-04-17T04:45:14.230 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.236+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=CX9WF84Y3435 tag=DJ7JIFW3PVGT type=write size=1302553 2024-04-17T04:45:14.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.240+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~1B8FBLENONXV.0 size=1048576 2024-04-17T04:45:14.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~1B8FBLENONXV.1 size=1048576 2024-04-17T04:45:14.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.244+0000 7fd8ef2218c0 1 > prepared part key=_multipart_BCG05NIQFA53.2~1B8FBLENONXV.2 size=180456 2024-04-17T04:45:14.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.244+0000 7fd8ef2218c0 1 > prepared multipart upload key=BCG05NIQFA53 upload=1B8FBLENONXV tag=FTH7HC74E1RR type=write size=3326184 2024-04-17T04:45:14.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.248+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=CISRZIGAMEXL type=del 2024-04-17T04:45:14.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.252+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=OKLDMGSWNYYD type=del size=4014815 2024-04-17T04:45:14.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.252+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=Q9A3Q1U8CGNA type=del 2024-04-17T04:45:14.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.260+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5K50O96P7OGD.2~EEO9N1KNFO6D.0 size=189699 2024-04-17T04:45:14.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.260+0000 7fd8ef2218c0 1 > prepared multipart upload key=5K50O96P7OGD upload=EEO9N1KNFO6D tag=OAQDTJIJ8Y7F type=write size=1238275 2024-04-17T04:45:14.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~0VN1KD6ZXYYU.0 size=1048576 2024-04-17T04:45:14.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.264+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~0VN1KD6ZXYYU.1 size=29577 2024-04-17T04:45:14.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.268+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=0VN1KD6ZXYYU tag=N0LA02ZRHOX3 type=write size=2126729 2024-04-17T04:45:14.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.272+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=0M7BYGDWZL0P type=del size=341999 2024-04-17T04:45:14.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.276+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=J7XJ3BRSCGFO type=del 2024-04-17T04:45:14.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.280+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=H42Q91S7MH9E type=write size=184674 2024-04-17T04:45:14.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.284+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=N5V08UH6VPZT type=del 2024-04-17T04:45:14.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.288+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~W88HQAR1534K.0 size=1048576 2024-04-17T04:45:14.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.292+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~W88HQAR1534K.1 size=1048576 2024-04-17T04:45:14.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.296+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1ZTJP8DRITEK.2~W88HQAR1534K.2 size=785584 2024-04-17T04:45:14.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.296+0000 7fd8ef2218c0 1 > prepared multipart upload key=1ZTJP8DRITEK upload=W88HQAR1534K tag=C4HMZN5EG357 type=write size=3931312 2024-04-17T04:45:14.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.296+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=4KQWC3OYOE94 type=del 2024-04-17T04:45:14.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.300+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=DSL5TYG4OA28 type=del size=4168923 2024-04-17T04:45:14.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.304+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=TF4V28WDAZXI type=del 2024-04-17T04:45:14.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.308+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=383YURYIS851 type=del size=1734693 2024-04-17T04:45:14.307 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.312+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=3KB3P1A2XQD2 type=del 2024-04-17T04:45:14.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.316+0000 7fd8ef2218c0 1 > prepared operation key=M3U8L7M8JKG9 tag=DRF5C86K30VP type=del size=968573 2024-04-17T04:45:14.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.320+0000 7fd8ef2218c0 1 < completed delete operation key=UEWLJ3E9ODUL tag=6TS709XHTNY5 type=del 2024-04-17T04:45:14.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.324+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=2GUXGXDFXXQ9 type=del size=3740937 2024-04-17T04:45:14.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.324+0000 7fd8ef2218c0 1 < canceled operation key=R3C8PAVPVGBY tag=AFLP1ALCD58X type=del size=1657476 2024-04-17T04:45:14.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.328+0000 7fd8ef2218c0 1 > prepared operation key=B63EZS0LAC30 tag=E3HM4SIFZ0VV type=del size=858303 2024-04-17T04:45:14.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.332+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=WIVW5KE5KKWF type=write size=137522 2024-04-17T04:45:14.331 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.336+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=1ZMPULBWTV8P type=write size=760571 2024-04-17T04:45:14.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.340+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=RQLGJGQNDDSN type=del 2024-04-17T04:45:14.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.344+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=VC7JKUVPOVEZ type=del size=2099638 2024-04-17T04:45:14.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.344+0000 7fd8ef2218c0 1 < canceled operation key=MS3T476JOEON tag=U9YNXYSJ45X1 type=del size=2686501 2024-04-17T04:45:14.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~ZOGRTIMW0UGD.0 size=1048576 2024-04-17T04:45:14.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.352+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~ZOGRTIMW0UGD.1 size=1048576 2024-04-17T04:45:14.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.356+0000 7fd8ef2218c0 1 > prepared part key=_multipart_A5ZUB3YK06J3.2~ZOGRTIMW0UGD.2 size=511059 2024-04-17T04:45:14.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.356+0000 7fd8ef2218c0 1 > prepared multipart upload key=A5ZUB3YK06J3 upload=ZOGRTIMW0UGD tag=B8N1YIQAH1O7 type=write size=3656787 2024-04-17T04:45:14.355 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.360+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=OWN2HUWBN2QT type=del 2024-04-17T04:45:14.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.364+0000 7fd8ef2218c0 1 > prepared operation key=1ZTJP8DRITEK tag=3EWA1D2EYI52 type=del size=1376930 2024-04-17T04:45:14.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.368+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=APTA9FE561RS type=write size=856793 2024-04-17T04:45:14.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.372+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~ZHUFLEMCSYLC.0 size=289201 2024-04-17T04:45:14.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.372+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=ZHUFLEMCSYLC tag=XXPH3NISNX3C type=write size=1337777 2024-04-17T04:45:14.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~1B8FBLENONXV.0 size=1048576 2024-04-17T04:45:14.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.380+0000 7fd8ef2218c0 1 < completed part key=_multipart_BCG05NIQFA53.2~1B8FBLENONXV.1 size=1048576 2024-04-17T04:45:14.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.380+0000 7fd8ef2218c0 1 < canceled part key=_multipart_BCG05NIQFA53.2~1B8FBLENONXV.2 size=180456 2024-04-17T04:45:14.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.388+0000 7fd8ef2218c0 1 < canceled multipart upload key=BCG05NIQFA53 upload=1B8FBLENONXV tag=FTH7HC74E1RR type=write size=3326184 2024-04-17T04:45:14.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.396+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~CI4EW9GFC8U6.0 size=125532 2024-04-17T04:45:14.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.396+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=CI4EW9GFC8U6 tag=Z6MRB2HXAZ9O type=write size=1174108 2024-04-17T04:45:14.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.400+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=OKLDMGSWNYYD type=del 2024-04-17T04:45:14.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.404+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=BNRPIWCC0OS5 type=del size=189666 2024-04-17T04:45:14.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.408+0000 7fd8ef2218c0 1 < completed part key=_multipart_5K50O96P7OGD.2~EEO9N1KNFO6D.0 size=189699 2024-04-17T04:45:14.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.412+0000 7fd8ef2218c0 1 < completed multipart upload key=5K50O96P7OGD upload=EEO9N1KNFO6D tag=OAQDTJIJ8Y7F type=write size=1238275 2024-04-17T04:45:14.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.416+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=PQ0PUBMLOJVP type=del size=3287543 2024-04-17T04:45:14.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.420+0000 7fd8ef2218c0 1 < completed delete operation key=G03VLS09F3BT tag=0M7BYGDWZL0P type=del 2024-04-17T04:45:14.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.424+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=GCXA42ML9QAD type=write size=196757 2024-04-17T04:45:14.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.428+0000 7fd8ef2218c0 1 < completed write operation key=1ZTJP8DRITEK tag=H42Q91S7MH9E type=write size=184674 2024-04-17T04:45:14.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.428+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=9I5WOB6KDW29 type=write size=511893 2024-04-17T04:45:14.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.432+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~W88HQAR1534K.0 size=1048576 2024-04-17T04:45:14.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.440+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~W88HQAR1534K.1 size=1048576 2024-04-17T04:45:14.438 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.444+0000 7fd8ef2218c0 1 < completed part key=_multipart_1ZTJP8DRITEK.2~W88HQAR1534K.2 size=785584 2024-04-17T04:45:14.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.448+0000 7fd8ef2218c0 1 < completed multipart upload key=1ZTJP8DRITEK upload=W88HQAR1534K tag=C4HMZN5EG357 type=write size=3931312 2024-04-17T04:45:14.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.452+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=PXCS2U297TC7 type=write size=70242 2024-04-17T04:45:14.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.456+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=DSL5TYG4OA28 type=del 2024-04-17T04:45:14.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.460+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~YP1K53NU2CZW.0 size=1048576 2024-04-17T04:45:14.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.464+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UEWLJ3E9ODUL.2~YP1K53NU2CZW.1 size=238661 2024-04-17T04:45:14.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.464+0000 7fd8ef2218c0 1 > prepared multipart upload key=UEWLJ3E9ODUL upload=YP1K53NU2CZW tag=OS02VRZRHQ8X type=write size=2335813 2024-04-17T04:45:14.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.468+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=383YURYIS851 type=del 2024-04-17T04:45:14.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.472+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=0TML1WHW56YY type=del size=1981200 2024-04-17T04:45:14.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.476+0000 7fd8ef2218c0 1 < completed delete operation key=M3U8L7M8JKG9 tag=DRF5C86K30VP type=del 2024-04-17T04:45:14.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.480+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5FR1NEWE0LD1.2~CBWWWBTH833J.0 size=408766 2024-04-17T04:45:14.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.480+0000 7fd8ef2218c0 1 > prepared multipart upload key=5FR1NEWE0LD1 upload=CBWWWBTH833J tag=S5F2Q21943DF type=write size=1457342 2024-04-17T04:45:14.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.484+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=2GUXGXDFXXQ9 type=del 2024-04-17T04:45:14.484 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.488+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=TN0P6KUJ3SUR type=del size=698893 2024-04-17T04:45:14.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.492+0000 7fd8ef2218c0 1 < canceled operation key=B63EZS0LAC30 tag=E3HM4SIFZ0VV type=del size=858303 2024-04-17T04:45:14.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.496+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=1KY7DJCMJ7X3 type=del size=2007350 2024-04-17T04:45:14.495 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.500+0000 7fd8ef2218c0 1 < completed write operation key=R3C8PAVPVGBY tag=1ZMPULBWTV8P type=write size=760571 2024-04-17T04:45:14.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.504+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=EVUTBATPQ2OV type=del size=1287517 2024-04-17T04:45:14.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.508+0000 7fd8ef2218c0 1 < suggested update operation key=R3C8PAVPVGBY tag=VC7JKUVPOVEZ type=del size=2099638 2024-04-17T04:45:14.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.516+0000 7fd8ef2218c0 1 > prepared part key=_multipart_R3C8PAVPVGBY.2~CCHF1QVS8UTK.0 size=790463 2024-04-17T04:45:14.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.516+0000 7fd8ef2218c0 1 > prepared multipart upload key=R3C8PAVPVGBY upload=CCHF1QVS8UTK tag=8PJP1J2KJR4P type=write size=1839039 2024-04-17T04:45:14.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.520+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~ZOGRTIMW0UGD.0 size=1048576 2024-04-17T04:45:14.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.520+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~ZOGRTIMW0UGD.1 size=1048576 2024-04-17T04:45:14.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.524+0000 7fd8ef2218c0 1 < completed part key=_multipart_A5ZUB3YK06J3.2~ZOGRTIMW0UGD.2 size=511059 2024-04-17T04:45:14.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.528+0000 7fd8ef2218c0 1 < completed multipart upload key=A5ZUB3YK06J3 upload=ZOGRTIMW0UGD tag=B8N1YIQAH1O7 type=write size=3656787 2024-04-17T04:45:14.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.536+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~6K2I5JFB62OC.0 size=454477 2024-04-17T04:45:14.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.536+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=6K2I5JFB62OC tag=BZO2FBVVLFIG type=write size=1503053 2024-04-17T04:45:14.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.544+0000 7fd8ef2218c0 1 < suggested update operation key=1ZTJP8DRITEK tag=3EWA1D2EYI52 type=del size=1376930 2024-04-17T04:45:14.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.544+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=D89NSD92GXPP type=del size=2944311 2024-04-17T04:45:14.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.548+0000 7fd8ef2218c0 1 < completed part key=_multipart_AE4CYL334ONO.2~ZHUFLEMCSYLC.0 size=289201 2024-04-17T04:45:14.550 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.552+0000 7fd8ef2218c0 1 < completed multipart upload key=AE4CYL334ONO upload=ZHUFLEMCSYLC tag=XXPH3NISNX3C type=write size=1337777 2024-04-17T04:45:14.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.556+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=V6Y4R0PHR4N4 type=del size=779431 2024-04-17T04:45:14.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.560+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~CI4EW9GFC8U6.0 size=125532 2024-04-17T04:45:14.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.568+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=CI4EW9GFC8U6 tag=Z6MRB2HXAZ9O type=write size=1174108 2024-04-17T04:45:14.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.568+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=FOTXP1YD0TXL type=del size=867335 2024-04-17T04:45:14.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.572+0000 7fd8ef2218c0 1 < completed delete operation key=5EFSN2FPUQB6 tag=BNRPIWCC0OS5 type=del 2024-04-17T04:45:14.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.580+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~AKWSTRXWBB55.0 size=1048576 2024-04-17T04:45:14.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.584+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~AKWSTRXWBB55.1 size=558751 2024-04-17T04:45:14.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.584+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=AKWSTRXWBB55 tag=JVQ1FB4EJJ11 type=write size=2655903 2024-04-17T04:45:14.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.588+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=PQ0PUBMLOJVP type=del 2024-04-17T04:45:14.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.588+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=Q895RILDODTA type=del size=1088515 2024-04-17T04:45:14.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.596+0000 7fd8ef2218c0 1 < suggested remove operation key=WCIJICR0FT2C tag=GCXA42ML9QAD type=write 2024-04-17T04:45:14.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.600+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=MJ6TR8FC2FTK type=del size=772871 2024-04-17T04:45:14.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.604+0000 7fd8ef2218c0 -1 < failed to complete operation key=R3C8PAVPVGBY tag=9I5WOB6KDW29 type=write size=511893: (22) Invalid argument 2024-04-17T04:45:14.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.608+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=LYT2CEP5SX17 type=write size=730369 2024-04-17T04:45:14.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.612+0000 7fd8ef2218c0 1 < completed write operation key=7BV3V2XMLSPH tag=PXCS2U297TC7 type=write size=70242 2024-04-17T04:45:14.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.612+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=6DWC3JDAJ1HU type=del size=1156009 2024-04-17T04:45:14.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.616+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~YP1K53NU2CZW.0 size=1048576 2024-04-17T04:45:14.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.620+0000 7fd8ef2218c0 1 < completed part key=_multipart_UEWLJ3E9ODUL.2~YP1K53NU2CZW.1 size=238661 2024-04-17T04:45:14.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.624+0000 7fd8ef2218c0 1 < completed multipart upload key=UEWLJ3E9ODUL upload=YP1K53NU2CZW tag=OS02VRZRHQ8X type=write size=2335813 2024-04-17T04:45:14.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.628+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=4UB661A7S3LS type=write size=877036 2024-04-17T04:45:14.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.632+0000 7fd8ef2218c0 1 < completed delete operation key=N11Y8FPN9XES tag=0TML1WHW56YY type=del 2024-04-17T04:45:14.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.636+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=YFL2G7TVA2UH type=del size=1509549 2024-04-17T04:45:14.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.636+0000 7fd8ef2218c0 1 < canceled part key=_multipart_5FR1NEWE0LD1.2~CBWWWBTH833J.0 size=408766 2024-04-17T04:45:14.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.640+0000 7fd8ef2218c0 1 < canceled multipart upload key=5FR1NEWE0LD1 upload=CBWWWBTH833J tag=S5F2Q21943DF type=write size=1457342 2024-04-17T04:45:14.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.644+0000 7fd8ef2218c0 1 > prepared operation key=2S1IT3TO1BPU tag=57L6R9MYC6O2 type=del size=3478833 2024-04-17T04:45:14.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.648+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=TN0P6KUJ3SUR type=del 2024-04-17T04:45:14.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~HO08VZRS3J13.0 size=1048576 2024-04-17T04:45:14.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.656+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~HO08VZRS3J13.1 size=510944 2024-04-17T04:45:14.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.656+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=HO08VZRS3J13 tag=TE6J6WD2H5NU type=write size=2608096 2024-04-17T04:45:14.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.660+0000 7fd8ef2218c0 -1 < failed to complete operation key=R3C8PAVPVGBY tag=1KY7DJCMJ7X3 type=del size=2007350: (22) Invalid argument 2024-04-17T04:45:14.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.668+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~PYGBX2ZKXVTH.0 size=1048576 2024-04-17T04:45:14.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.672+0000 7fd8ef2218c0 1 > prepared part key=_multipart_1JOWGWFR7R6P.2~PYGBX2ZKXVTH.1 size=511328 2024-04-17T04:45:14.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.672+0000 7fd8ef2218c0 1 > prepared multipart upload key=1JOWGWFR7R6P upload=PYGBX2ZKXVTH tag=7718W5IKQGC6 type=write size=2608480 2024-04-17T04:45:14.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.676+0000 7fd8ef2218c0 1 < canceled operation key=VUYM3JC57QRP tag=EVUTBATPQ2OV type=del size=1287517 2024-04-17T04:45:14.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.676+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=ZHWNL67LL6GU type=del size=3988662 2024-04-17T04:45:14.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.680+0000 7fd8ef2218c0 1 < completed part key=_multipart_R3C8PAVPVGBY.2~CCHF1QVS8UTK.0 size=790463 2024-04-17T04:45:14.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.688+0000 7fd8ef2218c0 1 < completed multipart upload key=R3C8PAVPVGBY upload=CCHF1QVS8UTK tag=8PJP1J2KJR4P type=write size=1839039 2024-04-17T04:45:14.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.692+0000 7fd8ef2218c0 1 > prepared operation key=WCIJICR0FT2C tag=PWXU0VCS9CO0 type=del size=2079615 2024-04-17T04:45:14.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.696+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~6K2I5JFB62OC.0 size=454477 2024-04-17T04:45:14.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.700+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=6K2I5JFB62OC tag=BZO2FBVVLFIG type=write size=1503053 2024-04-17T04:45:14.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.708+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~X51XU0TBL22J.0 size=182761 2024-04-17T04:45:14.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.708+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=X51XU0TBL22J tag=AF6KEFCS11E4 type=write size=1231337 2024-04-17T04:45:14.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.708+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=D89NSD92GXPP type=del 2024-04-17T04:45:14.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.716+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~1O0M0ZZY9AB4.0 size=1048576 2024-04-17T04:45:14.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.720+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~1O0M0ZZY9AB4.1 size=486597 2024-04-17T04:45:14.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.720+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=1O0M0ZZY9AB4 tag=D1K0TK3LNETV type=write size=2583749 2024-04-17T04:45:14.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.728+0000 7fd8ef2218c0 1 < suggested update operation key=UNOVYXDS5JEB tag=V6Y4R0PHR4N4 type=del size=779431 2024-04-17T04:45:14.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.732+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~W443IH7413R2.0 size=1048576 2024-04-17T04:45:14.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.736+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~W443IH7413R2.1 size=40409 2024-04-17T04:45:14.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.736+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=W443IH7413R2 tag=9UXTC7DCR220 type=write size=2137561 2024-04-17T04:45:14.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.740+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=FOTXP1YD0TXL type=del 2024-04-17T04:45:14.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.744+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=8LJVG0WG7J1T type=del size=40263 2024-04-17T04:45:14.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.744+0000 7fd8ef2218c0 1 < canceled part key=_multipart_JVWRJBMFIRLL.2~AKWSTRXWBB55.0 size=1048576 2024-04-17T04:45:14.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.744+0000 7fd8ef2218c0 1 < completed part key=_multipart_JVWRJBMFIRLL.2~AKWSTRXWBB55.1 size=558751 2024-04-17T04:45:14.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.752+0000 7fd8ef2218c0 1 < canceled multipart upload key=JVWRJBMFIRLL upload=AKWSTRXWBB55 tag=JVQ1FB4EJJ11 type=write size=2655903 2024-04-17T04:45:14.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.756+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=A29TMK8O44NG type=write size=359210 2024-04-17T04:45:14.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.756+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=Q895RILDODTA type=del 2024-04-17T04:45:14.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.768+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~C4X6BKR5AD65.0 size=1048576 2024-04-17T04:45:14.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.772+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~C4X6BKR5AD65.1 size=868324 2024-04-17T04:45:14.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.772+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=C4X6BKR5AD65 tag=KCE7WXZT0R5N type=write size=2965476 2024-04-17T04:45:14.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.776+0000 7fd8ef2218c0 -1 < failed to complete operation key=UNOVYXDS5JEB tag=MJ6TR8FC2FTK type=del size=772871: (22) Invalid argument 2024-04-17T04:45:14.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.780+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=9A3XM45KOKSB type=del size=3801193 2024-04-17T04:45:14.777 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.780+0000 7fd8ef2218c0 1 < canceled operation key=AE4CYL334ONO tag=LYT2CEP5SX17 type=write size=730369 2024-04-17T04:45:14.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.784+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=7ZXOL3YXQGXQ type=del size=77603 2024-04-17T04:45:14.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.788+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=6DWC3JDAJ1HU type=del 2024-04-17T04:45:14.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.792+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~NNZLT6T537GF.0 size=385462 2024-04-17T04:45:14.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.792+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=NNZLT6T537GF tag=7ZT2SABK53T0 type=write size=1434038 2024-04-17T04:45:14.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.800+0000 7fd8ef2218c0 1 < suggested update operation key=MS3T476JOEON tag=4UB661A7S3LS type=write size=877036 2024-04-17T04:45:14.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.804+0000 7fd8ef2218c0 1 > prepared operation key=1JOWGWFR7R6P tag=RLVXHB7A7QBX type=del size=926597 2024-04-17T04:45:14.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.804+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=YFL2G7TVA2UH type=del 2024-04-17T04:45:14.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.808+0000 7fd8ef2218c0 1 > prepared operation key=EWAW6EBW6W6U tag=BUO04D2BQ997 type=del size=2924373 2024-04-17T04:45:14.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.812+0000 7fd8ef2218c0 1 < completed delete operation key=2S1IT3TO1BPU tag=57L6R9MYC6O2 type=del 2024-04-17T04:45:14.810 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.816+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=87PTBV8MAUNM type=del size=1806853 2024-04-17T04:45:14.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.816+0000 7fd8ef2218c0 1 < canceled part key=_multipart_2S1IT3TO1BPU.2~HO08VZRS3J13.0 size=1048576 2024-04-17T04:45:14.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.820+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~HO08VZRS3J13.1 size=510944 2024-04-17T04:45:14.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.824+0000 7fd8ef2218c0 1 < canceled multipart upload key=2S1IT3TO1BPU upload=HO08VZRS3J13 tag=TE6J6WD2H5NU type=write size=2608096 2024-04-17T04:45:14.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.832+0000 7fd8ef2218c0 1 > prepared part key=_multipart_GX9X0D45GC2I.2~W103DIEFDB0D.0 size=964370 2024-04-17T04:45:14.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.832+0000 7fd8ef2218c0 1 > prepared multipart upload key=GX9X0D45GC2I upload=W103DIEFDB0D tag=G7G53LP807L2 type=write size=2012946 2024-04-17T04:45:14.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.836+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~PYGBX2ZKXVTH.0 size=1048576 2024-04-17T04:45:14.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.836+0000 7fd8ef2218c0 1 < completed part key=_multipart_1JOWGWFR7R6P.2~PYGBX2ZKXVTH.1 size=511328 2024-04-17T04:45:14.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.844+0000 7fd8ef2218c0 1 < completed multipart upload key=1JOWGWFR7R6P upload=PYGBX2ZKXVTH tag=7718W5IKQGC6 type=write size=2608480 2024-04-17T04:45:14.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.848+0000 7fd8ef2218c0 1 > prepared operation key=82D6YWKJ1TFQ tag=ZJMASNF481OD type=del size=3363396 2024-04-17T04:45:14.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.852+0000 7fd8ef2218c0 1 < suggested update operation key=2IY8DB37FI9H tag=ZHWNL67LL6GU type=del size=3988662 2024-04-17T04:45:14.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.860+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~B6SQZ2SLT24U.0 size=1048576 2024-04-17T04:45:14.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~B6SQZ2SLT24U.1 size=1048576 2024-04-17T04:45:14.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.864+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~B6SQZ2SLT24U.2 size=856854 2024-04-17T04:45:14.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.864+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=B6SQZ2SLT24U tag=7ZG33PX0ON5Z type=write size=4002582 2024-04-17T04:45:14.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.868+0000 7fd8ef2218c0 1 < completed delete operation key=WCIJICR0FT2C tag=PWXU0VCS9CO0 type=del 2024-04-17T04:45:14.869 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.872+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=PUS3QTHO25UK type=del size=2225707 2024-04-17T04:45:14.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.876+0000 7fd8ef2218c0 1 < completed part key=_multipart_2S1IT3TO1BPU.2~X51XU0TBL22J.0 size=182761 2024-04-17T04:45:14.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.884+0000 7fd8ef2218c0 1 < completed multipart upload key=2S1IT3TO1BPU upload=X51XU0TBL22J tag=AF6KEFCS11E4 type=write size=1231337 2024-04-17T04:45:14.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.888+0000 7fd8ef2218c0 1 > prepared operation key=UNOVYXDS5JEB tag=0MUJTIDYW43Q type=del size=2423768 2024-04-17T04:45:14.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~1O0M0ZZY9AB4.0 size=1048576 2024-04-17T04:45:14.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.892+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~1O0M0ZZY9AB4.1 size=486597 2024-04-17T04:45:14.897 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.900+0000 7fd8ef2218c0 1 < completed multipart upload key=B63EZS0LAC30 upload=1O0M0ZZY9AB4 tag=D1K0TK3LNETV type=write size=2583749 2024-04-17T04:45:14.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.904+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~57IZWQ1Y5LXC.0 size=823100 2024-04-17T04:45:14.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.904+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=57IZWQ1Y5LXC tag=CCOOMR77PCO2 type=write size=1871676 2024-04-17T04:45:14.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.908+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~W443IH7413R2.0 size=1048576 2024-04-17T04:45:14.907 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.912+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~W443IH7413R2.1 size=40409 2024-04-17T04:45:14.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.916+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=W443IH7413R2 tag=9UXTC7DCR220 type=write size=2137561 2024-04-17T04:45:14.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.920+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=VC9FKVYQPBE9 type=write size=167011 2024-04-17T04:45:14.921 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.924+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=8LJVG0WG7J1T type=del 2024-04-17T04:45:14.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.932+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~HWGFGZRE5Q01.0 size=1048576 2024-04-17T04:45:14.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.936+0000 7fd8ef2218c0 1 > prepared part key=_multipart_MS3T476JOEON.2~HWGFGZRE5Q01.1 size=284335 2024-04-17T04:45:14.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.936+0000 7fd8ef2218c0 1 > prepared multipart upload key=MS3T476JOEON upload=HWGFGZRE5Q01 tag=ODA1KMA70EX1 type=write size=2381487 2024-04-17T04:45:14.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.944+0000 7fd8ef2218c0 1 < suggested remove operation key=5FR1NEWE0LD1 tag=A29TMK8O44NG type=write 2024-04-17T04:45:14.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.944+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=P1WPL30TXCND type=write size=1045319 2024-04-17T04:45:14.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.948+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~C4X6BKR5AD65.0 size=1048576 2024-04-17T04:45:14.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.952+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~C4X6BKR5AD65.1 size=868324 2024-04-17T04:45:14.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.956+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=C4X6BKR5AD65 tag=KCE7WXZT0R5N type=write size=2965476 2024-04-17T04:45:14.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.960+0000 7fd8ef2218c0 1 > prepared operation key=BCG05NIQFA53 tag=ZU2N6MDK7CKF type=del size=123894 2024-04-17T04:45:14.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.964+0000 7fd8ef2218c0 1 < completed delete operation key=VUYM3JC57QRP tag=9A3XM45KOKSB type=del 2024-04-17T04:45:14.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.968+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=C0JUGTQH1FXG type=write size=447036 2024-04-17T04:45:14.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.968+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=7ZXOL3YXQGXQ type=del 2024-04-17T04:45:14.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.972+0000 7fd8ef2218c0 1 > prepared operation key=5EFSN2FPUQB6 tag=YF3VE2HX6YQV type=del size=2426851 2024-04-17T04:45:14.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.976+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~NNZLT6T537GF.0 size=385462 2024-04-17T04:45:14.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.980+0000 7fd8ef2218c0 1 < completed multipart upload key=N11Y8FPN9XES upload=NNZLT6T537GF tag=7ZT2SABK53T0 type=write size=1434038 2024-04-17T04:45:14.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.984+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~HK52R2BYSB3X.0 size=1048576 2024-04-17T04:45:14.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.988+0000 7fd8ef2218c0 1 > prepared part key=_multipart_EWAW6EBW6W6U.2~HK52R2BYSB3X.1 size=499936 2024-04-17T04:45:14.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.988+0000 7fd8ef2218c0 1 > prepared multipart upload key=EWAW6EBW6W6U upload=HK52R2BYSB3X tag=ZN0WM6MK943J type=write size=2597088 2024-04-17T04:45:14.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.988+0000 7fd8ef2218c0 1 < completed delete operation key=1JOWGWFR7R6P tag=RLVXHB7A7QBX type=del 2024-04-17T04:45:14.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.992+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=C1QOO26EBUG0 type=del size=1783023 2024-04-17T04:45:14.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.996+0000 7fd8ef2218c0 1 < completed delete operation key=EWAW6EBW6W6U tag=BUO04D2BQ997 type=del 2024-04-17T04:45:14.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:14.996+0000 7fd8ef2218c0 1 > prepared operation key=VUYM3JC57QRP tag=9PQUTKHM6YL8 type=del size=2481196 2024-04-17T04:45:14.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.000+0000 7fd8ef2218c0 -1 < failed to complete operation key=2IY8DB37FI9H tag=87PTBV8MAUNM type=del size=1806853: (22) Invalid argument 2024-04-17T04:45:14.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.004+0000 7fd8ef2218c0 1 > prepared operation key=509DZ9DBJWC5 tag=ZXJHSZF7GR8X type=del size=4139113 2024-04-17T04:45:15.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.008+0000 7fd8ef2218c0 1 < completed part key=_multipart_GX9X0D45GC2I.2~W103DIEFDB0D.0 size=964370 2024-04-17T04:45:15.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.012+0000 7fd8ef2218c0 1 < completed multipart upload key=GX9X0D45GC2I upload=W103DIEFDB0D tag=G7G53LP807L2 type=write size=2012946 2024-04-17T04:45:15.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.020+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~FPS23JXUV419.0 size=464395 2024-04-17T04:45:15.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.020+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=FPS23JXUV419 tag=GM21DVDALXUH type=write size=1512971 2024-04-17T04:45:15.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.020+0000 7fd8ef2218c0 1 < completed delete operation key=82D6YWKJ1TFQ tag=ZJMASNF481OD type=del 2024-04-17T04:45:15.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.024+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=H7D06VQJD4QK type=del size=2080386 2024-04-17T04:45:15.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.024+0000 7fd8ef2218c0 1 < canceled part key=_multipart_N11Y8FPN9XES.2~B6SQZ2SLT24U.0 size=1048576 2024-04-17T04:45:15.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~B6SQZ2SLT24U.1 size=1048576 2024-04-17T04:45:15.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.028+0000 7fd8ef2218c0 1 < completed part key=_multipart_N11Y8FPN9XES.2~B6SQZ2SLT24U.2 size=856854 2024-04-17T04:45:15.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.036+0000 7fd8ef2218c0 1 < canceled multipart upload key=N11Y8FPN9XES upload=B6SQZ2SLT24U tag=7ZG33PX0ON5Z type=write size=4002582 2024-04-17T04:45:15.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.036+0000 7fd8ef2218c0 1 > prepared operation key=UYCKMXE6LBUH tag=66Y6YV93Q40F type=del size=2829087 2024-04-17T04:45:15.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.044+0000 7fd8ef2218c0 1 < suggested remove operation key=VUYM3JC57QRP tag=PUS3QTHO25UK type=del 2024-04-17T04:45:15.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.048+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=C9XNKJ93G5BS type=del size=1128615 2024-04-17T04:45:15.046 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.052+0000 7fd8ef2218c0 1 < completed delete operation key=UNOVYXDS5JEB tag=0MUJTIDYW43Q type=del 2024-04-17T04:45:15.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.052+0000 7fd8ef2218c0 1 > prepared operation key=6219DPY72LWC tag=DTV6H77FCC19 type=del size=2337262 2024-04-17T04:45:15.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.056+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~57IZWQ1Y5LXC.0 size=823100 2024-04-17T04:45:15.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.064+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=57IZWQ1Y5LXC tag=CCOOMR77PCO2 type=write size=1871676 2024-04-17T04:45:15.063 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.068+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~OMN9JFSZV4TN.0 size=1048576 2024-04-17T04:45:15.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~OMN9JFSZV4TN.1 size=1048576 2024-04-17T04:45:15.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.076+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~OMN9JFSZV4TN.2 size=296235 2024-04-17T04:45:15.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.076+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=OMN9JFSZV4TN tag=9GI03SM29DGU type=write size=3441963 2024-04-17T04:45:15.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.080+0000 7fd8ef2218c0 1 < completed write operation key=N11Y8FPN9XES tag=VC9FKVYQPBE9 type=write size=167011 2024-04-17T04:45:15.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.084+0000 7fd8ef2218c0 1 > prepared operation key=5K50O96P7OGD tag=BWRIDJ3FJZVO type=del size=1345234 2024-04-17T04:45:15.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~HWGFGZRE5Q01.0 size=1048576 2024-04-17T04:45:15.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.088+0000 7fd8ef2218c0 1 < completed part key=_multipart_MS3T476JOEON.2~HWGFGZRE5Q01.1 size=284335 2024-04-17T04:45:15.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.096+0000 7fd8ef2218c0 1 < completed multipart upload key=MS3T476JOEON upload=HWGFGZRE5Q01 tag=ODA1KMA70EX1 type=write size=2381487 2024-04-17T04:45:15.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.100+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~ZQLMQ268MYJ5.0 size=1048576 2024-04-17T04:45:15.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.104+0000 7fd8ef2218c0 1 > prepared part key=_multipart_7BV3V2XMLSPH.2~ZQLMQ268MYJ5.1 size=32610 2024-04-17T04:45:15.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.104+0000 7fd8ef2218c0 1 > prepared multipart upload key=7BV3V2XMLSPH upload=ZQLMQ268MYJ5 tag=SJIDCI08YSPE type=write size=2129762 2024-04-17T04:45:15.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.108+0000 7fd8ef2218c0 1 < completed write operation key=OZ1IST8DOGXW tag=P1WPL30TXCND type=write size=1045319 2024-04-17T04:45:15.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.112+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~N60G8TEBC2FX.0 size=1048576 2024-04-17T04:45:15.111 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.116+0000 7fd8ef2218c0 1 > prepared part key=_multipart_B63EZS0LAC30.2~N60G8TEBC2FX.1 size=50226 2024-04-17T04:45:15.111 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.116+0000 7fd8ef2218c0 1 > prepared multipart upload key=B63EZS0LAC30 upload=N60G8TEBC2FX tag=EUQRN3Z8TQBN type=write size=2147378 2024-04-17T04:45:15.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.116+0000 7fd8ef2218c0 1 < completed delete operation key=BCG05NIQFA53 tag=ZU2N6MDK7CKF type=del 2024-04-17T04:45:15.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.120+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=OSJSB6N7IY3A type=del size=1476020 2024-04-17T04:45:15.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.124+0000 7fd8ef2218c0 1 < canceled operation key=OZ1IST8DOGXW tag=C0JUGTQH1FXG type=write size=447036 2024-04-17T04:45:15.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.128+0000 7fd8ef2218c0 1 > prepared operation key=MS3T476JOEON tag=IYFIY8RDFMQJ type=del size=1688565 2024-04-17T04:45:15.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.132+0000 7fd8ef2218c0 1 < canceled operation key=5EFSN2FPUQB6 tag=YF3VE2HX6YQV type=del size=2426851 2024-04-17T04:45:15.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.136+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~KB3JZ0ZR6E0S.0 size=1048576 2024-04-17T04:45:15.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.140+0000 7fd8ef2218c0 1 > prepared part key=_multipart_5ZULD0PHLS4V.2~KB3JZ0ZR6E0S.1 size=680908 2024-04-17T04:45:15.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.140+0000 7fd8ef2218c0 1 > prepared multipart upload key=5ZULD0PHLS4V upload=KB3JZ0ZR6E0S tag=MUK4TMEJCQMK type=write size=2778060 2024-04-17T04:45:15.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.144+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~HK52R2BYSB3X.0 size=1048576 2024-04-17T04:45:15.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.148+0000 7fd8ef2218c0 1 < completed part key=_multipart_EWAW6EBW6W6U.2~HK52R2BYSB3X.1 size=499936 2024-04-17T04:45:15.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.152+0000 7fd8ef2218c0 1 < completed multipart upload key=EWAW6EBW6W6U upload=HK52R2BYSB3X tag=ZN0WM6MK943J type=write size=2597088 2024-04-17T04:45:15.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.156+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=841OCTB2Z51W type=del size=1710781 2024-04-17T04:45:15.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.160+0000 7fd8ef2218c0 1 < completed delete operation key=7BV3V2XMLSPH tag=C1QOO26EBUG0 type=del 2024-04-17T04:45:15.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.164+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~0L6F18ZBCQVC.0 size=1048576 2024-04-17T04:45:15.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.168+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~0L6F18ZBCQVC.1 size=1048576 2024-04-17T04:45:15.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.168+0000 7fd8ef2218c0 1 > prepared part key=_multipart_UNOVYXDS5JEB.2~0L6F18ZBCQVC.2 size=287672 2024-04-17T04:45:15.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.168+0000 7fd8ef2218c0 1 > prepared multipart upload key=UNOVYXDS5JEB upload=0L6F18ZBCQVC tag=K0URICCV3GTE type=write size=3433400 2024-04-17T04:45:15.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.168+0000 7fd8ef2218c0 -1 < no bi entry to suggest for operation key=VUYM3JC57QRP tag=9PQUTKHM6YL8 type=del size=2481196: (2) No such file or directory 2024-04-17T04:45:15.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.172+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=O8MCETD9MJES type=write size=1007064 2024-04-17T04:45:15.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.176+0000 7fd8ef2218c0 1 < completed delete operation key=509DZ9DBJWC5 tag=ZXJHSZF7GR8X type=del 2024-04-17T04:45:15.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.180+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=M1KTB3MH01VF type=del size=1757223 2024-04-17T04:45:15.178 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.184+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~FPS23JXUV419.0 size=464395 2024-04-17T04:45:15.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.188+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=FPS23JXUV419 tag=GM21DVDALXUH type=write size=1512971 2024-04-17T04:45:15.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.192+0000 7fd8ef2218c0 1 > prepared operation key=UEWLJ3E9ODUL tag=6R5W33EVTSII type=del size=1057337 2024-04-17T04:45:15.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.196+0000 7fd8ef2218c0 1 < completed delete operation key=GX9X0D45GC2I tag=H7D06VQJD4QK type=del 2024-04-17T04:45:15.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.196+0000 7fd8ef2218c0 1 > prepared operation key=R3C8PAVPVGBY tag=9O21LTZHL9GA type=del size=3558391 2024-04-17T04:45:15.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.200+0000 7fd8ef2218c0 1 < completed delete operation key=UYCKMXE6LBUH tag=66Y6YV93Q40F type=del 2024-04-17T04:45:15.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.204+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=44K21N9O4BRL type=del size=4173303 2024-04-17T04:45:15.205 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.208+0000 7fd8ef2218c0 1 < suggested update operation key=5FR1NEWE0LD1 tag=C9XNKJ93G5BS type=del size=1128615 2024-04-17T04:45:15.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.212+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=SGPRR8B0VD6G type=del size=2505659 2024-04-17T04:45:15.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.216+0000 7fd8ef2218c0 1 < completed delete operation key=6219DPY72LWC tag=DTV6H77FCC19 type=del 2024-04-17T04:45:15.214 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.220+0000 7fd8ef2218c0 1 > prepared operation key=AE4CYL334ONO tag=2UM7P59KA90Q type=del size=2646014 2024-04-17T04:45:15.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.220+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~OMN9JFSZV4TN.0 size=1048576 2024-04-17T04:45:15.219 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.224+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~OMN9JFSZV4TN.1 size=1048576 2024-04-17T04:45:15.222 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.228+0000 7fd8ef2218c0 1 < completed part key=_multipart_VUYM3JC57QRP.2~OMN9JFSZV4TN.2 size=296235 2024-04-17T04:45:15.226 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.232+0000 7fd8ef2218c0 1 < completed multipart upload key=VUYM3JC57QRP upload=OMN9JFSZV4TN tag=9GI03SM29DGU type=write size=3441963 2024-04-17T04:45:15.229 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.232+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=YV4S45GZSZ3T type=del size=1455702 2024-04-17T04:45:15.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.236+0000 7fd8ef2218c0 1 < completed delete operation key=5K50O96P7OGD tag=BWRIDJ3FJZVO type=del 2024-04-17T04:45:15.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.240+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=J0PWQ4DJZYHX type=del size=72912 2024-04-17T04:45:15.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.244+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~ZQLMQ268MYJ5.0 size=1048576 2024-04-17T04:45:15.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.244+0000 7fd8ef2218c0 1 < completed part key=_multipart_7BV3V2XMLSPH.2~ZQLMQ268MYJ5.1 size=32610 2024-04-17T04:45:15.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.252+0000 7fd8ef2218c0 1 < completed multipart upload key=7BV3V2XMLSPH upload=ZQLMQ268MYJ5 tag=SJIDCI08YSPE type=write size=2129762 2024-04-17T04:45:15.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.252+0000 7fd8ef2218c0 1 > prepared operation key=G03VLS09F3BT tag=ARIAL56W7J53 type=del size=2880184 2024-04-17T04:45:15.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.256+0000 7fd8ef2218c0 1 < completed part key=_multipart_B63EZS0LAC30.2~N60G8TEBC2FX.0 size=1048576 2024-04-17T04:45:15.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.256+0000 7fd8ef2218c0 1 < canceled part key=_multipart_B63EZS0LAC30.2~N60G8TEBC2FX.1 size=50226 2024-04-17T04:45:15.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.264+0000 7fd8ef2218c0 1 < canceled multipart upload key=B63EZS0LAC30 upload=N60G8TEBC2FX tag=EUQRN3Z8TQBN type=write size=2147378 2024-04-17T04:45:15.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.264+0000 7fd8ef2218c0 1 > prepared operation key=N11Y8FPN9XES tag=2NO95ZL2ZMFM type=del size=4094329 2024-04-17T04:45:15.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.268+0000 7fd8ef2218c0 1 < completed delete operation key=5ZULD0PHLS4V tag=OSJSB6N7IY3A type=del 2024-04-17T04:45:15.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.272+0000 7fd8ef2218c0 1 > prepared operation key=5ZULD0PHLS4V tag=276HBNWS14MH type=del size=1554624 2024-04-17T04:45:15.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.276+0000 7fd8ef2218c0 1 < completed delete operation key=MS3T476JOEON tag=IYFIY8RDFMQJ type=del 2024-04-17T04:45:15.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.280+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~WOXDHPFK45F6.0 size=1048576 2024-04-17T04:45:15.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~WOXDHPFK45F6.1 size=1048576 2024-04-17T04:45:15.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.284+0000 7fd8ef2218c0 1 > prepared part key=_multipart_VUYM3JC57QRP.2~WOXDHPFK45F6.2 size=368977 2024-04-17T04:45:15.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.284+0000 7fd8ef2218c0 1 > prepared multipart upload key=VUYM3JC57QRP upload=WOXDHPFK45F6 tag=2NRXTSM2WTPN type=write size=3514705 2024-04-17T04:45:15.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.288+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~KB3JZ0ZR6E0S.0 size=1048576 2024-04-17T04:45:15.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.292+0000 7fd8ef2218c0 1 < completed part key=_multipart_5ZULD0PHLS4V.2~KB3JZ0ZR6E0S.1 size=680908 2024-04-17T04:45:15.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.296+0000 7fd8ef2218c0 1 < completed multipart upload key=5ZULD0PHLS4V upload=KB3JZ0ZR6E0S tag=MUK4TMEJCQMK type=write size=2778060 2024-04-17T04:45:15.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.296+0000 7fd8ef2218c0 1 > prepared operation key=7BV3V2XMLSPH tag=6FCKM9FCU0ZF type=del size=240202 2024-04-17T04:45:15.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.300+0000 7fd8ef2218c0 1 < completed delete operation key=3OFQH4KVYN18 tag=841OCTB2Z51W type=del 2024-04-17T04:45:15.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.304+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~1WF642YG0A2S.0 size=1048576 2024-04-17T04:45:15.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.308+0000 7fd8ef2218c0 1 > prepared part key=_multipart_JVWRJBMFIRLL.2~1WF642YG0A2S.1 size=1001159 2024-04-17T04:45:15.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.308+0000 7fd8ef2218c0 1 > prepared multipart upload key=JVWRJBMFIRLL upload=1WF642YG0A2S tag=CKNM08Y88CW0 type=write size=3098311 2024-04-17T04:45:15.306 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~0L6F18ZBCQVC.0 size=1048576 2024-04-17T04:45:15.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.312+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~0L6F18ZBCQVC.1 size=1048576 2024-04-17T04:45:15.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.316+0000 7fd8ef2218c0 1 < completed part key=_multipart_UNOVYXDS5JEB.2~0L6F18ZBCQVC.2 size=287672 2024-04-17T04:45:15.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.320+0000 7fd8ef2218c0 1 < completed multipart upload key=UNOVYXDS5JEB upload=0L6F18ZBCQVC tag=K0URICCV3GTE type=write size=3433400 2024-04-17T04:45:15.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.324+0000 7fd8ef2218c0 1 > prepared operation key=6WSZ5UL0V1Y1 tag=N9XQ3HBW8VE0 type=write size=925304 2024-04-17T04:45:15.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.324+0000 7fd8ef2218c0 1 < completed write operation key=3OFQH4KVYN18 tag=O8MCETD9MJES type=write size=1007064 2024-04-17T04:45:15.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.332+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~2BE9ZFXAQOVX.0 size=363530 2024-04-17T04:45:15.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.332+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=2BE9ZFXAQOVX tag=7GB3J6RGBPSI type=write size=1412106 2024-04-17T04:45:15.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.336+0000 7fd8ef2218c0 1 < completed delete operation key=2IY8DB37FI9H tag=M1KTB3MH01VF type=del 2024-04-17T04:45:15.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.336+0000 7fd8ef2218c0 1 > prepared operation key=2IY8DB37FI9H tag=P4HZ0DQ0LDAC type=del size=3175889 2024-04-17T04:45:15.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.344+0000 7fd8ef2218c0 1 < suggested remove operation key=UEWLJ3E9ODUL tag=6R5W33EVTSII type=del 2024-04-17T04:45:15.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.344+0000 7fd8ef2218c0 1 > prepared operation key=3OFQH4KVYN18 tag=WZRFRXMIEEML type=del size=3043911 2024-04-17T04:45:15.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.348+0000 7fd8ef2218c0 1 < completed delete operation key=R3C8PAVPVGBY tag=9O21LTZHL9GA type=del 2024-04-17T04:45:15.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.352+0000 7fd8ef2218c0 1 > prepared operation key=5FR1NEWE0LD1 tag=E3HJ1CE4FQV8 type=del size=3968706 2024-04-17T04:45:15.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.356+0000 7fd8ef2218c0 1 < completed delete operation key=6WSZ5UL0V1Y1 tag=44K21N9O4BRL type=del 2024-04-17T04:45:15.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.356+0000 7fd8ef2218c0 1 > prepared operation key=GX9X0D45GC2I tag=W80UWBMSHMKQ type=del size=978108 2024-04-17T04:45:15.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.364+0000 7fd8ef2218c0 1 < completed delete operation key=OZ1IST8DOGXW tag=SGPRR8B0VD6G type=del 2024-04-17T04:45:15.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.368+0000 7fd8ef2218c0 1 > prepared operation key=OZ1IST8DOGXW tag=WPB9SS01UY3W type=del size=3078730 2024-04-17T04:45:15.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.372+0000 7fd8ef2218c0 1 < completed delete operation key=AE4CYL334ONO tag=2UM7P59KA90Q type=del 2024-04-17T04:45:15.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.376+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~A6AAVF1PD4EG.0 size=1048576 2024-04-17T04:45:15.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.376+0000 7fd8ef2218c0 1 > prepared part key=_multipart_AE4CYL334ONO.2~A6AAVF1PD4EG.1 size=414633 2024-04-17T04:45:15.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.376+0000 7fd8ef2218c0 1 > prepared multipart upload key=AE4CYL334ONO upload=A6AAVF1PD4EG tag=KMLNDJ3O5IJZ type=write size=2511785 2024-04-17T04:45:15.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.380+0000 7fd8ef2218c0 1 < completed delete operation key=5FR1NEWE0LD1 tag=YV4S45GZSZ3T type=del 2024-04-17T04:45:15.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~V2DF82MIZ49O.0 size=1048576 2024-04-17T04:45:15.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.388+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~V2DF82MIZ49O.1 size=1048576 2024-04-17T04:45:15.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.392+0000 7fd8ef2218c0 1 > prepared part key=_multipart_2S1IT3TO1BPU.2~V2DF82MIZ49O.2 size=496497 2024-04-17T04:45:15.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.392+0000 7fd8ef2218c0 1 > prepared multipart upload key=2S1IT3TO1BPU upload=V2DF82MIZ49O tag=CEY5A940NKHU type=write size=3642225 2024-04-17T04:45:15.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.396+0000 7fd8ef2218c0 1 < suggested remove operation key=OZ1IST8DOGXW tag=J0PWQ4DJZYHX type=del 2024-04-17T04:45:15.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.400+0000 7fd8ef2218c0 1 > prepared part key=_multipart_N11Y8FPN9XES.2~D9TVZ3HIMVVK.0 size=923704 2024-04-17T04:45:15.396 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:45:15.400+0000 7fd8ef2218c0 1 > prepared multipart upload key=N11Y8FPN9XES upload=D9TVZ3HIMVVK tag=D4B12F88RCF8 type=write size=1972280 2024-04-17T04:45:15.397 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] cls_rgw_stats.simulate (22546 ms) 2024-04-17T04:45:15.397 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 1 test from cls_rgw_stats (22546 ms total) 2024-04-17T04:45:15.397 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:45:15.397 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:45:15.953 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 1 test from 1 test suite ran. (24993 ms total) 2024-04-17T04:45:15.953 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 1 test. 2024-04-17T04:45:15.961 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:45:15.962 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:45:16.019 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-04-17T04:45:16.020 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-04-17T04:45:16.020 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-04-17T04:45:16.084 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:45:16.084 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 31 tests from 1 test suite. 2024-04-17T04:45:16.084 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:45:17.993 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 31 tests from CmpOmap 2024-04-17T04:45:17.993 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-04-17T04:45:18.007 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (15 ms) 2024-04-17T04:45:18.007 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-04-17T04:45:18.015 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (8 ms) 2024-04-17T04:45:18.015 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-04-17T04:45:18.020 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (5 ms) 2024-04-17T04:45:18.020 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-04-17T04:45:18.026 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (6 ms) 2024-04-17T04:45:18.026 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-04-17T04:45:18.044 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_str (18 ms) 2024-04-17T04:45:18.044 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-04-17T04:45:18.055 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_u64 (11 ms) 2024-04-17T04:45:18.055 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-04-17T04:45:18.061 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (6 ms) 2024-04-17T04:45:18.061 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-04-17T04:45:18.072 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (11 ms) 2024-04-17T04:45:18.072 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-04-17T04:45:18.084 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (12 ms) 2024-04-17T04:45:18.085 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-04-17T04:45:18.096 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (12 ms) 2024-04-17T04:45:18.096 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-04-17T04:45:18.098 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (2 ms) 2024-04-17T04:45:18.098 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-04-17T04:45:18.100 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (2 ms) 2024-04-17T04:45:18.100 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-04-17T04:45:18.118 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (18 ms) 2024-04-17T04:45:18.118 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-04-17T04:45:18.135 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (17 ms) 2024-04-17T04:45:18.135 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-04-17T04:45:18.154 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (19 ms) 2024-04-17T04:45:18.154 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-04-17T04:45:18.170 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (16 ms) 2024-04-17T04:45:18.170 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-04-17T04:45:18.192 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_str (22 ms) 2024-04-17T04:45:18.192 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-04-17T04:45:18.212 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (20 ms) 2024-04-17T04:45:18.213 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-04-17T04:45:18.220 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (7 ms) 2024-04-17T04:45:18.220 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-04-17T04:45:18.226 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (7 ms) 2024-04-17T04:45:18.226 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-04-17T04:45:18.229 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-04-17T04:45:18.229 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-04-17T04:45:18.231 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (2 ms) 2024-04-17T04:45:18.231 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-04-17T04:45:18.249 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (18 ms) 2024-04-17T04:45:18.249 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-04-17T04:45:18.265 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (16 ms) 2024-04-17T04:45:18.266 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-04-17T04:45:18.285 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (20 ms) 2024-04-17T04:45:18.285 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-04-17T04:45:18.307 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (21 ms) 2024-04-17T04:45:18.307 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-04-17T04:45:18.312 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (6 ms) 2024-04-17T04:45:18.313 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-04-17T04:45:18.319 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (7 ms) 2024-04-17T04:45:18.319 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-04-17T04:45:18.322 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-04-17T04:45:18.322 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-04-17T04:45:18.324 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (2 ms) 2024-04-17T04:45:18.324 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-04-17T04:45:18.344 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (19 ms) 2024-04-17T04:45:18.344 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 31 tests from CmpOmap (351 ms total) 2024-04-17T04:45:18.344 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:45:18.344 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:45:18.977 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 31 tests from 1 test suite ran. (2893 ms total) 2024-04-17T04:45:18.977 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 31 tests. 2024-04-17T04:45:18.983 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:45:18.983 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:45:19.039 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-04-17T04:45:19.041 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-04-17T04:45:19.042 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-04-17T04:45:19.105 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:45:19.105 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-17T04:45:19.105 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:45:19.106 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from TestCls2PCQueue 2024-04-17T04:45:19.106 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-04-17T04:45:22.014 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2909 ms) 2024-04-17T04:45:22.014 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-04-17T04:45:25.068 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3053 ms) 2024-04-17T04:45:25.068 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-04-17T04:45:28.096 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.Reserve (3028 ms) 2024-04-17T04:45:28.096 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-04-17T04:45:31.146 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3051 ms) 2024-04-17T04:45:31.146 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-04-17T04:45:35.214 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.Commit (4064 ms) 2024-04-17T04:45:35.214 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-04-17T04:45:39.265 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.Stats (4054 ms) 2024-04-17T04:45:39.265 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-04-17T04:45:43.339 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (4074 ms) 2024-04-17T04:45:43.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-04-17T04:45:46.366 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.Abort (3027 ms) 2024-04-17T04:45:46.367 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-04-17T04:45:49.380 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.ReserveError (3014 ms) 2024-04-17T04:45:49.380 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-04-17T04:45:52.452 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.CommitError (3072 ms) 2024-04-17T04:45:52.452 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-04-17T04:45:55.515 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.AbortError (3062 ms) 2024-04-17T04:45:55.515 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-04-17T04:45:58.543 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3028 ms) 2024-04-17T04:45:58.543 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-04-17T04:46:01.604 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3062 ms) 2024-04-17T04:46:01.604 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-04-17T04:46:04.615 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3011 ms) 2024-04-17T04:46:04.615 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-04-17T04:46:07.622 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3006 ms) 2024-04-17T04:46:07.622 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-04-17T04:46:10.650 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3027 ms) 2024-04-17T04:46:10.651 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-04-17T04:46:13.685 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3036 ms) 2024-04-17T04:46:13.685 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-04-17T04:46:16.712 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.Cleanup (3028 ms) 2024-04-17T04:46:16.713 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-04-17T04:46:28.874 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.MultiProducer (12156 ms) 2024-04-17T04:46:28.874 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-04-17T04:46:33.951 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (5083 ms) 2024-04-17T04:46:33.952 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-04-17T04:46:46.873 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (12922 ms) 2024-04-17T04:46:46.874 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-04-17T04:46:52.981 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (6104 ms) 2024-04-17T04:46:52.982 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-04-17T04:47:02.876 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (9899 ms) 2024-04-17T04:47:02.876 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-04-17T04:47:12.882 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (10005 ms) 2024-04-17T04:47:12.882 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from TestCls2PCQueue (113776 ms total) 2024-04-17T04:47:12.882 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:12.882 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:12.882 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 24 tests from 1 test suite ran. (113777 ms total) 2024-04-17T04:47:12.882 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 24 tests. 2024-04-17T04:47:12.887 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:47:12.888 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:47:12.946 INFO:tasks.workunit:Running workunits matching cls/test_cls_user.sh on client.0... 2024-04-17T04:47:12.947 INFO:tasks.workunit:Running workunit cls/test_cls_user.sh... 2024-04-17T04:47:12.947 DEBUG:teuthology.orchestra.run.smithi088:workunit test cls/test_cls_user.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh 2024-04-17T04:47:13.024 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:13.024 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:13.024 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-17T04:47:13.025 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:14.904 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 4 tests from ClsAccount 2024-04-17T04:47:14.904 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsAccount.add 2024-04-17T04:47:14.931 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsAccount.add (27 ms) 2024-04-17T04:47:14.931 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsAccount.get 2024-04-17T04:47:14.939 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsAccount.get (9 ms) 2024-04-17T04:47:14.939 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsAccount.rm 2024-04-17T04:47:14.952 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsAccount.rm (13 ms) 2024-04-17T04:47:14.952 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] ClsAccount.list 2024-04-17T04:47:14.992 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] ClsAccount.list (39 ms) 2024-04-17T04:47:14.992 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 4 tests from ClsAccount (88 ms total) 2024-04-17T04:47:14.992 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:14.992 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:15.914 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 4 tests from 1 test suite ran. (2889 ms total) 2024-04-17T04:47:15.915 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 4 tests. 2024-04-17T04:47:15.921 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:47:15.921 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:47:15.976 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-04-17T04:47:15.977 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-04-17T04:47:15.977 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-04-17T04:47:16.054 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:16.055 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:16.055 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 6 tests from 1 test suite. 2024-04-17T04:47:16.055 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:17.953 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 6 tests from rgw_gc_log 2024-04-17T04:47:17.953 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-04-17T04:47:17.981 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] rgw_gc_log.init_existing_queue (28 ms) 2024-04-17T04:47:17.981 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-04-17T04:47:17.988 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] rgw_gc_log.init_existing_omap (7 ms) 2024-04-17T04:47:17.988 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-04-17T04:47:17.994 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (6 ms) 2024-04-17T04:47:17.994 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-04-17T04:47:17.997 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (3 ms) 2024-04-17T04:47:17.997 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-04-17T04:47:18.003 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] rgw_gc_log.defer1_after_init (6 ms) 2024-04-17T04:47:18.003 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-04-17T04:47:18.006 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] rgw_gc_log.defer2_before_init (4 ms) 2024-04-17T04:47:18.007 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 6 tests from rgw_gc_log (55 ms total) 2024-04-17T04:47:18.007 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:18.007 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:18.957 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 6 tests from 1 test suite ran. (2902 ms total) 2024-04-17T04:47:18.957 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 6 tests. 2024-04-17T04:47:18.963 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:47:18.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:47:19.018 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-04-17T04:47:19.020 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-04-17T04:47:19.020 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:Running main() from gmock_main.cc 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 4 tests from TestRGWObj 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestRGWObj.underscore 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns= i= 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.092 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns= i= 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns= i= 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.093 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.094 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.095 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.096 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.097 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns=ns i= 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns=ns i= 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns=ns i= 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.099 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.100 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.101 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.102 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.103 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns= i=v1 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns= i=v1 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.104 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns= i=v1 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.105 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.106 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.107 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.108 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.109 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns=ns i=v1 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns=ns i=v1 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: _obj[v1] 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: _obj ns=ns i=v1 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.110 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.111 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.112 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.113 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.114 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_obj", 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.115 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestRGWObj.underscore (0 ms) 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestRGWObj.no_underscore 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns= i= 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns= i= 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns= i= 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.116 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.117 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.118 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.121 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.121 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.122 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.123 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.124 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns=ns i= 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns=ns i= 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns=ns i= 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.125 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.126 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.127 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.128 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.129 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns= i=v1 2024-04-17T04:47:19.130 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns= i=v1 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns= i=v1 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.131 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.132 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.133 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.134 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.135 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns=ns i=v1 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns=ns i=v1 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:parsed: obj[v1] 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:stripped: obj ns=ns i=v1 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout: "obj1": { 2024-04-17T04:47:19.136 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.137 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "obj2": { 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.138 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3": { 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.139 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "obj3-2": { 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.140 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "obj", 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "v1", 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestRGWObj.no_underscore (1 ms) 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-04-17T04:47:19.141 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.142 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.143 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker_myobj", 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.144 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.145 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.146 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.147 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker_myobj", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.148 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.149 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.150 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.151 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns_myobj", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.152 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.153 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.154 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns_myobj", 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.155 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.156 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.157 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.158 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst_myobj", 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.159 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.160 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.161 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.162 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst_myobj", 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.163 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.164 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.165 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.166 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst_myobj", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.167 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.168 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.169 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.170 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst_myobj", 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.171 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.172 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.173 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.174 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj", 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.175 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.176 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.177 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.178 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj", 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.179 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.180 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.181 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.182 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj", 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.183 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.184 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.185 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.186 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj", 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.187 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.188 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.189 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.190 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj", 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.191 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.192 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.193 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.194 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj", 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.195 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.196 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.197 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.198 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj", 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.199 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.200 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.201 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.202 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj", 2024-04-17T04:47:19.203 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.204 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.205 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.206 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.207 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj_", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.208 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.209 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.210 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.211 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj_", 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.212 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.213 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.214 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.215 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj_", 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.216 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.217 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.218 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.219 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj_", 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.220 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.221 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.222 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.223 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj_", 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.224 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.225 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.226 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.227 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj_", 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.228 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.229 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.230 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.231 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.233 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.233 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.233 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.234 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": "default-placement.data", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "test", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": "", 2024-04-17T04:47:19.235 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": "" 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.236 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "obj": { 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.237 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.238 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.239 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestRGWObj.obj_to_raw (0 ms) 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.240 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker_myobj", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.241 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker_myobj 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker_myobj", 2024-04-17T04:47:19.242 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.243 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker_myobj 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns_myobj", 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.244 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.245 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__ns_myobj 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns_myobj", 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.246 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.247 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__ns_myobj 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst_myobj", 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.248 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.249 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__:inst_myobj 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst_myobj", 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.250 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.251 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__:inst_myobj 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst_myobj", 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.252 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.253 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst_myobj", 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.254 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.255 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "myobj", 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj", 2024-04-17T04:47:19.256 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.257 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker___myobj 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj", 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.258 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.259 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker___myobj 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj", 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.260 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.261 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__ns__myobj 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj", 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.262 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.263 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__ns__myobj 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj", 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.264 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.265 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__:inst__myobj 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj", 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj" 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.266 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__:inst__myobj 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.267 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.268 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj", 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.269 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj", 2024-04-17T04:47:19.270 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj_", 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.271 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.272 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker___myobj_ 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker___myobj_", 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.273 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.274 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker___myobj_ 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj_", 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.275 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.276 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__ns__myobj_ 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns__myobj_", 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.277 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "", 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-04-17T04:47:19.278 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj_", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.279 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__:inst__myobj_", 2024-04-17T04:47:19.280 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "marker__myobj_" 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.281 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "" 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj": { 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-17T04:47:19.282 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj": { 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.283 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "raw_obj2": { 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "pool": ".data-pool", 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: "loc": "" 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.284 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "new_obj2": { 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket": { 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "ebtest", 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "marker": "marker", 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_id": "bucket-id", 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "tenant": "", 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "explicit_placement": { 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "data_pool": ".data-pool", 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "data_extra_pool": "", 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "index_pool": ".index-pool" 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "key": { 2024-04-17T04:47:19.285 INFO:tasks.workunit.client.0.smithi088.stdout: "name": "_myobj_", 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout: "instance": "inst", 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout: "ns": "ns" 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] TestRGWObj.old_to_raw (0 ms) 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 4 tests from TestRGWObj (1 ms total) 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 4 tests from 1 test suite ran. (1 ms total) 2024-04-17T04:47:19.286 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 4 tests. 2024-04-17T04:47:19.287 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:47:19.287 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:47:19.301 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-04-17T04:47:19.303 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-04-17T04:47:19.303 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-04-17T04:47:19.461 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:19.461 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:19.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 realm 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.516 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.518+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.522+0000 7fd4354f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:47:19.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.522+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.522+0000 7fd4354f2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:47:19.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.522+0000 7fd4354f2980 20 searching for the correct realm 2024-04-17T04:47:19.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.526+0000 7fd43280c640 0 -- 172.21.15.88:0/2420078056 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fd424027040 msgr2=0x7fd42404f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:47:19.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.742+0000 7fd43300d640 0 -- 172.21.15.88:0/2420078056 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fd41c078980 msgr2=0x7fd41c00eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:47:19.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:47:19.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 zone default found 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 4 Realm: () 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 zonegroup default 2024-04-17T04:47:19.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.961 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.961 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.962+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.962 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.966+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.966+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.966+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.966+0000 7fd4354f2980 20 started sync module instance, tier type = 2024-04-17T04:47:19.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.966+0000 7fd4354f2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=0 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=1 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=2 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=3 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=4 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=5 2024-04-17T04:47:19.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.990+0000 7fd4354f2980 20 add_watcher() i=6 2024-04-17T04:47:19.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.994+0000 7fd4354f2980 20 add_watcher() i=7 2024-04-17T04:47:19.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.994+0000 7fd4354f2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:47:19.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd2e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:47:19.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:47:19.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:47:19.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 5 note: GC not initialized 2024-04-17T04:47:19.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd2e8fe1640 20 reqs_thread_entry: start 2024-04-17T04:47:19.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 init_complete bucket index max shards: 1 2024-04-17T04:47:19.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 Filter name: none 2024-04-17T04:47:19.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd2627fc640 20 reqs_thread_entry: start 2024-04-17T04:47:19.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-04-17T04:47:19.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-04-17T04:47:19.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-04-17T04:47:19.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:19.998+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 cache get: name=default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= : miss 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 20 rados->read ofs=0 len=0 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 cache put: name=default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= info.flags=0x0 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 adding default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= to cache LRU end 2024-04-17T04:47:19.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 cache get: name=default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= : hit (negative entry) 2024-04-17T04:47:20.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 cache get: name=default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= : hit (negative entry) 2024-04-17T04:47:20.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-04-17T04:47:20.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-04-17T04:47:20.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.002+0000 7fd4354f2980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-04-17T04:47:20.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.006+0000 7fd316ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824002 cookie 94272477903520 notifier 5532 bl.length()=697 2024-04-17T04:47:20.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.006+0000 7fd316ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-04-17T04:47:20.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.006+0000 7fd316ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-04-17T04:47:20.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.010+0000 7fd4354f2980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-04-17T04:47:20.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.010+0000 7fd4354f2980 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-04-17T04:47:20.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.010+0000 7fd4354f2980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-04-17T04:47:20.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.010+0000 7fd316ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824000 cookie 94272477899632 notifier 5532 bl.length()=194 2024-04-17T04:47:20.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.010+0000 7fd316ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-04-17T04:47:20.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.010+0000 7fd316ffd640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-04-17T04:47:20.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.018+0000 7fd4354f2980 10 cache put: name=default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= info.flags=0x7 2024-04-17T04:47:20.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.018+0000 7fd4354f2980 10 moving default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= to cache LRU end 2024-04-17T04:47:20.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.018+0000 7fd4354f2980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:ogCnKmzxsEZN3PSz91mO/GmqQ3s=, ofs0, ns] 2024-04-17T04:47:20.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.018+0000 7fd3167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824003 cookie 94272477903520 notifier 5532 bl.length()=203 2024-04-17T04:47:20.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.018+0000 7fd3167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= info.flags=0x7 2024-04-17T04:47:20.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.018+0000 7fd3167fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ogCnKmzxsEZN3PSz91mO/GmqQ3s= to cache LRU end 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout: "user_id": "ceph-test-librgw-file", 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout: "display_name": "librgw test user", 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout: "email": "librgw@example.com", 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout: "suspended": 0, 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout: "max_buckets": 1000, 2024-04-17T04:47:20.017 INFO:tasks.workunit.client.0.smithi088.stdout: "subusers": [], 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "keys": [ 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: { 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "user": "ceph-test-librgw-file", 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "access_key": "ogCnKmzxsEZN3PSz91mO/GmqQ3s=", 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "secret_key": "erKuyY5Eno/sPvqdHDPPFl7s2zVNTVXR95yIFkw20La61qX2Fh1Ing==", 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "active": true, 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "create_date": "2024-04-17T04:47:20.005604Z" 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: ], 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "swift_keys": [], 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "caps": [], 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "default_placement": "", 2024-04-17T04:47:20.018 INFO:tasks.workunit.client.0.smithi088.stdout: "default_storage_class": "", 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "placement_tags": [], 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_quota": { 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "enabled": false, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size": -1, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "max_objects": -1 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "user_quota": { 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "enabled": false, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size": -1, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "max_objects": -1 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "temp_url_keys": [], 2024-04-17T04:47:20.019 INFO:tasks.workunit.client.0.smithi088.stdout: "type": "rgw", 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: "mfa_ids": [], 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: "account_id": "", 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: "path": "/", 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: "create_date": "2024-04-17T04:47:20.005588Z", 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: "tags": [], 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: "group_ids": [] 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:20.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.022+0000 7fd2e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:47:20.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.026+0000 7fd4354f2980 20 remove_watcher() i=0 2024-04-17T04:47:20.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.026+0000 7fd4354f2980 2 removed watcher, disabling cache 2024-04-17T04:47:20.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.030+0000 7fd4354f2980 20 remove_watcher() i=1 2024-04-17T04:47:20.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.030+0000 7fd4354f2980 20 remove_watcher() i=2 2024-04-17T04:47:20.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.034+0000 7fd4354f2980 20 remove_watcher() i=3 2024-04-17T04:47:20.032 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.034+0000 7fd4354f2980 20 remove_watcher() i=4 2024-04-17T04:47:20.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.038+0000 7fd4354f2980 20 remove_watcher() i=5 2024-04-17T04:47:20.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.038+0000 7fd4354f2980 20 remove_watcher() i=6 2024-04-17T04:47:20.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:20.038+0000 7fd4354f2980 20 remove_watcher() i=7 2024-04-17T04:47:20.050 INFO:tasks.workunit.client.0.smithi088.stdout:phase 1.1 2024-04-17T04:47:20.071 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-17T04:47:20.071 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:20.071 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from LibRGW 2024-04-17T04:47:20.071 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:20.084 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:20.084 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:20.838 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (768 ms) 2024-04-17T04:47:20.839 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-04-17T04:47:20.841 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-04-17T04:47:20.841 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:20.841 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-17T04:47:20.841 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-04-17T04:47:20.860 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/quux 2024-04-17T04:47:20.869 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/f1 2024-04-17T04:47:20.877 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/f2 2024-04-17T04:47:20.886 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/f1 2024-04-17T04:47:20.894 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/d1/ 2024-04-17T04:47:20.903 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-04-17T04:47:20.912 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-04-17T04:47:20.921 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/momma 2024-04-17T04:47:20.930 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-04-17T04:47:20.938 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-04-17T04:47:20.949 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-04-17T04:47:20.958 INFO:tasks.workunit.client.0.smithi088.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-04-17T04:47:20.966 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETUP_HIER1 (125 ms) 2024-04-17T04:47:20.966 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-04-17T04:47:21.216 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETUP_DIRS1 (250 ms) 2024-04-17T04:47:21.216 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETATTR 2024-04-17T04:47:21.216 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-04-17T04:47:21.216 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-04-17T04:47:21.256 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (40 ms) 2024-04-17T04:47:21.256 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-04-17T04:47:21.366 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (108 ms) 2024-04-17T04:47:21.366 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-04-17T04:47:21.366 INFO:tasks.workunit.client.0.smithi088.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-04-17T04:47:21.386 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (21 ms) 2024-04-17T04:47:21.386 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-04-17T04:47:21.386 INFO:tasks.workunit.client.0.smithi088.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-04-17T04:47:21.404 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (18 ms) 2024-04-17T04:47:21.404 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout:scanning objects in /bdirs1/dir_0 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout:scanning objects in /bdirs1/dir_1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout:scanning objects in /bdirs1/dir_2 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:21.405 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-17T04:47:21.406 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (1 ms) 2024-04-17T04:47:21.406 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-04-17T04:47:21.406 INFO:tasks.workunit.client.0.smithi088.stdout:read back objects in /bdirs1/dir_0 2024-04-17T04:47:21.406 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_0/sfile_0 2024-04-17T04:47:21.409 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-17T04:47:21.409 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_0/sfile_1 2024-04-17T04:47:21.412 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-17T04:47:21.413 INFO:tasks.workunit.client.0.smithi088.stdout:read back objects in /bdirs1/dir_1 2024-04-17T04:47:21.413 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_1/sfile_0 2024-04-17T04:47:21.416 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-17T04:47:21.416 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_1/sfile_1 2024-04-17T04:47:21.419 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-17T04:47:21.419 INFO:tasks.workunit.client.0.smithi088.stdout:read back objects in /bdirs1/dir_2 2024-04-17T04:47:21.419 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_2/sfile_0 2024-04-17T04:47:21.423 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-17T04:47:21.423 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_2/sfile_1 2024-04-17T04:47:21.427 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-17T04:47:21.427 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READ_DIRS1 (22 ms) 2024-04-17T04:47:21.427 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READF_SETUP1 (227 ms) 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_0 type: DIR refs: 2 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_0 type: FILE refs: 2 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_1 type: DIR refs: 2 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_1 type: FILE refs: 2 2024-04-17T04:47:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_0 type: DIR refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_0 type: FILE refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_1 type: DIR refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_1 type: FILE refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_0 type: DIR refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_0 type: FILE refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_1 type: DIR refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_1 type: FILE refs: 2 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-04-17T04:47:21.655 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.HIER1 2024-04-17T04:47:21.673 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/f2 (f2): FILE 2024-04-17T04:47:21.674 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/f1 (f1): FILE 2024-04-17T04:47:21.681 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-04-17T04:47:21.686 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-04-17T04:47:21.693 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-17T04:47:21.695 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-17T04:47:21.696 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-04-17T04:47:21.697 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-04-17T04:47:21.699 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-17T04:47:21.700 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-17T04:47:21.701 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar (bar): DIR 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo (foo): DIR 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot (nfsroot): DIR 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.HIER1 (52 ms) 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-17T04:47:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:21.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:21.710+0000 7fb5fef63a80 -1 shutting down 2024-04-17T04:47:21.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:21.710+0000 7fb4da2d5640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56358, si_uid : 1000, si_addr0x3e80000dc26, si_status0 2024-04-17T04:47:21.772 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (66 ms) 2024-04-17T04:47:21.772 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from LibRGW (1701 ms total) 2024-04-17T04:47:21.772 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:21.772 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:21.773 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 24 tests from 1 test suite ran. (1701 ms total) 2024-04-17T04:47:21.773 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 24 tests. 2024-04-17T04:47:21.792 INFO:tasks.workunit.client.0.smithi088.stdout:phase 1.2 2024-04-17T04:47:21.813 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-17T04:47:21.813 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:21.813 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from LibRGW 2024-04-17T04:47:21.814 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:21.823 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:21.823 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:22.337 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (524 ms) 2024-04-17T04:47:22.337 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-04-17T04:47:22.339 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-04-17T04:47:22.339 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:22.339 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-17T04:47:22.339 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-04-17T04:47:22.339 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETATTR 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-04-17T04:47:22.340 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.HIER1 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-04-17T04:47:22.341 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:22.342 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:22.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:22.342+0000 7fed8d78ba80 -1 shutting down 2024-04-17T04:47:22.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:22.346+0000 7fec68afd640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57002, si_uid : 1000, si_addr0x3e80000deaa, si_status0 2024-04-17T04:47:22.497 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (158 ms) 2024-04-17T04:47:22.497 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from LibRGW (684 ms total) 2024-04-17T04:47:22.497 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:22.497 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:22.497 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 24 tests from 1 test suite ran. (684 ms total) 2024-04-17T04:47:22.497 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 24 tests. 2024-04-17T04:47:22.512 INFO:tasks.workunit.client.0.smithi088.stdout:phase 1.3 2024-04-17T04:47:22.534 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-17T04:47:22.534 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:22.534 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from LibRGW 2024-04-17T04:47:22.534 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:22.546 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:22.546 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:23.072 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (538 ms) 2024-04-17T04:47:23.072 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-04-17T04:47:23.074 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-04-17T04:47:23.074 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:23.074 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-17T04:47:23.074 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-04-17T04:47:23.083 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETUP_HIER1 (9 ms) 2024-04-17T04:47:23.083 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETUP_DIRS1 (110 ms) 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SETATTR 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-04-17T04:47:23.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout:scanning objects in /bdirs1/dir_0 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout:scanning objects in /bdirs1/dir_1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout:scanning objects in /bdirs1/dir_2 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:23.194 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-17T04:47:23.195 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-17T04:47:23.195 INFO:tasks.workunit.client.0.smithi088.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-17T04:47:23.195 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-04-17T04:47:23.195 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-04-17T04:47:23.195 INFO:tasks.workunit.client.0.smithi088.stdout:read back objects in /bdirs1/dir_0 2024-04-17T04:47:23.195 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_0/sfile_0 2024-04-17T04:47:23.196 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-17T04:47:23.196 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_0/sfile_1 2024-04-17T04:47:23.198 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-17T04:47:23.198 INFO:tasks.workunit.client.0.smithi088.stdout:read back objects in /bdirs1/dir_1 2024-04-17T04:47:23.198 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_1/sfile_0 2024-04-17T04:47:23.200 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-17T04:47:23.200 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_1/sfile_1 2024-04-17T04:47:23.202 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-17T04:47:23.202 INFO:tasks.workunit.client.0.smithi088.stdout:read back objects in /bdirs1/dir_2 2024-04-17T04:47:23.203 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_2/sfile_0 2024-04-17T04:47:23.205 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-17T04:47:23.205 INFO:tasks.workunit.client.0.smithi088.stdout:reading 0,256 dir_2/sfile_1 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READ_DIRS1 (15 ms) 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-04-17T04:47:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_0 type: DIR refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_0 type: FILE refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_1 type: DIR refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_1 type: FILE refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_0 type: DIR refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_0 type: FILE refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_1 type: DIR refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_1 type: FILE refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_0 type: DIR refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_0 type: FILE refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sdir_1 type: DIR refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:release sfile_1 type: FILE refs: 2 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-04-17T04:47:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.HIER1 2024-04-17T04:47:23.223 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/f2 (f2): FILE 2024-04-17T04:47:23.224 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/f1 (f1): FILE 2024-04-17T04:47:23.229 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-04-17T04:47:23.233 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-04-17T04:47:23.238 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-17T04:47:23.238 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-17T04:47:23.239 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-04-17T04:47:23.240 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-04-17T04:47:23.242 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-17T04:47:23.243 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-17T04:47:23.244 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo/bar (bar): DIR 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot/foo (foo): DIR 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:/nfsroot (nfsroot): DIR 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.HIER1 (39 ms) 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-17T04:47:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:23.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:23.250+0000 7eff8bb03a80 -1 shutting down 2024-04-17T04:47:23.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:23.254+0000 7efe66e75640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57646, si_uid : 1000, si_addr0x3e80000e12e, si_status0 2024-04-17T04:47:23.311 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (64 ms) 2024-04-17T04:47:23.311 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 24 tests from LibRGW (777 ms total) 2024-04-17T04:47:23.311 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:23.311 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:23.311 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 24 tests from 1 test suite ran. (777 ms total) 2024-04-17T04:47:23.311 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 24 tests. 2024-04-17T04:47:23.320 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2.1 2024-04-17T04:47:23.338 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-17T04:47:23.338 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:23.338 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 9 tests from LibRGW 2024-04-17T04:47:23.338 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:23.348 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:23.348 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:23.636 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (298 ms) 2024-04-17T04:47:23.636 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:23.637 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-17T04:47:23.637 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-17T04:47:23.653 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET (16 ms) 2024-04-17T04:47:23.653 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-17T04:47:23.653 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-04-17T04:47:23.653 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-04-17T04:47:23.667 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave0 2024-04-17T04:47:23.682 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave1 2024-04-17T04:47:23.699 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave2 2024-04-17T04:47:23.716 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave3 2024-04-17T04:47:23.735 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave4 2024-04-17T04:47:23.754 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave5 2024-04-17T04:47:23.776 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave6 2024-04-17T04:47:23.797 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave7 2024-04-17T04:47:23.815 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave8 2024-04-17T04:47:23.843 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave9 2024-04-17T04:47:23.843 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (190 ms) 2024-04-17T04:47:23.843 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-04-17T04:47:24.315 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (472 ms) 2024-04-17T04:47:24.315 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:24.315 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:24.315 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:24.316 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:24.316 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:24.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:24.318+0000 7fa1e0b0da80 -1 shutting down 2024-04-17T04:47:24.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:24.322+0000 7fa0bbe7f640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58290, si_uid : 1000, si_addr0x3e80000e3b2, si_status0 2024-04-17T04:47:24.487 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (172 ms) 2024-04-17T04:47:24.487 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 9 tests from LibRGW (1149 ms total) 2024-04-17T04:47:24.487 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:24.487 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:24.487 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 9 tests from 1 test suite ran. (1149 ms total) 2024-04-17T04:47:24.488 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 9 tests. 2024-04-17T04:47:24.501 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2.2 2024-04-17T04:47:24.524 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-17T04:47:24.525 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:24.525 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 9 tests from LibRGW 2024-04-17T04:47:24.525 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:24.536 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:24.536 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:24.853 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (328 ms) 2024-04-17T04:47:24.853 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:24.854 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-17T04:47:24.854 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-17T04:47:24.854 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-04-17T04:47:24.854 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-17T04:47:24.886 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET (32 ms) 2024-04-17T04:47:24.886 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-04-17T04:47:24.902 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave0 2024-04-17T04:47:24.919 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave1 2024-04-17T04:47:24.938 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave2 2024-04-17T04:47:24.956 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave3 2024-04-17T04:47:24.972 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave4 2024-04-17T04:47:24.995 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave5 2024-04-17T04:47:25.016 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave6 2024-04-17T04:47:25.046 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave7 2024-04-17T04:47:25.067 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave8 2024-04-17T04:47:25.087 INFO:tasks.workunit.client.0.smithi088.stdout:created: sorrydave9 2024-04-17T04:47:25.087 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (201 ms) 2024-04-17T04:47:25.087 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-04-17T04:47:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (481 ms) 2024-04-17T04:47:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:25.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:25.570+0000 7f1a7aa2da80 -1 shutting down 2024-04-17T04:47:25.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:25.574+0000 7f1955d9f640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58934, si_uid : 1000, si_addr0x3e80000e636, si_status0 2024-04-17T04:47:25.701 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (133 ms) 2024-04-17T04:47:25.701 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 9 tests from LibRGW (1176 ms total) 2024-04-17T04:47:25.701 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:25.701 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:25.701 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 9 tests from 1 test suite ran. (1176 ms total) 2024-04-17T04:47:25.701 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 9 tests. 2024-04-17T04:47:25.715 INFO:tasks.workunit.client.0.smithi088.stdout:phase 3.1 2024-04-17T04:47:26.057 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 13 tests from 1 test suite. 2024-04-17T04:47:26.057 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:26.057 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 13 tests from LibRGW 2024-04-17T04:47:26.057 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:26.067 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:26.067 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:26.346 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (290 ms) 2024-04-17T04:47:26.347 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:26.348 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-17T04:47:26.348 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-17T04:47:26.366 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET (18 ms) 2024-04-17T04:47:26.366 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-17T04:47:26.373 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (7 ms) 2024-04-17T04:47:26.373 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-17T04:47:26.376 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (3 ms) 2024-04-17T04:47:26.376 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LARGE1 2024-04-17T04:47:26.448 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LARGE1 (72 ms) 2024-04-17T04:47:26.448 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LARGE2 2024-04-17T04:47:26.527 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LARGE2 (79 ms) 2024-04-17T04:47:26.527 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-17T04:47:26.527 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-17T04:47:26.527 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-17T04:47:26.527 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:26.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:26.530+0000 7fbcc12daa80 -1 shutting down 2024-04-17T04:47:26.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:26.534+0000 7fbb9c64c640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59578, si_uid : 1000, si_addr0x3e80000e8ba, si_status0 2024-04-17T04:47:26.714 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (187 ms) 2024-04-17T04:47:26.714 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 13 tests from LibRGW (657 ms total) 2024-04-17T04:47:26.714 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:26.714 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:26.714 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 13 tests from 1 test suite ran. (657 ms total) 2024-04-17T04:47:26.715 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 13 tests. 2024-04-17T04:47:26.728 INFO:tasks.workunit.client.0.smithi088.stdout:phase 3.2 2024-04-17T04:47:27.071 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 13 tests from 1 test suite. 2024-04-17T04:47:27.071 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:27.071 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 13 tests from LibRGW 2024-04-17T04:47:27.071 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:27.079 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:27.080 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:27.375 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (304 ms) 2024-04-17T04:47:27.375 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:27.376 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-17T04:47:27.376 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-17T04:47:27.376 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-04-17T04:47:27.376 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-17T04:47:27.385 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (9 ms) 2024-04-17T04:47:27.385 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-17T04:47:27.393 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (9 ms) 2024-04-17T04:47:27.393 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LARGE1 2024-04-17T04:47:27.483 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LARGE1 (89 ms) 2024-04-17T04:47:27.483 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LARGE2 2024-04-17T04:47:27.483 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-04-17T04:47:27.483 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-17T04:47:27.483 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-17T04:47:27.483 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-17T04:47:27.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_OBJECT (12 ms) 2024-04-17T04:47:27.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET (43 ms) 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:27.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:27.542+0000 7fb436b50a80 -1 shutting down 2024-04-17T04:47:27.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:27.542+0000 7fb311ec2640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60222, si_uid : 1000, si_addr0x3e80000eb3e, si_status0 2024-04-17T04:47:27.608 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (71 ms) 2024-04-17T04:47:27.609 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 13 tests from LibRGW (538 ms total) 2024-04-17T04:47:27.609 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:27.609 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:27.609 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 13 tests from 1 test suite ran. (538 ms total) 2024-04-17T04:47:27.609 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 13 tests. 2024-04-17T04:47:27.620 INFO:tasks.workunit.client.0.smithi088.stdout:phase 4.1 2024-04-17T04:47:27.641 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-17T04:47:27.641 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:27.641 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 11 tests from LibRGW 2024-04-17T04:47:27.642 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.TVAR 2024-04-17T04:47:27.642 INFO:tasks.workunit.client.0.smithi088.stdout:read i1: 64001 2024-04-17T04:47:27.642 INFO:tasks.workunit.client.0.smithi088.stdout:read s1: blunderbuss 2024-04-17T04:47:27.642 INFO:tasks.workunit.client.0.smithi088.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-04-17T04:47:27.642 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:27.655 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:27.655 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:28.192 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (550 ms) 2024-04-17T04:47:28.192 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:28.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-04-17T04:47:28.193 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-17T04:47:28.226 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (33 ms) 2024-04-17T04:47:28.227 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_0 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_1 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_2 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_3 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_4 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_5 2024-04-17T04:47:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_6 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_7 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_8 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_9 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_10 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_11 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_12 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_13 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_14 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_15 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_16 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_17 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_18 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_19 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_20 2024-04-17T04:47:29.702 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_21 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_22 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_23 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_24 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_25 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_26 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_27 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_28 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_29 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_30 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_31 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_32 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_33 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_34 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_35 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_36 2024-04-17T04:47:29.703 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_37 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_38 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_39 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_40 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_41 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_42 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_43 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_44 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_45 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_46 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_47 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_48 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_49 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_50 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_51 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_52 2024-04-17T04:47:29.704 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_53 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_54 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_55 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_56 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_57 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_58 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_59 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_60 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_61 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_62 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_63 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_64 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_65 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_66 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_67 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_68 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_69 2024-04-17T04:47:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_70 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_71 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_72 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_73 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_74 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_75 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_76 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_77 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_78 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_79 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_80 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_81 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_82 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_83 2024-04-17T04:47:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_84 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_85 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_86 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_87 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_88 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_89 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_90 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_91 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_92 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_93 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_94 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_95 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_96 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_97 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_98 2024-04-17T04:47:29.707 INFO:tasks.workunit.client.0.smithi088.stdout:created: dmarker:f_99 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (1474 ms) 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-04-17T04:47:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-04-17T04:47:29.709 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-04-17T04:47:29.710 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-04-17T04:47:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-04-17T04:47:29.712 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-04-17T04:47:29.713 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-04-17T04:47:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_READDIR (7 ms) 2024-04-17T04:47:29.715 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-04-17T04:47:29.716 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-04-17T04:47:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-04-17T04:47:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-04-17T04:47:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-04-17T04:47:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-04-17T04:47:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER2_READDIR (5 ms) 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:29.722 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:29.723 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:29.723 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:29.723 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:29.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:29.714+0000 7fe15995fa80 -1 shutting down 2024-04-17T04:47:29.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:29.718+0000 7fe034cd1640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60866, si_uid : 1000, si_addr0x3e80000edc2, si_status0 2024-04-17T04:47:29.853 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (140 ms) 2024-04-17T04:47:29.853 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 11 tests from LibRGW (2211 ms total) 2024-04-17T04:47:29.853 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:29.853 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:29.853 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 11 tests from 1 test suite ran. (2211 ms total) 2024-04-17T04:47:29.853 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 11 tests. 2024-04-17T04:47:29.868 INFO:tasks.workunit.client.0.smithi088.stdout:phase 4.2 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 11 tests from LibRGW 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.TVAR 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:read i1: 64001 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:read s1: blunderbuss 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-04-17T04:47:29.891 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:29.900 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:29.900 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:30.476 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (585 ms) 2024-04-17T04:47:30.476 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:30.478 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-04-17T04:47:30.478 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-17T04:47:30.497 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (20 ms) 2024-04-17T04:47:30.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-17T04:47:30.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-17T04:47:30.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-17T04:47:30.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-17T04:47:30.498 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-04-17T04:47:30.505 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-04-17T04:47:30.505 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-04-17T04:47:30.505 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-04-17T04:47:30.505 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-04-17T04:47:30.506 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-04-17T04:47:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-04-17T04:47:30.508 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-04-17T04:47:30.509 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-04-17T04:47:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-04-17T04:47:30.511 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER2_READDIR (8 ms) 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:30.512 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:30.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:30.510+0000 7f3313232a80 -1 shutting down 2024-04-17T04:47:30.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:30.510+0000 7f31ee5a4640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 61510, si_uid : 1000, si_addr0x3e80000f046, si_status0 2024-04-17T04:47:30.594 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (88 ms) 2024-04-17T04:47:30.594 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 11 tests from LibRGW (703 ms total) 2024-04-17T04:47:30.595 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:30.595 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:30.595 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 11 tests from 1 test suite ran. (703 ms total) 2024-04-17T04:47:30.595 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 11 tests. 2024-04-17T04:47:30.603 INFO:tasks.workunit.client.0.smithi088.stdout:phase 5.1 2024-04-17T04:47:30.632 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 19 tests from 1 test suite. 2024-04-17T04:47:30.632 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:30.633 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 19 tests from LibRGW 2024-04-17T04:47:30.633 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:30.643 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:30.643 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:30.924 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (292 ms) 2024-04-17T04:47:30.924 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:30.925 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-04-17T04:47:30.925 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-17T04:47:30.940 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET (14 ms) 2024-04-17T04:47:30.940 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-17T04:47:30.945 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (6 ms) 2024-04-17T04:47:30.946 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-04-17T04:47:30.946 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-04-17T04:47:30.946 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-17T04:47:30.948 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (3 ms) 2024-04-17T04:47:30.948 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-04-17T04:47:30.948 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-04-17T04:47:30.948 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-04-17T04:47:30.957 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.PUT_OBJECT (9 ms) 2024-04-17T04:47:30.957 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-04-17T04:47:30.961 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.GET_OBJECT (4 ms) 2024-04-17T04:47:30.961 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITEV 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READV 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READV (0 ms) 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-17T04:47:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-04-17T04:47:30.963 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:30.963 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:30.963 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:30.963 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:30.963 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:30.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:30.966+0000 7fd8df15ca80 -1 shutting down 2024-04-17T04:47:30.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:30.966+0000 7fd7ba4ce640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 62154, si_uid : 1000, si_addr0x3e80000f2ca, si_status0 2024-04-17T04:47:31.078 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (116 ms) 2024-04-17T04:47:31.079 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 19 tests from LibRGW (446 ms total) 2024-04-17T04:47:31.079 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:31.079 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:31.079 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 19 tests from 1 test suite ran. (446 ms total) 2024-04-17T04:47:31.079 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 19 tests. 2024-04-17T04:47:31.090 INFO:tasks.workunit.client.0.smithi088.stdout:phase 5.2 2024-04-17T04:47:31.124 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 19 tests from 1 test suite. 2024-04-17T04:47:31.124 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:31.124 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 19 tests from LibRGW 2024-04-17T04:47:31.124 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:31.132 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:31.132 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:31.416 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (292 ms) 2024-04-17T04:47:31.416 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:31.418 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-04-17T04:47:31.418 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-17T04:47:31.418 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-04-17T04:47:31.418 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-17T04:47:31.428 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (10 ms) 2024-04-17T04:47:31.428 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-04-17T04:47:31.428 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-04-17T04:47:31.428 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-17T04:47:31.428 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.WRITEV 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READV 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READV (0 ms) 2024-04-17T04:47:31.429 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-04-17T04:47:31.430 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-04-17T04:47:31.430 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-17T04:47:31.439 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_OBJECT (11 ms) 2024-04-17T04:47:31.439 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-17T04:47:31.472 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.DELETE_BUCKET (33 ms) 2024-04-17T04:47:31.472 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:31.472 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:31.472 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:31.472 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:31.472 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:31.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:31.474+0000 7f0a70046a80 -1 shutting down 2024-04-17T04:47:31.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:31.478+0000 7f094b3b8640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 62798, si_uid : 1000, si_addr0x3e80000f54e, si_status0 2024-04-17T04:47:31.561 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (90 ms) 2024-04-17T04:47:31.562 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 19 tests from LibRGW (438 ms total) 2024-04-17T04:47:31.562 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:31.562 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:31.562 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 19 tests from 1 test suite ran. (438 ms total) 2024-04-17T04:47:31.562 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 19 tests. 2024-04-17T04:47:31.569 INFO:tasks.workunit.client.0.smithi088.stdout:phase 6.1 2024-04-17T04:47:31.587 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] Running 10 tests from 1 test suite. 2024-04-17T04:47:31.587 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment set-up. 2024-04-17T04:47:31.587 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 10 tests from LibRGW 2024-04-17T04:47:31.587 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.INIT 2024-04-17T04:47:31.596 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:31.596 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:32.087 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.INIT (500 ms) 2024-04-17T04:47:32.087 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.MOUNT 2024-04-17T04:47:32.088 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-04-17T04:47:32.088 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CREATE_BUCKETS 2024-04-17T04:47:32.125 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CREATE_BUCKETS (36 ms) 2024-04-17T04:47:32.125 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.LOOKUP_BUCKETS 2024-04-17T04:47:32.140 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.LOOKUP_BUCKETS (15 ms) 2024-04-17T04:47:32.140 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.TOPDIR_RENAME 2024-04-17T04:47:32.183 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.TOPDIR_RENAME (43 ms) 2024-04-17T04:47:32.183 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SUBDIR_RENAME 2024-04-17T04:47:32.240 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SUBDIR_RENAME (58 ms) 2024-04-17T04:47:32.241 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CROSS_BUCKET_RENAME 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CROSS_BUCKET_RENAME (52 ms) 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-17T04:47:32.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.294+0000 7f8b9079aa80 -1 shutting down 2024-04-17T04:47:32.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.298+0000 7f8a6bb0c640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 63442, si_uid : 1000, si_addr0x3e80000f7d2, si_status0 2024-04-17T04:47:32.458 INFO:tasks.workunit.client.0.smithi088.stdout:[ OK ] LibRGW.SHUTDOWN (165 ms) 2024-04-17T04:47:32.458 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] 10 tests from LibRGW (871 ms total) 2024-04-17T04:47:32.458 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:32.458 INFO:tasks.workunit.client.0.smithi088.stdout:[----------] Global test environment tear-down 2024-04-17T04:47:32.458 INFO:tasks.workunit.client.0.smithi088.stdout:[==========] 10 tests from 1 test suite ran. (871 ms total) 2024-04-17T04:47:32.458 INFO:tasks.workunit.client.0.smithi088.stdout:[ PASSED ] 10 tests. 2024-04-17T04:47:32.473 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:47:32.473 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:47:32.488 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-04-17T04:47:32.489 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-04-17T04:47:32.489 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-04-17T04:47:32.538 INFO:tasks.workunit.client.0.smithi088.stderr:+ [ -z ] 2024-04-17T04:47:32.538 INFO:tasks.workunit.client.0.smithi088.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-04-17T04:47:32.538 INFO:tasks.workunit.client.0.smithi088.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-04-17T04:47:32.538 INFO:tasks.workunit.client.0.smithi088.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2024-04-17T04:47:32.577 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:47:32.577 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:47:32.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.830+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 realm 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:47:32.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.834+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.838+0000 7f23613bc980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:47:32.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.838+0000 7f23613bc980 20 searching for the correct realm 2024-04-17T04:47:32.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.842+0000 7f235e6d6640 0 -- 172.21.15.88:0/3431930458 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f235006f040 msgr2=0x7f2350072040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:47:32.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.850+0000 7f235f6d8640 0 -- 172.21.15.88:0/3431930458 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f2348013030 msgr2=0x7f2348013410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:47:32.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:47:32.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:47:32.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:47:32.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.858+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:47:32.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:47:32.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 zone default found 2024-04-17T04:47:32.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 4 Realm: () 2024-04-17T04:47:32.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:47:32.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:47:32.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:47:32.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 zonegroup default 2024-04-17T04:47:32.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 started sync module instance, tier type = 2024-04-17T04:47:32.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.862+0000 7f23613bc980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:47:32.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=0 2024-04-17T04:47:32.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=1 2024-04-17T04:47:32.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=2 2024-04-17T04:47:32.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=3 2024-04-17T04:47:32.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=4 2024-04-17T04:47:32.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=5 2024-04-17T04:47:32.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.878+0000 7f23613bc980 20 add_watcher() i=6 2024-04-17T04:47:32.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.882+0000 7f23613bc980 20 add_watcher() i=7 2024-04-17T04:47:32.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.882+0000 7f23613bc980 2 all 8 watchers are set, enabling cache 2024-04-17T04:47:32.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.882+0000 7f22157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:47:32.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:47:32.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:47:32.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 5 note: GC not initialized 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f218ffff640 20 reqs_thread_entry: start 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 init_complete bucket index max shards: 1 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 Filter name: none 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f218dffb640 20 reqs_thread_entry: start 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-04-17T04:47:32.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rados->read ofs=0 len=0 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-04-17T04:47:32.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-04-17T04:47:32.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-04-17T04:47:32.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-04-17T04:47:32.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.886+0000 7f23613bc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-04-17T04:47:32.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.890+0000 7f22427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824002 cookie 94777451977040 notifier 5522 bl.length()=677 2024-04-17T04:47:32.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.890+0000 7f22427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-04-17T04:47:32.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.890+0000 7f22427fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-04-17T04:47:32.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.894+0000 7f23613bc980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-04-17T04:47:32.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.894+0000 7f23613bc980 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-04-17T04:47:32.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.894+0000 7f23613bc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-04-17T04:47:32.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.894+0000 7f22427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824013 cookie 94777451962528 notifier 5522 bl.length()=187 2024-04-17T04:47:32.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.894+0000 7f22427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-04-17T04:47:32.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.894+0000 7f22427fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-04-17T04:47:32.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.898+0000 7f23613bc980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-04-17T04:47:32.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.898+0000 7f23613bc980 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-04-17T04:47:32.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.898+0000 7f23613bc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-04-17T04:47:32.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.898+0000 7f22427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824014 cookie 94777451962528 notifier 5522 bl.length()=197 2024-04-17T04:47:32.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.898+0000 7f22427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-04-17T04:47:32.896 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.898+0000 7f22427fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-04-17T04:47:32.898 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:47:32.898 INFO:tasks.workunit.client.0.smithi088.stdout: "user_id": "ceph-test-maven", 2024-04-17T04:47:32.898 INFO:tasks.workunit.client.0.smithi088.stdout: "display_name": "maven test user", 2024-04-17T04:47:32.898 INFO:tasks.workunit.client.0.smithi088.stdout: "email": "sigv4@example.com", 2024-04-17T04:47:32.898 INFO:tasks.workunit.client.0.smithi088.stdout: "suspended": 0, 2024-04-17T04:47:32.898 INFO:tasks.workunit.client.0.smithi088.stdout: "max_buckets": 1000, 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "subusers": [], 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "keys": [ 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: { 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "user": "ceph-test-maven", 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "active": true, 2024-04-17T04:47:32.899 INFO:tasks.workunit.client.0.smithi088.stdout: "create_date": "2024-04-17T04:47:32.890979Z" 2024-04-17T04:47:32.900 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:47:32.900 INFO:tasks.workunit.client.0.smithi088.stdout: ], 2024-04-17T04:47:32.900 INFO:tasks.workunit.client.0.smithi088.stdout: "swift_keys": [], 2024-04-17T04:47:32.900 INFO:tasks.workunit.client.0.smithi088.stdout: "caps": [], 2024-04-17T04:47:32.900 INFO:tasks.workunit.client.0.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T04:47:32.900 INFO:tasks.workunit.client.0.smithi088.stdout: "default_placement": "", 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "default_storage_class": "", 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "placement_tags": [], 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_quota": { 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "enabled": false, 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size": -1, 2024-04-17T04:47:32.901 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: "max_objects": -1 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: "user_quota": { 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: "enabled": false, 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size": -1, 2024-04-17T04:47:32.902 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "max_objects": -1 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "temp_url_keys": [], 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "type": "rgw", 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "mfa_ids": [], 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "account_id": "", 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "path": "/", 2024-04-17T04:47:32.903 INFO:tasks.workunit.client.0.smithi088.stdout: "create_date": "2024-04-17T04:47:32.890965Z", 2024-04-17T04:47:32.904 INFO:tasks.workunit.client.0.smithi088.stdout: "tags": [], 2024-04-17T04:47:32.904 INFO:tasks.workunit.client.0.smithi088.stdout: "group_ids": [] 2024-04-17T04:47:32.904 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:47:32.904 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:47:32.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.902+0000 7f22157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:47:32.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.906+0000 7f23613bc980 20 remove_watcher() i=0 2024-04-17T04:47:32.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.906+0000 7f23613bc980 2 removed watcher, disabling cache 2024-04-17T04:47:32.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.910+0000 7f23613bc980 20 remove_watcher() i=1 2024-04-17T04:47:32.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.910+0000 7f23613bc980 20 remove_watcher() i=2 2024-04-17T04:47:32.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.914+0000 7f23613bc980 20 remove_watcher() i=3 2024-04-17T04:47:32.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.914+0000 7f23613bc980 20 remove_watcher() i=4 2024-04-17T04:47:32.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.918+0000 7f23613bc980 20 remove_watcher() i=5 2024-04-17T04:47:32.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.918+0000 7f23613bc980 20 remove_watcher() i=6 2024-04-17T04:47:32.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:47:32.918+0000 7f23613bc980 20 remove_watcher() i=7 2024-04-17T04:47:32.930 INFO:tasks.workunit.client.0.smithi088.stderr:+ [ -z ] 2024-04-17T04:47:32.930 INFO:tasks.workunit.client.0.smithi088.stderr:+ [ -z /home/ubuntu/cephtest ] 2024-04-17T04:47:32.931 INFO:tasks.workunit.client.0.smithi088.stderr:+ cat /home/ubuntu/cephtest/url_file 2024-04-17T04:47:32.931 INFO:tasks.workunit.client.0.smithi088.stderr:+ export RGW_HTTP_ENDPOINT_URL=http://smithi088.front.sepia.ceph.com:80 2024-04-17T04:47:32.931 INFO:tasks.workunit.client.0.smithi088.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2024-04-17T04:47:32.931 INFO:tasks.workunit.client.0.smithi088.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-04-17T04:47:32.931 INFO:tasks.workunit.client.0.smithi088.stderr:+ ./mvnw clean package 2024-04-17T04:47:32.938 INFO:tasks.workunit.client.0.smithi088.stdout:Warning: JAVA_HOME environment variable is not set. 2024-04-17T04:47:38.003 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Scanning for projects... 2024-04-17T04:47:38.157 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-04-17T04:47:38.267 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 50 kB/s) 2024-04-17T04:47:38.314 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-04-17T04:47:38.358 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/83 kB Progress (1): 2.8/83 kB Progress (1): 4.1/83 kB Progress (1): 5.5/83 kB Progress (1): 6.9/83 kB Progress (1): 8.3/83 kB Progress (1): 9.7/83 kB Progress (1): 11/83 kB Progress (1): 12/83 kB Progress (1): 14/83 kB Progress (1): 15/83 kB Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 66/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 1.9 MB/s) 2024-04-17T04:47:38.383 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-04-17T04:47:38.409 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/77 kB Progress (1): 33/77 kB Progress (1): 49/77 kB Progress (1): 66/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 3.0 MB/s) 2024-04-17T04:47:38.458 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:38.458 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-04-17T04:47:38.458 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Building jcksum 1.0 2024-04-17T04:47:38.458 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] from pom.xml 2024-04-17T04:47:38.458 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-04-17T04:47:38.465 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-04-17T04:47:38.484 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 265 kB/s) 2024-04-17T04:47:38.495 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-04-17T04:47:38.515 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 495 kB/s) 2024-04-17T04:47:38.521 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-04-17T04:47:38.541 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 2.3 MB/s) 2024-04-17T04:47:38.551 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-04-17T04:47:38.571 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 978 kB/s) 2024-04-17T04:47:38.592 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-04-17T04:47:38.612 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 1.7 MB/s) 2024-04-17T04:47:38.625 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-04-17T04:47:38.645 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 408 kB/s) 2024-04-17T04:47:38.651 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-04-17T04:47:38.672 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 385 kB/s) 2024-04-17T04:47:38.676 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-04-17T04:47:38.700 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/48 kB Progress (1): 33/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 1.9 MB/s) 2024-04-17T04:47:38.709 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-04-17T04:47:38.741 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 648 kB/s) 2024-04-17T04:47:38.752 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-04-17T04:47:38.778 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 1.2 MB/s) 2024-04-17T04:47:38.789 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-04-17T04:47:38.809 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 491 kB/s) 2024-04-17T04:47:38.821 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-04-17T04:47:38.845 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/66 kB Progress (1): 33/66 kB Progress (1): 49/66 kB Progress (1): 66/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 2.6 MB/s) 2024-04-17T04:47:38.858 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-04-17T04:47:38.878 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 266 kB/s) 2024-04-17T04:47:38.882 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-04-17T04:47:38.902 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 1.1 MB/s) 2024-04-17T04:47:38.907 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-04-17T04:47:38.928 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/50 kB Progress (1): 33/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 2.3 MB/s) 2024-04-17T04:47:38.935 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-04-17T04:47:38.954 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 1.2 MB/s) 2024-04-17T04:47:38.962 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-04-17T04:47:38.981 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 313 kB/s) 2024-04-17T04:47:38.986 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-04-17T04:47:39.008 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 1.9 MB/s) 2024-04-17T04:47:39.016 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-04-17T04:47:39.035 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 376 kB/s) 2024-04-17T04:47:39.038 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-04-17T04:47:39.057 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 550 kB/s) 2024-04-17T04:47:39.060 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-04-17T04:47:39.081 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 2.1 MB/s) 2024-04-17T04:47:39.087 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-04-17T04:47:39.108 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 1.0 MB/s) 2024-04-17T04:47:39.116 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-04-17T04:47:39.137 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 1.4 MB/s) 2024-04-17T04:47:39.165 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-04-17T04:47:39.185 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 304 kB/s) 2024-04-17T04:47:39.189 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-04-17T04:47:39.209 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 1.1 MB/s) 2024-04-17T04:47:39.218 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-04-17T04:47:39.238 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 953 kB/s) 2024-04-17T04:47:39.253 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-04-17T04:47:39.273 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 200 kB/s) 2024-04-17T04:47:39.277 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-04-17T04:47:39.296 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 78 kB/s) 2024-04-17T04:47:39.300 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-04-17T04:47:39.319 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 111 kB/s) 2024-04-17T04:47:39.328 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-04-17T04:47:39.347 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 191 kB/s) 2024-04-17T04:47:39.355 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-04-17T04:47:39.374 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 161 kB/s) 2024-04-17T04:47:39.384 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-04-17T04:47:39.403 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 577 kB/s) 2024-04-17T04:47:39.408 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-04-17T04:47:39.426 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 94 kB/s) 2024-04-17T04:47:39.430 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-04-17T04:47:39.450 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 191 kB/s) 2024-04-17T04:47:39.458 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-04-17T04:47:39.486 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 175 kB/s) 2024-04-17T04:47:39.493 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-04-17T04:47:39.511 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 61 kB/s) 2024-04-17T04:47:39.516 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-04-17T04:47:39.534 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 202 kB/s) 2024-04-17T04:47:39.538 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-04-17T04:47:39.558 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 690 kB/s) 2024-04-17T04:47:39.565 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-04-17T04:47:39.585 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 184 kB/s) 2024-04-17T04:47:39.595 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-04-17T04:47:39.615 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 117 kB/s) 2024-04-17T04:47:39.626 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-04-17T04:47:39.646 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 146 kB/s) 2024-04-17T04:47:39.656 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-04-17T04:47:39.675 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 370 kB/s) 2024-04-17T04:47:39.685 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-04-17T04:47:39.707 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 248 kB/s) 2024-04-17T04:47:39.717 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-04-17T04:47:39.736 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 174 kB/s) 2024-04-17T04:47:39.746 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-04-17T04:47:39.766 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 220 kB/s) 2024-04-17T04:47:39.772 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-04-17T04:47:39.793 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 115 kB/s) 2024-04-17T04:47:39.801 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-04-17T04:47:39.823 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 293 kB/s) 2024-04-17T04:47:39.832 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-04-17T04:47:39.852 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 319 kB/s) 2024-04-17T04:47:39.861 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-04-17T04:47:39.881 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 149 kB/s) 2024-04-17T04:47:39.890 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-04-17T04:47:39.910 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 174 kB/s) 2024-04-17T04:47:39.920 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-04-17T04:47:39.942 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 195 kB/s) 2024-04-17T04:47:39.944 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-04-17T04:47:39.964 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 94 kB/s) 2024-04-17T04:47:39.975 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-04-17T04:47:39.995 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 330 kB/s) 2024-04-17T04:47:40.000 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-04-17T04:47:40.020 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 780 kB/s) 2024-04-17T04:47:40.026 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-04-17T04:47:40.048 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/35 kB Progress (1): 33/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 1.6 MB/s) 2024-04-17T04:47:40.056 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-04-17T04:47:40.075 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 855 kB/s) 2024-04-17T04:47:40.084 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-04-17T04:47:40.105 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 237 kB/s) 2024-04-17T04:47:40.110 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-04-17T04:47:40.129 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 693 kB/s) 2024-04-17T04:47:40.138 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-04-17T04:47:40.159 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 635 kB/s) 2024-04-17T04:47:40.164 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-04-17T04:47:40.189 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/68 kB Progress (1): 33/68 kB Progress (1): 49/68 kB Progress (1): 62/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 2.7 MB/s) 2024-04-17T04:47:40.211 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-04-17T04:47:40.230 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 825 kB/s) 2024-04-17T04:47:40.240 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-04-17T04:47:40.259 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 772 kB/s) 2024-04-17T04:47:40.264 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-04-17T04:47:40.289 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/79 kB Progress (1): 32/79 kB Progress (1): 49/79 kB Progress (1): 65/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 3.0 MB/s) 2024-04-17T04:47:40.300 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-04-17T04:47:40.323 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 768 kB/s) 2024-04-17T04:47:40.331 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-04-17T04:47:40.352 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 998 kB/s) 2024-04-17T04:47:40.358 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-04-17T04:47:40.383 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 3.1 MB/s) 2024-04-17T04:47:40.397 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-04-17T04:47:40.418 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.7/3.8 kB Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 174 kB/s) 2024-04-17T04:47:40.423 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-04-17T04:47:40.442 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 690 kB/s) 2024-04-17T04:47:40.449 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-04-17T04:47:40.472 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 139 kB/s) 2024-04-17T04:47:40.478 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-04-17T04:47:40.498 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 152 kB/s) 2024-04-17T04:47:40.503 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-04-17T04:47:40.522 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 107 kB/s) 2024-04-17T04:47:40.527 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-04-17T04:47:40.546 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 149 kB/s) 2024-04-17T04:47:40.551 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-04-17T04:47:40.590 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 38 kB/s) 2024-04-17T04:47:40.595 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-04-17T04:47:40.614 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 158 kB/s) 2024-04-17T04:47:40.620 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-04-17T04:47:40.639 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 169 kB/s) 2024-04-17T04:47:40.644 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-04-17T04:47:40.663 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 160 kB/s) 2024-04-17T04:47:40.670 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-04-17T04:47:40.689 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 51 kB/s) 2024-04-17T04:47:40.698 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-04-17T04:47:40.716 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 51 kB/s) 2024-04-17T04:47:40.723 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-04-17T04:47:40.742 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 161 kB/s) 2024-04-17T04:47:40.747 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-04-17T04:47:40.766 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 160 kB/s) 2024-04-17T04:47:40.771 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-04-17T04:47:40.791 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 172 kB/s) 2024-04-17T04:47:40.796 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-04-17T04:47:40.815 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 172 kB/s) 2024-04-17T04:47:40.821 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-04-17T04:47:40.840 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 159 kB/s) 2024-04-17T04:47:40.889 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-04-17T04:47:41.133 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 13 MB/s) 2024-04-17T04:47:41.138 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-04-17T04:47:41.138 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-04-17T04:47:41.138 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-04-17T04:47:41.139 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-04-17T04:47:41.139 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-04-17T04:47:41.159 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/35 kB Progress (1): 33/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 1.7 MB/s) 2024-04-17T04:47:41.159 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-04-17T04:47:41.177 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 422 kB/s) 2024-04-17T04:47:41.177 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-04-17T04:47:41.209 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/782 kB Progress (1): 33/782 kB Progress (1): 49/782 kB Progress (1): 63/782 kB Progress (1): 79/782 kB Progress (1): 96/782 kB Progress (1): 112/782 kB Progress (1): 128/782 kB Progress (2): 128/782 kB | 1.4/98 kB Progress (3): 128/782 kB | 1.4/98 kB | 1.4/49 kB Progress (4): 128/782 kB | 1.4/98 kB | 1.4/49 kB | 1.4/71 kB Progress (4): 128/782 kB | 2.8/98 kB | 1.4/49 kB | 1.4/71 kB Progress (4): 128/782 kB | 2.8/98 kB | 1.4/49 kB | 2.8/71 kB Progress (4): 128/782 kB | 2.8/98 kB | 2.8/49 kB | 2.8/71 kB Progress (5): 128/782 kB | 2.8/98 kB | 2.8/49 kB | 2.8/71 kB | 1.4/16 kB Progress (5): 128/782 kB | 2.8/98 kB | 4.1/49 kB | 2.8/71 kB | 1.4/16 kB Progress (5): 128/782 kB | 2.8/98 kB | 4.1/49 kB | 4.1/71 kB | 1.4/16 kB Progress (5): 128/782 kB | 3.1/98 kB | 4.1/49 kB | 4.1/71 kB | 1.4/16 kB Progress (5): 145/782 kB | 3.1/98 kB | 4.1/49 kB | 4.1/71 kB | 1.4/16 kB Progress (5): 145/782 kB | 3.1/98 kB | 4.1/49 kB | 5.5/71 kB | 1.4/16 kB Progress (5): 145/782 kB | 3.1/98 kB | 5.5/49 kB | 5.5/71 kB | 1.4/16 kB Progress (5): 145/782 kB | 3.1/98 kB | 5.5/49 kB | 5.5/71 kB | 2.8/16 kB Progress (5): 161/782 kB | 3.1/98 kB | 5.5/49 kB | 5.5/71 kB | 2.8/16 kB Progress (5): 161/782 kB | 3.1/98 kB | 6.9/49 kB | 5.5/71 kB | 2.8/16 kB Progress (5): 161/782 kB | 3.1/98 kB | 6.9/49 kB | 6.9/71 kB | 2.8/16 kB Progress (5): 161/782 kB | 4.4/98 kB | 6.9/49 kB | 6.9/71 kB | 2.8/16 kB Progress (5): 178/782 kB | 4.4/98 kB | 6.9/49 kB | 6.9/71 kB | 2.8/16 kB Progress (5): 178/782 kB | 4.4/98 kB | 8.3/49 kB | 6.9/71 kB | 2.8/16 kB Progress (5): 178/782 kB | 4.4/98 kB | 8.3/49 kB | 6.9/71 kB | 4.1/16 kB Progress (5): 194/782 kB | 4.4/98 kB | 8.3/49 kB | 6.9/71 kB | 4.1/16 kB Progress (5): 194/782 kB | 4.4/98 kB | 9.7/49 kB | 6.9/71 kB | 4.1/16 kB Progress (5): 194/782 kB | 5.8/98 kB | 9.7/49 kB | 6.9/71 kB | 4.1/16 kB Progress (5): 194/782 kB | 5.8/98 kB | 9.7/49 kB | 8.3/71 kB | 4.1/16 kB Progress (5): 194/782 kB | 7.2/98 kB | 9.7/49 kB | 8.3/71 kB | 4.1/16 kB Progress (5): 210/782 kB | 7.2/98 kB | 9.7/49 kB | 8.3/71 kB | 4.1/16 kB Progress (5): 210/782 kB | 7.2/98 kB | 11/49 kB | 8.3/71 kB | 4.1/16 kB Progress (5): 210/782 kB | 7.2/98 kB | 11/49 kB | 8.3/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 7.2/98 kB | 11/49 kB | 8.3/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 7.2/98 kB | 12/49 kB | 8.3/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 8.6/98 kB | 12/49 kB | 8.3/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 8.6/98 kB | 12/49 kB | 9.7/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 10.0/98 kB | 12/49 kB | 9.7/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 10.0/98 kB | 14/49 kB | 9.7/71 kB | 5.5/16 kB Progress (5): 227/782 kB | 10.0/98 kB | 14/49 kB | 9.7/71 kB | 6.9/16 kB Progress (5): 227/782 kB | 10.0/98 kB | 15/49 kB | 9.7/71 kB | 6.9/16 kB Progress (5): 227/782 kB | 11/98 kB | 15/49 kB | 9.7/71 kB | 6.9/16 kB Progress (5): 227/782 kB | 11/98 kB | 15/49 kB | 11/71 kB | 6.9/16 kB Progress (5): 243/782 kB | 11/98 kB | 15/49 kB | 11/71 kB | 6.9/16 kB Progress (5): 243/782 kB | 11/98 kB | 15/49 kB | 12/71 kB | 6.9/16 kB Progress (5): 243/782 kB | 13/98 kB | 15/49 kB | 12/71 kB | 6.9/16 kB Progress (5): 243/782 kB | 13/98 kB | 16/49 kB | 12/71 kB | 6.9/16 kB Progress (5): 243/782 kB | 13/98 kB | 16/49 kB | 12/71 kB | 8.3/16 kB Progress (5): 243/782 kB | 13/98 kB | 17/49 kB | 12/71 kB | 8.3/16 kB Progress (5): 243/782 kB | 14/98 kB | 17/49 kB | 12/71 kB | 8.3/16 kB Progress (5): 260/782 kB | 14/98 kB | 17/49 kB | 12/71 kB | 8.3/16 kB Progress (5): 260/782 kB | 14/98 kB | 17/49 kB | 14/71 kB | 8.3/16 kB Progress (5): 260/782 kB | 15/98 kB | 17/49 kB | 14/71 kB | 8.3/16 kB Progress (5): 260/782 kB | 15/98 kB | 17/49 kB | 15/71 kB | 8.3/16 kB Progress (5): 260/782 kB | 15/98 kB | 19/49 kB | 15/71 kB | 8.3/16 kB Progress (5): 260/782 kB | 15/98 kB | 19/49 kB | 15/71 kB | 9.7/16 kB Progress (5): 260/782 kB | 15/98 kB | 20/49 kB | 15/71 kB | 9.7/16 kB Progress (5): 276/782 kB | 15/98 kB | 20/49 kB | 15/71 kB | 9.7/16 kB Progress (5): 276/782 kB | 15/98 kB | 20/49 kB | 16/71 kB | 9.7/16 kB Progress (5): 276/782 kB | 17/98 kB | 20/49 kB | 16/71 kB | 9.7/16 kB Progress (5): 276/782 kB | 17/98 kB | 21/49 kB | 16/71 kB | 9.7/16 kB Progress (5): 276/782 kB | 17/98 kB | 21/49 kB | 16/71 kB | 11/16 kB Progress (5): 276/782 kB | 17/98 kB | 23/49 kB | 16/71 kB | 11/16 kB Progress (5): 276/782 kB | 18/98 kB | 23/49 kB | 16/71 kB | 11/16 kB Progress (5): 276/782 kB | 18/98 kB | 23/49 kB | 33/71 kB | 11/16 kB Progress (5): 276/782 kB | 18/98 kB | 23/49 kB | 33/71 kB | 12/16 kB Progress (5): 292/782 kB | 18/98 kB | 23/49 kB | 33/71 kB | 12/16 kB Progress (5): 292/782 kB | 18/98 kB | 23/49 kB | 33/71 kB | 14/16 kB Progress (5): 292/782 kB | 19/98 kB | 23/49 kB | 33/71 kB | 14/16 kB Progress (5): 292/782 kB | 19/98 kB | 24/49 kB | 33/71 kB | 14/16 kB Progress (5): 292/782 kB | 19/98 kB | 24/49 kB | 33/71 kB | 15/16 kB Progress (5): 292/782 kB | 19/98 kB | 25/49 kB | 33/71 kB | 15/16 kB Progress (5): 309/782 kB | 19/98 kB | 25/49 kB | 33/71 kB | 15/16 kB Progress (5): 309/782 kB | 19/98 kB | 25/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 25/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 27/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 28/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 30/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 31/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 32/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 34/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 35/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 36/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 38/49 kB | 33/71 kB | 16 kB Progress (5): 309/782 kB | 36/98 kB | 39/49 kB | 33/71 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 229 kB/s) 2024-04-17T04:47:41.209 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-04-17T04:47:41.215 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (4): 325/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 341/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 358/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 374/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 391/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 407/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 423/782 kB | 36/98 kB | 39/49 kB | 33/71 kB Progress (4): 423/782 kB | 36/98 kB | 41/49 kB | 33/71 kB Progress (4): 423/782 kB | 36/98 kB | 42/49 kB | 33/71 kB Progress (4): 440/782 kB | 36/98 kB | 42/49 kB | 33/71 kB Progress (4): 440/782 kB | 36/98 kB | 43/49 kB | 33/71 kB Progress (4): 440/782 kB | 36/98 kB | 45/49 kB | 33/71 kB Progress (4): 456/782 kB | 36/98 kB | 45/49 kB | 33/71 kB Progress (4): 456/782 kB | 36/98 kB | 46/49 kB | 33/71 kB Progress (4): 456/782 kB | 36/98 kB | 48/49 kB | 33/71 kB Progress (4): 456/782 kB | 36/98 kB | 49/49 kB | 33/71 kB Progress (4): 456/782 kB | 36/98 kB | 49 kB | 33/71 kB Progress (4): 456/782 kB | 36/98 kB | 49 kB | 49/71 kB Progress (4): 473/782 kB | 36/98 kB | 49 kB | 49/71 kB Progress (4): 473/782 kB | 52/98 kB | 49 kB | 49/71 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 639 kB/s) 2024-04-17T04:47:41.215 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 489/782 kB | 52/98 kB | 49/71 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-04-17T04:47:41.220 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 505/782 kB | 52/98 kB | 49/71 kB Progress (3): 522/782 kB | 52/98 kB | 49/71 kB Progress (3): 538/782 kB | 52/98 kB | 49/71 kB Progress (3): 538/782 kB | 52/98 kB | 66/71 kB Progress (3): 554/782 kB | 52/98 kB | 66/71 kB Progress (3): 571/782 kB | 52/98 kB | 66/71 kB Progress (3): 587/782 kB | 52/98 kB | 66/71 kB Progress (3): 587/782 kB | 69/98 kB | 66/71 kB Progress (3): 604/782 kB | 69/98 kB | 66/71 kB Progress (3): 604/782 kB | 69/98 kB | 71 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 840 kB/s) 2024-04-17T04:47:41.220 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-04-17T04:47:41.226 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 620/782 kB | 69/98 kB Progress (2): 636/782 kB | 69/98 kB Progress (2): 653/782 kB | 69/98 kB Progress (3): 653/782 kB | 69/98 kB | 1.4/11 kB Progress (3): 653/782 kB | 69/98 kB | 2.8/11 kB Progress (3): 669/782 kB | 69/98 kB | 2.8/11 kB Progress (3): 669/782 kB | 69/98 kB | 4.1/11 kB Progress (3): 669/782 kB | 69/98 kB | 5.5/11 kB Progress (3): 669/782 kB | 69/98 kB | 6.9/11 kB Progress (3): 686/782 kB | 69/98 kB | 6.9/11 kB Progress (3): 702/782 kB | 69/98 kB | 6.9/11 kB Progress (3): 702/782 kB | 69/98 kB | 8.3/11 kB Progress (3): 702/782 kB | 69/98 kB | 9.7/11 kB Progress (3): 702/782 kB | 69/98 kB | 11/11 kB Progress (3): 702/782 kB | 69/98 kB | 11 kB Progress (3): 718/782 kB | 69/98 kB | 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 129 kB/s) 2024-04-17T04:47:41.227 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-04-17T04:47:41.229 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 735/782 kB | 69/98 kB Progress (2): 751/782 kB | 69/98 kB Progress (2): 767/782 kB | 69/98 kB Progress (2): 767/782 kB | 85/98 kB Progress (2): 782 kB | 85/98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 8.7 MB/s) 2024-04-17T04:47:41.229 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-04-17T04:47:41.230 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 1.0 MB/s) 2024-04-17T04:47:41.230 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-04-17T04:47:41.238 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/205 kB Progress (1): 32/205 kB Progress (1): 49/205 kB Progress (1): 49/205 kB Progress (1): 66/205 kB Progress (2): 66/205 kB | 16/30 kB Progress (2): 66/205 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 305 kB/s) 2024-04-17T04:47:41.239 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-04-17T04:47:41.248 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 66/205 kB | 1.4/81 kB Progress (2): 66/205 kB | 2.8/81 kB Progress (2): 66/205 kB | 4.1/81 kB Progress (2): 66/205 kB | 5.5/81 kB Progress (2): 66/205 kB | 6.9/81 kB Progress (2): 66/205 kB | 8.3/81 kB Progress (2): 66/205 kB | 9.7/81 kB Progress (2): 66/205 kB | 11/81 kB Progress (2): 66/205 kB | 12/81 kB Progress (2): 66/205 kB | 14/81 kB Progress (2): 66/205 kB | 15/81 kB Progress (2): 66/205 kB | 16/81 kB Progress (2): 80/205 kB | 16/81 kB Progress (2): 96/205 kB | 16/81 kB Progress (3): 96/205 kB | 16/81 kB | 15/798 kB Progress (4): 96/205 kB | 16/81 kB | 15/798 kB | 13 kB Progress (4): 112/205 kB | 16/81 kB | 15/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 124 kB/s) 2024-04-17T04:47:41.248 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 112/205 kB | 16/81 kB | 31/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-04-17T04:47:41.254 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 112/205 kB | 16/81 kB | 48/798 kB Progress (3): 112/205 kB | 33/81 kB | 48/798 kB Progress (3): 129/205 kB | 33/81 kB | 48/798 kB Progress (3): 129/205 kB | 33/81 kB | 64/798 kB Progress (3): 145/205 kB | 33/81 kB | 64/798 kB Progress (3): 145/205 kB | 33/81 kB | 78/798 kB Progress (3): 162/205 kB | 33/81 kB | 78/798 kB Progress (3): 178/205 kB | 33/81 kB | 78/798 kB Progress (3): 178/205 kB | 33/81 kB | 95/798 kB Progress (3): 194/205 kB | 33/81 kB | 95/798 kB Progress (3): 205 kB | 33/81 kB | 95/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 1.8 MB/s) 2024-04-17T04:47:41.254 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-04-17T04:47:41.264 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 33/81 kB | 95/798 kB | 16/182 kB Progress (3): 33/81 kB | 95/798 kB | 33/182 kB Progress (3): 33/81 kB | 95/798 kB | 49/182 kB Progress (3): 33/81 kB | 95/798 kB | 66/182 kB Progress (3): 49/81 kB | 95/798 kB | 66/182 kB Progress (3): 49/81 kB | 95/798 kB | 82/182 kB Progress (3): 49/81 kB | 95/798 kB | 98/182 kB Progress (3): 49/81 kB | 111/798 kB | 98/182 kB Progress (3): 49/81 kB | 128/798 kB | 98/182 kB Progress (3): 66/81 kB | 128/798 kB | 98/182 kB Progress (3): 66/81 kB | 144/798 kB | 98/182 kB Progress (3): 66/81 kB | 160/798 kB | 98/182 kB Progress (3): 81 kB | 160/798 kB | 98/182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 650 kB/s) 2024-04-17T04:47:41.264 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-04-17T04:47:41.271 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 177/798 kB | 98/182 kB Progress (3): 177/798 kB | 98/182 kB | 15/151 kB Progress (3): 193/798 kB | 98/182 kB | 15/151 kB Progress (3): 193/798 kB | 98/182 kB | 31/151 kB Progress (3): 193/798 kB | 98/182 kB | 48/151 kB Progress (3): 210/798 kB | 98/182 kB | 48/151 kB Progress (3): 210/798 kB | 98/182 kB | 64/151 kB Progress (3): 226/798 kB | 98/182 kB | 64/151 kB Progress (3): 226/798 kB | 98/182 kB | 81/151 kB Progress (3): 242/798 kB | 98/182 kB | 81/151 kB Progress (3): 242/798 kB | 98/182 kB | 97/151 kB Progress (3): 242/798 kB | 98/182 kB | 113/151 kB Progress (3): 259/798 kB | 98/182 kB | 113/151 kB Progress (3): 275/798 kB | 98/182 kB | 113/151 kB Progress (3): 275/798 kB | 98/182 kB | 130/151 kB Progress (3): 291/798 kB | 98/182 kB | 130/151 kB Progress (3): 291/798 kB | 98/182 kB | 146/151 kB Progress (3): 291/798 kB | 115/182 kB | 146/151 kB Progress (3): 308/798 kB | 115/182 kB | 146/151 kB Progress (3): 308/798 kB | 115/182 kB | 151 kB Progress (3): 308/798 kB | 131/182 kB | 151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 1.2 MB/s) 2024-04-17T04:47:41.271 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-04-17T04:47:41.272 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 308/798 kB | 131/182 kB | 16/27 kB Progress (3): 308/798 kB | 147/182 kB | 16/27 kB Progress (3): 308/798 kB | 147/182 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 208 kB/s) 2024-04-17T04:47:41.272 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-04-17T04:47:41.274 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 308/798 kB | 164/182 kB Progress (2): 308/798 kB | 180/182 kB Progress (2): 308/798 kB | 182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 1.4 MB/s) 2024-04-17T04:47:41.274 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-04-17T04:47:41.283 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 324/798 kB Progress (1): 341/798 kB Progress (1): 357/798 kB Progress (1): 373/798 kB Progress (1): 390/798 kB Progress (1): 406/798 kB Progress (1): 423/798 kB Progress (1): 439/798 kB Progress (1): 455/798 kB Progress (1): 472/798 kB Progress (1): 488/798 kB Progress (1): 504/798 kB Progress (1): 521/798 kB Progress (1): 537/798 kB Progress (2): 537/798 kB | 16/30 kB Progress (2): 554/798 kB | 16/30 kB Progress (2): 554/798 kB | 30 kB Progress (2): 570/798 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 212 kB/s) 2024-04-17T04:47:41.283 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-04-17T04:47:41.290 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 586/798 kB Progress (1): 603/798 kB Progress (1): 619/798 kB Progress (1): 636/798 kB Progress (1): 652/798 kB Progress (1): 668/798 kB Progress (1): 685/798 kB Progress (1): 701/798 kB Progress (2): 701/798 kB | 16/385 kB Progress (3): 701/798 kB | 16/385 kB | 13 kB Progress (3): 701/798 kB | 33/385 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 88 kB/s) 2024-04-17T04:47:41.290 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-04-17T04:47:41.293 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 701/798 kB | 49/385 kB Progress (2): 701/798 kB | 62/385 kB Progress (3): 701/798 kB | 62/385 kB | 16/75 kB Progress (3): 717/798 kB | 62/385 kB | 16/75 kB Progress (3): 734/798 kB | 62/385 kB | 16/75 kB Progress (3): 734/798 kB | 62/385 kB | 33/75 kB Progress (3): 734/798 kB | 79/385 kB | 33/75 kB Progress (3): 734/798 kB | 79/385 kB | 49/75 kB Progress (3): 734/798 kB | 95/385 kB | 49/75 kB Progress (3): 750/798 kB | 95/385 kB | 49/75 kB Progress (3): 750/798 kB | 95/385 kB | 66/75 kB Progress (3): 750/798 kB | 111/385 kB | 66/75 kB Progress (3): 767/798 kB | 111/385 kB | 66/75 kB Progress (3): 767/798 kB | 111/385 kB | 75 kB Progress (3): 767/798 kB | 128/385 kB | 75 kB Progress (3): 783/798 kB | 128/385 kB | 75 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 490 kB/s) 2024-04-17T04:47:41.293 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 798 kB | 128/385 kB Progress (2): 798 kB | 144/385 kB Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-04-17T04:47:41.293 INFO:tasks.workunit.client.0.smithi088.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 5.2 MB/s) 2024-04-17T04:47:41.293 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-04-17T04:47:41.305 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 161/385 kB Progress (1): 177/385 kB Progress (1): 193/385 kB Progress (1): 210/385 kB Progress (1): 226/385 kB Progress (1): 242/385 kB Progress (1): 259/385 kB Progress (1): 275/385 kB Progress (1): 292/385 kB Progress (1): 308/385 kB Progress (1): 324/385 kB Progress (1): 341/385 kB Progress (2): 341/385 kB | 16/780 kB Progress (2): 341/385 kB | 33/780 kB Progress (2): 357/385 kB | 33/780 kB Progress (2): 357/385 kB | 49/780 kB Progress (2): 374/385 kB | 49/780 kB Progress (2): 385 kB | 49/780 kB Progress (2): 385 kB | 66/780 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 2.3 MB/s) 2024-04-17T04:47:41.305 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-04-17T04:47:41.331 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 82/780 kB Progress (1): 98/780 kB Progress (2): 98/780 kB | 16/329 kB Progress (2): 115/780 kB | 16/329 kB Progress (2): 115/780 kB | 33/329 kB Progress (2): 115/780 kB | 49/329 kB Progress (2): 115/780 kB | 66/329 kB Progress (2): 115/780 kB | 82/329 kB Progress (2): 115/780 kB | 98/329 kB Progress (2): 115/780 kB | 115/329 kB Progress (2): 115/780 kB | 131/329 kB Progress (3): 115/780 kB | 131/329 kB | 15/501 kB Progress (3): 115/780 kB | 131/329 kB | 31/501 kB Progress (3): 115/780 kB | 131/329 kB | 48/501 kB Progress (3): 115/780 kB | 131/329 kB | 64/501 kB Progress (3): 115/780 kB | 147/329 kB | 64/501 kB Progress (3): 115/780 kB | 147/329 kB | 81/501 kB Progress (3): 115/780 kB | 164/329 kB | 81/501 kB Progress (3): 115/780 kB | 164/329 kB | 97/501 kB Progress (4): 115/780 kB | 164/329 kB | 97/501 kB | 15/354 kB Progress (4): 115/780 kB | 164/329 kB | 110/501 kB | 15/354 kB Progress (4): 115/780 kB | 180/329 kB | 110/501 kB | 15/354 kB Progress (4): 115/780 kB | 180/329 kB | 127/501 kB | 15/354 kB Progress (4): 131/780 kB | 180/329 kB | 127/501 kB | 15/354 kB Progress (4): 131/780 kB | 180/329 kB | 127/501 kB | 31/354 kB Progress (4): 131/780 kB | 197/329 kB | 127/501 kB | 31/354 kB Progress (4): 131/780 kB | 197/329 kB | 143/501 kB | 31/354 kB Progress (4): 147/780 kB | 197/329 kB | 143/501 kB | 31/354 kB Progress (4): 147/780 kB | 197/329 kB | 143/501 kB | 48/354 kB Progress (4): 147/780 kB | 213/329 kB | 143/501 kB | 48/354 kB Progress (5): 147/780 kB | 213/329 kB | 143/501 kB | 48/354 kB | 16/41 kB Progress (5): 147/780 kB | 213/329 kB | 160/501 kB | 48/354 kB | 16/41 kB Progress (5): 164/780 kB | 213/329 kB | 160/501 kB | 48/354 kB | 16/41 kB Progress (5): 164/780 kB | 229/329 kB | 160/501 kB | 48/354 kB | 16/41 kB Progress (5): 164/780 kB | 229/329 kB | 160/501 kB | 48/354 kB | 33/41 kB Progress (5): 164/780 kB | 229/329 kB | 176/501 kB | 48/354 kB | 33/41 kB Progress (5): 180/780 kB | 229/329 kB | 176/501 kB | 48/354 kB | 33/41 kB Progress (5): 180/780 kB | 246/329 kB | 176/501 kB | 48/354 kB | 33/41 kB Progress (5): 180/780 kB | 246/329 kB | 176/501 kB | 48/354 kB | 41 kB Progress (5): 180/780 kB | 246/329 kB | 192/501 kB | 48/354 kB | 41 kB Progress (5): 197/780 kB | 246/329 kB | 192/501 kB | 48/354 kB | 41 kB Progress (5): 197/780 kB | 262/329 kB | 192/501 kB | 48/354 kB | 41 kB Progress (5): 197/780 kB | 262/329 kB | 209/501 kB | 48/354 kB | 41 kB Progress (5): 213/780 kB | 262/329 kB | 209/501 kB | 48/354 kB | 41 kB Progress (5): 213/780 kB | 279/329 kB | 209/501 kB | 48/354 kB | 41 kB Progress (5): 213/780 kB | 279/329 kB | 225/501 kB | 48/354 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 222 kB/s) 2024-04-17T04:47:41.331 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (4): 229/780 kB | 279/329 kB | 225/501 kB | 48/354 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-04-17T04:47:41.331 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (4): 229/780 kB | 279/329 kB | 241/501 kB | 48/354 kB Progress (4): 229/780 kB | 295/329 kB | 241/501 kB | 48/354 kB Progress (4): 229/780 kB | 295/329 kB | 241/501 kB | 64/354 kB Progress (4): 246/780 kB | 295/329 kB | 241/501 kB | 64/354 kB Progress (4): 246/780 kB | 295/329 kB | 258/501 kB | 64/354 kB Progress (4): 246/780 kB | 311/329 kB | 258/501 kB | 64/354 kB Progress (4): 246/780 kB | 311/329 kB | 258/501 kB | 79/354 kB Progress (4): 262/780 kB | 311/329 kB | 258/501 kB | 79/354 kB Progress (4): 262/780 kB | 311/329 kB | 274/501 kB | 79/354 kB Progress (4): 262/780 kB | 328/329 kB | 274/501 kB | 79/354 kB Progress (4): 262/780 kB | 328/329 kB | 274/501 kB | 96/354 kB Progress (4): 279/780 kB | 328/329 kB | 274/501 kB | 96/354 kB Progress (4): 279/780 kB | 329 kB | 274/501 kB | 96/354 kB Progress (4): 279/780 kB | 329 kB | 291/501 kB | 96/354 kB Progress (4): 279/780 kB | 329 kB | 291/501 kB | 112/354 kB Progress (4): 279/780 kB | 329 kB | 307/501 kB | 112/354 kB Progress (4): 295/780 kB | 329 kB | 307/501 kB | 112/354 kB Progress (4): 295/780 kB | 329 kB | 323/501 kB | 112/354 kB Progress (4): 295/780 kB | 329 kB | 323/501 kB | 128/354 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 1.8 MB/s) 2024-04-17T04:47:41.331 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 295/780 kB | 340/501 kB | 128/354 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-04-17T04:47:41.332 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 311/780 kB | 340/501 kB | 128/354 kB Progress (3): 311/780 kB | 340/501 kB | 145/354 kB Progress (3): 311/780 kB | 356/501 kB | 145/354 kB Progress (3): 328/780 kB | 356/501 kB | 145/354 kB Progress (3): 328/780 kB | 373/501 kB | 145/354 kB Progress (3): 328/780 kB | 373/501 kB | 161/354 kB Progress (3): 328/780 kB | 389/501 kB | 161/354 kB Progress (3): 344/780 kB | 389/501 kB | 161/354 kB Progress (3): 344/780 kB | 389/501 kB | 178/354 kB Progress (3): 344/780 kB | 405/501 kB | 178/354 kB Progress (3): 344/780 kB | 405/501 kB | 194/354 kB Progress (3): 344/780 kB | 422/501 kB | 194/354 kB Progress (3): 344/780 kB | 422/501 kB | 210/354 kB Progress (3): 344/780 kB | 438/501 kB | 210/354 kB Progress (3): 344/780 kB | 438/501 kB | 227/354 kB Progress (3): 344/780 kB | 454/501 kB | 227/354 kB Progress (3): 344/780 kB | 454/501 kB | 243/354 kB Progress (3): 344/780 kB | 471/501 kB | 243/354 kB Progress (3): 344/780 kB | 471/501 kB | 259/354 kB Progress (3): 344/780 kB | 487/501 kB | 259/354 kB Progress (3): 344/780 kB | 487/501 kB | 276/354 kB Progress (3): 344/780 kB | 501 kB | 276/354 kB Progress (3): 344/780 kB | 501 kB | 292/354 kB Progress (3): 344/780 kB | 501 kB | 309/354 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 2.6 MB/s) 2024-04-17T04:47:41.332 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-04-17T04:47:41.333 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 344/780 kB | 325/354 kB Progress (2): 344/780 kB | 341/354 kB Progress (2): 344/780 kB | 354 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 1.9 MB/s) 2024-04-17T04:47:41.333 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-04-17T04:47:41.344 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 360/780 kB Progress (1): 377/780 kB Progress (1): 393/780 kB Progress (1): 410/780 kB Progress (1): 426/780 kB Progress (1): 442/780 kB Progress (1): 459/780 kB Progress (1): 475/780 kB Progress (1): 492/780 kB Progress (1): 508/780 kB Progress (1): 524/780 kB Progress (1): 541/780 kB Progress (1): 557/780 kB Progress (1): 573/780 kB Progress (1): 585/780 kB Progress (1): 601/780 kB Progress (1): 617/780 kB Progress (1): 634/780 kB Progress (1): 650/780 kB Progress (1): 667/780 kB Progress (1): 683/780 kB Progress (1): 699/780 kB Progress (1): 716/780 kB Progress (1): 732/780 kB Progress (1): 749/780 kB Progress (1): 765/780 kB Progress (2): 765/780 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 31 kB/s) 2024-04-17T04:47:41.344 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-04-17T04:47:41.350 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 765/780 kB | 16/586 kB Progress (2): 765/780 kB | 33/586 kB Progress (2): 765/780 kB | 49/586 kB Progress (2): 765/780 kB | 66/586 kB Progress (2): 765/780 kB | 82/586 kB Progress (2): 765/780 kB | 98/586 kB Progress (2): 765/780 kB | 113/586 kB Progress (2): 765/780 kB | 129/586 kB Progress (2): 780 kB | 129/586 kB Progress (3): 780 kB | 129/586 kB | 16/211 kB Progress (3): 780 kB | 145/586 kB | 16/211 kB Progress (3): 780 kB | 145/586 kB | 33/211 kB Progress (4): 780 kB | 145/586 kB | 33/211 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 3.8 MB/s) 2024-04-17T04:47:41.350 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 162/586 kB | 33/211 kB | 14 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-04-17T04:47:41.350 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 162/586 kB | 49/211 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 69 kB/s) 2024-04-17T04:47:41.350 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-04-17T04:47:41.355 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 178/586 kB | 49/211 kB Progress (2): 178/586 kB | 62/211 kB Progress (2): 194/586 kB | 62/211 kB Progress (2): 194/586 kB | 79/211 kB Progress (2): 211/586 kB | 79/211 kB Progress (2): 211/586 kB | 95/211 kB Progress (2): 227/586 kB | 95/211 kB Progress (2): 227/586 kB | 112/211 kB Progress (2): 244/586 kB | 112/211 kB Progress (2): 244/586 kB | 128/211 kB Progress (2): 260/586 kB | 128/211 kB Progress (2): 260/586 kB | 144/211 kB Progress (2): 276/586 kB | 144/211 kB Progress (2): 293/586 kB | 144/211 kB Progress (2): 293/586 kB | 161/211 kB Progress (2): 293/586 kB | 169/211 kB Progress (2): 309/586 kB | 169/211 kB Progress (2): 309/586 kB | 185/211 kB Progress (2): 326/586 kB | 185/211 kB Progress (2): 326/586 kB | 202/211 kB Progress (2): 342/586 kB | 202/211 kB Progress (2): 342/586 kB | 210/211 kB Progress (2): 342/586 kB | 211 kB Progress (2): 358/586 kB | 211 kB Progress (2): 375/586 kB | 211 kB Progress (2): 391/586 kB | 211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 990 kB/s) 2024-04-17T04:47:41.355 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 407/586 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-04-17T04:47:41.358 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 424/586 kB Progress (1): 440/586 kB Progress (1): 457/586 kB Progress (1): 473/586 kB Progress (1): 489/586 kB Progress (1): 506/586 kB Progress (1): 522/586 kB Progress (1): 539/586 kB Progress (1): 555/586 kB Progress (1): 571/586 kB Progress (1): 586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 2.7 MB/s) 2024-04-17T04:47:41.358 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-04-17T04:47:41.364 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/106 kB Progress (1): 33/106 kB Progress (1): 49/106 kB Progress (1): 66/106 kB Progress (1): 82/106 kB Progress (1): 97/106 kB Progress (1): 106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 477 kB/s) 2024-04-17T04:47:41.365 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-04-17T04:47:41.366 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 30 kB/s) 2024-04-17T04:47:41.366 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-04-17T04:47:41.374 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/245 kB Progress (1): 33/245 kB Progress (1): 49/245 kB Progress (1): 61/245 kB Progress (1): 78/245 kB Progress (1): 94/245 kB Progress (1): 111/245 kB Progress (1): 127/245 kB Progress (1): 143/245 kB Progress (1): 160/245 kB Progress (1): 176/245 kB Progress (1): 193/245 kB Progress (1): 209/245 kB Progress (2): 209/245 kB | 16/205 kB Progress (2): 225/245 kB | 16/205 kB Progress (2): 225/245 kB | 33/205 kB Progress (2): 242/245 kB | 33/205 kB Progress (2): 245 kB | 33/205 kB Progress (2): 245 kB | 49/205 kB Progress (2): 245 kB | 62/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 1.1 MB/s) 2024-04-17T04:47:41.374 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 78/205 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-04-17T04:47:41.375 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 95/205 kB Progress (1): 111/205 kB Progress (1): 128/205 kB Progress (2): 128/205 kB | 15 kB Progress (2): 144/205 kB | 15 kB Progress (2): 160/205 kB | 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 65 kB/s) 2024-04-17T04:47:41.375 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-04-17T04:47:41.377 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 177/205 kB Progress (1): 193/205 kB Progress (1): 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 872 kB/s) 2024-04-17T04:47:41.377 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-04-17T04:47:41.382 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/16 kB Progress (1): 16 kB Progress (2): 16 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 69 kB/s) 2024-04-17T04:47:41.383 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-04-17T04:47:41.383 INFO:tasks.workunit.client.0.smithi088.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 26 kB/s) 2024-04-17T04:47:41.392 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 91 kB/s) 2024-04-17T04:47:41.393 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 96 kB/s) 2024-04-17T04:47:41.394 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 69 kB/s) 2024-04-17T04:47:41.405 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/184 kB Progress (1): 33/184 kB Progress (1): 49/184 kB Progress (1): 66/184 kB Progress (1): 81/184 kB Progress (1): 98/184 kB Progress (1): 114/184 kB Progress (1): 130/184 kB Progress (1): 147/184 kB Progress (1): 163/184 kB Progress (1): 180/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 699 kB/s) 2024-04-17T04:47:41.431 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:41.431 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-04-17T04:47:41.439 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-04-17T04:47:41.459 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 292 kB/s) 2024-04-17T04:47:41.462 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-04-17T04:47:41.480 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 283 kB/s) 2024-04-17T04:47:41.483 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-04-17T04:47:41.502 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 2.3 MB/s) 2024-04-17T04:47:41.508 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-04-17T04:47:41.527 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 792 kB/s) 2024-04-17T04:47:41.532 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-04-17T04:47:41.552 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/153 kB Progress (1): 33/153 kB Progress (1): 49/153 kB Progress (1): 64/153 kB Progress (1): 81/153 kB Progress (1): 97/153 kB Progress (1): 113/153 kB Progress (1): 130/153 kB Progress (1): 146/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 7.7 MB/s) 2024-04-17T04:47:41.552 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-04-17T04:47:41.573 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/215 kB Progress (1): 33/215 kB Progress (1): 49/215 kB Progress (1): 64/215 kB Progress (1): 81/215 kB Progress (1): 97/215 kB Progress (1): 113/215 kB Progress (1): 130/215 kB Progress (1): 146/215 kB Progress (1): 163/215 kB Progress (1): 179/215 kB Progress (1): 195/215 kB Progress (1): 212/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 9.8 MB/s) 2024-04-17T04:47:41.669 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:41.669 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-04-17T04:47:41.676 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-04-17T04:47:41.694 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 148 kB/s) 2024-04-17T04:47:41.697 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-04-17T04:47:41.715 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 1.2 MB/s) 2024-04-17T04:47:41.720 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-04-17T04:47:41.738 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 487 kB/s) 2024-04-17T04:47:41.741 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-04-17T04:47:41.759 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 1.4 MB/s) 2024-04-17T04:47:41.764 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-04-17T04:47:41.783 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 302 kB/s) 2024-04-17T04:47:41.786 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-04-17T04:47:41.804 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.9/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 179 kB/s) 2024-04-17T04:47:41.810 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-04-17T04:47:41.827 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 36 kB/s) 2024-04-17T04:47:41.831 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-04-17T04:47:41.849 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 152 kB/s) 2024-04-17T04:47:41.853 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-04-17T04:47:41.872 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 742 kB/s) 2024-04-17T04:47:41.879 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-04-17T04:47:41.899 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 169 kB/s) 2024-04-17T04:47:41.903 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-04-17T04:47:41.922 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 440 kB/s) 2024-04-17T04:47:41.926 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-04-17T04:47:41.945 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 440 kB/s) 2024-04-17T04:47:41.951 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-04-17T04:47:41.970 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 422 kB/s) 2024-04-17T04:47:41.977 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-04-17T04:47:41.996 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 1.0 MB/s) 2024-04-17T04:47:42.009 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-04-17T04:47:42.029 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 255 kB/s) 2024-04-17T04:47:42.036 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-04-17T04:47:42.056 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 1.5 MB/s) 2024-04-17T04:47:42.069 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-04-17T04:47:42.089 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 255 kB/s) 2024-04-17T04:47:42.103 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-04-17T04:47:42.125 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/85 kB Progress (1): 33/85 kB Progress (1): 49/85 kB Progress (1): 66/85 kB Progress (1): 82/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 3.9 MB/s) 2024-04-17T04:47:42.125 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-04-17T04:47:42.126 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-04-17T04:47:42.126 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-04-17T04:47:42.126 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-04-17T04:47:42.127 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-04-17T04:47:42.146 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/269 kB Progress (1): 33/269 kB Progress (1): 49/269 kB Progress (2): 49/269 kB | 16/55 kB Progress (2): 62/269 kB | 16/55 kB Progress (2): 78/269 kB | 16/55 kB Progress (2): 78/269 kB | 33/55 kB Progress (3): 78/269 kB | 33/55 kB | 2.5 kB Progress (4): 78/269 kB | 33/55 kB | 2.5 kB | 16/41 kB Progress (4): 95/269 kB | 33/55 kB | 2.5 kB | 16/41 kB Progress (4): 95/269 kB | 49/55 kB | 2.5 kB | 16/41 kB Progress (4): 95/269 kB | 49/55 kB | 2.5 kB | 32/41 kB Progress (5): 95/269 kB | 49/55 kB | 2.5 kB | 32/41 kB | 8.5 kB Progress (5): 95/269 kB | 55 kB | 2.5 kB | 32/41 kB | 8.5 kB Progress (5): 95/269 kB | 55 kB | 2.5 kB | 41 kB | 8.5 kB Progress (5): 111/269 kB | 55 kB | 2.5 kB | 41 kB | 8.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 125 kB/s) 2024-04-17T04:47:42.146 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-04-17T04:47:42.147 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (4): 127/269 kB | 55 kB | 41 kB | 8.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 446 kB/s) 2024-04-17T04:47:42.147 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-04-17T04:47:42.147 INFO:tasks.workunit.client.0.smithi088.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 2.5 MB/s) 2024-04-17T04:47:42.147 INFO:tasks.workunit.client.0.smithi088.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 2.0 MB/s) 2024-04-17T04:47:42.150 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 144/269 kB Progress (1): 160/269 kB Progress (1): 177/269 kB Progress (1): 193/269 kB Progress (1): 209/269 kB Progress (1): 226/269 kB Progress (1): 242/269 kB Progress (1): 258/269 kB Progress (1): 269 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 11 MB/s) 2024-04-17T04:47:42.176 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/587 kB Progress (2): 16/587 kB | 16/327 kB Progress (2): 33/587 kB | 16/327 kB Progress (2): 33/587 kB | 33/327 kB Progress (2): 49/587 kB | 33/327 kB Progress (2): 49/587 kB | 49/327 kB Progress (2): 66/587 kB | 49/327 kB Progress (2): 66/587 kB | 66/327 kB Progress (2): 82/587 kB | 66/327 kB Progress (2): 82/587 kB | 82/327 kB Progress (2): 98/587 kB | 82/327 kB Progress (2): 98/587 kB | 98/327 kB Progress (2): 115/587 kB | 98/327 kB Progress (2): 115/587 kB | 115/327 kB Progress (2): 131/587 kB | 115/327 kB Progress (2): 131/587 kB | 131/327 kB Progress (2): 147/587 kB | 131/327 kB Progress (2): 147/587 kB | 147/327 kB Progress (2): 164/587 kB | 147/327 kB Progress (2): 164/587 kB | 164/327 kB Progress (2): 180/587 kB | 164/327 kB Progress (2): 180/587 kB | 180/327 kB Progress (2): 197/587 kB | 180/327 kB Progress (2): 197/587 kB | 197/327 kB Progress (2): 213/587 kB | 197/327 kB Progress (2): 213/587 kB | 213/327 kB Progress (2): 229/587 kB | 213/327 kB Progress (2): 229/587 kB | 229/327 kB Progress (2): 229/587 kB | 246/327 kB Progress (2): 246/587 kB | 246/327 kB Progress (2): 246/587 kB | 262/327 kB Progress (2): 262/587 kB | 262/327 kB Progress (2): 262/587 kB | 279/327 kB Progress (2): 279/587 kB | 279/327 kB Progress (2): 279/587 kB | 295/327 kB Progress (2): 295/587 kB | 295/327 kB Progress (2): 295/587 kB | 311/327 kB Progress (2): 311/587 kB | 311/327 kB Progress (2): 311/587 kB | 327 kB Progress (2): 328/587 kB | 327 kB Progress (2): 344/587 kB | 327 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 6.7 MB/s) 2024-04-17T04:47:42.182 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 360/587 kB Progress (1): 377/587 kB Progress (1): 393/587 kB Progress (1): 410/587 kB Progress (1): 426/587 kB Progress (1): 442/587 kB Progress (1): 459/587 kB Progress (1): 475/587 kB Progress (1): 492/587 kB Progress (1): 508/587 kB Progress (1): 524/587 kB Progress (1): 541/587 kB Progress (1): 557/587 kB Progress (1): 573/587 kB Progress (1): 587 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 11 MB/s) 2024-04-17T04:47:42.237 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-04-17T04:47:42.238 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:42.238 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-04-17T04:47:42.245 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-04-17T04:47:42.263 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 263 kB/s) 2024-04-17T04:47:42.266 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-04-17T04:47:42.284 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 353 kB/s) 2024-04-17T04:47:42.287 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-04-17T04:47:42.304 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.8 MB/s) 2024-04-17T04:47:42.307 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-04-17T04:47:42.324 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 822 kB/s) 2024-04-17T04:47:42.328 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-04-17T04:47:42.346 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 48 kB/s) 2024-04-17T04:47:42.349 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-04-17T04:47:42.367 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 236 kB/s) 2024-04-17T04:47:42.369 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-04-17T04:47:42.388 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 961 kB/s) 2024-04-17T04:47:42.391 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-04-17T04:47:42.409 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 276 kB/s) 2024-04-17T04:47:42.412 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-04-17T04:47:42.430 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 230 kB/s) 2024-04-17T04:47:42.434 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-04-17T04:47:42.452 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 132 kB/s) 2024-04-17T04:47:42.455 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-04-17T04:47:42.473 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 627 kB/s) 2024-04-17T04:47:42.476 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-04-17T04:47:42.494 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 907 kB/s) 2024-04-17T04:47:42.497 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-04-17T04:47:42.515 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 386 kB/s) 2024-04-17T04:47:42.518 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-04-17T04:47:42.536 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 59 kB/s) 2024-04-17T04:47:42.539 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-04-17T04:47:42.557 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 466 kB/s) 2024-04-17T04:47:42.561 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-04-17T04:47:42.579 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 141 kB/s) 2024-04-17T04:47:42.585 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-04-17T04:47:42.603 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 296 kB/s) 2024-04-17T04:47:42.610 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-04-17T04:47:42.629 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 60 kB/s) 2024-04-17T04:47:42.636 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-04-17T04:47:42.653 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 43 kB/s) 2024-04-17T04:47:42.656 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-04-17T04:47:42.674 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 318 kB/s) 2024-04-17T04:47:42.677 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-04-17T04:47:42.695 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 1.4 MB/s) 2024-04-17T04:47:42.702 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-04-17T04:47:42.720 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 67 kB/s) 2024-04-17T04:47:42.722 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-04-17T04:47:42.740 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 74 kB/s) 2024-04-17T04:47:42.750 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-04-17T04:47:42.768 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 752 kB/s) 2024-04-17T04:47:42.769 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-04-17T04:47:42.769 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-04-17T04:47:42.770 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-04-17T04:47:42.770 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-04-17T04:47:42.770 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-04-17T04:47:42.788 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.2 kB Progress (2): 4.2 kB | 16/55 kB Progress (3): 4.2 kB | 16/55 kB | 16/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 222 kB/s) 2024-04-17T04:47:42.788 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-04-17T04:47:42.789 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 33/55 kB | 16/334 kB Progress (2): 49/55 kB | 16/334 kB Progress (3): 49/55 kB | 16/334 kB | 16/122 kB Progress (3): 49/55 kB | 33/334 kB | 16/122 kB Progress (3): 55 kB | 33/334 kB | 16/122 kB Progress (4): 55 kB | 33/334 kB | 16/122 kB | 16/27 kB Progress (4): 55 kB | 49/334 kB | 16/122 kB | 16/27 kB Progress (4): 55 kB | 49/334 kB | 33/122 kB | 16/27 kB Progress (4): 55 kB | 49/334 kB | 33/122 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 2.7 MB/s) 2024-04-17T04:47:42.789 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-04-17T04:47:42.790 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 64/334 kB | 33/122 kB | 27 kB Progress (3): 64/334 kB | 49/122 kB | 27 kB Progress (3): 80/334 kB | 49/122 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 1.4 MB/s) 2024-04-17T04:47:42.790 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-04-17T04:47:42.792 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 80/334 kB | 65/122 kB Progress (2): 96/334 kB | 65/122 kB Progress (2): 96/334 kB | 82/122 kB Progress (2): 113/334 kB | 82/122 kB Progress (2): 113/334 kB | 98/122 kB Progress (2): 129/334 kB | 98/122 kB Progress (2): 129/334 kB | 114/122 kB Progress (2): 129/334 kB | 122 kB Progress (2): 146/334 kB | 122 kB Progress (2): 162/334 kB | 122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 5.6 MB/s) 2024-04-17T04:47:42.798 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 178/334 kB Progress (1): 195/334 kB Progress (1): 211/334 kB Progress (1): 227/334 kB Progress (1): 244/334 kB Progress (1): 260/334 kB Progress (1): 277/334 kB Progress (1): 293/334 kB Progress (1): 309/334 kB Progress (1): 326/334 kB Progress (1): 334 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 12 MB/s) 2024-04-17T04:47:42.807 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15/267 kB Progress (1): 31/267 kB Progress (1): 48/267 kB Progress (1): 64/267 kB Progress (2): 64/267 kB | 4.7 kB Progress (2): 81/267 kB | 4.7 kB Progress (2): 97/267 kB | 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 129 kB/s) 2024-04-17T04:47:42.808 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 97/267 kB | 16/23 kB Progress (2): 113/267 kB | 16/23 kB Progress (2): 113/267 kB | 23 kB Progress (2): 130/267 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 600 kB/s) 2024-04-17T04:47:42.812 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 146/267 kB Progress (1): 162/267 kB Progress (1): 179/267 kB Progress (1): 195/267 kB Progress (1): 212/267 kB Progress (1): 228/267 kB Progress (1): 244/267 kB Progress (1): 261/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 6.5 MB/s) 2024-04-17T04:47:42.941 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Changes detected - recompiling the module! :source 2024-04-17T04:47:42.943 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-04-17T04:47:44.062 INFO:tasks.workunit.client.0.smithi088.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-04-17T04:47:44.063 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:44.063 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-04-17T04:47:44.063 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-04-17T04:47:44.063 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:44.063 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-04-17T04:47:44.068 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-04-17T04:47:44.069 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-04-17T04:47:44.248 INFO:tasks.workunit.client.0.smithi088.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-04-17T04:47:44.248 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:44.248 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-04-17T04:47:44.253 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-04-17T04:47:44.270 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 367 kB/s) 2024-04-17T04:47:44.274 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-04-17T04:47:44.292 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 196 kB/s) 2024-04-17T04:47:44.296 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-04-17T04:47:44.313 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 181 kB/s) 2024-04-17T04:47:44.317 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-04-17T04:47:44.335 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 239 kB/s) 2024-04-17T04:47:44.338 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-04-17T04:47:44.356 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 194 kB/s) 2024-04-17T04:47:44.360 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-04-17T04:47:44.377 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 262 kB/s) 2024-04-17T04:47:44.381 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-04-17T04:47:44.398 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 104 kB/s) 2024-04-17T04:47:44.402 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-04-17T04:47:44.419 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 129 kB/s) 2024-04-17T04:47:44.420 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-04-17T04:47:44.437 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 1.7 MB/s) 2024-04-17T04:47:44.441 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-04-17T04:47:44.458 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 105 kB/s) 2024-04-17T04:47:44.462 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-04-17T04:47:44.480 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 305 kB/s) 2024-04-17T04:47:44.485 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-04-17T04:47:44.502 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 130 kB/s) 2024-04-17T04:47:44.505 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-04-17T04:47:44.523 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 1.2 MB/s) 2024-04-17T04:47:44.526 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-04-17T04:47:44.544 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 2.0 MB/s) 2024-04-17T04:47:44.548 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-04-17T04:47:44.565 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 846 kB/s) 2024-04-17T04:47:44.569 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-04-17T04:47:44.586 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 509 kB/s) 2024-04-17T04:47:44.588 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-04-17T04:47:44.606 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 1.6 MB/s) 2024-04-17T04:47:44.608 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-04-17T04:47:44.625 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 474 kB/s) 2024-04-17T04:47:44.630 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-04-17T04:47:44.650 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 109 kB/s) 2024-04-17T04:47:44.654 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-04-17T04:47:44.672 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 144 kB/s) 2024-04-17T04:47:44.677 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-04-17T04:47:44.695 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 86 kB/s) 2024-04-17T04:47:44.697 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-04-17T04:47:44.715 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 180 kB/s) 2024-04-17T04:47:44.717 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-04-17T04:47:44.735 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 1.2 MB/s) 2024-04-17T04:47:44.738 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-04-17T04:47:44.756 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 398 kB/s) 2024-04-17T04:47:44.759 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-04-17T04:47:44.777 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 753 kB/s) 2024-04-17T04:47:44.783 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-04-17T04:47:44.802 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 156 kB/s) 2024-04-17T04:47:44.805 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-04-17T04:47:44.823 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 358 kB/s) 2024-04-17T04:47:44.827 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-04-17T04:47:44.845 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 466 kB/s) 2024-04-17T04:47:44.850 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-04-17T04:47:44.869 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 115 kB/s) 2024-04-17T04:47:44.878 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-04-17T04:47:44.893 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 117 kB/s) 2024-04-17T04:47:44.901 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-04-17T04:47:44.921 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 151 kB/s) 2024-04-17T04:47:44.928 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-04-17T04:47:44.947 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 788 kB/s) 2024-04-17T04:47:44.951 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-04-17T04:47:44.970 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 725 kB/s) 2024-04-17T04:47:44.977 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-04-17T04:47:44.996 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 672 kB/s) 2024-04-17T04:47:45.000 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-04-17T04:47:45.018 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 266 kB/s) 2024-04-17T04:47:45.023 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-04-17T04:47:45.042 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 349 kB/s) 2024-04-17T04:47:45.046 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-04-17T04:47:45.065 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 1.7 MB/s) 2024-04-17T04:47:45.071 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-04-17T04:47:45.090 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 913 kB/s) 2024-04-17T04:47:45.094 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-04-17T04:47:45.113 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 779 kB/s) 2024-04-17T04:47:45.119 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-04-17T04:47:45.137 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 39 kB/s) 2024-04-17T04:47:45.141 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-04-17T04:47:45.159 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 253 kB/s) 2024-04-17T04:47:45.164 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-04-17T04:47:45.185 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 396 kB/s) 2024-04-17T04:47:45.190 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-04-17T04:47:45.209 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 166 kB/s) 2024-04-17T04:47:45.217 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-04-17T04:47:45.235 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 224 kB/s) 2024-04-17T04:47:45.243 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-04-17T04:47:45.261 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 108 kB/s) 2024-04-17T04:47:45.267 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-04-17T04:47:45.286 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 183 kB/s) 2024-04-17T04:47:45.292 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-04-17T04:47:45.311 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 576 kB/s) 2024-04-17T04:47:45.315 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-04-17T04:47:45.334 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 709 kB/s) 2024-04-17T04:47:45.337 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-04-17T04:47:45.356 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 986 kB/s) 2024-04-17T04:47:45.362 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-04-17T04:47:45.380 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 21 kB/s) 2024-04-17T04:47:45.384 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-04-17T04:47:45.402 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 339 kB/s) 2024-04-17T04:47:45.406 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-04-17T04:47:45.424 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 386 kB/s) 2024-04-17T04:47:45.428 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-04-17T04:47:45.446 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 268 kB/s) 2024-04-17T04:47:45.451 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-04-17T04:47:45.470 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 384 kB/s) 2024-04-17T04:47:45.476 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-04-17T04:47:45.495 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 1.1 MB/s) 2024-04-17T04:47:45.499 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-04-17T04:47:45.521 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 64/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 3.5 MB/s) 2024-04-17T04:47:45.529 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-04-17T04:47:45.548 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 1.2 MB/s) 2024-04-17T04:47:45.555 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-04-17T04:47:45.573 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 314 kB/s) 2024-04-17T04:47:45.578 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-04-17T04:47:45.596 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 225 kB/s) 2024-04-17T04:47:45.600 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-04-17T04:47:45.619 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 168 kB/s) 2024-04-17T04:47:45.622 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-04-17T04:47:45.642 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 1.5 MB/s) 2024-04-17T04:47:45.648 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-04-17T04:47:45.668 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 113 kB/s) 2024-04-17T04:47:45.681 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-04-17T04:47:45.711 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/308 kB Progress (1): 33/308 kB Progress (1): 49/308 kB Progress (1): 63/308 kB Progress (1): 80/308 kB Progress (1): 96/308 kB Progress (1): 112/308 kB Progress (1): 129/308 kB Progress (1): 145/308 kB Progress (1): 162/308 kB Progress (1): 178/308 kB Progress (1): 194/308 kB Progress (1): 211/308 kB Progress (1): 227/308 kB Progress (1): 244/308 kB Progress (1): 260/308 kB Progress (1): 276/308 kB Progress (1): 293/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 10 MB/s) 2024-04-17T04:47:45.711 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-04-17T04:47:45.712 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-04-17T04:47:45.712 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-04-17T04:47:45.713 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-04-17T04:47:45.713 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-04-17T04:47:45.733 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/171 kB Progress (1): 33/171 kB Progress (1): 49/171 kB Progress (1): 66/171 kB Progress (1): 82/171 kB Progress (1): 98/171 kB Progress (2): 98/171 kB | 16/26 kB Progress (3): 98/171 kB | 16/26 kB | 14 kB Progress (3): 114/171 kB | 16/26 kB | 14 kB Progress (3): 114/171 kB | 26 kB | 14 kB Progress (4): 114/171 kB | 26 kB | 14 kB | 16/118 kB Progress (4): 130/171 kB | 26 kB | 14 kB | 16/118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 679 kB/s) 2024-04-17T04:47:45.733 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (4): 130/171 kB | 26 kB | 16/118 kB | 8.2 kB Progress (4): 147/171 kB | 26 kB | 16/118 kB | 8.2 kB Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-04-17T04:47:45.733 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (4): 147/171 kB | 26 kB | 33/118 kB | 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 1.2 MB/s) 2024-04-17T04:47:45.733 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 163/171 kB | 33/118 kB | 8.2 kB Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-04-17T04:47:45.734 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 171 kB | 33/118 kB | 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 409 kB/s) 2024-04-17T04:47:45.734 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-04-17T04:47:45.734 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 171 kB | 49/118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 7.8 MB/s) 2024-04-17T04:47:45.734 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-04-17T04:47:45.735 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 66/118 kB Progress (1): 82/118 kB Progress (1): 98/118 kB Progress (1): 115/118 kB Progress (1): 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 5.3 MB/s) 2024-04-17T04:47:45.735 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-04-17T04:47:45.753 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/146 kB Progress (2): 16/146 kB | 16/136 kB Progress (2): 33/146 kB | 16/136 kB Progress (2): 33/146 kB | 33/136 kB Progress (2): 49/146 kB | 33/136 kB Progress (2): 49/146 kB | 49/136 kB Progress (2): 63/146 kB | 49/136 kB Progress (2): 80/146 kB | 49/136 kB Progress (2): 80/146 kB | 65/136 kB Progress (3): 80/146 kB | 65/136 kB | 16/61 kB Progress (4): 80/146 kB | 65/136 kB | 16/61 kB | 16/501 kB Progress (4): 96/146 kB | 65/136 kB | 16/61 kB | 16/501 kB Progress (4): 96/146 kB | 65/136 kB | 33/61 kB | 16/501 kB Progress (4): 96/146 kB | 65/136 kB | 33/61 kB | 33/501 kB Progress (4): 96/146 kB | 82/136 kB | 33/61 kB | 33/501 kB Progress (4): 113/146 kB | 82/136 kB | 33/61 kB | 33/501 kB Progress (4): 113/146 kB | 82/136 kB | 33/61 kB | 49/501 kB Progress (4): 113/146 kB | 82/136 kB | 49/61 kB | 49/501 kB Progress (4): 113/146 kB | 98/136 kB | 49/61 kB | 49/501 kB Progress (4): 113/146 kB | 98/136 kB | 49/61 kB | 64/501 kB Progress (4): 129/146 kB | 98/136 kB | 49/61 kB | 64/501 kB Progress (4): 129/146 kB | 98/136 kB | 61 kB | 64/501 kB Progress (4): 129/146 kB | 98/136 kB | 61 kB | 80/501 kB Progress (4): 145/146 kB | 98/136 kB | 61 kB | 80/501 kB Progress (4): 145/146 kB | 98/136 kB | 61 kB | 97/501 kB Progress (4): 146 kB | 98/136 kB | 61 kB | 97/501 kB Progress (4): 146 kB | 114/136 kB | 61 kB | 97/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 1.6 MB/s) 2024-04-17T04:47:45.753 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 146 kB | 114/136 kB | 113/501 kB Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-04-17T04:47:45.753 INFO:tasks.workunit.client.0.smithi088.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 3.6 MB/s) 2024-04-17T04:47:45.753 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 131/136 kB | 113/501 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-04-17T04:47:45.754 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 131/136 kB | 130/501 kB Progress (2): 136 kB | 130/501 kB Progress (2): 136 kB | 146/501 kB Progress (2): 136 kB | 162/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 3.4 MB/s) 2024-04-17T04:47:45.757 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 179/501 kB Progress (1): 195/501 kB Progress (1): 212/501 kB Progress (1): 228/501 kB Progress (1): 244/501 kB Progress (2): 244/501 kB | 16/58 kB Progress (2): 261/501 kB | 16/58 kB Progress (2): 277/501 kB | 16/58 kB Progress (2): 293/501 kB | 16/58 kB Progress (2): 293/501 kB | 33/58 kB Progress (2): 310/501 kB | 33/58 kB Progress (2): 310/501 kB | 49/58 kB Progress (2): 326/501 kB | 49/58 kB Progress (2): 326/501 kB | 58 kB Progress (2): 343/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 1.3 MB/s) 2024-04-17T04:47:45.761 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 359/501 kB Progress (1): 375/501 kB Progress (1): 392/501 kB Progress (1): 408/501 kB Progress (1): 425/501 kB Progress (1): 441/501 kB Progress (1): 457/501 kB Progress (1): 474/501 kB Progress (1): 490/501 kB Progress (1): 501 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 10 MB/s) 2024-04-17T04:47:45.772 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15/124 kB Progress (1): 31/124 kB Progress (1): 48/124 kB Progress (2): 48/124 kB | 0/2.4 MB Progress (2): 64/124 kB | 0/2.4 MB Progress (2): 64/124 kB | 0/2.4 MB Progress (2): 81/124 kB | 0/2.4 MB Progress (2): 81/124 kB | 0/2.4 MB Progress (2): 97/124 kB | 0/2.4 MB Progress (2): 97/124 kB | 0.1/2.4 MB Progress (2): 113/124 kB | 0.1/2.4 MB Progress (2): 113/124 kB | 0.1/2.4 MB Progress (2): 124 kB | 0.1/2.4 MB Progress (2): 124 kB | 0.1/2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 2.1 MB/s) 2024-04-17T04:47:45.812 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 25 MB/s) 2024-04-17T04:47:45.932 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:45.932 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-04-17T04:47:45.936 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-04-17T04:47:45.953 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 264 kB/s) 2024-04-17T04:47:45.956 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-04-17T04:47:45.973 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 272 kB/s) 2024-04-17T04:47:45.975 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-04-17T04:47:45.994 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 2.4 MB/s) 2024-04-17T04:47:45.997 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-04-17T04:47:46.016 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 1.1 MB/s) 2024-04-17T04:47:46.021 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-04-17T04:47:46.037 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 305 kB/s) 2024-04-17T04:47:46.039 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-04-17T04:47:46.056 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 245 kB/s) 2024-04-17T04:47:46.058 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-04-17T04:47:46.075 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 376 kB/s) 2024-04-17T04:47:46.077 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-04-17T04:47:46.094 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 391 kB/s) 2024-04-17T04:47:46.096 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-04-17T04:47:46.113 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 1.2 MB/s) 2024-04-17T04:47:46.117 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-04-17T04:47:46.135 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 808 kB/s) 2024-04-17T04:47:46.139 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-04-17T04:47:46.156 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 114 kB/s) 2024-04-17T04:47:46.160 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-04-17T04:47:46.177 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 456 kB/s) 2024-04-17T04:47:46.180 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-04-17T04:47:46.198 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 15/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 1.4 MB/s) 2024-04-17T04:47:46.205 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-04-17T04:47:46.223 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 2.0 MB/s) 2024-04-17T04:47:46.224 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-04-17T04:47:46.224 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-04-17T04:47:46.224 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-04-17T04:47:46.225 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-04-17T04:47:46.225 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-04-17T04:47:46.243 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/26 kB Progress (2): 16/26 kB | 16/211 kB Progress (2): 26 kB | 16/211 kB Progress (3): 26 kB | 16/211 kB | 16/58 kB Progress (3): 26 kB | 33/211 kB | 16/58 kB Progress (3): 26 kB | 33/211 kB | 33/58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 1.4 MB/s) 2024-04-17T04:47:46.243 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 33/211 kB | 33/58 kB | 16/79 kB Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-04-17T04:47:46.246 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 49/211 kB | 33/58 kB | 16/79 kB Progress (3): 49/211 kB | 49/58 kB | 16/79 kB Progress (3): 49/211 kB | 49/58 kB | 33/79 kB Progress (3): 49/211 kB | 58 kB | 33/79 kB Progress (3): 49/211 kB | 58 kB | 49/79 kB Progress (3): 66/211 kB | 58 kB | 49/79 kB Progress (3): 66/211 kB | 58 kB | 66/79 kB Progress (3): 82/211 kB | 58 kB | 66/79 kB Progress (3): 95/211 kB | 58 kB | 66/79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 3.1 MB/s) 2024-04-17T04:47:46.246 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 95/211 kB | 66/79 kB | 0/1.0 MB Progress (3): 95/211 kB | 79 kB | 0/1.0 MB Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-04-17T04:47:46.246 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 111/211 kB | 79 kB | 0/1.0 MB Progress (3): 111/211 kB | 79 kB | 0/1.0 MB Progress (3): 127/211 kB | 79 kB | 0/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 3.8 MB/s) 2024-04-17T04:47:46.247 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 127/211 kB | 0/1.0 MB Progress (2): 144/211 kB | 0/1.0 MB Progress (2): 144/211 kB | 0.1/1.0 MB Progress (2): 160/211 kB | 0.1/1.0 MB Progress (2): 160/211 kB | 0.1/1.0 MB Progress (2): 176/211 kB | 0.1/1.0 MB Progress (2): 176/211 kB | 0.1/1.0 MB Progress (2): 176/211 kB | 0.1/1.0 MB Progress (2): 193/211 kB | 0.1/1.0 MB Progress (2): 209/211 kB | 0.1/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 9.2 MB/s) 2024-04-17T04:47:46.262 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 0.1/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (2): 0.4/1.0 MB | 16/116 kB Progress (2): 0.4/1.0 MB | 33/116 kB Progress (2): 0.4/1.0 MB | 33/116 kB Progress (2): 0.4/1.0 MB | 49/116 kB Progress (2): 0.4/1.0 MB | 49/116 kB Progress (2): 0.4/1.0 MB | 49/116 kB Progress (2): 0.4/1.0 MB | 66/116 kB Progress (2): 0.4/1.0 MB | 66/116 kB Progress (2): 0.4/1.0 MB | 82/116 kB Progress (2): 0.5/1.0 MB | 82/116 kB Progress (2): 0.5/1.0 MB | 82/116 kB Progress (2): 0.5/1.0 MB | 98/116 kB Progress (2): 0.5/1.0 MB | 115/116 kB Progress (2): 0.5/1.0 MB | 116 kB Progress (2): 0.5/1.0 MB | 116 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 3.1 MB/s) 2024-04-17T04:47:46.271 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 0.5/1.0 MB | 16/267 kB Progress (2): 0.5/1.0 MB | 33/267 kB Progress (2): 0.5/1.0 MB | 33/267 kB Progress (2): 0.5/1.0 MB | 49/267 kB Progress (2): 0.5/1.0 MB | 49/267 kB Progress (2): 0.5/1.0 MB | 49/267 kB Progress (2): 0.5/1.0 MB | 64/267 kB Progress (2): 0.5/1.0 MB | 81/267 kB Progress (2): 0.6/1.0 MB | 81/267 kB Progress (2): 0.6/1.0 MB | 81/267 kB Progress (2): 0.6/1.0 MB | 97/267 kB Progress (2): 0.6/1.0 MB | 97/267 kB Progress (2): 0.6/1.0 MB | 97/267 kB Progress (2): 0.6/1.0 MB | 113/267 kB Progress (2): 0.6/1.0 MB | 130/267 kB Progress (2): 0.6/1.0 MB | 130/267 kB Progress (2): 0.6/1.0 MB | 146/267 kB Progress (2): 0.6/1.0 MB | 146/267 kB Progress (2): 0.6/1.0 MB | 146/267 kB Progress (2): 0.6/1.0 MB | 163/267 kB Progress (2): 0.7/1.0 MB | 163/267 kB Progress (2): 0.7/1.0 MB | 179/267 kB Progress (2): 0.7/1.0 MB | 195/267 kB Progress (2): 0.7/1.0 MB | 195/267 kB Progress (2): 0.7/1.0 MB | 212/267 kB Progress (2): 0.7/1.0 MB | 212/267 kB Progress (2): 0.7/1.0 MB | 228/267 kB Progress (2): 0.7/1.0 MB | 228/267 kB Progress (2): 0.7/1.0 MB | 244/267 kB Progress (2): 0.7/1.0 MB | 244/267 kB Progress (2): 0.7/1.0 MB | 261/267 kB Progress (2): 0.7/1.0 MB | 267 kB Progress (2): 0.7/1.0 MB | 267 kB Progress (2): 0.8/1.0 MB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 5.7 MB/s) 2024-04-17T04:47:46.282 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 18 MB/s) 2024-04-17T04:47:46.464 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-04-17T04:47:46.501 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-17T04:47:46.501 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] BUILD SUCCESS 2024-04-17T04:47:46.501 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-17T04:47:46.502 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Total time: 8.540 s 2024-04-17T04:47:46.502 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Finished at: 2024-04-17T04:47:46Z 2024-04-17T04:47:46.502 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-17T04:47:46.530 INFO:tasks.workunit.client.0.smithi088.stderr:+ ./mvnw test -Dtest=PutObjects 2024-04-17T04:47:46.536 INFO:tasks.workunit.client.0.smithi088.stdout:Warning: JAVA_HOME environment variable is not set. 2024-04-17T04:47:47.727 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Scanning for projects... 2024-04-17T04:47:47.844 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:47.844 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-04-17T04:47:47.844 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Building jcksum 1.0 2024-04-17T04:47:47.844 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] from pom.xml 2024-04-17T04:47:47.844 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-04-17T04:47:48.217 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:48.217 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-04-17T04:47:48.362 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-04-17T04:47:48.363 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:48.363 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-04-17T04:47:48.477 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Nothing to compile - all classes are up to date 2024-04-17T04:47:48.477 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:48.477 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-04-17T04:47:48.480 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-04-17T04:47:48.480 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:48.480 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-04-17T04:47:48.487 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Nothing to compile - all classes are up to date 2024-04-17T04:47:48.487 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:48.487 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-04-17T04:47:48.711 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-04-17T04:47:48.928 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-04-17T04:47:49.131 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 23 kB/s) 2024-04-17T04:47:49.140 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-04-17T04:47:49.161 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 124 kB/s) 2024-04-17T04:47:49.167 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-04-17T04:47:49.187 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 141 kB/s) 2024-04-17T04:47:49.192 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-04-17T04:47:49.212 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 159 kB/s) 2024-04-17T04:47:49.215 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-04-17T04:47:49.238 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 133 kB/s) 2024-04-17T04:47:49.243 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-04-17T04:47:49.264 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 80 kB/s) 2024-04-17T04:47:49.268 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-04-17T04:47:49.290 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 129 kB/s) 2024-04-17T04:47:49.300 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-04-17T04:47:49.333 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 823 kB/s) 2024-04-17T04:47:49.335 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-04-17T04:47:49.336 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-04-17T04:47:49.336 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-04-17T04:47:49.337 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-04-17T04:47:49.339 INFO:tasks.workunit.client.0.smithi088.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-04-17T04:47:49.370 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 474 kB/s) 2024-04-17T04:47:49.399 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 1.4/7.7 kB Progress (2): 1.4/7.7 kB | 1.4/103 kB Progress (2): 2.8/7.7 kB | 1.4/103 kB Progress (2): 2.8/7.7 kB | 2.8/103 kB Progress (2): 4.1/7.7 kB | 2.8/103 kB Progress (2): 4.1/7.7 kB | 4.1/103 kB Progress (2): 5.5/7.7 kB | 4.1/103 kB Progress (3): 5.5/7.7 kB | 4.1/103 kB | 1.4/169 kB Progress (3): 5.5/7.7 kB | 5.5/103 kB | 1.4/169 kB Progress (3): 6.9/7.7 kB | 5.5/103 kB | 1.4/169 kB Progress (3): 6.9/7.7 kB | 5.5/103 kB | 2.8/169 kB Progress (3): 7.7 kB | 5.5/103 kB | 2.8/169 kB Progress (3): 7.7 kB | 6.9/103 kB | 2.8/169 kB Progress (3): 7.7 kB | 6.9/103 kB | 4.1/169 kB Progress (4): 7.7 kB | 6.9/103 kB | 4.1/169 kB | 16/189 kB Progress (4): 7.7 kB | 8.3/103 kB | 4.1/169 kB | 16/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 121 kB/s) 2024-04-17T04:47:49.429 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (3): 8.3/103 kB | 5.5/169 kB | 16/189 kB Progress (3): 9.7/103 kB | 5.5/169 kB | 16/189 kB Progress (3): 9.7/103 kB | 6.9/169 kB | 16/189 kB Progress (3): 11/103 kB | 6.9/169 kB | 16/189 kB Progress (3): 11/103 kB | 8.3/169 kB | 16/189 kB Progress (3): 12/103 kB | 8.3/169 kB | 16/189 kB Progress (3): 12/103 kB | 9.7/169 kB | 16/189 kB Progress (3): 14/103 kB | 9.7/169 kB | 16/189 kB Progress (3): 14/103 kB | 11/169 kB | 16/189 kB Progress (3): 15/103 kB | 11/169 kB | 16/189 kB Progress (3): 15/103 kB | 11/169 kB | 33/189 kB Progress (3): 16/103 kB | 11/169 kB | 33/189 kB Progress (3): 16/103 kB | 12/169 kB | 33/189 kB Progress (3): 18/103 kB | 12/169 kB | 33/189 kB Progress (3): 18/103 kB | 14/169 kB | 33/189 kB Progress (3): 19/103 kB | 14/169 kB | 33/189 kB Progress (3): 19/103 kB | 15/169 kB | 33/189 kB Progress (3): 21/103 kB | 15/169 kB | 33/189 kB Progress (3): 21/103 kB | 16/169 kB | 33/189 kB Progress (3): 22/103 kB | 16/169 kB | 33/189 kB Progress (3): 22/103 kB | 18/169 kB | 33/189 kB Progress (3): 23/103 kB | 18/169 kB | 33/189 kB Progress (3): 23/103 kB | 19/169 kB | 33/189 kB Progress (3): 25/103 kB | 19/169 kB | 33/189 kB Progress (3): 25/103 kB | 21/169 kB | 33/189 kB Progress (3): 26/103 kB | 21/169 kB | 33/189 kB Progress (3): 26/103 kB | 22/169 kB | 33/189 kB Progress (3): 27/103 kB | 22/169 kB | 33/189 kB Progress (3): 27/103 kB | 23/169 kB | 33/189 kB Progress (3): 29/103 kB | 23/169 kB | 33/189 kB Progress (3): 29/103 kB | 25/169 kB | 33/189 kB Progress (3): 30/103 kB | 25/169 kB | 33/189 kB Progress (3): 30/103 kB | 26/169 kB | 33/189 kB Progress (3): 32/103 kB | 26/169 kB | 33/189 kB Progress (3): 32/103 kB | 27/169 kB | 33/189 kB Progress (3): 33/103 kB | 27/169 kB | 33/189 kB Progress (3): 33/103 kB | 29/169 kB | 33/189 kB Progress (3): 34/103 kB | 29/169 kB | 33/189 kB Progress (3): 34/103 kB | 30/169 kB | 33/189 kB Progress (3): 36/103 kB | 30/169 kB | 33/189 kB Progress (3): 36/103 kB | 32/169 kB | 33/189 kB Progress (3): 37/103 kB | 32/169 kB | 33/189 kB Progress (3): 37/103 kB | 33/169 kB | 33/189 kB Progress (3): 38/103 kB | 33/169 kB | 33/189 kB Progress (3): 38/103 kB | 34/169 kB | 33/189 kB Progress (3): 40/103 kB | 34/169 kB | 33/189 kB Progress (3): 40/103 kB | 36/169 kB | 33/189 kB Progress (3): 41/103 kB | 36/169 kB | 33/189 kB Progress (3): 41/103 kB | 36/169 kB | 49/189 kB Progress (3): 43/103 kB | 36/169 kB | 49/189 kB Progress (3): 43/103 kB | 37/169 kB | 49/189 kB Progress (3): 44/103 kB | 37/169 kB | 49/189 kB Progress (3): 44/103 kB | 38/169 kB | 49/189 kB Progress (3): 45/103 kB | 38/169 kB | 49/189 kB Progress (3): 45/103 kB | 38/169 kB | 66/189 kB Progress (3): 45/103 kB | 40/169 kB | 66/189 kB Progress (3): 47/103 kB | 40/169 kB | 66/189 kB Progress (3): 47/103 kB | 41/169 kB | 66/189 kB Progress (3): 48/103 kB | 41/169 kB | 66/189 kB Progress (3): 48/103 kB | 43/169 kB | 66/189 kB Progress (3): 49/103 kB | 43/169 kB | 66/189 kB Progress (3): 49/103 kB | 44/169 kB | 66/189 kB Progress (3): 49/103 kB | 44/169 kB | 82/189 kB Progress (3): 51/103 kB | 44/169 kB | 82/189 kB Progress (3): 51/103 kB | 45/169 kB | 82/189 kB Progress (3): 52/103 kB | 45/169 kB | 82/189 kB Progress (3): 52/103 kB | 47/169 kB | 82/189 kB Progress (3): 53/103 kB | 47/169 kB | 82/189 kB Progress (3): 53/103 kB | 47/169 kB | 98/189 kB Progress (3): 55/103 kB | 47/169 kB | 98/189 kB Progress (3): 55/103 kB | 48/169 kB | 98/189 kB Progress (3): 56/103 kB | 48/169 kB | 98/189 kB Progress (3): 56/103 kB | 49/169 kB | 98/189 kB Progress (3): 57/103 kB | 49/169 kB | 98/189 kB Progress (3): 57/103 kB | 51/169 kB | 98/189 kB Progress (3): 59/103 kB | 51/169 kB | 98/189 kB Progress (3): 59/103 kB | 52/169 kB | 98/189 kB Progress (3): 60/103 kB | 52/169 kB | 98/189 kB Progress (3): 60/103 kB | 54/169 kB | 98/189 kB Progress (3): 62/103 kB | 54/169 kB | 98/189 kB Progress (3): 62/103 kB | 55/169 kB | 98/189 kB Progress (3): 63/103 kB | 55/169 kB | 98/189 kB Progress (3): 63/103 kB | 56/169 kB | 98/189 kB Progress (3): 64/103 kB | 56/169 kB | 98/189 kB Progress (3): 64/103 kB | 58/169 kB | 98/189 kB Progress (3): 64/103 kB | 59/169 kB | 98/189 kB Progress (3): 64/103 kB | 61/169 kB | 98/189 kB Progress (3): 64/103 kB | 62/169 kB | 98/189 kB Progress (3): 64/103 kB | 63/169 kB | 98/189 kB Progress (3): 64/103 kB | 64/169 kB | 98/189 kB Progress (3): 78/103 kB | 64/169 kB | 98/189 kB Progress (3): 94/103 kB | 64/169 kB | 98/189 kB Progress (3): 103 kB | 64/169 kB | 98/189 kB Progress (3): 103 kB | 64/169 kB | 115/189 kB Progress (3): 103 kB | 80/169 kB | 115/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 1.1 MB/s) 2024-04-17T04:47:49.434 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (2): 80/169 kB | 131/189 kB Progress (2): 97/169 kB | 131/189 kB Progress (2): 97/169 kB | 147/189 kB Progress (2): 113/169 kB | 147/189 kB Progress (2): 113/169 kB | 164/189 kB Progress (2): 113/169 kB | 180/189 kB Progress (2): 129/169 kB | 180/189 kB Progress (2): 129/169 kB | 189 kB Progress (2): 146/169 kB | 189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 1.9 MB/s) 2024-04-17T04:47:49.436 INFO:tasks.workunit.client.0.smithi088.stdout:Progress (1): 162/169 kB Progress (1): 169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 1.7 MB/s) 2024-04-17T04:47:49.458 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:47:49.459 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------- 2024-04-17T04:47:49.459 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] T E S T S 2024-04-17T04:47:49.459 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------- 2024-04-17T04:47:50.037 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-04-17T04:47:50.109 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjects.java: starting test run: 2024-04-17T04:47:50.109 INFO:tasks.workunit.client.0.smithi088.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-04-17T04:47:50.109 INFO:tasks.workunit.client.0.smithi088.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-04-17T04:47:50.109 INFO:tasks.workunit.client.0.smithi088.stdout: EndpointUrl=http://smithi088.front.sepia.ceph.com:80 2024-04-17T04:47:50.109 INFO:tasks.workunit.client.0.smithi088.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-04-17T04:47:50.306 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Environment Variables Read 2024-04-17T04:47:50.708 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: S3 Client Initialized 2024-04-17T04:47:50.708 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generating File 2024-04-17T04:47:50.708 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: File Generated 2024-04-17T04:47:50.708 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generating File 2024-04-17T04:47:50.708 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: File Generated 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generating File 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: File Generated 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generating File 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: File Generated 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generating File 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: File Generated 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generating File 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: File Generated 2024-04-17T04:47:50.709 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Generated Big Files 2024-04-17T04:47:51.104 INFO:tasks.workunit.client.0.smithi088.stdout:HeadBucketResponse() 2024-04-17T04:47:51.105 INFO:tasks.workunit.client.0.smithi088.stdout:sheik is ready 2024-04-17T04:47:51.105 INFO:tasks.workunit.client.0.smithi088.stdout:DEBUG: Test Bucket Created 2024-04-17T04:47:51.208 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-0b 2024-04-17T04:47:51.509 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-8b 2024-04-17T04:47:51.810 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-200b 2024-04-17T04:47:52.110 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-21983b 2024-04-17T04:47:52.612 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-5519b 2024-04-17T04:47:52.913 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-204329b 2024-04-17T04:47:53.413 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-256k 2024-04-17T04:47:53.814 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-1m 2024-04-17T04:47:54.516 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-04-17T04:47:55.521 INFO:tasks.workunit.client.0.smithi088.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-04-17T04:48:00.131 INFO:tasks.workunit.client.0.smithi088.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-04-17T04:48:06.545 INFO:tasks.workunit.client.0.smithi088.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-04-17T04:48:43.758 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-0b 2024-04-17T04:48:43.760 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-8b 2024-04-17T04:48:43.762 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-200b 2024-04-17T04:48:43.764 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-21983b 2024-04-17T04:48:43.765 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-5519b 2024-04-17T04:48:43.767 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-204329b 2024-04-17T04:48:43.769 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-256k 2024-04-17T04:48:43.771 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-1m 2024-04-17T04:48:43.772 INFO:tasks.workunit.client.0.smithi088.stdout:arg: file-1038757b 2024-04-17T04:48:43.809 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-0b 2024-04-17T04:48:43.984 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:44.009 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-8b 2024-04-17T04:48:44.109 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:44.210 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-200b 2024-04-17T04:48:44.309 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:44.510 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-21983b 2024-04-17T04:48:44.610 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:44.811 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-5519b 2024-04-17T04:48:44.910 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:45.011 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-204329b 2024-04-17T04:48:45.311 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:45.411 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-256k 2024-04-17T04:48:45.712 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:45.812 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-1m 2024-04-17T04:48:46.313 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:46.513 INFO:tasks.workunit.client.0.smithi088.stdout:putObjectFromFileCksum called with file-1038757b 2024-04-17T04:48:47.214 INFO:tasks.workunit.client.0.smithi088.stdout:PutObjectResponse 2024-04-17T04:48:47.414 INFO:tasks.workunit.client.0.smithi088.stdout:mpuObjectFromFileCksum called with file-5m 2024-04-17T04:48:49.818 INFO:tasks.workunit.client.0.smithi088.stdout:mpuObjectFromFileCksum called with file-10m 2024-04-17T04:48:54.524 INFO:tasks.workunit.client.0.smithi088.stdout:mpuObjectFromFileCksum called with file-100m 2024-04-17T04:49:33.371 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 103.3 s -- in io.ceph.jcksum.PutObjects 2024-04-17T04:49:33.413 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:49:33.414 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Results: 2024-04-17T04:49:33.414 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:49:33.414 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-04-17T04:49:33.414 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] 2024-04-17T04:49:33.416 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-17T04:49:33.416 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] BUILD SUCCESS 2024-04-17T04:49:33.416 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-17T04:49:33.418 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Total time: 01:45 min 2024-04-17T04:49:33.418 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] Finished at: 2024-04-17T04:49:33Z 2024-04-17T04:49:33.418 INFO:tasks.workunit.client.0.smithi088.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-17T04:49:33.439 INFO:tasks.workunit.client.0.smithi088.stderr:+ exit 2024-04-17T04:49:33.440 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:49:33.440 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:49:33.493 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-04-17T04:49:33.493 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T04:49:33.883 DEBUG:teuthology.parallel:result is None 2024-04-17T04:49:33.883 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:49:33.896 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:49:33.897 DEBUG:teuthology.orchestra.run.smithi088:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:49:33.945 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:49:33.945 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T04:49:33.955 INFO:tasks.workunit:Pulling workunits from ref 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:49:33.955 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T04:49:33.956 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:49:33.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:49:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T04:49:33.993 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:49:34.041 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-17T04:49:34.042 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-17T04:49:34.089 INFO:tasks.workunit:timeout=3h 2024-04-17T04:49:34.089 INFO:tasks.workunit:cleanup=True 2024-04-17T04:49:34.090 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:49:34.141 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T04:50:41.917 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '41c500dd60c93c8d83053b72cffc3dd1a952a336'. 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T04:50:41.918 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:50:41.919 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 41c500dd60c Merge pull request #56902 from adk3798/wip-65415-squid 2024-04-17T04:50:41.921 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T04:50:41.971 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T04:50:41.975 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:50:41.975 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T04:50:42.036 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T04:50:42.087 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T04:50:42.125 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:50:42.126 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:50:42.126 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T04:50:42.173 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:50:42.177 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:50:42.177 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T04:50:42.228 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-04-17T04:50:42.229 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-04-17T04:50:42.229 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-04-17T04:50:42.284 INFO:tasks.workunit.client.0.smithi088.stdout:INFO: fully qualified domain name: smithi088.front.sepia.ceph.com 2024-04-17T04:50:42.284 INFO:tasks.workunit.client.0.smithi088.stdout:== TESTING *ABORTING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-04-17T04:50:42.284 INFO:tasks.workunit.client.0.smithi088.stdout:== TESTING NON-VERSIONED BUCKET == 2024-04-17T04:50:45.570 INFO:tasks.workunit.client.0.smithi088.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-04-17T04:50:45.783 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting pip 2024-04-17T04:50:45.790 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-17T04:50:45.825 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: pip 2024-04-17T04:50:45.826 INFO:tasks.workunit.client.0.smithi088.stdout: Attempting uninstall: pip 2024-04-17T04:50:45.826 INFO:tasks.workunit.client.0.smithi088.stdout: Found existing installation: pip 22.0.2 2024-04-17T04:50:46.081 INFO:tasks.workunit.client.0.smithi088.stdout: Uninstalling pip-22.0.2: 2024-04-17T04:50:46.086 INFO:tasks.workunit.client.0.smithi088.stdout: Successfully uninstalled pip-22.0.2 2024-04-17T04:50:47.119 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed pip-24.0 2024-04-17T04:50:48.145 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting boto3 2024-04-17T04:50:48.146 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached boto3-1.34.85-py3-none-any.whl.metadata (6.6 kB) 2024-04-17T04:50:48.696 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting botocore<1.35.0,>=1.34.85 (from boto3) 2024-04-17T04:50:48.697 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached botocore-1.34.85-py3-none-any.whl.metadata (5.7 kB) 2024-04-17T04:50:48.752 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-04-17T04:50:48.753 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-04-17T04:50:48.809 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-04-17T04:50:48.810 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-04-17T04:50:48.881 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:50:48.883 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-04-17T04:50:48.987 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:50:48.988 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-04-17T04:50:49.164 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:50:49.165 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-17T04:50:49.177 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached boto3-1.34.85-py3-none-any.whl (139 kB) 2024-04-17T04:50:49.179 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached botocore-1.34.85-py3-none-any.whl (12.1 MB) 2024-04-17T04:50:49.203 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-04-17T04:50:49.204 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-04-17T04:50:49.205 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-17T04:50:49.207 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-04-17T04:50:49.209 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-17T04:50:49.301 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-04-17T04:50:50.423 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed boto3-1.34.85 botocore-1.34.85 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-04-17T04:50:50.971 INFO:tasks.workunit.client.0.smithi088.stdout:INFO: creating user 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout:{ 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "user_id": "testid", 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "display_name": "M. Tester", 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "email": "tester@ceph.com", 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "suspended": 0, 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "max_buckets": 1000, 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "subusers": [], 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: "keys": [ 2024-04-17T04:50:51.110 INFO:tasks.workunit.client.0.smithi088.stdout: { 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "user": "testid", 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "access_key": "0555b35654ad1656d804", 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "active": true, 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "create_date": "2024-04-17T04:50:51.104717Z" 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: } 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: ], 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "swift_keys": [], 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "caps": [], 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "default_placement": "", 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "default_storage_class": "", 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "placement_tags": [], 2024-04-17T04:50:51.111 INFO:tasks.workunit.client.0.smithi088.stdout: "bucket_quota": { 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "enabled": false, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size": -1, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "max_objects": -1 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "user_quota": { 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "enabled": false, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size": -1, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "max_objects": -1 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: }, 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "temp_url_keys": [], 2024-04-17T04:50:51.112 INFO:tasks.workunit.client.0.smithi088.stdout: "type": "rgw", 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: "mfa_ids": [], 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: "account_id": "", 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: "path": "/", 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: "create_date": "2024-04-17T04:50:51.104708Z", 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: "tags": [], 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: "group_ids": [] 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout:} 2024-04-17T04:50:51.113 INFO:tasks.workunit.client.0.smithi088.stdout: 2024-04-17T04:50:51.489 INFO:tasks.workunit.client.0.smithi088.stderr:5+0 records in 2024-04-17T04:50:51.490 INFO:tasks.workunit.client.0.smithi088.stderr:5+0 records out 2024-04-17T04:50:51.490 INFO:tasks.workunit.client.0.smithi088.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0228571 s, 229 MB/s 2024-04-17T04:51:04.937 INFO:tasks.workunit.client.0.smithi088.stdout:bucket nams is reupload-bkt-131848 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:endpoint is http://smithi088.front.sepia.ceph.com:80 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:start UploadId=2~Y5XiQwAplZQW7dp5ofeW4_QLhpkdxOR 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 1 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 1 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 2} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.938 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.939 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.939 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.939 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.939 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.939 INFO:tasks.workunit.client.0.smithi088.stdout:phase 2 uploaded part {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1} 2024-04-17T04:51:04.939 INFO:tasks.workunit.client.0.smithi088.stdout:aborting multipart upload, parts=[{'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 1}, {'ETag': '"b126fe144d27d8e1a58bfc0b1aa30c32"', 'PartNumber': 2}] 2024-04-17T04:51:06.431 INFO:tasks.workunit.client.0.smithi088.stdout:radosgw-admin bucket radoslist: 2024-04-17T04:51:06.520 INFO:tasks.workunit.client.0.smithi088.stdout:rados ls: 2024-04-17T04:51:06.540 INFO:tasks.workunit.client.0.smithi088.stdout:== PASSED == 2024-04-17T04:51:06.540 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:51:06.540 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:51:06.591 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-04-17T04:51:06.591 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T04:51:06.997 DEBUG:teuthology.parallel:result is None 2024-04-17T04:51:06.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:51:07.011 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:51:07.011 DEBUG:teuthology.orchestra.run.smithi088:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:51:07.058 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:51:07.058 INFO:teuthology.run_tasks:Running task ragweed... 2024-04-17T04:51:07.073 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-squid', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-04-17T04:51:07.073 INFO:tasks.ragweed:Downloading ragweed... 2024-04-17T04:51:07.073 INFO:tasks.ragweed:Using branch 'squid' for ragweed 2024-04-17T04:51:07.074 DEBUG:teuthology.orchestra.run.smithi088:> git clone -b squid https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-04-17T04:51:07.105 INFO:teuthology.orchestra.run.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-04-17T04:51:07.158 INFO:teuthology.orchestra.run.smithi088.stderr:fatal: Remote branch squid not found in upstream origin 2024-04-17T04:51:07.159 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-04-17T04:51:07.161 INFO:tasks.ragweed:Using branch 'ceph-squid' for ragweed 2024-04-17T04:51:07.162 DEBUG:teuthology.orchestra.run.smithi088:> git clone -b ceph-squid https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-04-17T04:51:07.209 INFO:teuthology.orchestra.run.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-04-17T04:51:07.399 INFO:tasks.ragweed:Creating rgw users... 2024-04-17T04:51:07.399 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-04-17T04:51:07.399 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key CCCMNWTHRRWHWGQWBKOS --secret XfSIt7Fsq7VfaPnRxsO3UaxteyuaIsJdwRZ9nyooWnKvuoIceBcvUw== --email ragweed.client.0+test@test.test --system false 2024-04-17T04:51:07.486 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:51:07.486 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:51:07.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.530+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 realm 2024-04-17T04:51:07.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:51:07.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:51:07.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:51:07.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.534+0000 7f9b84ab8980 20 searching for the correct realm 2024-04-17T04:51:07.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.542+0000 7f9b81dd2640 0 -- 172.21.15.88:0/1656913391 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f9b7404f040 msgr2=0x7f9b74050430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:51:07.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.546+0000 7f9b81dd2640 0 -- 172.21.15.88:0/1656913391 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f9a4c074720 msgr2=0x7f9a4c076b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:51:07.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:51:07.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.554+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:51:07.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 zone default found 2024-04-17T04:51:07.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 4 Realm: () 2024-04-17T04:51:07.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 zonegroup default 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 started sync module instance, tier type = 2024-04-17T04:51:07.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.558+0000 7f9b84ab8980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:51:07.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=0 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=1 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=2 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=3 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=4 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=5 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=6 2024-04-17T04:51:07.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 20 add_watcher() i=7 2024-04-17T04:51:07.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.574+0000 7f9b84ab8980 2 all 8 watchers are set, enabling cache 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9a38fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 5 note: GC not initialized 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f99b37fe640 20 reqs_thread_entry: start 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 init_complete bucket index max shards: 1 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 Filter name: none 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f99b17fa640 20 reqs_thread_entry: start 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-04-17T04:51:07.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-04-17T04:51:07.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-17T04:51:07.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-04-17T04:51:07.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache get: name=default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS : miss 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache put: name=default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS info.flags=0x0 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 adding default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS to cache LRU end 2024-04-17T04:51:07.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache get: name=default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS : hit (negative entry) 2024-04-17T04:51:07.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.578+0000 7f9b84ab8980 10 cache get: name=default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS : hit (negative entry) 2024-04-17T04:51:07.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.582+0000 7f9b84ab8980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-04-17T04:51:07.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.582+0000 7f9b84ab8980 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-17T04:51:07.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.582+0000 7f9b84ab8980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-04-17T04:51:07.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.586+0000 7f9a65ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1086626725911 cookie 94754970734816 notifier 5892 bl.length()=675 2024-04-17T04:51:07.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.586+0000 7f9a65ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-04-17T04:51:07.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.586+0000 7f9a65ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-17T04:51:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.590+0000 7f9b84ab8980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-04-17T04:51:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.590+0000 7f9b84ab8980 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-04-17T04:51:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.590+0000 7f9b84ab8980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-04-17T04:51:07.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.590+0000 7f9a65ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1086626725904 cookie 94754970731088 notifier 5892 bl.length()=202 2024-04-17T04:51:07.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.590+0000 7f9a65ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-04-17T04:51:07.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.590+0000 7f9a65ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-04-17T04:51:07.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.594+0000 7f9b84ab8980 10 cache put: name=default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS info.flags=0x7 2024-04-17T04:51:07.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.594+0000 7f9b84ab8980 10 moving default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS to cache LRU end 2024-04-17T04:51:07.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.594+0000 7f9b84ab8980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:CCCMNWTHRRWHWGQWBKOS, ofs0, ns] 2024-04-17T04:51:07.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.594+0000 7f9a657fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1086626725902 cookie 94754970744496 notifier 5892 bl.length()=190 2024-04-17T04:51:07.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.594+0000 7f9a657fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS info.flags=0x7 2024-04-17T04:51:07.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.594+0000 7f9a657fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+CCCMNWTHRRWHWGQWBKOS to cache LRU end 2024-04-17T04:51:07.592 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T04:51:07.592 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "ragweed.client.0", 2024-04-17T04:51:07.592 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr. ragweed.client.0", 2024-04-17T04:51:07.592 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "ragweed.client.0+test@test.test", 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "ragweed.client.0", 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "CCCMNWTHRRWHWGQWBKOS", 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "XfSIt7Fsq7VfaPnRxsO3UaxteyuaIsJdwRZ9nyooWnKvuoIceBcvUw==", 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T04:51:07.584318Z" 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T04:51:07.593 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:51:07.594 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "rgw", 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "", 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T04:51:07.584304Z", 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T04:51:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:51:07.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.598+0000 7f9a38fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:51:07.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.602+0000 7f9b84ab8980 20 remove_watcher() i=0 2024-04-17T04:51:07.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.602+0000 7f9b84ab8980 2 removed watcher, disabling cache 2024-04-17T04:51:07.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.606+0000 7f9b84ab8980 20 remove_watcher() i=1 2024-04-17T04:51:07.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.606+0000 7f9b84ab8980 20 remove_watcher() i=2 2024-04-17T04:51:07.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.610+0000 7f9b84ab8980 20 remove_watcher() i=3 2024-04-17T04:51:07.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.614+0000 7f9b84ab8980 20 remove_watcher() i=4 2024-04-17T04:51:07.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.614+0000 7f9b84ab8980 20 remove_watcher() i=5 2024-04-17T04:51:07.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.614+0000 7f9b84ab8980 20 remove_watcher() i=6 2024-04-17T04:51:07.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.618+0000 7f9b84ab8980 20 remove_watcher() i=7 2024-04-17T04:51:07.627 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-04-17T04:51:07.627 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key MIEJRQVUOIIIKAGFYPXB --secret 4l4WRvIkk7L6xOZ+lUdhAFvyj2EbisnbtPnbzb6D3iTcgn4jk4kZnw== --email sysuser.client.0+test@test.test --system true 2024-04-17T04:51:07.668 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:51:07.668 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:51:07.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 realm 2024-04-17T04:51:07.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:51:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.714+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.718+0000 7f5d6db03980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:51:07.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.718+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.718+0000 7f5d6db03980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:51:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.718+0000 7f5d6db03980 20 searching for the correct realm 2024-04-17T04:51:07.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.722+0000 7f5d6ae1d640 0 -- 172.21.15.88:0/4132399146 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5d640d5040 msgr2=0x7f5d640f3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:51:07.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.730+0000 7f5d6be1f640 0 -- 172.21.15.88:0/4132399146 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f5d54008030 msgr2=0x7f5d54055740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:51:07.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:51:07.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:51:07.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:51:07.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 zone default found 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 4 Realm: () 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 zonegroup default 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 started sync module instance, tier type = 2024-04-17T04:51:07.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.738+0000 7f5d6db03980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:51:07.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=0 2024-04-17T04:51:07.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=1 2024-04-17T04:51:07.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=2 2024-04-17T04:51:07.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=3 2024-04-17T04:51:07.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=4 2024-04-17T04:51:07.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=5 2024-04-17T04:51:07.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.754+0000 7f5d6db03980 20 add_watcher() i=6 2024-04-17T04:51:07.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 20 add_watcher() i=7 2024-04-17T04:51:07.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 2 all 8 watchers are set, enabling cache 2024-04-17T04:51:07.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5c21fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:51:07.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:51:07.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:51:07.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 5 note: GC not initialized 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5c20fe1640 20 reqs_thread_entry: start 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 20 init_complete bucket index max shards: 1 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 20 Filter name: none 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5b9a7fc640 20 reqs_thread_entry: start 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.758+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-04-17T04:51:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache get: name=default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB : miss 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 20 rados->read ofs=0 len=0 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache put: name=default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB info.flags=0x0 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 adding default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB to cache LRU end 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache get: name=default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB : hit (negative entry) 2024-04-17T04:51:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.762+0000 7f5d6db03980 10 cache get: name=default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB : hit (negative entry) 2024-04-17T04:51:07.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.766+0000 7f5d6db03980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-04-17T04:51:07.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.766+0000 7f5d6db03980 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-17T04:51:07.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.766+0000 7f5d6db03980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-04-17T04:51:07.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.766+0000 7f5c4e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1086626725905 cookie 94581182000848 notifier 5910 bl.length()=675 2024-04-17T04:51:07.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.766+0000 7f5c4e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-04-17T04:51:07.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.766+0000 7f5c4e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-17T04:51:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.770+0000 7f5d6db03980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-04-17T04:51:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.770+0000 7f5d6db03980 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-04-17T04:51:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.770+0000 7f5d6db03980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-04-17T04:51:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.770+0000 7f5c4e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1086626725912 cookie 94581181993856 notifier 5910 bl.length()=202 2024-04-17T04:51:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.770+0000 7f5c4e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-04-17T04:51:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.770+0000 7f5c4e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-04-17T04:51:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.774+0000 7f5d6db03980 10 cache put: name=default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB info.flags=0x7 2024-04-17T04:51:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.774+0000 7f5d6db03980 10 moving default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB to cache LRU end 2024-04-17T04:51:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.774+0000 7f5d6db03980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MIEJRQVUOIIIKAGFYPXB, ofs0, ns] 2024-04-17T04:51:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.778+0000 7f5c4e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1086626725906 cookie 94581182000848 notifier 5910 bl.length()=190 2024-04-17T04:51:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.778+0000 7f5c4e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB info.flags=0x7 2024-04-17T04:51:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.778+0000 7f5c4e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MIEJRQVUOIIIKAGFYPXB to cache LRU end 2024-04-17T04:51:07.774 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T04:51:07.774 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "sysuser.client.0", 2024-04-17T04:51:07.774 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr. sysuser.client.0", 2024-04-17T04:51:07.774 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "sysuser.client.0+test@test.test", 2024-04-17T04:51:07.774 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T04:51:07.774 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "sysuser.client.0", 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "MIEJRQVUOIIIKAGFYPXB", 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "4l4WRvIkk7L6xOZ+lUdhAFvyj2EbisnbtPnbzb6D3iTcgn4jk4kZnw==", 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T04:51:07.766466Z" 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "system": true, 2024-04-17T04:51:07.775 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T04:51:07.776 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "rgw", 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "", 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T04:51:07.766452Z", 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:51:07.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.778+0000 7f5c21fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:51:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.782+0000 7f5d6db03980 20 remove_watcher() i=0 2024-04-17T04:51:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.782+0000 7f5d6db03980 2 removed watcher, disabling cache 2024-04-17T04:51:07.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.786+0000 7f5d6db03980 20 remove_watcher() i=1 2024-04-17T04:51:07.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.790+0000 7f5d6db03980 20 remove_watcher() i=2 2024-04-17T04:51:07.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.790+0000 7f5d6db03980 20 remove_watcher() i=3 2024-04-17T04:51:07.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.794+0000 7f5d6db03980 20 remove_watcher() i=4 2024-04-17T04:51:07.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.794+0000 7f5d6db03980 20 remove_watcher() i=5 2024-04-17T04:51:07.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.794+0000 7f5d6db03980 20 remove_watcher() i=6 2024-04-17T04:51:07.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:51:07.794+0000 7f5d6db03980 20 remove_watcher() i=7 2024-04-17T04:51:07.806 INFO:tasks.ragweed:Configuring ragweed... 2024-04-17T04:51:07.806 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:51:07.806 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-04-17T04:51:07.853 INFO:tasks.ragweed:Configuring boto... 2024-04-17T04:51:07.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:51:07.854 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-04-17T04:51:07.901 DEBUG:teuthology.orchestra.run.smithi088:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-04-17T04:51:08.077 INFO:teuthology.orchestra.run.smithi088.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-04-17T04:51:08.258 INFO:teuthology.orchestra.run.smithi088.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-04-17T04:51:08.818 INFO:teuthology.orchestra.run.smithi088.stdout:py installdeps: -rrequirements.txt 2024-04-17T04:51:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-04-17T04:51:15.014 INFO:teuthology.orchestra.run.smithi088.stdout:py installed: appdirs==1.4.4,asyncssh==2.5.0,attrs==21.2.0,Babel==2.8.0,bcrypt==3.2.0,beautifulsoup4==4.10.0,beniget==0.4.1,blinker==1.4,boto==2.49.0,Brotli==1.0.9,cachetools==5.0.0,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2020.6.20,chardet==4.0.0,cheroot==8.5.2+ds1,CherryPy==18.6.1,command-not-found==0.3,configobj==5.0.6,cryptography==3.4.8,cycler==0.11.0,dbus-python==1.2.18,decorator==4.4.2,distlib==0.3.4,distro==1.7.0,distro-info===1.1build1,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,fonttools==4.29.1,fs==2.4.12,gast==0.5.2,gevent==21.8.0,google-auth==1.5.1,greenlet==1.1.2,html5lib==1.1,httplib2==0.20.2,idna==3.3,importlib-metadata==4.6.4,iniconfig==1.1.1,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.4.0,jaraco.functools==3.4.0,jaraco.text==3.6.0,jeepney==0.7.1,Jinja2==3.0.3,joblib==0.17.0,keyring==23.5.0,kiwisolver==1.3.2,kubernetes==12.0.1,language-selector==0.1,launchpadlib==1.10.16,lazr.restfulclient==0.14.4,lazr.uri==1.0.6,logutils==0.3.3,lxml==4.8.0,lz4==3.1.3+dfsg,Mako==1.1.3,MarkupSafe==2.0.1,matplotlib==3.5.1,more-itertools==8.10.0,mpmath==0.0.0,munch==4.0.0,natsort==8.0.2,netifaces==0.11.0,nose==1.3.7,numpy==1.21.5,oauthlib==3.2.0,olefile==0.46,packaging==21.3,Paste==3.5.0,PasteDeploy==2.1.1,PasteScript==2.0.2,pecan==1.3.3,Pillow==9.0.1,platformdirs==2.5.1,pluggy==0.13.0,ply==3.11,portend==3.0.0,prettytable==2.5.0,psutil==5.9.0,py==1.10.0,pyasn1==0.4.8,pyasn1-modules==0.2.1,Pygments==2.11.2,PyGObject==3.42.1,pyinotify==0.9.6,PyJWT==2.3.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyOpenSSL==21.0.0,pyparsing==2.4.7,pytest==6.2.5,python-apt==2.3.0+ubuntu2.1,python-dateutil==2.8.1,python-magic==0.4.24,pythran==0.10.0,pytz==2022.1,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=55d77e02f5c2ba19f3a5065ad7469856d0f822b43a47b20ebce2c726a1782abc,rbd==2.0.0,registries-conf-ctl @ git+https://github.com/sebastian-philipp/registries-conf-ctl@0e9bf2dc202aa9be9b0bf8e8265aac3f2308f9db,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rsa==4.8,s3cmd==2.2.0,scikit-learn==0.23.2,scipy==1.8.0,SecretStorage==3.3.1,simplegeneric==0.8.1,simplejson==3.17.6,six==1.16.0,soupsieve==2.3.1,ssh-import-id==5.11,sympy==1.9,Tempita==0.5.2,tempora==4.1.2,threadpoolctl==3.1.0,toml==0.10.2,ubuntu-advantage-tools==27.12,ufoLib2==0.13.1,ufw==0.36.1,unattended-upgrades==0.1,unicodedata2==14.0.0,urllib3==1.26.5,virtualenv==20.13.0+ds,wadllib==1.3.6,waitress==1.4.4,wcwidth==0.2.5,webencodings==0.5.1,WebOb==1.8.6,websocket-client==1.2.3,WebTest==2.0.35,Werkzeug==2.0.2,zc.lockfile==2.0,zipp==1.0.0,zope.event==4.4,zope.interface==5.4.0 2024-04-17T04:51:15.014 INFO:teuthology.orchestra.run.smithi088.stdout:py run-test-pre: PYTHONHASHSEED='2355142134' 2024-04-17T04:51:15.014 INFO:teuthology.orchestra.run.smithi088.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-04-17T04:51:15.179 INFO:teuthology.orchestra.run.smithi088.stdout:============================= test session starts ============================== 2024-04-17T04:51:15.179 INFO:teuthology.orchestra.run.smithi088.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-04-17T04:51:15.179 INFO:teuthology.orchestra.run.smithi088.stdout:cachedir: .tox/py/.pytest_cache 2024-04-17T04:51:15.179 INFO:teuthology.orchestra.run.smithi088.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2024-04-17T04:51:15.352 INFO:teuthology.orchestra.run.smithi088.stdout:collecting ... collected 9 items 2024-04-17T04:51:15.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:51:20.114 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-04-17T04:51:24.118 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-04-17T04:51:29.037 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-04-17T04:51:34.577 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-04-17T04:51:38.234 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-04-17T04:51:49.948 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-04-17T04:51:58.551 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-04-17T04:52:08.798 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-04-17T04:52:26.436 INFO:teuthology.orchestra.run.smithi088.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout:=============================== warnings summary =============================== 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout: import imp 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-04-17T04:52:26.437 INFO:teuthology.orchestra.run.smithi088.stdout:=================== 9 passed, 1 warning in 71.26s (0:01:11) ==================== 2024-04-17T04:52:26.486 INFO:teuthology.orchestra.run.smithi088.stdout:___________________________________ summary ____________________________________ 2024-04-17T04:52:26.486 INFO:teuthology.orchestra.run.smithi088.stdout: py: commands succeeded 2024-04-17T04:52:26.486 INFO:teuthology.orchestra.run.smithi088.stdout: congratulations :) 2024-04-17T04:52:26.505 INFO:tasks.ragweed:Cleaning up boto... 2024-04-17T04:52:26.506 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-04-17T04:52:26.512 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-04-17T04:52:26.591 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:52:26.591 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:52:26.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 realm 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.636+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.640+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:52:26.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.640+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.640+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:52:26.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.640+0000 7f1f83c68980 20 searching for the correct realm 2024-04-17T04:52:26.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.644+0000 7f1f81f84640 0 -- 172.21.15.88:0/4123473657 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5568be495140 msgr2=0x5568be4975d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:52:26.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.652+0000 7f1f80f82640 0 -- 172.21.15.88:0/4123473657 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f1f7407c640 msgr2=0x7f1f7403b040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:52:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:52:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:52:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:52:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:52:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:52:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:52:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:52:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:52:26.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 zone default found 2024-04-17T04:52:26.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 4 Realm: () 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 zonegroup default 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.660+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.664+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.664+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.664+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.664+0000 7f1f83c68980 20 started sync module instance, tier type = 2024-04-17T04:52:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.664+0000 7f1f83c68980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:52:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=0 2024-04-17T04:52:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=1 2024-04-17T04:52:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=2 2024-04-17T04:52:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=3 2024-04-17T04:52:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=4 2024-04-17T04:52:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=5 2024-04-17T04:52:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=6 2024-04-17T04:52:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 20 add_watcher() i=7 2024-04-17T04:52:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.680+0000 7f1f83c68980 2 all 8 watchers are set, enabling cache 2024-04-17T04:52:26.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.684+0000 7f1e377e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:52:26.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.684+0000 7f1f83c68980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:52:26.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.684+0000 7f1f83c68980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:52:26.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.756+0000 7f1e367e4640 20 reqs_thread_entry: start 2024-04-17T04:52:26.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.756+0000 7f1f83c68980 20 init_complete bucket index max shards: 1 2024-04-17T04:52:26.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.756+0000 7f1f83c68980 20 Filter name: none 2024-04-17T04:52:26.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1da7fff640 20 reqs_thread_entry: start 2024-04-17T04:52:26.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-04-17T04:52:26.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=486 2024-04-17T04:52:26.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-04-17T04:52:26.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-17T04:52:26.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:26.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 : miss 2024-04-17T04:52:26.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=418 2024-04-17T04:52:26.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 info.flags=0x17 2024-04-17T04:52:26.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 to cache LRU end 2024-04-17T04:52:26.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:26.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:26.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.760+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-04-17T04:52:26.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x5568be54f948 s->prefetch_data=0 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 10 manifest: total_size = 7547 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16640278832355630616 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x5568be54f948 s->prefetch_data=0 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be54f948 2024-04-17T04:52:26.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.764+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=7547 stripe_ofs=7547 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16640278832355630616 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.fonW3ZalRYXQO0vg355LPSbZqHRARzi_0 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16640278832355630616 2024-04-17T04:52:26.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-04-17T04:52:26.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.772+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.776+0000 7f1f83c68980 10 manifest: total_size = 5243 2024-04-17T04:52:26.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.776+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18366126528739179819 2024-04-17T04:52:26.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.776+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.776+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.776+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5243 stripe_ofs=5243 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18366126528739179819 2024-04-17T04:52:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.sFNrTzas1xmKINBBE_o66wU_iPkrhDP_0 2024-04-17T04:52:26.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.780+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18366126528739179819 2024-04-17T04:52:26.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-04-17T04:52:26.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1f83c68980 10 manifest: total_size = 5113 2024-04-17T04:52:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16376418569850011000 2024-04-17T04:52:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.784+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5113 stripe_ofs=5113 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16376418569850011000 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.Kaew7lqgXnaN-I0r3V1QDLlnHCluBJk_0 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.788+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16376418569850011000 2024-04-17T04:52:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-04-17T04:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1f83c68980 10 manifest: total_size = 7504 2024-04-17T04:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9064418187252474800 2024-04-17T04:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068df0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.792+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=7504 stripe_ofs=7504 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9064418187252474800 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.Ij9YJGQP06x-jPBzDGxTSpP8sO_NsPW_0 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9064418187252474800 2024-04-17T04:52:26.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be5503a0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.800+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-04-17T04:52:26.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.804+0000 7f1f83c68980 10 manifest: total_size = 13549 2024-04-17T04:52:26.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.804+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17570049470131095556 2024-04-17T04:52:26.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.804+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be5503a0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.804+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.804+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=13549 stripe_ofs=13549 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17570049470131095556 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.O5F4XerjAiJyE3muqnfMylvBEyIAszv_0 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:26.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.808+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17570049470131095556 2024-04-17T04:52:26.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-04-17T04:52:26.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0c2f60 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1f83c68980 10 manifest: total_size = 5272 2024-04-17T04:52:26.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9997103251580419038 2024-04-17T04:52:26.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0c2f60 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.812+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5272 stripe_ofs=5272 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9997103251580419038 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.xqnYdAf-rRPuD35yidAZk_0HzexJCy9_0 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:26.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.816+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9997103251580419038 2024-04-17T04:52:26.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-04-17T04:52:26.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0c2f60 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1f83c68980 10 manifest: total_size = 5407 2024-04-17T04:52:26.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1965224351320529598 2024-04-17T04:52:26.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0c2f60 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.820+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5407 stripe_ofs=5407 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1965224351320529598 2024-04-17T04:52:26.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.d1NUwcs_-cB39LSG7IqJaKYVw0dvaLP_0 2024-04-17T04:52:26.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:26.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1965224351320529598 2024-04-17T04:52:26.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0c2f60 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-04-17T04:52:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 10 manifest: total_size = 7742 2024-04-17T04:52:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12781097983849889692 2024-04-17T04:52:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0c2f60 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.828+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=7742 stripe_ofs=7742 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12781097983849889692 2024-04-17T04:52:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.ErXdAMiNIvrHJNR5vHHBbLCfOj_INP8_0 2024-04-17T04:52:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:26.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12781097983849889692 2024-04-17T04:52:26.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.836+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78087a50 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.840+0000 7f1f83c68980 10 manifest: total_size = 8757 2024-04-17T04:52:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.840+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12016678185059146316 2024-04-17T04:52:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.840+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78087a50 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.840+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.840+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-04-17T04:52:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.840+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5.0 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=8757 stripe_ofs=8757 part_ofs=0 rule->part_size=0 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12016678185059146316 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.q55BlF2Q4ouIRUB_Li3Jpu2XEMrVVPD_0 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.844+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12016678185059146316 2024-04-17T04:52:26.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-04-17T04:52:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:26.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:26.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:26.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.848+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.852+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-04-17T04:52:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.852+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.852+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=204 2024-04-17T04:52:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.852+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-04-17T04:52:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.852+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-04-17T04:52:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.856+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-04-17T04:52:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.856+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-04-17T04:52:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.856+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464288 cookie 93908357443792 notifier 5669 bl.length()=162 2024-04-17T04:52:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.860+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 from cache 2024-04-17T04:52:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.860+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5, ofs0, ns] 2024-04-17T04:52:26.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.860+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464289 cookie 93908357443792 notifier 5669 bl.length()=219 2024-04-17T04:52:26.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.868+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 : miss 2024-04-17T04:52:26.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.868+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.868+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.868+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 info.flags=0x0 2024-04-17T04:52:26.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.868+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5 to cache LRU end 2024-04-17T04:52:26.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 : miss 2024-04-17T04:52:26.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=449 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 info.flags=0x17 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 to cache LRU end 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.872+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo 2024-04-17T04:52:26.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78067740 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 10 manifest: total_size = 26078 2024-04-17T04:52:26.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1645654568369969872 2024-04-17T04:52:26.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78067740 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.876+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12.0 2024-04-17T04:52:26.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=7667 rules.size()=4 2024-04-17T04:52:26.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=7667 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=7667 stripe_ofs=7667 part_ofs=7667 rule->part_size=7674 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=4 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6299123 part_ofs=7667 rule->part_size=7674 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=15341 stripe_ofs=15341 part_ofs=15341 rule->part_size=7670 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=7670 rules.size()=4 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6306797 part_ofs=15341 rule->part_size=7670 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=23011 stripe_ofs=23011 part_ofs=23011 rule->part_size=3067 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6314467 part_ofs=23011 rule->part_size=3067 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=26078 stripe_ofs=26078 part_ofs=26078 rule->part_size=3067 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1645654568369969872 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.1 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.2 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.3 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.4 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.880+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1645654568369969872 2024-04-17T04:52:26.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo 2024-04-17T04:52:26.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:26.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:26.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:26.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:26.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:26.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:26.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.884+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:26.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.888+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-04-17T04:52:26.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.888+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.888+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=235 2024-04-17T04:52:26.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.888+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-04-17T04:52:26.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.888+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-04-17T04:52:26.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.892+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-04-17T04:52:26.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.892+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-04-17T04:52:26.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.892+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464356 cookie 93908357487888 notifier 5669 bl.length()=191 2024-04-17T04:52:26.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.896+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 from cache 2024-04-17T04:52:26.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.896+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12, ofs0, ns] 2024-04-17T04:52:26.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.896+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464280 cookie 93908357441584 notifier 5669 bl.length()=249 2024-04-17T04:52:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.900+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 : miss 2024-04-17T04:52:26.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.900+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.900+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.900+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 info.flags=0x0 2024-04-17T04:52:26.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.900+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12 to cache LRU end 2024-04-17T04:52:26.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.904+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 : miss 2024-04-17T04:52:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.904+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.904+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=456 2024-04-17T04:52:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.904+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 info.flags=0x17 2024-04-17T04:52:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.904+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 to cache LRU end 2024-04-17T04:52:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 2024-04-17T04:52:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo 2024-04-17T04:52:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4da1e0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 10 manifest: total_size = 1667 2024-04-17T04:52:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18049114420975794800 2024-04-17T04:52:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4da1e0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d7e18 2024-04-17T04:52:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.908+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13.0 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-04-17T04:52:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18049114420975794800 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.1 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.2 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.3 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.4 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18049114420975794800 2024-04-17T04:52:26.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:26.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.912+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:26.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo 2024-04-17T04:52:26.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:26.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:26.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:26.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:26.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:26.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-04-17T04:52:26.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.916+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.920+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=242 2024-04-17T04:52:26.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.920+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-04-17T04:52:26.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.920+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-04-17T04:52:26.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.920+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-04-17T04:52:26.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.920+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-04-17T04:52:26.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.924+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464281 cookie 93908357441584 notifier 5669 bl.length()=198 2024-04-17T04:52:26.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.924+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 from cache 2024-04-17T04:52:26.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.924+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13, ofs0, ns] 2024-04-17T04:52:26.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.924+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464262 cookie 93908357505760 notifier 5669 bl.length()=256 2024-04-17T04:52:26.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.928+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 : miss 2024-04-17T04:52:26.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.928+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.928+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.928+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 info.flags=0x0 2024-04-17T04:52:26.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.928+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13 to cache LRU end 2024-04-17T04:52:26.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.932+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 : miss 2024-04-17T04:52:26.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.932+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=451 2024-04-17T04:52:26.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 info.flags=0x17 2024-04-17T04:52:26.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 to cache LRU end 2024-04-17T04:52:26.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:26.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:26.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 2024-04-17T04:52:26.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo[81X7vDW1SOgC2oehdwGvbveTZ5LcfY6] 2024-04-17T04:52:26.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11.0 2024-04-17T04:52:26.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78087a50 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to yx1dq664o55q9sspnb3vi5l3maq7263s 2024-04-17T04:52:26.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 20 get_obj_state_impl: setting s->olh_tag to hs4afdvvjto9c9ntcuez1lq8tf81ypik 2024-04-17T04:52:26.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.936+0000 7f1f83c68980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=hs4afdvvjto9c9ntcuez1lq8tf81ypik 2024-04-17T04:52:26.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.940+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11.0 2024-04-17T04:52:26.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11.0 2024-04-17T04:52:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-04-17T04:52:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 olh_log_entry: epoch=3 op=3 key=foo[81X7vDW1SOgC2oehdwGvbveTZ5LcfY6] 2024-04-17T04:52:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78087a50 obj=test-client.0-r-test-multipart-index-versioning.1:_:81X7vDW1SOgC2oehdwGvbveTZ5LcfY6_foo state=0x5568be4d6f28 s->prefetch_data=0 2024-04-17T04:52:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 10 manifest: total_size = 26082 2024-04-17T04:52:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2996146780926949517 2024-04-17T04:52:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78087a50 obj=test-client.0-r-test-multipart-index-versioning.1:_:81X7vDW1SOgC2oehdwGvbveTZ5LcfY6_foo state=0x5568be4d6f28 s->prefetch_data=0 2024-04-17T04:52:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d6f28 2024-04-17T04:52:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.944+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11.0 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=7671 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7672 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6299127 part_ofs=7671 rule->part_size=7672 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=15343 stripe_ofs=15343 part_ofs=15343 rule->part_size=7672 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6306799 part_ofs=15343 rule->part_size=7672 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=23015 stripe_ofs=23015 part_ofs=23015 rule->part_size=3067 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=3 2024-04-17T04:52:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6314471 part_ofs=23015 rule->part_size=3067 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=26082 stripe_ofs=26082 part_ofs=26082 rule->part_size=3067 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2996146780926949517 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.1 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.2 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.3 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.4 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.952+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2996146780926949517 2024-04-17T04:52:26.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.956+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo[81X7vDW1SOgC2oehdwGvbveTZ5LcfY6] 2024-04-17T04:52:26.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.960+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78087a50 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x5568be4d7e18 s->prefetch_data=0 2024-04-17T04:52:26.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.964+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11.0 2024-04-17T04:52:26.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11.0 2024-04-17T04:52:26.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:26.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:26.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:26.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:26.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:26.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:26.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.968+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-04-17T04:52:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=237 2024-04-17T04:52:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-04-17T04:52:26.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-04-17T04:52:26.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-04-17T04:52:26.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.972+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-04-17T04:52:26.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.976+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464282 cookie 93908357441584 notifier 5669 bl.length()=193 2024-04-17T04:52:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.980+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 from cache 2024-04-17T04:52:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.980+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11, ofs0, ns] 2024-04-17T04:52:26.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.980+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464275 cookie 93908357500000 notifier 5669 bl.length()=251 2024-04-17T04:52:26.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.984+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 : miss 2024-04-17T04:52:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.984+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.984+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:26.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.984+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 info.flags=0x0 2024-04-17T04:52:26.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.984+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11 to cache LRU end 2024-04-17T04:52:26.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.988+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 : miss 2024-04-17T04:52:26.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.988+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=441 2024-04-17T04:52:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 info.flags=0x17 2024-04-17T04:52:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 to cache LRU end 2024-04-17T04:52:26.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 2024-04-17T04:52:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo 2024-04-17T04:52:26.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068a00 obj=test-client.0-r-test-multipart-simple.1:foo state=0x5568be4d6f28 s->prefetch_data=0 2024-04-17T04:52:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 10 manifest: total_size = 886297 2024-04-17T04:52:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16703826128154381260 2024-04-17T04:52:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068a00 obj=test-client.0-r-test-multipart-simple.1:foo state=0x5568be4d6f28 s->prefetch_data=0 2024-04-17T04:52:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d6f28 2024-04-17T04:52:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:26.992+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10.0 2024-04-17T04:52:26.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=246247 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6537703 part_ofs=246247 rule->part_size=246247 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6783950 part_ofs=492494 rule->part_size=246247 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=7030197 part_ofs=738741 rule->part_size=147556 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16703826128154381260 2024-04-17T04:52:26.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.1 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.2 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.3 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.4 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:26.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16703826128154381260 2024-04-17T04:52:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.000+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo 2024-04-17T04:52:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:26.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:26.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-04-17T04:52:27.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.004+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.008+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=227 2024-04-17T04:52:27.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.008+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-04-17T04:52:27.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.008+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-04-17T04:52:27.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.008+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-04-17T04:52:27.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.008+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-04-17T04:52:27.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.012+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464290 cookie 93908357497696 notifier 5669 bl.length()=183 2024-04-17T04:52:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.016+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 from cache 2024-04-17T04:52:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.016+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10, ofs0, ns] 2024-04-17T04:52:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.016+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464274 cookie 93908357494928 notifier 5669 bl.length()=241 2024-04-17T04:52:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.024+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 : miss 2024-04-17T04:52:27.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.024+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.024+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.024+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 info.flags=0x0 2024-04-17T04:52:27.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.024+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10 to cache LRU end 2024-04-17T04:52:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.028+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 : miss 2024-04-17T04:52:27.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.028+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=447 2024-04-17T04:52:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 info.flags=0x17 2024-04-17T04:52:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 to cache LRU end 2024-04-17T04:52:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-04-17T04:52:27.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4b31f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x5568be4de828 s->prefetch_data=0 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2580317577252748330 2024-04-17T04:52:27.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4b31f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x5568be4de828 s->prefetch_data=0 2024-04-17T04:52:27.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4de828 2024-04-17T04:52:27.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.032+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2580317577252748330 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_0 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 337 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.040+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2580317577252748330 2024-04-17T04:52:27.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-17T04:52:27.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4b31f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x5568be4de828 s->prefetch_data=0 2024-04-17T04:52:27.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 10 generate_fake_tag new tag=52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_1_a1899668ab9f8d7c2428d9bc89c6752c 2024-04-17T04:52:27.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_1_a1899668ab9f8d7c2428d9bc89c6752c 2024-04-17T04:52:27.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4b31f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x5568be4de828 s->prefetch_data=0 2024-04-17T04:52:27.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4de828 2024-04-17T04:52:27.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.044+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_1_a1899668ab9f8d7c2428d9bc89c6752c 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_0 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.048+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_1_a1899668ab9f8d7c2428d9bc89c6752c 2024-04-17T04:52:27.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.052+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-04-17T04:52:27.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.052+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4b31f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x5568be4de828 s->prefetch_data=0 2024-04-17T04:52:27.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.056+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.056+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _EhqgM5HZk1dyxdxKctGAafe5epntHXX 2024-04-17T04:52:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.056+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4b31f0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x5568be4de828 s->prefetch_data=0 2024-04-17T04:52:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.056+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4de828 2024-04-17T04:52:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.056+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _EhqgM5HZk1dyxdxKctGAafe5epntHXX 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.E7v-s0wKdtBdaDKpo5XtMgQZXMFflo8_0 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.060+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _EhqgM5HZk1dyxdxKctGAafe5epntHXX 2024-04-17T04:52:27.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-04-17T04:52:27.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4e1fe0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _8bp469jdtlGeYoQgAE9gI_xK-ZXFqdO 2024-04-17T04:52:27.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4e1fe0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.064+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _8bp469jdtlGeYoQgAE9gI_xK-ZXFqdO 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.PpI8eG1cAFn1bjAqzxlH6ZM5pkSj5up_0 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _8bp469jdtlGeYoQgAE9gI_xK-ZXFqdO 2024-04-17T04:52:27.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.072+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-04-17T04:52:27.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.076+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.076+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.076+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17561353937998813697 2024-04-17T04:52:27.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.076+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.076+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.076+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17561353937998813697 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_0 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 340 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17561353937998813697 2024-04-17T04:52:27.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dacb0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-17T04:52:27.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _-V1FlikXDUi-9chE4YMhYLnDOiBb7nJ 2024-04-17T04:52:27.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dacb0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.084+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _-V1FlikXDUi-9chE4YMhYLnDOiBb7nJ 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.7fSDuzhJXGxYdqchNRXF_xUX_WxKU6s_0 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.092+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _-V1FlikXDUi-9chE4YMhYLnDOiBb7nJ 2024-04-17T04:52:27.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-04-17T04:52:27.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 10 generate_fake_tag new tag=52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_1_4ebae0e1fc765aed257bffed04a65a77 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_1_4ebae0e1fc765aed257bffed04a65a77 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.096+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_1_4ebae0e1fc765aed257bffed04a65a77 2024-04-17T04:52:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_0 2024-04-17T04:52:27.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-04-17T04:52:27.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.100+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_1_4ebae0e1fc765aed257bffed04a65a77 2024-04-17T04:52:27.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-04-17T04:52:27.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1f83c68980 10 manifest: total_size = 5242880 2024-04-17T04:52:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _QUGYfh2ApQ5zk3wJTrbBNhTiUhihPSQ 2024-04-17T04:52:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.104+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _QUGYfh2ApQ5zk3wJTrbBNhTiUhihPSQ 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.E86CI1pIPArFwa_rZEjoG9_6gs-dvAo_0 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _QUGYfh2ApQ5zk3wJTrbBNhTiUhihPSQ 2024-04-17T04:52:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-04-17T04:52:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.112+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.116+0000 7f1f83c68980 10 manifest: total_size = 27199 2024-04-17T04:52:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.116+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6707506847384385726 2024-04-17T04:52:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.116+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7805fe00 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.116+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.116+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=27199 stripe_ofs=27199 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6707506847384385726 2024-04-17T04:52:27.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_0 2024-04-17T04:52:27.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 330 2024-04-17T04:52:27.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.120+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6707506847384385726 2024-04-17T04:52:27.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-17T04:52:27.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78067740 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1f83c68980 10 manifest: total_size = 27199 2024-04-17T04:52:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _LY6HrTs7mM1Y4yqUFvFmvHeIMQ6IUpM 2024-04-17T04:52:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78067740 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.124+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=27199 stripe_ofs=27199 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _LY6HrTs7mM1Y4yqUFvFmvHeIMQ6IUpM 2024-04-17T04:52:27.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.boCaB13pvgUrsPl2KGBx6fLli2TTrBb_0 2024-04-17T04:52:27.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-17T04:52:27.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: _LY6HrTs7mM1Y4yqUFvFmvHeIMQ6IUpM 2024-04-17T04:52:27.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.132+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-04-17T04:52:27.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.136+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78067740 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.136+0000 7f1f83c68980 10 manifest: total_size = 27199 2024-04-17T04:52:27.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.136+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _U4T2PiAsjz-TATo8NaDsNGAVNbFsbvi 2024-04-17T04:52:27.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.136+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78067740 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.136+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.136+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=27199 stripe_ofs=27199 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _U4T2PiAsjz-TATo8NaDsNGAVNbFsbvi 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.ph0evQ7r_AHKUcWwqxxcq-5wqY3EydL_0 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.140+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: _U4T2PiAsjz-TATo8NaDsNGAVNbFsbvi 2024-04-17T04:52:27.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 10 manifest: total_size = 27199 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=27199 stripe_ofs=27199 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 10 generate_fake_tag new tag=52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_1_e7c424f2e243a1fae9dde9d9a5985710 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_1_e7c424f2e243a1fae9dde9d9a5985710 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x5568be4d5e68 s->prefetch_data=0 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e68 2024-04-17T04:52:27.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.144+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16.0 2024-04-17T04:52:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=27199 stripe_ofs=27199 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_1_e7c424f2e243a1fae9dde9d9a5985710 2024-04-17T04:52:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_0 2024-04-17T04:52:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 389 2024-04-17T04:52:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_1_e7c424f2e243a1fae9dde9d9a5985710 2024-04-17T04:52:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.152+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-04-17T04:52:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-04-17T04:52:27.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.156+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.160+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=233 2024-04-17T04:52:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.160+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-04-17T04:52:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.160+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-04-17T04:52:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.160+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-04-17T04:52:27.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.160+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-04-17T04:52:27.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.160+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464283 cookie 93908357441584 notifier 5669 bl.length()=189 2024-04-17T04:52:27.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.168+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 from cache 2024-04-17T04:52:27.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.168+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16, ofs0, ns] 2024-04-17T04:52:27.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.168+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464276 cookie 93908357500000 notifier 5669 bl.length()=247 2024-04-17T04:52:27.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.172+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 : miss 2024-04-17T04:52:27.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.172+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.172+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.172+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 info.flags=0x0 2024-04-17T04:52:27.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.172+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16 to cache LRU end 2024-04-17T04:52:27.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 : miss 2024-04-17T04:52:27.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=457 2024-04-17T04:52:27.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 info.flags=0x17 2024-04-17T04:52:27.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 to cache LRU end 2024-04-17T04:52:27.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 2024-04-17T04:52:27.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.176+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-04-17T04:52:27.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.180+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068a00 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x5568be4d5e08 s->prefetch_data=0 2024-04-17T04:52:27.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.180+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.180+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18405166338911737529 2024-04-17T04:52:27.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.180+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068a00 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x5568be4d5e08 s->prefetch_data=0 2024-04-17T04:52:27.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.180+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e08 2024-04-17T04:52:27.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.180+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18405166338911737529 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.1 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 635 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.3 2024-04-17T04:52:27.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 902 2024-04-17T04:52:27.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.184+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18405166338911737529 2024-04-17T04:52:27.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-17T04:52:27.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068a00 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x5568be4d5e08 s->prefetch_data=0 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 10 generate_fake_tag new tag=52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2_26279c4c239ea20dd1a5af4444a52a63 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2_26279c4c239ea20dd1a5af4444a52a63 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f78068a00 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x5568be4d5e08 s->prefetch_data=0 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5e08 2024-04-17T04:52:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.188+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2_26279c4c239ea20dd1a5af4444a52a63 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.1 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 441 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 708 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.3 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 975 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.192+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2_26279c4c239ea20dd1a5af4444a52a63 2024-04-17T04:52:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-04-17T04:52:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4e8c80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x5568be355768 s->prefetch_data=0 2024-04-17T04:52:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _z0ZGbg98k2337uxANkID0FhY7DB1Hy5 2024-04-17T04:52:27.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4e8c80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x5568be355768 s->prefetch_data=0 2024-04-17T04:52:27.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be355768 2024-04-17T04:52:27.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.196+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.200+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.200+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.200+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.200+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _z0ZGbg98k2337uxANkID0FhY7DB1Hy5 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.ETw-d3CGi03o-JtEpg0TF4nTXh2PUX2_0 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.ETw-d3CGi03o-JtEpg0TF4nTXh2PUX2_1 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: _z0ZGbg98k2337uxANkID0FhY7DB1Hy5 2024-04-17T04:52:27.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-04-17T04:52:27.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.204+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x5568be3556c8 s->prefetch_data=0 2024-04-17T04:52:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.208+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.208+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _HgWEf97JlN_kIuCSFqQENwZetHNCxuF 2024-04-17T04:52:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.208+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x5568be3556c8 s->prefetch_data=0 2024-04-17T04:52:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.208+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be3556c8 2024-04-17T04:52:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.208+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _HgWEf97JlN_kIuCSFqQENwZetHNCxuF 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.qN0M0IdOsOvNS9YhDggjYSVl3txI6rV_0 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.qN0M0IdOsOvNS9YhDggjYSVl3txI6rV_1 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: _HgWEf97JlN_kIuCSFqQENwZetHNCxuF 2024-04-17T04:52:27.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.212+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-04-17T04:52:27.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.216+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4e8a20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x5568be3556c8 s->prefetch_data=0 2024-04-17T04:52:27.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.216+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.216+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4481332236259749016 2024-04-17T04:52:27.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.216+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4e8a20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x5568be3556c8 s->prefetch_data=0 2024-04-17T04:52:27.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.216+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be3556c8 2024-04-17T04:52:27.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.216+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4481332236259749016 2024-04-17T04:52:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.1 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 646 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.3 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 919 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.220+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4481332236259749016 2024-04-17T04:52:27.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x5568be4e0078 s->prefetch_data=0 2024-04-17T04:52:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-17T04:52:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _18HCggrqkEE58hUl9SbVBTApWCkp0U9 2024-04-17T04:52:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x5568be4e0078 s->prefetch_data=0 2024-04-17T04:52:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e0078 2024-04-17T04:52:27.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.224+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _18HCggrqkEE58hUl9SbVBTApWCkp0U9 2024-04-17T04:52:27.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.TufLWn4UXpTVhO8LntmtEB7mEB-DAWS_0 2024-04-17T04:52:27.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-17T04:52:27.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.TufLWn4UXpTVhO8LntmtEB7mEB-DAWS_1 2024-04-17T04:52:27.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-04-17T04:52:27.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.228+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _18HCggrqkEE58hUl9SbVBTApWCkp0U9 2024-04-17T04:52:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-04-17T04:52:27.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 10 generate_fake_tag new tag=52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2_d6820f5c88590181fad3abbf067c4e41 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2_d6820f5c88590181fad3abbf067c4e41 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5cf8 2024-04-17T04:52:27.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.232+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2_d6820f5c88590181fad3abbf067c4e41 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.1 2024-04-17T04:52:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 449 2024-04-17T04:52:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2 2024-04-17T04:52:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 722 2024-04-17T04:52:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.3 2024-04-17T04:52:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 995 2024-04-17T04:52:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2_d6820f5c88590181fad3abbf067c4e41 2024-04-17T04:52:27.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-04-17T04:52:27.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.240+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0ab4e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.244+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.244+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _EKeb5Or2IZwNkLinXXJwswxcvgMzMZ4 2024-04-17T04:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.244+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0ab4e0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.244+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5cf8 2024-04-17T04:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.244+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _EKeb5Or2IZwNkLinXXJwswxcvgMzMZ4 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gnlngG6-OauNzyfJdw9-pffaEVCjjf9_0 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gnlngG6-OauNzyfJdw9-pffaEVCjjf9_1 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.248+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _EKeb5Or2IZwNkLinXXJwswxcvgMzMZ4 2024-04-17T04:52:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-04-17T04:52:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17813352488194049708 2024-04-17T04:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5cf8 2024-04-17T04:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.252+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17813352488194049708 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.1 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.3 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.256+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17813352488194049708 2024-04-17T04:52:27.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4da6c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-17T04:52:27.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _mgfb8vGBLtmT82VV4IL41m0wVaDmY4n 2024-04-17T04:52:27.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4da6c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5cf8 2024-04-17T04:52:27.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.260+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _mgfb8vGBLtmT82VV4IL41m0wVaDmY4n 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gXHePTiGIPfQEeXmywLmTgP3NtCzkPe_0 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gXHePTiGIPfQEeXmywLmTgP3NtCzkPe_1 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.264+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _mgfb8vGBLtmT82VV4IL41m0wVaDmY4n 2024-04-17T04:52:27.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-04-17T04:52:27.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to _0aVcWC8hwpu6Ufq_KhFiM42sNg3pwhW 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5cf8 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.268+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: _0aVcWC8hwpu6Ufq_KhFiM42sNg3pwhW 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.f1afeEr3VOBzl8RIIJdkaFb_dOO3zj2_0 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.f1afeEr3VOBzl8RIIJdkaFb_dOO3zj2_1 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: _0aVcWC8hwpu6Ufq_KhFiM42sNg3pwhW 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.276+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-04-17T04:52:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 10 generate_fake_tag new tag=52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2_cda98037d30c48c027fd08c4e7844713 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2_cda98037d30c48c027fd08c4e7844713 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x5568be4d5cf8 s->prefetch_data=0 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4d5cf8 2024-04-17T04:52:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.280+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17.0 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2_cda98037d30c48c027fd08c4e7844713 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.1 2024-04-17T04:52:27.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-04-17T04:52:27.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2 2024-04-17T04:52:27.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-04-17T04:52:27.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.3 2024-04-17T04:52:27.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-04-17T04:52:27.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2_cda98037d30c48c027fd08c4e7844713 2024-04-17T04:52:27.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-04-17T04:52:27.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.288+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-04-17T04:52:27.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.292+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.296+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=243 2024-04-17T04:52:27.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.296+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-04-17T04:52:27.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.296+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-04-17T04:52:27.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.296+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-04-17T04:52:27.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.296+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-04-17T04:52:27.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.296+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464275 cookie 93908357494928 notifier 5669 bl.length()=199 2024-04-17T04:52:27.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.304+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 from cache 2024-04-17T04:52:27.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.304+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17, ofs0, ns] 2024-04-17T04:52:27.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.304+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464291 cookie 93908357443792 notifier 5669 bl.length()=257 2024-04-17T04:52:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.308+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 : miss 2024-04-17T04:52:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.308+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.308+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.308+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 info.flags=0x0 2024-04-17T04:52:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.308+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17 to cache LRU end 2024-04-17T04:52:27.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.312+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 : miss 2024-04-17T04:52:27.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.312+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=452 2024-04-17T04:52:27.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 info.flags=0x17 2024-04-17T04:52:27.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 to cache LRU end 2024-04-17T04:52:27.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 2024-04-17T04:52:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-17T04:52:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-17T04:52:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-17T04:52:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dacb0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7625213628631826171 2024-04-17T04:52:27.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dacb0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb728 2024-04-17T04:52:27.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.316+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15.0 2024-04-17T04:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7625213628631826171 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-FROZEN.2~SFqZrD-YUf5hmB8Re8Z6_qCwcQgYVTD.1 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 367 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-FROZEN.2~SFqZrD-YUf5hmB8Re8Z6_qCwcQgYVTD.2 2024-04-17T04:52:27.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 634 2024-04-17T04:52:27.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-FROZEN.2~SFqZrD-YUf5hmB8Re8Z6_qCwcQgYVTD.3 2024-04-17T04:52:27.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 901 2024-04-17T04:52:27.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7625213628631826171 2024-04-17T04:52:27.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-17T04:52:27.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.324+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7407a6e0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.328+0000 7f1f83c68980 10 manifest: total_size = 11534336 2024-04-17T04:52:27.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.328+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13842275789272848975 2024-04-17T04:52:27.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.328+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7407a6e0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.328+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb728 2024-04-17T04:52:27.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.328+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15.0 2024-04-17T04:52:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-04-17T04:52:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-17T04:52:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13842275789272848975 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-LUKEWARM.2~gCQP2AqwKNkXq_36mLsiUUTd8RS1JYT.1 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 374 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-LUKEWARM.2~gCQP2AqwKNkXq_36mLsiUUTd8RS1JYT.2 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 647 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-LUKEWARM.2~gCQP2AqwKNkXq_36mLsiUUTd8RS1JYT.3 2024-04-17T04:52:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 920 2024-04-17T04:52:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.332+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13842275789272848975 2024-04-17T04:52:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-17T04:52:27.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be356040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1f83c68980 10 manifest: total_size = 45360 2024-04-17T04:52:27.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15258549198467422455 2024-04-17T04:52:27.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be356040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb728 2024-04-17T04:52:27.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.336+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15.0 2024-04-17T04:52:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-04-17T04:52:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2024-04-17T04:52:27.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=4134 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15258549198467422455 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-STANDARD.2~EzlEk1NaqjvssmU90zWgfR83tFInWvt.1 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-STANDARD.2~EzlEk1NaqjvssmU90zWgfR83tFInWvt.2 2024-04-17T04:52:27.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-04-17T04:52:27.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-STANDARD.2~EzlEk1NaqjvssmU90zWgfR83tFInWvt.3 2024-04-17T04:52:27.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-04-17T04:52:27.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.344+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15258549198467422455 2024-04-17T04:52:27.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.348+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-17T04:52:27.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.348+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.348+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.352+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.352+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.352+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.352+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.352+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.356+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-04-17T04:52:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.356+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.356+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=238 2024-04-17T04:52:27.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.356+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-04-17T04:52:27.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.356+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-04-17T04:52:27.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.360+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-04-17T04:52:27.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.360+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-04-17T04:52:27.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.360+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464277 cookie 93908357500000 notifier 5669 bl.length()=194 2024-04-17T04:52:27.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.364+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 from cache 2024-04-17T04:52:27.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.364+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15, ofs0, ns] 2024-04-17T04:52:27.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.364+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464292 cookie 93908357443792 notifier 5669 bl.length()=252 2024-04-17T04:52:27.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.372+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 : miss 2024-04-17T04:52:27.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.372+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.372+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.372+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 info.flags=0x0 2024-04-17T04:52:27.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.372+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15 to cache LRU end 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 : miss 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=442 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 info.flags=0x17 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 to cache LRU end 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-17T04:52:27.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.376+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be356190 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.380+0000 7f1f83c68980 10 manifest: total_size = 10485760 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.380+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7354409550113537632 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.380+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be356190 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.380+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb728 2024-04-17T04:52:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.380+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14.0 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7354409550113537632 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.dTo_b8Djw-TIWzoOQQniTyBQuNYDB6w_0 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 337 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.dTo_b8Djw-TIWzoOQQniTyBQuNYDB6w_1 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 574 2024-04-17T04:52:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.384+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7354409550113537632 2024-04-17T04:52:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0ab4e0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-17T04:52:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1f83c68980 10 manifest: total_size = 10485760 2024-04-17T04:52:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16636855843335856865 2024-04-17T04:52:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c0ab4e0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb728 2024-04-17T04:52:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.388+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14.0 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16636855843335856865 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.t0IlW5XZgDMzeIbSRRyicNcJMVLi_nV_0 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 340 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.t0IlW5XZgDMzeIbSRRyicNcJMVLi_nV_1 2024-04-17T04:52:27.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 579 2024-04-17T04:52:27.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.396+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16636855843335856865 2024-04-17T04:52:27.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-17T04:52:27.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1f83c68980 10 manifest: total_size = 53499 2024-04-17T04:52:27.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14993780938410216773 2024-04-17T04:52:27.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1f83c68980 20 get_obj_state: octx=0x7f1f7c10e4b0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x5568be4eb728 s->prefetch_data=0 2024-04-17T04:52:27.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb728 2024-04-17T04:52:27.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.400+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14.0 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=53499 stripe_ofs=53499 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14993780938410216773 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.AjiiJM_XqKo0_HZquO4_vXWE8NXCeKV_0 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 331 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14993780938410216773 2024-04-17T04:52:27.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-17T04:52:27.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.408+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.412+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.412+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.412+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.412+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.412+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-04-17T04:52:27.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=228 2024-04-17T04:52:27.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-04-17T04:52:27.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-04-17T04:52:27.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-04-17T04:52:27.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-04-17T04:52:27.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.416+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464284 cookie 93908357441584 notifier 5669 bl.length()=184 2024-04-17T04:52:27.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.424+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 from cache 2024-04-17T04:52:27.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.424+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14, ofs0, ns] 2024-04-17T04:52:27.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.424+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464263 cookie 93908357505760 notifier 5669 bl.length()=242 2024-04-17T04:52:27.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.428+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 : miss 2024-04-17T04:52:27.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.428+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.428+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.428+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 info.flags=0x0 2024-04-17T04:52:27.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.428+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14 to cache LRU end 2024-04-17T04:52:27.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.432+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 : miss 2024-04-17T04:52:27.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.432+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=437 2024-04-17T04:52:27.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 info.flags=0x17 2024-04-17T04:52:27.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 to cache LRU end 2024-04-17T04:52:27.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 2024-04-17T04:52:27.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 cls_bucket_list_unordered: got __ 2024-04-17T04:52:27.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 cls_bucket_list_unordered: got __ _ 2024-04-17T04:52:27.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 cls_bucket_list_unordered: got ___ 2024-04-17T04:52:27.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 cls_bucket_list_unordered: got obj 2024-04-17T04:52:27.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x5568be4eb6d8 s->prefetch_data=0 2024-04-17T04:52:27.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 10 manifest: total_size = 0 2024-04-17T04:52:27.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15396973207880473179 2024-04-17T04:52:27.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x5568be4eb6d8 s->prefetch_data=0 2024-04-17T04:52:27.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb6d8 2024-04-17T04:52:27.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.436+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7.0 2024-04-17T04:52:27.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.444+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x5568be4eb668 s->prefetch_data=0 2024-04-17T04:52:27.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.444+0000 7f1f83c68980 10 manifest: total_size = 0 2024-04-17T04:52:27.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.444+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9958003785992777441 2024-04-17T04:52:27.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.444+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x5568be4eb668 s->prefetch_data=0 2024-04-17T04:52:27.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.444+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb668 2024-04-17T04:52:27.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.444+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7.0 2024-04-17T04:52:27.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.448+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=__ 2024-04-17T04:52:27.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.452+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x5568be4eb668 s->prefetch_data=0 2024-04-17T04:52:27.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.452+0000 7f1f83c68980 10 manifest: total_size = 0 2024-04-17T04:52:27.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.452+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1360160991894645134 2024-04-17T04:52:27.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.452+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x5568be4eb668 s->prefetch_data=0 2024-04-17T04:52:27.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.452+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb668 2024-04-17T04:52:27.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.452+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7.0 2024-04-17T04:52:27.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.456+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=__ _ 2024-04-17T04:52:27.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x5568be4eb668 s->prefetch_data=0 2024-04-17T04:52:27.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1f83c68980 10 manifest: total_size = 0 2024-04-17T04:52:27.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11528006034452288164 2024-04-17T04:52:27.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ecad0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x5568be4eb668 s->prefetch_data=0 2024-04-17T04:52:27.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4eb668 2024-04-17T04:52:27.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7.0 2024-04-17T04:52:27.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.460+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=___ 2024-04-17T04:52:27.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.464+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.464+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.464+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=obj 2024-04-17T04:52:27.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-04-17T04:52:27.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=223 2024-04-17T04:52:27.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-04-17T04:52:27.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.468+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-04-17T04:52:27.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.472+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-04-17T04:52:27.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.472+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-04-17T04:52:27.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.472+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464357 cookie 93908357487888 notifier 5669 bl.length()=181 2024-04-17T04:52:27.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.476+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 from cache 2024-04-17T04:52:27.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.476+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7, ofs0, ns] 2024-04-17T04:52:27.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.480+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464276 cookie 93908357494928 notifier 5669 bl.length()=238 2024-04-17T04:52:27.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.484+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 : miss 2024-04-17T04:52:27.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.484+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.484+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.484+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 info.flags=0x0 2024-04-17T04:52:27.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.484+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7 to cache LRU end 2024-04-17T04:52:27.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 : miss 2024-04-17T04:52:27.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=437 2024-04-17T04:52:27.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 info.flags=0x17 2024-04-17T04:52:27.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 to cache LRU end 2024-04-17T04:52:27.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 2024-04-17T04:52:27.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 cls_bucket_list_unordered: got __ 2024-04-17T04:52:27.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 cls_bucket_list_unordered: got __ _ 2024-04-17T04:52:27.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 cls_bucket_list_unordered: got ___ 2024-04-17T04:52:27.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 cls_bucket_list_unordered: got obj 2024-04-17T04:52:27.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 10 manifest: total_size = 24595 2024-04-17T04:52:27.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6068203785831278025 2024-04-17T04:52:27.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.492+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8.0 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=24595 stripe_ofs=24595 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6068203785831278025 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.Kdx4g7r8XT7qpyUmrB-7sLdwojoMYHs_0 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.500+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6068203785831278025 2024-04-17T04:52:27.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=__ 2024-04-17T04:52:27.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1f83c68980 10 manifest: total_size = 524288 2024-04-17T04:52:27.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10442656189416446325 2024-04-17T04:52:27.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.504+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8.0 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=524288 stripe_ofs=524288 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10442656189416446325 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.7q-iMuNHOm-srE_a-99unVQKo-A2dHR_0 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10442656189416446325 2024-04-17T04:52:27.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=__ _ 2024-04-17T04:52:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.512+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.516+0000 7f1f83c68980 10 manifest: total_size = 38 2024-04-17T04:52:27.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.516+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13978424803278354288 2024-04-17T04:52:27.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.516+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.516+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.516+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8.0 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13978424803278354288 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.VmegyrY_OgQNvKX1V73f6aylQoUaPC9_0 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.520+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13978424803278354288 2024-04-17T04:52:27.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=___ 2024-04-17T04:52:27.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1f83c68980 10 manifest: total_size = 524288 2024-04-17T04:52:27.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5720787998017603528 2024-04-17T04:52:27.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4ef1c0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.524+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8.0 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=524288 stripe_ofs=524288 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5720787998017603528 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.V5LEjPRr1jIETBN1wSti84YEn40R-bt_0 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5720787998017603528 2024-04-17T04:52:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.532+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=obj 2024-04-17T04:52:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.536+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-04-17T04:52:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=223 2024-04-17T04:52:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-04-17T04:52:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-04-17T04:52:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-04-17T04:52:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.540+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-04-17T04:52:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.544+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464358 cookie 93908357487888 notifier 5669 bl.length()=181 2024-04-17T04:52:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.548+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 from cache 2024-04-17T04:52:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.548+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8, ofs0, ns] 2024-04-17T04:52:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.548+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464277 cookie 93908357494928 notifier 5669 bl.length()=238 2024-04-17T04:52:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.552+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 : miss 2024-04-17T04:52:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.552+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.552+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.552+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 info.flags=0x0 2024-04-17T04:52:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.552+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8 to cache LRU end 2024-04-17T04:52:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 : miss 2024-04-17T04:52:27.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=437 2024-04-17T04:52:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 info.flags=0x17 2024-04-17T04:52:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 to cache LRU end 2024-04-17T04:52:27.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 cls_bucket_list_unordered: got __ 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 cls_bucket_list_unordered: got __ _ 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 cls_bucket_list_unordered: got ___ 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 cls_bucket_list_unordered: got obj 2024-04-17T04:52:27.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4d9480 obj=test-client.0-r-test-small-obj-data.3:__ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 10 manifest: total_size = 49187 2024-04-17T04:52:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8378322104445853895 2024-04-17T04:52:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4d9480 obj=test-client.0-r-test-small-obj-data.3:__ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.560+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9.0 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8378322104445853895 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.8kBECyHRDV9-IwGBGjaZI4KOS-YrcSk_0 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.568+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8378322104445853895 2024-04-17T04:52:27.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=__ 2024-04-17T04:52:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dd050 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1f83c68980 10 manifest: total_size = 49187 2024-04-17T04:52:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4630922249441394948 2024-04-17T04:52:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dd050 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.572+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9.0 2024-04-17T04:52:27.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4630922249441394948 2024-04-17T04:52:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.-XRt1dI4-DJOuR8rp5euc68yHKT0a6l_0 2024-04-17T04:52:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-17T04:52:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.580+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4630922249441394948 2024-04-17T04:52:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=__ _ 2024-04-17T04:52:27.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dd050 obj=test-client.0-r-test-small-obj-data.3:___ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1f83c68980 10 manifest: total_size = 55 2024-04-17T04:52:27.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13369702719412952550 2024-04-17T04:52:27.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dd050 obj=test-client.0-r-test-small-obj-data.3:___ state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.584+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9.0 2024-04-17T04:52:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=55 stripe_ofs=55 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13369702719412952550 2024-04-17T04:52:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.8vxzc6SUlNNaYlqsp5dr5F80PjOVyo1_0 2024-04-17T04:52:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.592+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13369702719412952550 2024-04-17T04:52:27.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1e53fff640 20 handle_completion(): completion ok for obj=___ 2024-04-17T04:52:27.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dd050 obj=test-client.0-r-test-small-obj-data.3:obj state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1f83c68980 10 manifest: total_size = 49187 2024-04-17T04:52:27.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1f83c68980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10624819203853602309 2024-04-17T04:52:27.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1f83c68980 20 get_obj_state: octx=0x5568be4dd050 obj=test-client.0-r-test-small-obj-data.3:obj state=0x5568be4e7528 s->prefetch_data=0 2024-04-17T04:52:27.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1f83c68980 20 prepare_atomic_modification: state is not atomic. state=0x5568be4e7528 2024-04-17T04:52:27.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.596+0000 7f1f83c68980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9.0 2024-04-17T04:52:27.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10624819203853602309 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.wFpa93mvpTcqjhUlrQCGUUefTOD6NCX_0 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.600+0000 7f1f83c68980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-17T04:52:27.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.604+0000 7f1f83c68980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10624819203853602309 2024-04-17T04:52:27.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.604+0000 7f1e64ff9640 20 handle_completion(): completion ok for obj=obj 2024-04-17T04:52:27.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.604+0000 7f1f83c68980 20 list_objects_ordered: starting attempt 1 2024-04-17T04:52:27.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.604+0000 7f1f83c68980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-17T04:52:27.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-17T04:52:27.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-17T04:52:27.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-17T04:52:27.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-17T04:52:27.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9])"; uploads.size()=0; is_truncated=0 2024-04-17T04:52:27.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-04-17T04:52:27.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.608+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.612+0000 7f1f83c68980 20 rados_obj.operate() r=0 bl.length=223 2024-04-17T04:52:27.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.612+0000 7f1f83c68980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-04-17T04:52:27.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.612+0000 7f1f83c68980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-04-17T04:52:27.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.612+0000 7f1f83c68980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-04-17T04:52:27.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.612+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-04-17T04:52:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.612+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464359 cookie 93908357487888 notifier 5669 bl.length()=181 2024-04-17T04:52:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.620+0000 7f1f83c68980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 from cache 2024-04-17T04:52:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.620+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9, ofs0, ns] 2024-04-17T04:52:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.620+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464278 cookie 93908357494928 notifier 5669 bl.length()=238 2024-04-17T04:52:27.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.624+0000 7f1f83c68980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 : miss 2024-04-17T04:52:27.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.624+0000 7f1f83c68980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.628+0000 7f1f83c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.628+0000 7f1f83c68980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 info.flags=0x0 2024-04-17T04:52:27.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.628+0000 7f1f83c68980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9 to cache LRU end 2024-04-17T04:52:27.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.632+0000 7f1f83c68980 10 removing key index: CCCMNWTHRRWHWGQWBKOS 2024-04-17T04:52:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.632+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:CCCMNWTHRRWHWGQWBKOS, ofs0, ns] 2024-04-17T04:52:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.632+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464278 cookie 93908357503552 notifier 5669 bl.length()=170 2024-04-17T04:52:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.636+0000 7f1f83c68980 10 removing email index: ragweed.client.0+test@test.test 2024-04-17T04:52:27.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.640+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-04-17T04:52:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.640+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464285 cookie 93908357441584 notifier 5669 bl.length()=182 2024-04-17T04:52:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.644+0000 7f1f83c68980 10 removing user buckets index 2024-04-17T04:52:27.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.648+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-04-17T04:52:27.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.648+0000 7f1e64ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464279 cookie 93908357500000 notifier 5669 bl.length()=173 2024-04-17T04:52:27.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.648+0000 7f1f83c68980 10 removing user index: ragweed.client.0 2024-04-17T04:52:27.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.652+0000 7f1f83c68980 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-04-17T04:52:27.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.652+0000 7f1f83c68980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-04-17T04:52:27.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.652+0000 7f1e53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464293 cookie 93908357497696 notifier 5669 bl.length()=165 2024-04-17T04:52:27.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.664+0000 7f1e377e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:52:27.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.668+0000 7f1f83c68980 20 remove_watcher() i=0 2024-04-17T04:52:27.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.668+0000 7f1f83c68980 2 removed watcher, disabling cache 2024-04-17T04:52:27.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.672+0000 7f1f83c68980 20 remove_watcher() i=1 2024-04-17T04:52:27.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.672+0000 7f1f83c68980 20 remove_watcher() i=2 2024-04-17T04:52:27.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.676+0000 7f1f83c68980 20 remove_watcher() i=3 2024-04-17T04:52:27.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.680+0000 7f1f83c68980 20 remove_watcher() i=4 2024-04-17T04:52:27.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.680+0000 7f1f83c68980 20 remove_watcher() i=5 2024-04-17T04:52:27.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.684+0000 7f1f83c68980 20 remove_watcher() i=6 2024-04-17T04:52:27.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.684+0000 7f1f83c68980 20 remove_watcher() i=7 2024-04-17T04:52:27.700 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-04-17T04:52:27.781 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:52:27.781 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:52:27.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.828+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.828+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.828+0000 7f55ef01f980 20 realm 2024-04-17T04:52:27.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.828+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.828+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.828+0000 7f55ef01f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:52:27.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:52:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:52:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.832+0000 7f55ef01f980 20 searching for the correct realm 2024-04-17T04:52:27.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.840+0000 7f55ec339640 0 -- 172.21.15.88:0/3615757738 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f55e4120c20 msgr2=0x7f55e40cd440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:52:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.848+0000 7f55ed33b640 0 -- 172.21.15.88:0/3615757738 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f55e00083c0 msgr2=0x7f55e0091050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.852+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:52:27.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:52:27.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 zone default found 2024-04-17T04:52:27.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 4 Realm: () 2024-04-17T04:52:27.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:52:27.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:52:27.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:52:27.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 zonegroup default 2024-04-17T04:52:27.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:52:27.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 started sync module instance, tier type = 2024-04-17T04:52:27.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.856+0000 7f55ef01f980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=0 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=1 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=2 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=3 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=4 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=5 2024-04-17T04:52:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=6 2024-04-17T04:52:27.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 20 add_watcher() i=7 2024-04-17T04:52:27.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.872+0000 7f55ef01f980 2 all 8 watchers are set, enabling cache 2024-04-17T04:52:27.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.876+0000 7f54a2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:52:27.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.876+0000 7f55ef01f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:52:27.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.876+0000 7f55ef01f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:52:27.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.956+0000 7f54a1fe3640 20 reqs_thread_entry: start 2024-04-17T04:52:27.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.956+0000 7f55ef01f980 20 init_complete bucket index max shards: 1 2024-04-17T04:52:27.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 20 Filter name: none 2024-04-17T04:52:27.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f54137fe640 20 reqs_thread_entry: start 2024-04-17T04:52:27.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-04-17T04:52:27.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 20 rados->read ofs=0 len=0 2024-04-17T04:52:27.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 20 rados_obj.operate() r=0 bl.length=486 2024-04-17T04:52:27.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-04-17T04:52:27.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-17T04:52:27.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:52:27.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.960+0000 7f55ef01f980 10 removing key index: MIEJRQVUOIIIKAGFYPXB 2024-04-17T04:52:27.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.964+0000 7f55ef01f980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:MIEJRQVUOIIIKAGFYPXB, ofs0, ns] 2024-04-17T04:52:27.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.964+0000 7f54c3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464286 cookie 93861836456144 notifier 5687 bl.length()=170 2024-04-17T04:52:27.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.968+0000 7f55ef01f980 10 removing email index: sysuser.client.0+test@test.test 2024-04-17T04:52:27.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.968+0000 7f55ef01f980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-04-17T04:52:27.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.968+0000 7f54c3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464294 cookie 93861836471120 notifier 5687 bl.length()=182 2024-04-17T04:52:27.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.972+0000 7f55ef01f980 10 removing user buckets index 2024-04-17T04:52:27.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.976+0000 7f55ef01f980 10 removing user index: sysuser.client.0 2024-04-17T04:52:27.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.976+0000 7f55ef01f980 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-04-17T04:52:27.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.976+0000 7f55ef01f980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-04-17T04:52:27.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.980+0000 7f54c3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464287 cookie 93861836456144 notifier 5687 bl.length()=165 2024-04-17T04:52:27.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.980+0000 7f54a2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:52:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.988+0000 7f55ef01f980 20 remove_watcher() i=0 2024-04-17T04:52:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.988+0000 7f55ef01f980 2 removed watcher, disabling cache 2024-04-17T04:52:27.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.992+0000 7f55ef01f980 20 remove_watcher() i=1 2024-04-17T04:52:27.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.992+0000 7f55ef01f980 20 remove_watcher() i=2 2024-04-17T04:52:27.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.996+0000 7f55ef01f980 20 remove_watcher() i=3 2024-04-17T04:52:27.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:27.996+0000 7f55ef01f980 20 remove_watcher() i=4 2024-04-17T04:52:27.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:28.000+0000 7f55ef01f980 20 remove_watcher() i=5 2024-04-17T04:52:27.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:28.000+0000 7f55ef01f980 20 remove_watcher() i=6 2024-04-17T04:52:27.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:52:28.004+0000 7f55ef01f980 20 remove_watcher() i=7 2024-04-17T04:52:28.017 INFO:tasks.ragweed:Removing ragweed... 2024-04-17T04:52:28.018 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-04-17T04:52:28.112 INFO:teuthology.run_tasks:Running task workunit... 2024-04-17T04:52:28.124 INFO:tasks.workunit:Pulling workunits from ref 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:52:28.124 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-17T04:52:28.124 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:52:28.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T04:52:28.130 INFO:teuthology.orchestra.run.smithi088.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-17T04:52:28.130 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:52:28.177 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-17T04:52:28.177 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-17T04:52:28.225 INFO:tasks.workunit:timeout=3h 2024-04-17T04:52:28.226 INFO:tasks.workunit:cleanup=True 2024-04-17T04:52:28.226 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 41c500dd60c93c8d83053b72cffc3dd1a952a336 2024-04-17T04:52:28.276 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '41c500dd60c93c8d83053b72cffc3dd1a952a336'. 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-17T04:53:38.100 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:38.101 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 41c500dd60c Merge pull request #56902 from adk3798/wip-65415-squid 2024-04-17T04:53:38.103 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-17T04:53:38.150 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-17T04:53:38.153 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:53:38.153 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-17T04:53:38.212 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-17T04:53:38.273 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-17T04:53:38.320 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-17T04:53:38.321 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:53:38.321 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-17T04:53:38.364 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-17T04:53:38.367 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:53:38.367 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-17T04:53:38.412 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-04-17T04:53:38.414 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-04-17T04:53:38.414 DEBUG:teuthology.orchestra.run.smithi088:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41c500dd60c93c8d83053b72cffc3dd1a952a336 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-04-17T04:53:38.465 INFO:tasks.workunit.client.0.smithi088.stderr:+ ulimit -c 0 2024-04-17T04:53:38.465 INFO:tasks.workunit.client.0.smithi088.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-04-17T04:53:38.466 INFO:tasks.workunit.client.0.smithi088.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:53:38.466 INFO:tasks.workunit.client.0.smithi088.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:53:41.414 INFO:tasks.workunit.client.0.smithi088.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-04-17T04:53:41.414 INFO:tasks.workunit.client.0.smithi088.stderr:++ deactivate nondestructive 2024-04-17T04:53:41.414 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:53:41.414 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:53:41.414 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:53:41.414 INFO:tasks.workunit.client.0.smithi088.stderr:++ hash -r 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ unset VIRTUAL_ENV 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ export VIRTUAL_ENV 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ export PATH 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n '' ']' 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -z '' ']' 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ _OLD_VIRTUAL_PS1= 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ PS1='(rgw) ' 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ export PS1 2024-04-17T04:53:41.415 INFO:tasks.workunit.client.0.smithi088.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-04-17T04:53:41.416 INFO:tasks.workunit.client.0.smithi088.stderr:++ export VIRTUAL_ENV_PROMPT 2024-04-17T04:53:41.416 INFO:tasks.workunit.client.0.smithi088.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:53:41.416 INFO:tasks.workunit.client.0.smithi088.stderr:++ hash -r 2024-04-17T04:53:41.416 INFO:tasks.workunit.client.0.smithi088.stderr:+ pip install pip --upgrade 2024-04-17T04:53:41.773 INFO:tasks.workunit.client.0.smithi088.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-04-17T04:53:41.979 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting pip 2024-04-17T04:53:41.986 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-17T04:53:42.022 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: pip 2024-04-17T04:53:42.022 INFO:tasks.workunit.client.0.smithi088.stdout: Attempting uninstall: pip 2024-04-17T04:53:42.023 INFO:tasks.workunit.client.0.smithi088.stdout: Found existing installation: pip 22.0.2 2024-04-17T04:53:42.304 INFO:tasks.workunit.client.0.smithi088.stdout: Uninstalling pip-22.0.2: 2024-04-17T04:53:42.310 INFO:tasks.workunit.client.0.smithi088.stdout: Successfully uninstalled pip-22.0.2 2024-04-17T04:53:43.293 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed pip-24.0 2024-04-17T04:53:43.360 INFO:tasks.workunit.client.0.smithi088.stderr:+ pip install boto3 2024-04-17T04:53:44.277 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting boto3 2024-04-17T04:53:44.278 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached boto3-1.34.85-py3-none-any.whl.metadata (6.6 kB) 2024-04-17T04:53:44.777 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting botocore<1.35.0,>=1.34.85 (from boto3) 2024-04-17T04:53:44.778 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached botocore-1.34.85-py3-none-any.whl.metadata (5.7 kB) 2024-04-17T04:53:44.836 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-04-17T04:53:44.838 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-04-17T04:53:44.899 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-04-17T04:53:44.900 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-04-17T04:53:44.975 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:53:44.976 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-04-17T04:53:45.094 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:53:45.096 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-04-17T04:53:45.266 INFO:tasks.workunit.client.0.smithi088.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.85->boto3) 2024-04-17T04:53:45.267 INFO:tasks.workunit.client.0.smithi088.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-17T04:53:45.279 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached boto3-1.34.85-py3-none-any.whl (139 kB) 2024-04-17T04:53:45.281 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached botocore-1.34.85-py3-none-any.whl (12.1 MB) 2024-04-17T04:53:45.306 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-04-17T04:53:45.308 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-04-17T04:53:45.309 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-17T04:53:45.311 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-04-17T04:53:45.312 INFO:tasks.workunit.client.0.smithi088.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-17T04:53:45.419 INFO:tasks.workunit.client.0.smithi088.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-04-17T04:53:46.575 INFO:tasks.workunit.client.0.smithi088.stdout:Successfully installed boto3-1.34.85 botocore-1.34.85 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-04-17T04:53:46.740 INFO:tasks.workunit.client.0.smithi088.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-04-17T04:53:46.880 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-04-17T04:53:46.919 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:46.924 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:46.961 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 realm 2024-04-17T04:53:46.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:46.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:46.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.967+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.971+0000 7f47119e0980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:46.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.971+0000 7f47119e0980 20 searching for the correct realm 2024-04-17T04:53:46.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.975+0000 7f470ecfa640 0 -- 172.21.15.88:0/3214232537 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f4700047040 msgr2=0x7f4700030400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:46.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.983+0000 7f470fcfc640 0 -- 172.21.15.88:0/3214232537 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f470405d6e0 msgr2=0x7f4704062760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:46.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 zone default found 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 4 Realm: () 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 zonegroup default 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:46.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:46.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 started sync module instance, tier type = 2024-04-17T04:53:46.988 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:46.991+0000 7f47119e0980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=0 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=1 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=2 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=3 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=4 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=5 2024-04-17T04:53:47.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.007+0000 7f47119e0980 20 add_watcher() i=6 2024-04-17T04:53:47.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 20 add_watcher() i=7 2024-04-17T04:53:47.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:47.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f45c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 5 note: GC not initialized 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f45c4fe1640 20 reqs_thread_entry: start 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 20 Filter name: none 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f453e7fc640 20 reqs_thread_entry: start 2024-04-17T04:53:47.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-04-17T04:53:47.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.011+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-04-17T04:53:47.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-04-17T04:53:47.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-04-17T04:53:47.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 20 rados_obj.operate() r=0 bl.length=22 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x5 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 cache get: name=default.rgw.meta+users.uid+rgw_datacache_user : miss 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 20 rados_obj.operate() r=0 bl.length=420 2024-04-17T04:53:47.010 INFO:tasks.workunit.client.0.smithi088.stderr:could not create user: unable to parse parameters, duplicate key provided2024-04-17T04:53:47.015+0000 7f47119e0980 10 cache put: name=default.rgw.meta+users.uid+rgw_datacache_user info.flags=0x13 2024-04-17T04:53:47.011 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:47.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 adding default.rgw.meta+users.uid+rgw_datacache_user to cache LRU end 2024-04-17T04:53:47.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:53:47.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+rgw_datacache_user 2024-04-17T04:53:47.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f47119e0980 20 get_user_info_from_index found 0 xattrs 2024-04-17T04:53:47.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.015+0000 7f45c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:47.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.019+0000 7f47119e0980 20 remove_watcher() i=0 2024-04-17T04:53:47.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.019+0000 7f47119e0980 2 removed watcher, disabling cache 2024-04-17T04:53:47.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.023+0000 7f47119e0980 20 remove_watcher() i=1 2024-04-17T04:53:47.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.023+0000 7f47119e0980 20 remove_watcher() i=2 2024-04-17T04:53:47.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.027+0000 7f47119e0980 20 remove_watcher() i=3 2024-04-17T04:53:47.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.027+0000 7f47119e0980 20 remove_watcher() i=4 2024-04-17T04:53:47.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.031+0000 7f47119e0980 20 remove_watcher() i=5 2024-04-17T04:53:47.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.031+0000 7f47119e0980 20 remove_watcher() i=6 2024-04-17T04:53:47.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.031+0000 7f47119e0980 20 remove_watcher() i=7 2024-04-17T04:53:47.733 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard add 2024-04-17T04:53:47.734 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:47.734 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-17T04:53:47.768 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:47.768 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:47.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 realm 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.819+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.823+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:47.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.823+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.823+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:47.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.823+0000 7f5d97c19980 20 searching for the correct realm 2024-04-17T04:53:47.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.827+0000 7f5d95f35640 0 -- 172.21.15.88:0/253961386 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55968f4095f0 msgr2=0x55968f40b9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:47.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.835+0000 7f5d94f33640 0 -- 172.21.15.88:0/253961386 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5d88019070 msgr2=0x7f5d88054300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:47.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:47.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 zone default found 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 4 Realm: () 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 zonegroup default 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 started sync module instance, tier type = 2024-04-17T04:53:47.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.843+0000 7f5d97c19980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:47.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=0 2024-04-17T04:53:47.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=1 2024-04-17T04:53:47.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=2 2024-04-17T04:53:47.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=3 2024-04-17T04:53:47.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=4 2024-04-17T04:53:47.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=5 2024-04-17T04:53:47.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.859+0000 7f5d97c19980 20 add_watcher() i=6 2024-04-17T04:53:47.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 20 add_watcher() i=7 2024-04-17T04:53:47.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:47.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5c4b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 5 note: GC not initialized 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5c4a7e4640 20 reqs_thread_entry: start 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 20 Filter name: none 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5bbbfff640 20 reqs_thread_entry: start 2024-04-17T04:53:47.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.863+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:47.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:47.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:47.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:47.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.867+0000 7f5d97c19980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.871+0000 7f5d97c19980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:47.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.871+0000 7f5d97c19980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:47.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.871+0000 7f5c4b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:47.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.879+0000 7f5d97c19980 20 remove_watcher() i=0 2024-04-17T04:53:47.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.879+0000 7f5d97c19980 2 removed watcher, disabling cache 2024-04-17T04:53:47.875 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.879+0000 7f5d97c19980 20 remove_watcher() i=1 2024-04-17T04:53:47.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.883+0000 7f5d97c19980 20 remove_watcher() i=2 2024-04-17T04:53:47.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.883+0000 7f5d97c19980 20 remove_watcher() i=3 2024-04-17T04:53:47.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.887+0000 7f5d97c19980 20 remove_watcher() i=4 2024-04-17T04:53:47.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.887+0000 7f5d97c19980 20 remove_watcher() i=5 2024-04-17T04:53:47.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.887+0000 7f5d97c19980 20 remove_watcher() i=6 2024-04-17T04:53:47.885 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.891+0000 7f5d97c19980 20 remove_watcher() i=7 2024-04-17T04:53:47.899 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:47.900 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:53:47.900 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: a-bucket id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 num_objects: 0 size_kb: 0 num_shards: 1 2024-04-17T04:53:47.900 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:47.900 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 2 2024-04-17T04:53:47.929 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:47.929 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:47.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.975+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 realm 2024-04-17T04:53:47.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:47.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:47.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:47.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:47.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:47.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.979+0000 7f215c021980 20 searching for the correct realm 2024-04-17T04:53:47.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:47.987+0000 7f215a33d640 0 -- 172.21.15.88:0/3055838087 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f215005b1d0 msgr2=0x7f21500430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:48.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.203+0000 7f2159b3c640 0 -- 172.21.15.88:0/3055838087 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55818d9e3eb0 msgr2=0x55818d9e62e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:48.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:48.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:48.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:48.206 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:48.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:48.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:48.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:48.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.211+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 zone default found 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 4 Realm: () 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 zonegroup default 2024-04-17T04:53:48.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 started sync module instance, tier type = 2024-04-17T04:53:48.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.215+0000 7f215c021980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=0 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=1 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=2 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=3 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=4 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=5 2024-04-17T04:53:48.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=6 2024-04-17T04:53:48.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 20 add_watcher() i=7 2024-04-17T04:53:48.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.239+0000 7f215c021980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:48.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f200ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:48.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f215c021980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:48.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f215c021980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:48.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f215c021980 5 note: GC not initialized 2024-04-17T04:53:48.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f200efe5640 20 reqs_thread_entry: start 2024-04-17T04:53:48.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f215c021980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:48.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f215c021980 20 Filter name: none 2024-04-17T04:53:48.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.243+0000 7f200cfe1640 20 reqs_thread_entry: start 2024-04-17T04:53:48.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-17T04:53:48.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:48.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-04-17T04:53:48.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-17T04:53:48.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : miss 2024-04-17T04:53:48.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:48.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:48.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:48.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:53:48.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:48.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.247+0000 7f215c021980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:53:48.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.251+0000 7f200ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:48.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.255+0000 7f215c021980 20 remove_watcher() i=0 2024-04-17T04:53:48.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.255+0000 7f215c021980 2 removed watcher, disabling cache 2024-04-17T04:53:48.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.259+0000 7f215c021980 20 remove_watcher() i=1 2024-04-17T04:53:48.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.263+0000 7f215c021980 20 remove_watcher() i=2 2024-04-17T04:53:48.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.263+0000 7f215c021980 20 remove_watcher() i=3 2024-04-17T04:53:48.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.267+0000 7f215c021980 20 remove_watcher() i=4 2024-04-17T04:53:48.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.267+0000 7f215c021980 20 remove_watcher() i=5 2024-04-17T04:53:48.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.271+0000 7f215c021980 20 remove_watcher() i=6 2024-04-17T04:53:48.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.271+0000 7f215c021980 20 remove_watcher() i=7 2024-04-17T04:53:48.288 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin reshard list 2024-04-17T04:53:48.318 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:48.318 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:48.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.367+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 realm 2024-04-17T04:53:48.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:48.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:48.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:48.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.371+0000 7ff146dcb980 20 searching for the correct realm 2024-04-17T04:53:48.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.379+0000 7ff13ffff640 0 -- 172.21.15.88:0/3918846193 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7ff1400e8020 msgr2=0x7ff1400030a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:48.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.387+0000 7ff1450e7640 0 -- 172.21.15.88:0/3918846193 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7ff134008460 msgr2=0x7ff1340088e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:48.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:48.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 zone default found 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 4 Realm: () 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 zonegroup default 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 started sync module instance, tier type = 2024-04-17T04:53:48.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.395+0000 7ff146dcb980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:48.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=0 2024-04-17T04:53:48.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=1 2024-04-17T04:53:48.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=2 2024-04-17T04:53:48.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=3 2024-04-17T04:53:48.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=4 2024-04-17T04:53:48.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=5 2024-04-17T04:53:48.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.411+0000 7ff146dcb980 20 add_watcher() i=6 2024-04-17T04:53:48.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 20 add_watcher() i=7 2024-04-17T04:53:48.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:48.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7feffafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:48.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:48.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:48.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 5 note: GC not initialized 2024-04-17T04:53:48.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7feff9fdb640 20 reqs_thread_entry: start 2024-04-17T04:53:48.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:48.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7ff146dcb980 20 Filter name: none 2024-04-17T04:53:48.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.415+0000 7fef6b7fe640 20 reqs_thread_entry: start 2024-04-17T04:53:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.427+0000 7feffafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:48.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.431+0000 7ff146dcb980 20 remove_watcher() i=0 2024-04-17T04:53:48.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.431+0000 7ff146dcb980 2 removed watcher, disabling cache 2024-04-17T04:53:48.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.435+0000 7ff146dcb980 20 remove_watcher() i=1 2024-04-17T04:53:48.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.435+0000 7ff146dcb980 20 remove_watcher() i=2 2024-04-17T04:53:48.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.439+0000 7ff146dcb980 20 remove_watcher() i=3 2024-04-17T04:53:48.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.439+0000 7ff146dcb980 20 remove_watcher() i=4 2024-04-17T04:53:48.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.443+0000 7ff146dcb980 20 remove_watcher() i=5 2024-04-17T04:53:48.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.443+0000 7ff146dcb980 20 remove_watcher() i=6 2024-04-17T04:53:48.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.447+0000 7ff146dcb980 20 remove_watcher() i=7 2024-04-17T04:53:48.455 INFO:tasks.workunit.client.0.smithi088.stderr:bucket name a-bucket 2024-04-17T04:53:48.455 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard process 2024-04-17T04:53:48.455 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:48.455 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin reshard process 2024-04-17T04:53:48.480 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:48.480 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:48.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.527+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 realm 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.531+0000 7fbd01205980 20 searching for the correct realm 2024-04-17T04:53:48.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.539+0000 7fbcff521640 0 -- 172.21.15.88:0/1238231183 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fbcf402b060 msgr2=0x7fbcf407d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:48.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.755+0000 7fbcfed20640 0 -- 172.21.15.88:0/1238231183 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55daf42deb40 msgr2=0x55daf42e0f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:48.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:48.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:48.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:48.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:48.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:48.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:48.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:48.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.763+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 zone default found 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 4 Realm: () 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 zonegroup default 2024-04-17T04:53:48.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:48.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 started sync module instance, tier type = 2024-04-17T04:53:48.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.767+0000 7fbd01205980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:48.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=0 2024-04-17T04:53:48.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=1 2024-04-17T04:53:48.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=2 2024-04-17T04:53:48.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=3 2024-04-17T04:53:48.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=4 2024-04-17T04:53:48.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=5 2024-04-17T04:53:48.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.791+0000 7fbd01205980 20 add_watcher() i=6 2024-04-17T04:53:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.795+0000 7fbd01205980 20 add_watcher() i=7 2024-04-17T04:53:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.795+0000 7fbd01205980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbbb57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbd01205980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbd01205980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbd01205980 5 note: GC not initialized 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbb2ffff640 20 reqs_thread_entry: start 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbd01205980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbd01205980 20 Filter name: none 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbb2dffb640 20 reqs_thread_entry: start 2024-04-17T04:53:48.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.799+0000 7fbd01205980 20 processing logshard = reshard.0000000000 2024-04-17T04:53:48.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.807+0000 7fbd01205980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-04-17T04:53:48.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.807+0000 7fbd01205980 20 processing logshard = reshard.0000000001 2024-04-17T04:53:48.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.811+0000 7fbd01205980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-04-17T04:53:48.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.811+0000 7fbd01205980 20 processing logshard = reshard.0000000002 2024-04-17T04:53:48.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.815+0000 7fbd01205980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-04-17T04:53:48.812 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.815+0000 7fbd01205980 20 processing logshard = reshard.0000000003 2024-04-17T04:53:48.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.823+0000 7fbd01205980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-04-17T04:53:48.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.823+0000 7fbd01205980 20 processing logshard = reshard.0000000004 2024-04-17T04:53:48.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.827+0000 7fbd01205980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-04-17T04:53:48.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.827+0000 7fbd01205980 20 processing logshard = reshard.0000000005 2024-04-17T04:53:48.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.831+0000 7fbd01205980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-04-17T04:53:48.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.831+0000 7fbd01205980 20 processing logshard = reshard.0000000006 2024-04-17T04:53:48.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 20 process_entry resharding a-bucket 2024-04-17T04:53:48.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-17T04:53:48.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:48.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-04-17T04:53:48.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-17T04:53:48.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : miss 2024-04-17T04:53:48.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.835+0000 7fbd01205980 20 rados->read ofs=0 len=0 2024-04-17T04:53:48.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.839+0000 7fbd01205980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:48.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.839+0000 7fbd01205980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:48.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.839+0000 7fbd01205980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:48.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.839+0000 7fbd01205980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:53:48.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.839+0000 7fbd01205980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:48.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.839+0000 7fbd01205980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:53:48.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.847+0000 7fbd01205980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:48.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.847+0000 7fbd01205980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:48.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.847+0000 7fbd01205980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:48.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.847+0000 7fbd01205980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18, ofs0, ns] 2024-04-17T04:53:48.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbbe27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464289 cookie 94398887917424 notifier 5774 bl.length()=903 2024-04-17T04:53:48.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbbe27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:48.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbbe27fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:48.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbbe27fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:48.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_sources= new_sources= 2024-04-17T04:53:48.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential sources added=[] removed=[] 2024-04-17T04:53:48.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_dests= new_dests= 2024-04-17T04:53:48.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.851+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential dests added=[] removed=[] 2024-04-17T04:53:48.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.0 2024-04-17T04:53:48.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.0 2024-04-17T04:53:48.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.1 2024-04-17T04:53:48.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.1 2024-04-17T04:53:48.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.0 2024-04-17T04:53:48.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : hit (requested=0x17, cached=0x17) 2024-04-17T04:53:48.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.855+0000 7fbd01205980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:53:48.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbd01205980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:48.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbd01205980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:48.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbd01205980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:48.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbd01205980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18, ofs0, ns] 2024-04-17T04:53:48.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbbe1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464290 cookie 94398887917424 notifier 5774 bl.length()=871 2024-04-17T04:53:48.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbbe1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:48.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbbe1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:48.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.859+0000 7fbbe1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:48.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.863+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_sources= new_sources= 2024-04-17T04:53:48.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.863+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential sources added=[] removed=[] 2024-04-17T04:53:48.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.863+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_dests= new_dests= 2024-04-17T04:53:48.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.863+0000 7fbd01205980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential dests added=[] removed=[] 2024-04-17T04:53:48.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.863+0000 7fbd01205980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-04-17T04:53:48.862 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.867+0000 7fbd01205980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-04-17T04:53:48.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.871+0000 7fbd01205980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-04-17T04:53:48.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.871+0000 7fbd01205980 20 processing logshard = reshard.0000000007 2024-04-17T04:53:48.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.875+0000 7fbd01205980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-04-17T04:53:48.871 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.875+0000 7fbd01205980 20 processing logshard = reshard.0000000008 2024-04-17T04:53:48.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.883+0000 7fbd01205980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-04-17T04:53:48.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.883+0000 7fbd01205980 20 processing logshard = reshard.0000000009 2024-04-17T04:53:48.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.887+0000 7fbd01205980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-04-17T04:53:48.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.887+0000 7fbd01205980 20 processing logshard = reshard.0000000010 2024-04-17T04:53:48.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.895+0000 7fbd01205980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-04-17T04:53:48.891 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.895+0000 7fbd01205980 20 processing logshard = reshard.0000000011 2024-04-17T04:53:48.897 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.903+0000 7fbd01205980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-04-17T04:53:48.897 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.903+0000 7fbd01205980 20 processing logshard = reshard.0000000012 2024-04-17T04:53:48.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.907+0000 7fbd01205980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-04-17T04:53:48.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.907+0000 7fbd01205980 20 processing logshard = reshard.0000000013 2024-04-17T04:53:48.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.915+0000 7fbd01205980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-04-17T04:53:48.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.915+0000 7fbd01205980 20 processing logshard = reshard.0000000014 2024-04-17T04:53:48.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.919+0000 7fbd01205980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-04-17T04:53:48.914 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.919+0000 7fbd01205980 20 processing logshard = reshard.0000000015 2024-04-17T04:53:48.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.927+0000 7fbd01205980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-04-17T04:53:48.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.927+0000 7fbbb57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:48.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.931+0000 7fbd01205980 20 remove_watcher() i=0 2024-04-17T04:53:48.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.931+0000 7fbd01205980 2 removed watcher, disabling cache 2024-04-17T04:53:48.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.935+0000 7fbd01205980 20 remove_watcher() i=1 2024-04-17T04:53:48.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.935+0000 7fbd01205980 20 remove_watcher() i=2 2024-04-17T04:53:48.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.939+0000 7fbd01205980 20 remove_watcher() i=3 2024-04-17T04:53:48.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.939+0000 7fbd01205980 20 remove_watcher() i=4 2024-04-17T04:53:48.938 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.943+0000 7fbd01205980 20 remove_watcher() i=5 2024-04-17T04:53:48.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.943+0000 7fbd01205980 20 remove_watcher() i=6 2024-04-17T04:53:48.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:48.947+0000 7fbd01205980 20 remove_watcher() i=7 2024-04-17T04:53:53.961 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-17T04:53:53.994 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:53.994 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:54.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 20 realm 2024-04-17T04:53:54.040 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:54.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.042+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.046+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.046+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.046+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:54.042 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.046+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.046+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:54.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.046+0000 7f87349b7980 20 searching for the correct realm 2024-04-17T04:53:54.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.054+0000 7f8732cd3640 0 -- 172.21.15.88:0/97919742 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f8728020910 msgr2=0x7f87280143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:54.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.270+0000 7f87324d2640 0 -- 172.21.15.88:0/97919742 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55a8a892cbe0 msgr2=0x55a8a892efd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:54.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:54.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.278+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:54.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 zone default found 2024-04-17T04:53:54.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 4 Realm: () 2024-04-17T04:53:54.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:54.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:54.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:54.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 zonegroup default 2024-04-17T04:53:54.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:54.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 started sync module instance, tier type = 2024-04-17T04:53:54.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.282+0000 7f87349b7980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:54.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=0 2024-04-17T04:53:54.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=1 2024-04-17T04:53:54.301 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=2 2024-04-17T04:53:54.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=3 2024-04-17T04:53:54.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=4 2024-04-17T04:53:54.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=5 2024-04-17T04:53:54.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=6 2024-04-17T04:53:54.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 20 add_watcher() i=7 2024-04-17T04:53:54.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.306+0000 7f87349b7980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:54.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.310+0000 7f85e8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:54.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:54.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:54.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 5 note: GC not initialized 2024-04-17T04:53:54.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f85637fe640 20 reqs_thread_entry: start 2024-04-17T04:53:54.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:54.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 Filter name: none 2024-04-17T04:53:54.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f85617fa640 20 reqs_thread_entry: start 2024-04-17T04:53:54.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:54.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.314+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:54.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:54.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:54.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 rados->read ofs=0 len=0 2024-04-17T04:53:54.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:54.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.318+0000 7f87349b7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:54.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.322+0000 7f85e8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:54.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.326+0000 7f87349b7980 20 remove_watcher() i=0 2024-04-17T04:53:54.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.326+0000 7f87349b7980 2 removed watcher, disabling cache 2024-04-17T04:53:54.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.330+0000 7f87349b7980 20 remove_watcher() i=1 2024-04-17T04:53:54.328 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.330+0000 7f87349b7980 20 remove_watcher() i=2 2024-04-17T04:53:54.330 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.334+0000 7f87349b7980 20 remove_watcher() i=3 2024-04-17T04:53:54.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.334+0000 7f87349b7980 20 remove_watcher() i=4 2024-04-17T04:53:54.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.338+0000 7f87349b7980 20 remove_watcher() i=5 2024-04-17T04:53:54.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.338+0000 7f87349b7980 20 remove_watcher() i=6 2024-04-17T04:53:54.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:54.342+0000 7f87349b7980 20 remove_watcher() i=7 2024-04-17T04:53:54.353 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:54.353 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:53:54.353 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: a-bucket id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 num_objects: 0 size_kb: 0 num_shards: 2 2024-04-17T04:53:54.353 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:54.353 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard add non empty bucket 2024-04-17T04:53:54.353 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:55.099 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-17T04:53:55.130 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:55.131 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:55.175 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.178+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 realm 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.177 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:55.179 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.182+0000 7f0e98fba980 20 searching for the correct realm 2024-04-17T04:53:55.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.190+0000 7f0e972d6640 0 -- 172.21.15.88:0/2851993349 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f0e8c0ce9b0 msgr2=0x7f0e8c128a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:53:55.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.406+0000 7f0e96ad5640 0 -- 172.21.15.88:0/2851993349 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5571bd499250 msgr2=0x5571bd49b710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:55.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.414+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 zone default found 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 4 Realm: () 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 zonegroup default 2024-04-17T04:53:55.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.414 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 started sync module instance, tier type = 2024-04-17T04:53:55.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.418+0000 7f0e98fba980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=0 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=1 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=2 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=3 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=4 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=5 2024-04-17T04:53:55.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.442+0000 7f0e98fba980 20 add_watcher() i=6 2024-04-17T04:53:55.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.446+0000 7f0e98fba980 20 add_watcher() i=7 2024-04-17T04:53:55.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.446+0000 7f0e98fba980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:55.446 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0d4d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:55.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0e98fba980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:55.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0e98fba980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:55.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0e98fba980 5 note: GC not initialized 2024-04-17T04:53:55.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0cc7fff640 20 reqs_thread_entry: start 2024-04-17T04:53:55.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0e98fba980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:55.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0e98fba980 20 Filter name: none 2024-04-17T04:53:55.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0cc5ffb640 20 reqs_thread_entry: start 2024-04-17T04:53:55.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.450+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:55.450 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:55.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:55.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:55.452 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.454+0000 7f0e98fba980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.453 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.458+0000 7f0e98fba980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:55.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.458+0000 7f0e98fba980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:55.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.458+0000 7f0d4d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:55.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.466+0000 7f0e98fba980 20 remove_watcher() i=0 2024-04-17T04:53:55.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.466+0000 7f0e98fba980 2 removed watcher, disabling cache 2024-04-17T04:53:55.464 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.470+0000 7f0e98fba980 20 remove_watcher() i=1 2024-04-17T04:53:55.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.470+0000 7f0e98fba980 20 remove_watcher() i=2 2024-04-17T04:53:55.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.474+0000 7f0e98fba980 20 remove_watcher() i=3 2024-04-17T04:53:55.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.474+0000 7f0e98fba980 20 remove_watcher() i=4 2024-04-17T04:53:55.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.478+0000 7f0e98fba980 20 remove_watcher() i=5 2024-04-17T04:53:55.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.478+0000 7f0e98fba980 20 remove_watcher() i=6 2024-04-17T04:53:55.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.482+0000 7f0e98fba980 20 remove_watcher() i=7 2024-04-17T04:53:55.495 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:55.495 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:53:55.495 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: a-bucket id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 num_objects: 8 size_kb: 1 num_shards: 2 2024-04-17T04:53:55.495 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:55.496 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 3 2024-04-17T04:53:55.528 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:55.528 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:55.572 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.574+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 realm 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:55.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.578+0000 7fa0ad823980 20 searching for the correct realm 2024-04-17T04:53:55.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.586+0000 7fa0aab3d640 0 -- 172.21.15.88:0/35672372 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fa0a40eb910 msgr2=0x7fa0a40c3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:55.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.594+0000 7fa0abb3f640 0 -- 172.21.15.88:0/35672372 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fa094083eb0 msgr2=0x7fa09404b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:55.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:55.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:55.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:55.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:55.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:55.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:55.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.598+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:55.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 zone default found 2024-04-17T04:53:55.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 4 Realm: () 2024-04-17T04:53:55.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:55.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:55.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:55.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 zonegroup default 2024-04-17T04:53:55.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 started sync module instance, tier type = 2024-04-17T04:53:55.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.602+0000 7fa0ad823980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:55.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=0 2024-04-17T04:53:55.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=1 2024-04-17T04:53:55.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=2 2024-04-17T04:53:55.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=3 2024-04-17T04:53:55.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=4 2024-04-17T04:53:55.616 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=5 2024-04-17T04:53:55.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.618+0000 7fa0ad823980 20 add_watcher() i=6 2024-04-17T04:53:55.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.622+0000 7fa0ad823980 20 add_watcher() i=7 2024-04-17T04:53:55.617 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.622+0000 7fa0ad823980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:55.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.622+0000 7f9f627e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:55.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.622+0000 7fa0ad823980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:55.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.622+0000 7fa0ad823980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:55.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.622+0000 7fa0ad823980 5 note: GC not initialized 2024-04-17T04:53:55.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7f9f617e2640 20 reqs_thread_entry: start 2024-04-17T04:53:55.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:55.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 20 Filter name: none 2024-04-17T04:53:55.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7f9f5a7fc640 20 reqs_thread_entry: start 2024-04-17T04:53:55.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-17T04:53:55.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : miss 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:55.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.626+0000 7fa0ad823980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:53:55.626 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.630+0000 7f9f627e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:55.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.634+0000 7fa0ad823980 20 remove_watcher() i=0 2024-04-17T04:53:55.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.634+0000 7fa0ad823980 2 removed watcher, disabling cache 2024-04-17T04:53:55.633 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.638+0000 7fa0ad823980 20 remove_watcher() i=1 2024-04-17T04:53:55.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.638+0000 7fa0ad823980 20 remove_watcher() i=2 2024-04-17T04:53:55.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.642+0000 7fa0ad823980 20 remove_watcher() i=3 2024-04-17T04:53:55.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.642+0000 7fa0ad823980 20 remove_watcher() i=4 2024-04-17T04:53:55.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.642+0000 7fa0ad823980 20 remove_watcher() i=5 2024-04-17T04:53:55.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.646+0000 7fa0ad823980 20 remove_watcher() i=6 2024-04-17T04:53:55.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.646+0000 7fa0ad823980 20 remove_watcher() i=7 2024-04-17T04:53:55.656 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin reshard list 2024-04-17T04:53:55.680 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:55.680 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:55.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.726+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 realm 2024-04-17T04:53:55.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:55.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:55.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.730+0000 7ff040fde980 20 searching for the correct realm 2024-04-17T04:53:55.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.738+0000 7ff03e2f8640 0 -- 172.21.15.88:0/3198318059 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7ff03804a040 msgr2=0x7ff03804a420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:55.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.746+0000 7ff03e2f8640 0 -- 172.21.15.88:0/3198318059 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fef080763a0 msgr2=0x7fef080787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.750+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 zone default found 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 4 Realm: () 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 zonegroup default 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 started sync module instance, tier type = 2024-04-17T04:53:55.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.754+0000 7ff040fde980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=0 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=1 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=2 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=3 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=4 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=5 2024-04-17T04:53:55.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=6 2024-04-17T04:53:55.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 20 add_watcher() i=7 2024-04-17T04:53:55.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.770+0000 7ff040fde980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:55.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7feef57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:55.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7ff040fde980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:55.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7ff040fde980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:55.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7ff040fde980 5 note: GC not initialized 2024-04-17T04:53:55.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7fee6ffff640 20 reqs_thread_entry: start 2024-04-17T04:53:55.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7ff040fde980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:55.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7ff040fde980 20 Filter name: none 2024-04-17T04:53:55.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.774+0000 7fee6dffb640 20 reqs_thread_entry: start 2024-04-17T04:53:55.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.786+0000 7feef57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:55.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.790+0000 7ff040fde980 20 remove_watcher() i=0 2024-04-17T04:53:55.785 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.790+0000 7ff040fde980 2 removed watcher, disabling cache 2024-04-17T04:53:55.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.790+0000 7ff040fde980 20 remove_watcher() i=1 2024-04-17T04:53:55.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.794+0000 7ff040fde980 20 remove_watcher() i=2 2024-04-17T04:53:55.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.794+0000 7ff040fde980 20 remove_watcher() i=3 2024-04-17T04:53:55.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.798+0000 7ff040fde980 20 remove_watcher() i=4 2024-04-17T04:53:55.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.798+0000 7ff040fde980 20 remove_watcher() i=5 2024-04-17T04:53:55.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.798+0000 7ff040fde980 20 remove_watcher() i=6 2024-04-17T04:53:55.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.802+0000 7ff040fde980 20 remove_watcher() i=7 2024-04-17T04:53:55.811 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard process non empty bucket 2024-04-17T04:53:55.811 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:55.811 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin reshard process 2024-04-17T04:53:55.835 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:55.835 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:55.878 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 realm 2024-04-17T04:53:55.879 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:55.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:55.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.882+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.886+0000 7fe20c21f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:55.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.886+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:55.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.886+0000 7fe20c21f980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:55.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.886+0000 7fe20c21f980 20 searching for the correct realm 2024-04-17T04:53:55.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:55.894+0000 7fe20a53b640 0 -- 172.21.15.88:0/1456075313 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fe200104040 msgr2=0x7fe2000ed7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:56.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.110+0000 7fe209d3a640 0 -- 172.21.15.88:0/1456075313 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55e097d1fdf0 msgr2=0x55e097d221e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:56.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.118+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:56.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:56.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 zone default found 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 4 Realm: () 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 zonegroup default 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 started sync module instance, tier type = 2024-04-17T04:53:56.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.122+0000 7fe20c21f980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:56.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=0 2024-04-17T04:53:56.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=1 2024-04-17T04:53:56.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=2 2024-04-17T04:53:56.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=3 2024-04-17T04:53:56.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=4 2024-04-17T04:53:56.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=5 2024-04-17T04:53:56.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.142+0000 7fe20c21f980 20 add_watcher() i=6 2024-04-17T04:53:56.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.146+0000 7fe20c21f980 20 add_watcher() i=7 2024-04-17T04:53:56.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.146+0000 7fe20c21f980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:56.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.150+0000 7fe0bb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:56.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.150+0000 7fe20c21f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:56.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.150+0000 7fe20c21f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:56.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.150+0000 7fe20c21f980 5 note: GC not initialized 2024-04-17T04:53:56.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.154+0000 7fe0ba7fc640 20 reqs_thread_entry: start 2024-04-17T04:53:56.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.154+0000 7fe20c21f980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:56.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.154+0000 7fe20c21f980 20 Filter name: none 2024-04-17T04:53:56.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.154+0000 7fe02bfff640 20 reqs_thread_entry: start 2024-04-17T04:53:56.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.154+0000 7fe20c21f980 20 processing logshard = reshard.0000000000 2024-04-17T04:53:56.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.158+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-04-17T04:53:56.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.158+0000 7fe20c21f980 20 processing logshard = reshard.0000000001 2024-04-17T04:53:56.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.162+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-04-17T04:53:56.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.162+0000 7fe20c21f980 20 processing logshard = reshard.0000000002 2024-04-17T04:53:56.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.170+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-04-17T04:53:56.165 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.170+0000 7fe20c21f980 20 processing logshard = reshard.0000000003 2024-04-17T04:53:56.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.174+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-04-17T04:53:56.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.174+0000 7fe20c21f980 20 processing logshard = reshard.0000000004 2024-04-17T04:53:56.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.178+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-04-17T04:53:56.176 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.178+0000 7fe20c21f980 20 processing logshard = reshard.0000000005 2024-04-17T04:53:56.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.182+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-04-17T04:53:56.180 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.182+0000 7fe20c21f980 20 processing logshard = reshard.0000000006 2024-04-17T04:53:56.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 20 process_entry resharding a-bucket 2024-04-17T04:53:56.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-17T04:53:56.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:56.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-04-17T04:53:56.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-17T04:53:56.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : miss 2024-04-17T04:53:56.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:56.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:56.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:56.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:53:56.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:56.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.190+0000 7fe20c21f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:53:56.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe20c21f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:56.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe20c21f980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:56.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe20c21f980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:56.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe20c21f980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18, ofs0, ns] 2024-04-17T04:53:56.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe0ecff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464291 cookie 94423108162800 notifier 6144 bl.length()=903 2024-04-17T04:53:56.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe0ecff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:56.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe0ecff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:56.200 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.202+0000 7fe0ecff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:56.203 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.206+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_sources= new_sources= 2024-04-17T04:53:56.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.206+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential sources added=[] removed=[] 2024-04-17T04:53:56.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.206+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_dests= new_dests= 2024-04-17T04:53:56.204 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.206+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential dests added=[] removed=[] 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.1 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.1 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.2 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.2 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.0 2024-04-17T04:53:56.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.0 2024-04-17T04:53:56.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.1 2024-04-17T04:53:56.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.210+0000 7fe20c21f980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.1.1 2024-04-17T04:53:56.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.218+0000 7fe20c21f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : hit (requested=0x17, cached=0x17) 2024-04-17T04:53:56.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.218+0000 7fe20c21f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:53:56.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.218+0000 7fe20c21f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:56.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.218+0000 7fe20c21f980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:56.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.218+0000 7fe20c21f980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:56.216 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.218+0000 7fe20c21f980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18, ofs0, ns] 2024-04-17T04:53:56.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.222+0000 7fe0ecff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464292 cookie 94423108162800 notifier 6144 bl.length()=871 2024-04-17T04:53:56.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.222+0000 7fe0ecff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:53:56.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.222+0000 7fe0ecff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:53:56.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.222+0000 7fe0ecff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:53:56.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.226+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_sources= new_sources= 2024-04-17T04:53:56.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.226+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential sources added=[] removed=[] 2024-04-17T04:53:56.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.226+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_dests= new_dests= 2024-04-17T04:53:56.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.226+0000 7fe20c21f980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential dests added=[] removed=[] 2024-04-17T04:53:56.224 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.226+0000 7fe20c21f980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-04-17T04:53:56.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.230+0000 7fe20c21f980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-04-17T04:53:56.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.234+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-04-17T04:53:56.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.234+0000 7fe20c21f980 20 processing logshard = reshard.0000000007 2024-04-17T04:53:56.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.242+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-04-17T04:53:56.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.242+0000 7fe20c21f980 20 processing logshard = reshard.0000000008 2024-04-17T04:53:56.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.250+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-04-17T04:53:56.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.250+0000 7fe20c21f980 20 processing logshard = reshard.0000000009 2024-04-17T04:53:56.252 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.258+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-04-17T04:53:56.253 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.258+0000 7fe20c21f980 20 processing logshard = reshard.0000000010 2024-04-17T04:53:56.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.262+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-04-17T04:53:56.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.262+0000 7fe20c21f980 20 processing logshard = reshard.0000000011 2024-04-17T04:53:56.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.266+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-04-17T04:53:56.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.266+0000 7fe20c21f980 20 processing logshard = reshard.0000000012 2024-04-17T04:53:56.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.274+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-04-17T04:53:56.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.274+0000 7fe20c21f980 20 processing logshard = reshard.0000000013 2024-04-17T04:53:56.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.282+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-04-17T04:53:56.277 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.282+0000 7fe20c21f980 20 processing logshard = reshard.0000000014 2024-04-17T04:53:56.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.286+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-04-17T04:53:56.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.286+0000 7fe20c21f980 20 processing logshard = reshard.0000000015 2024-04-17T04:53:56.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.294+0000 7fe20c21f980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-04-17T04:53:56.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.298+0000 7fe0bb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:56.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.302+0000 7fe20c21f980 20 remove_watcher() i=0 2024-04-17T04:53:56.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.302+0000 7fe20c21f980 2 removed watcher, disabling cache 2024-04-17T04:53:56.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.306+0000 7fe20c21f980 20 remove_watcher() i=1 2024-04-17T04:53:56.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.310+0000 7fe20c21f980 20 remove_watcher() i=2 2024-04-17T04:53:56.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.310+0000 7fe20c21f980 20 remove_watcher() i=3 2024-04-17T04:53:56.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.314+0000 7fe20c21f980 20 remove_watcher() i=4 2024-04-17T04:53:56.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.314+0000 7fe20c21f980 20 remove_watcher() i=5 2024-04-17T04:53:56.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.318+0000 7fe20c21f980 20 remove_watcher() i=6 2024-04-17T04:53:56.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.318+0000 7fe20c21f980 20 remove_watcher() i=7 2024-04-17T04:53:56.336 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-17T04:53:56.370 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:56.370 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:56.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 realm 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.418+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.422+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:56.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.422+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.422+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:56.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.422+0000 7f6ec14a4980 20 searching for the correct realm 2024-04-17T04:53:56.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.430+0000 7f6ebf7c0640 0 -- 172.21.15.88:0/1400120256 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f6eb404d150 msgr2=0x7f6eb4037ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:53:56.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.646+0000 7f6ebefbf640 0 -- 172.21.15.88:0/1400120256 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x560d8881f980 msgr2=0x560d88821e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:56.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:56.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.654+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 zone default found 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 4 Realm: () 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 zonegroup default 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:56.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 started sync module instance, tier type = 2024-04-17T04:53:56.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.658+0000 7f6ec14a4980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=0 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=1 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=2 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=3 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=4 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=5 2024-04-17T04:53:56.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=6 2024-04-17T04:53:56.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 20 add_watcher() i=7 2024-04-17T04:53:56.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.682+0000 7f6ec14a4980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:56.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6d757e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:56.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ec14a4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:56.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ec14a4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:56.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ec14a4980 5 note: GC not initialized 2024-04-17T04:53:56.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ceffff640 20 reqs_thread_entry: start 2024-04-17T04:53:56.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ec14a4980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:56.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ec14a4980 20 Filter name: none 2024-04-17T04:53:56.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6cedffb640 20 reqs_thread_entry: start 2024-04-17T04:53:56.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.686+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:56.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:56.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:56.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:53:56.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.690+0000 7f6ec14a4980 20 rados->read ofs=0 len=0 2024-04-17T04:53:56.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.694+0000 7f6ec14a4980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:53:56.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.694+0000 7f6ec14a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:56.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.694+0000 7f6d757e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:56.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.698+0000 7f6ec14a4980 20 remove_watcher() i=0 2024-04-17T04:53:56.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.698+0000 7f6ec14a4980 2 removed watcher, disabling cache 2024-04-17T04:53:56.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.702+0000 7f6ec14a4980 20 remove_watcher() i=1 2024-04-17T04:53:56.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.702+0000 7f6ec14a4980 20 remove_watcher() i=2 2024-04-17T04:53:56.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.706+0000 7f6ec14a4980 20 remove_watcher() i=3 2024-04-17T04:53:56.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.710+0000 7f6ec14a4980 20 remove_watcher() i=4 2024-04-17T04:53:56.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.710+0000 7f6ec14a4980 20 remove_watcher() i=5 2024-04-17T04:53:56.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.710+0000 7f6ec14a4980 20 remove_watcher() i=6 2024-04-17T04:53:56.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:56.714+0000 7f6ec14a4980 20 remove_watcher() i=7 2024-04-17T04:53:56.725 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:56.725 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:53:56.725 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: a-bucket id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 num_objects: 8 size_kb: 1 num_shards: 3 2024-04-17T04:53:56.725 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:56.725 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-04-17T04:53:56.725 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:58.929 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-17T04:53:58.961 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:58.961 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 realm 2024-04-17T04:53:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.010+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.014+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:59.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.014+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.010 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.014+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:59.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.014+0000 7f82fefc1980 20 searching for the correct realm 2024-04-17T04:53:59.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.022+0000 7f82fd2dd640 0 -- 172.21.15.88:0/628504262 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f82f004d040 msgr2=0x7f82f0031410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:53:59.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.238+0000 7f82fcadc640 0 -- 172.21.15.88:0/628504262 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55aba57f85e0 msgr2=0x55aba57faaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:59.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:59.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:59.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:59.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:59.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:59.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:59.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:59.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.246+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:59.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 zone default found 2024-04-17T04:53:59.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 4 Realm: () 2024-04-17T04:53:59.245 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:59.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:59.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:59.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 zonegroup default 2024-04-17T04:53:59.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.246 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 started sync module instance, tier type = 2024-04-17T04:53:59.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.250+0000 7f82fefc1980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:59.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=0 2024-04-17T04:53:59.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=1 2024-04-17T04:53:59.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=2 2024-04-17T04:53:59.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=3 2024-04-17T04:53:59.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=4 2024-04-17T04:53:59.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=5 2024-04-17T04:53:59.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.274+0000 7f82fefc1980 20 add_watcher() i=6 2024-04-17T04:53:59.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.278+0000 7f82fefc1980 20 add_watcher() i=7 2024-04-17T04:53:59.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.278+0000 7f82fefc1980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:59.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f81b2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:59.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f82fefc1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:59.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f82fefc1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:59.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f82fefc1980 5 note: GC not initialized 2024-04-17T04:53:59.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f81b1fe3640 20 reqs_thread_entry: start 2024-04-17T04:53:59.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f82fefc1980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:59.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.282+0000 7f82fefc1980 20 Filter name: none 2024-04-17T04:53:59.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f81237fe640 20 reqs_thread_entry: start 2024-04-17T04:53:59.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:53:59.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:53:59.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:59.282 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:53:59.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:53:59.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.286+0000 7f82fefc1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:53:59.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.290+0000 7f81b2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:59.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.294+0000 7f82fefc1980 20 remove_watcher() i=0 2024-04-17T04:53:59.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.294+0000 7f82fefc1980 2 removed watcher, disabling cache 2024-04-17T04:53:59.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.298+0000 7f82fefc1980 20 remove_watcher() i=1 2024-04-17T04:53:59.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.298+0000 7f82fefc1980 20 remove_watcher() i=2 2024-04-17T04:53:59.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.302+0000 7f82fefc1980 20 remove_watcher() i=3 2024-04-17T04:53:59.300 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.306+0000 7f82fefc1980 20 remove_watcher() i=4 2024-04-17T04:53:59.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.306+0000 7f82fefc1980 20 remove_watcher() i=5 2024-04-17T04:53:59.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.306+0000 7f82fefc1980 20 remove_watcher() i=6 2024-04-17T04:53:59.305 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.310+0000 7f82fefc1980 20 remove_watcher() i=7 2024-04-17T04:53:59.322 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:59.323 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:53:59.323 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:53:59.323 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:53:59.323 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-04-17T04:53:59.355 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:59.355 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:59.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 realm 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.402+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.406+0000 7f05d126e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:59.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.406+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.406+0000 7f05d126e980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:59.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.406+0000 7f05d126e980 20 searching for the correct realm 2024-04-17T04:53:59.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.414+0000 7f05cf58a640 0 -- 172.21.15.88:0/1527855320 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f05c4079460 msgr2=0x7f05c405a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:59.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.630+0000 7f05ced89640 0 -- 172.21.15.88:0/1527855320 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x564be8ccae40 msgr2=0x564be8ccd230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.638+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 zone default found 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 4 Realm: () 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 zonegroup default 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 started sync module instance, tier type = 2024-04-17T04:53:59.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.642+0000 7f05d126e980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:53:59.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=0 2024-04-17T04:53:59.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=1 2024-04-17T04:53:59.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=2 2024-04-17T04:53:59.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=3 2024-04-17T04:53:59.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=4 2024-04-17T04:53:59.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=5 2024-04-17T04:53:59.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=6 2024-04-17T04:53:59.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 20 add_watcher() i=7 2024-04-17T04:53:59.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.666+0000 7f05d126e980 2 all 8 watchers are set, enabling cache 2024-04-17T04:53:59.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.670+0000 7f04857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:59.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.670+0000 7f05d126e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:53:59.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.670+0000 7f05d126e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:53:59.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.670+0000 7f05d126e980 5 note: GC not initialized 2024-04-17T04:53:59.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.670+0000 7f03fffff640 20 reqs_thread_entry: start 2024-04-17T04:53:59.668 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.670+0000 7f05d126e980 20 init_complete bucket index max shards: 1 2024-04-17T04:53:59.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 20 Filter name: none 2024-04-17T04:53:59.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f03fdffb640 20 reqs_thread_entry: start 2024-04-17T04:53:59.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-04-17T04:53:59.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:53:59.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-04-17T04:53:59.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-04-17T04:53:59.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 : miss 2024-04-17T04:53:59.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:53:59.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 info.flags=0x17 2024-04-17T04:53:59.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 to cache LRU end 2024-04-17T04:53:59.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:53:59.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:53:59.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.674+0000 7f05d126e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 2024-04-17T04:53:59.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.678+0000 7f05d126e980 -1 Injecting error=-5 at location=set_target_layout 2024-04-17T04:53:59.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.678+0000 7f05d126e980 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-04-17T04:53:59.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.686+0000 7f04857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:53:59.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.690+0000 7f05d126e980 20 remove_watcher() i=0 2024-04-17T04:53:59.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.690+0000 7f05d126e980 2 removed watcher, disabling cache 2024-04-17T04:53:59.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.694+0000 7f05d126e980 20 remove_watcher() i=1 2024-04-17T04:53:59.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.694+0000 7f05d126e980 20 remove_watcher() i=2 2024-04-17T04:53:59.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.698+0000 7f05d126e980 20 remove_watcher() i=3 2024-04-17T04:53:59.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.698+0000 7f05d126e980 20 remove_watcher() i=4 2024-04-17T04:53:59.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.702+0000 7f05d126e980 20 remove_watcher() i=5 2024-04-17T04:53:59.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.702+0000 7f05d126e980 20 remove_watcher() i=6 2024-04-17T04:53:59.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.702+0000 7f05d126e980 20 remove_watcher() i=7 2024-04-17T04:53:59.715 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-17T04:53:59.743 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:53:59.743 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:53:59.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 realm 2024-04-17T04:53:59.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:53:59.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:53:59.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.790+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.794+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:53:59.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.794+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:53:59.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.794+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:53:59.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.794+0000 7fc2eca5c980 20 searching for the correct realm 2024-04-17T04:53:59.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:53:59.802+0000 7fc2ead78640 0 -- 172.21.15.88:0/1074142269 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fc2d4084700 msgr2=0x7fc2d408ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:00.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.018+0000 7fc2ea577640 0 -- 172.21.15.88:0/1074142269 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55a301db3c70 msgr2=0x55a301db6060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.022+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:00.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:00.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 zone default found 2024-04-17T04:54:00.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 4 Realm: () 2024-04-17T04:54:00.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:00.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 zonegroup default 2024-04-17T04:54:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 started sync module instance, tier type = 2024-04-17T04:54:00.024 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.026+0000 7fc2eca5c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=0 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=1 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=2 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=3 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=4 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=5 2024-04-17T04:54:00.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=6 2024-04-17T04:54:00.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 20 add_watcher() i=7 2024-04-17T04:54:00.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.050+0000 7fc2eca5c980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:00.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.054+0000 7fc1a0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:00.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:00.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:00.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 5 note: GC not initialized 2024-04-17T04:54:00.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc11b7fe640 20 reqs_thread_entry: start 2024-04-17T04:54:00.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:00.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 Filter name: none 2024-04-17T04:54:00.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc1197fa640 20 reqs_thread_entry: start 2024-04-17T04:54:00.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:00.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:00.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:00.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:00.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:00.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.058+0000 7fc2eca5c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:00.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.062+0000 7fc1a0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:00.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.066+0000 7fc2eca5c980 20 remove_watcher() i=0 2024-04-17T04:54:00.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.066+0000 7fc2eca5c980 2 removed watcher, disabling cache 2024-04-17T04:54:00.066 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.070+0000 7fc2eca5c980 20 remove_watcher() i=1 2024-04-17T04:54:00.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.070+0000 7fc2eca5c980 20 remove_watcher() i=2 2024-04-17T04:54:00.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.074+0000 7fc2eca5c980 20 remove_watcher() i=3 2024-04-17T04:54:00.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.078+0000 7fc2eca5c980 20 remove_watcher() i=4 2024-04-17T04:54:00.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.078+0000 7fc2eca5c980 20 remove_watcher() i=5 2024-04-17T04:54:00.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.078+0000 7fc2eca5c980 20 remove_watcher() i=6 2024-04-17T04:54:00.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.082+0000 7fc2eca5c980 20 remove_watcher() i=7 2024-04-17T04:54:00.093 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:00.093 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:00.093 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:00.093 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:00.194 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:54:00.228 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:00.228 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:00.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 20 realm 2024-04-17T04:54:00.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:00.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.274+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.278+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.278+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.278+0000 7f0bb906e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:00.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.278+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.278+0000 7f0bb906e980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:00.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.278+0000 7f0bb906e980 20 searching for the correct realm 2024-04-17T04:54:00.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.286+0000 7f0bb738a640 0 -- 172.21.15.88:0/2438047948 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f0ba006cc20 msgr2=0x7f0ba000b210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:00.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.290+0000 7f0bb6b89640 0 -- 172.21.15.88:0/2438047948 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f0ba000a760 msgr2=0x7f0ba0053ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:54:00.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:00.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:00.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:00.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:00.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:00.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:00.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.506+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:00.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 zone default found 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 4 Realm: () 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 zonegroup default 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.510+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.514+0000 7f0bb906e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.514+0000 7f0bb906e980 20 started sync module instance, tier type = 2024-04-17T04:54:00.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.514+0000 7f0bb906e980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=0 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=1 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=2 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=3 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=4 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=5 2024-04-17T04:54:00.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.534+0000 7f0bb906e980 20 add_watcher() i=6 2024-04-17T04:54:00.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.538+0000 7f0bb906e980 20 add_watcher() i=7 2024-04-17T04:54:00.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.538+0000 7f0bb906e980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:00.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0a6d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:00.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:00.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:00.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 5 note: GC not initialized 2024-04-17T04:54:00.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f09e7fff640 20 reqs_thread_entry: start 2024-04-17T04:54:00.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:00.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 Filter name: none 2024-04-17T04:54:00.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f09e5ffb640 20 reqs_thread_entry: start 2024-04-17T04:54:00.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-04-17T04:54:00.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:00.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-04-17T04:54:00.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-04-17T04:54:00.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 : miss 2024-04-17T04:54:00.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.542+0000 7f0bb906e980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.546+0000 7f0bb906e980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:00.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.546+0000 7f0bb906e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 info.flags=0x17 2024-04-17T04:54:00.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.546+0000 7f0bb906e980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 to cache LRU end 2024-04-17T04:54:00.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.546+0000 7f0bb906e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:00.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.546+0000 7f0bb906e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:00.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.546+0000 7f0bb906e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 2024-04-17T04:54:00.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.550+0000 7f0bb906e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 info.flags=0x17 2024-04-17T04:54:00.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.550+0000 7f0bb906e980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 to cache LRU end 2024-04-17T04:54:00.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.550+0000 7f0bb906e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:00.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.550+0000 7f0bb906e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20, ofs0, ns] 2024-04-17T04:54:00.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0a9a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464294 cookie 94898693769328 notifier 6240 bl.length()=1001 2024-04-17T04:54:00.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0a9a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 info.flags=0x17 2024-04-17T04:54:00.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0a9a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 to cache LRU end 2024-04-17T04:54:00.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0a9a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:00.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): orig_sources= new_sources= 2024-04-17T04:54:00.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): potential sources added=[] removed=[] 2024-04-17T04:54:00.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): orig_dests= new_dests= 2024-04-17T04:54:00.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.554+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): potential dests added=[] removed=[] 2024-04-17T04:54:00.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.558+0000 7f0bb906e980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20.1.0 2024-04-17T04:54:00.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.558+0000 7f0bb906e980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20.1.0 2024-04-17T04:54:00.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.558+0000 7f0bb906e980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20.1.1 2024-04-17T04:54:00.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.558+0000 7f0bb906e980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20.1.1 2024-04-17T04:54:00.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.558+0000 7f0bb906e980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20.0 2024-04-17T04:54:00.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.562+0000 7f0bb906e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 : hit (requested=0x17, cached=0x17) 2024-04-17T04:54:00.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.562+0000 7f0bb906e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 2024-04-17T04:54:00.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 info.flags=0x17 2024-04-17T04:54:00.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 to cache LRU end 2024-04-17T04:54:00.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:00.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20, ofs0, ns] 2024-04-17T04:54:00.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0a99ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464295 cookie 94898693769328 notifier 6240 bl.length()=969 2024-04-17T04:54:00.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0a99ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 info.flags=0x17 2024-04-17T04:54:00.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0a99ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 to cache LRU end 2024-04-17T04:54:00.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0a99ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:00.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): orig_sources= new_sources= 2024-04-17T04:54:00.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): potential sources added=[] removed=[] 2024-04-17T04:54:00.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): orig_dests= new_dests= 2024-04-17T04:54:00.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.566+0000 7f0bb906e980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20]): potential dests added=[] removed=[] 2024-04-17T04:54:00.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.570+0000 7f0bb906e980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-04-17T04:54:00.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.574+0000 7f0a6d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:00.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.578+0000 7f0bb906e980 20 remove_watcher() i=0 2024-04-17T04:54:00.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.578+0000 7f0bb906e980 2 removed watcher, disabling cache 2024-04-17T04:54:00.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.582+0000 7f0bb906e980 20 remove_watcher() i=1 2024-04-17T04:54:00.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.586+0000 7f0bb906e980 20 remove_watcher() i=2 2024-04-17T04:54:00.584 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.590+0000 7f0bb906e980 20 remove_watcher() i=3 2024-04-17T04:54:00.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.590+0000 7f0bb906e980 20 remove_watcher() i=4 2024-04-17T04:54:00.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.590+0000 7f0bb906e980 20 remove_watcher() i=5 2024-04-17T04:54:00.590 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.594+0000 7f0bb906e980 20 remove_watcher() i=6 2024-04-17T04:54:00.591 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.594+0000 7f0bb906e980 20 remove_watcher() i=7 2024-04-17T04:54:00.609 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-17T04:54:00.635 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:00.636 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:00.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 realm 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.682+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.686+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:00.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.686+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.686+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:00.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.686+0000 7f8fd38e1980 20 searching for the correct realm 2024-04-17T04:54:00.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.690+0000 7f8fd1bfd640 0 -- 172.21.15.88:0/2413099287 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x564840325920 msgr2=0x564840327d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:00.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.698+0000 7f8fd0bfb640 0 -- 172.21.15.88:0/2413099287 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f8fcc031040 msgr2=0x7f8fcc04b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:54:00.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:00.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 zone default found 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 4 Realm: () 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 zonegroup default 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.706+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:00.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 started sync module instance, tier type = 2024-04-17T04:54:00.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.710+0000 7f8fd38e1980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:00.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.726+0000 7f8fd38e1980 20 add_watcher() i=0 2024-04-17T04:54:00.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.726+0000 7f8fd38e1980 20 add_watcher() i=1 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.726+0000 7f8fd38e1980 20 add_watcher() i=2 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.726+0000 7f8fd38e1980 20 add_watcher() i=3 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.726+0000 7f8fd38e1980 20 add_watcher() i=4 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.726+0000 7f8fd38e1980 20 add_watcher() i=5 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.730+0000 7f8fd38e1980 20 add_watcher() i=6 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.730+0000 7f8fd38e1980 20 add_watcher() i=7 2024-04-17T04:54:00.725 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.730+0000 7f8fd38e1980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:00.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.730+0000 7f8e877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 5 note: GC not initialized 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8e867e4640 20 reqs_thread_entry: start 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 Filter name: none 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8df7fff640 20 reqs_thread_entry: start 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:00.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados->read ofs=0 len=0 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:00.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.734+0000 7f8fd38e1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:00.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.738+0000 7f8e877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:00.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.746+0000 7f8fd38e1980 20 remove_watcher() i=0 2024-04-17T04:54:00.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.746+0000 7f8fd38e1980 2 removed watcher, disabling cache 2024-04-17T04:54:00.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.746+0000 7f8fd38e1980 20 remove_watcher() i=1 2024-04-17T04:54:00.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.750+0000 7f8fd38e1980 20 remove_watcher() i=2 2024-04-17T04:54:00.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.750+0000 7f8fd38e1980 20 remove_watcher() i=3 2024-04-17T04:54:00.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.758+0000 7f8fd38e1980 20 remove_watcher() i=4 2024-04-17T04:54:00.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.758+0000 7f8fd38e1980 20 remove_watcher() i=5 2024-04-17T04:54:00.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.758+0000 7f8fd38e1980 20 remove_watcher() i=6 2024-04-17T04:54:00.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:00.762+0000 7f8fd38e1980 20 remove_watcher() i=7 2024-04-17T04:54:00.771 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:00.771 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:00.771 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:54:00.771 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:01.580 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-04-17T04:54:01.581 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:03.849 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-17T04:54:03.881 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:03.881 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:03.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:03.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 realm 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:03.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:03.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:03.928 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.930+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:03.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.934+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:03.929 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.934+0000 7f0255fdd980 20 searching for the correct realm 2024-04-17T04:54:03.936 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:03.938+0000 7f02542f9640 0 -- 172.21.15.88:0/1568283479 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f0248027020 msgr2=0x7f0248075070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:04.151 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.154+0000 7f0253af8640 0 -- 172.21.15.88:0/1568283479 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x559b7498b710 msgr2=0x559b7498db00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:04.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:04.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:04.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:04.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:04.158 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:04.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:04.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:04.159 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.162+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:04.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 zone default found 2024-04-17T04:54:04.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 4 Realm: () 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 zonegroup default 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.161 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 started sync module instance, tier type = 2024-04-17T04:54:04.163 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.166+0000 7f0255fdd980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=0 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=1 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=2 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=3 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=4 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=5 2024-04-17T04:54:04.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.190+0000 7f0255fdd980 20 add_watcher() i=6 2024-04-17T04:54:04.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.194+0000 7f0255fdd980 20 add_watcher() i=7 2024-04-17T04:54:04.189 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.194+0000 7f0255fdd980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:04.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0109fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:04.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0255fdd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:04.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0255fdd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:04.194 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0255fdd980 5 note: GC not initialized 2024-04-17T04:54:04.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0108fe1640 20 reqs_thread_entry: start 2024-04-17T04:54:04.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0255fdd980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:04.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0255fdd980 20 Filter name: none 2024-04-17T04:54:04.195 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f00827fc640 20 reqs_thread_entry: start 2024-04-17T04:54:04.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.198+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:04.196 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.198 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:04.198 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:04.198 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:04.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:04.199 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.202+0000 7f0255fdd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:04.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.206+0000 7f0109fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:04.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.210+0000 7f0255fdd980 20 remove_watcher() i=0 2024-04-17T04:54:04.207 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.210+0000 7f0255fdd980 2 removed watcher, disabling cache 2024-04-17T04:54:04.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.214+0000 7f0255fdd980 20 remove_watcher() i=1 2024-04-17T04:54:04.213 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.218+0000 7f0255fdd980 20 remove_watcher() i=2 2024-04-17T04:54:04.215 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.218+0000 7f0255fdd980 20 remove_watcher() i=3 2024-04-17T04:54:04.217 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.222+0000 7f0255fdd980 20 remove_watcher() i=4 2024-04-17T04:54:04.218 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.222+0000 7f0255fdd980 20 remove_watcher() i=5 2024-04-17T04:54:04.220 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.222+0000 7f0255fdd980 20 remove_watcher() i=6 2024-04-17T04:54:04.221 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.226+0000 7f0255fdd980 20 remove_watcher() i=7 2024-04-17T04:54:04.236 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:04.236 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:04.237 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:04.237 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:04.237 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-04-17T04:54:04.271 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:04.271 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:04.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 20 realm 2024-04-17T04:54:04.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:04.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.318+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.322+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.322+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.322+0000 7eff31ce6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:04.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.322+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.322+0000 7eff31ce6980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:04.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.322+0000 7eff31ce6980 20 searching for the correct realm 2024-04-17T04:54:04.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.330+0000 7eff2f000640 0 -- 172.21.15.88:0/3812452685 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7eff200a83f0 msgr2=0x7eff200a8810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:04.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.334+0000 7eff30002640 0 -- 172.21.15.88:0/3812452685 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7eff18055a70 msgr2=0x7eff18059a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:04.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:04.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:04.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:04.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:04.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:04.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:04.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:04.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.342+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 zone default found 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 4 Realm: () 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 zonegroup default 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 started sync module instance, tier type = 2024-04-17T04:54:04.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.346+0000 7eff31ce6980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=0 2024-04-17T04:54:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=1 2024-04-17T04:54:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=2 2024-04-17T04:54:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=3 2024-04-17T04:54:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=4 2024-04-17T04:54:04.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=5 2024-04-17T04:54:04.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=6 2024-04-17T04:54:04.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 20 add_watcher() i=7 2024-04-17T04:54:04.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7eff31ce6980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:04.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.362+0000 7efde67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 5 note: GC not initialized 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7efde57e2640 20 reqs_thread_entry: start 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 Filter name: none 2024-04-17T04:54:04.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7efdde7fc640 20 reqs_thread_entry: start 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 : miss 2024-04-17T04:54:04.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 info.flags=0x17 2024-04-17T04:54:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 to cache LRU end 2024-04-17T04:54:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:04.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.366+0000 7eff31ce6980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 2024-04-17T04:54:04.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.370+0000 7eff31ce6980 -1 Injecting error=-125 at location=set_target_layout 2024-04-17T04:54:04.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7eff31ce6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 info.flags=0x17 2024-04-17T04:54:04.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7eff31ce6980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 to cache LRU end 2024-04-17T04:54:04.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7eff31ce6980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:04.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7eff31ce6980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21, ofs0, ns] 2024-04-17T04:54:04.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7efe12ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464298 cookie 94355365724672 notifier 6297 bl.length()=1001 2024-04-17T04:54:04.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7efe12ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 info.flags=0x17 2024-04-17T04:54:04.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7efe12ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 to cache LRU end 2024-04-17T04:54:04.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.374+0000 7efe12ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:04.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.378+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): orig_sources= new_sources= 2024-04-17T04:54:04.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.378+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): potential sources added=[] removed=[] 2024-04-17T04:54:04.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.378+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): orig_dests= new_dests= 2024-04-17T04:54:04.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.378+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): potential dests added=[] removed=[] 2024-04-17T04:54:04.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.382+0000 7eff31ce6980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21.1.0 2024-04-17T04:54:04.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.382+0000 7eff31ce6980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21.1.0 2024-04-17T04:54:04.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.382+0000 7eff31ce6980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21.1.1 2024-04-17T04:54:04.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.382+0000 7eff31ce6980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21.1.1 2024-04-17T04:54:04.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.382+0000 7eff31ce6980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21.0 2024-04-17T04:54:04.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.386+0000 7eff31ce6980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 : hit (requested=0x17, cached=0x17) 2024-04-17T04:54:04.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.386+0000 7eff31ce6980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 2024-04-17T04:54:04.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 info.flags=0x17 2024-04-17T04:54:04.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 to cache LRU end 2024-04-17T04:54:04.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:04.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21, ofs0, ns] 2024-04-17T04:54:04.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7efe127fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464299 cookie 94355365724672 notifier 6297 bl.length()=969 2024-04-17T04:54:04.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7efe127fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 info.flags=0x17 2024-04-17T04:54:04.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7efe127fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 to cache LRU end 2024-04-17T04:54:04.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7efe127fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:04.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): orig_sources= new_sources= 2024-04-17T04:54:04.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): potential sources added=[] removed=[] 2024-04-17T04:54:04.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): orig_dests= new_dests= 2024-04-17T04:54:04.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.390+0000 7eff31ce6980 20 handle_bi_update(): bucket=:error-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21]): potential dests added=[] removed=[] 2024-04-17T04:54:04.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.394+0000 7eff31ce6980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-04-17T04:54:04.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.398+0000 7efde67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:04.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.402+0000 7eff31ce6980 20 remove_watcher() i=0 2024-04-17T04:54:04.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.402+0000 7eff31ce6980 2 removed watcher, disabling cache 2024-04-17T04:54:04.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.402+0000 7eff31ce6980 20 remove_watcher() i=1 2024-04-17T04:54:04.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.406+0000 7eff31ce6980 20 remove_watcher() i=2 2024-04-17T04:54:04.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.406+0000 7eff31ce6980 20 remove_watcher() i=3 2024-04-17T04:54:04.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.410+0000 7eff31ce6980 20 remove_watcher() i=4 2024-04-17T04:54:04.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.410+0000 7eff31ce6980 20 remove_watcher() i=5 2024-04-17T04:54:04.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.414+0000 7eff31ce6980 20 remove_watcher() i=6 2024-04-17T04:54:04.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.414+0000 7eff31ce6980 20 remove_watcher() i=7 2024-04-17T04:54:04.425 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-17T04:54:04.452 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:04.452 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:04.495 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.498+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 realm 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:04.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:04.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.502+0000 7fadd7474980 20 searching for the correct realm 2024-04-17T04:54:04.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.510+0000 7fadd5790640 0 -- 172.21.15.88:0/664577287 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fadc804d920 msgr2=0x7fadc803a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:54:04.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.726+0000 7fadd4f8f640 0 -- 172.21.15.88:0/664577287 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55ec851e35e0 msgr2=0x55ec851e59f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:04.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:04.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 zone default found 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 4 Realm: () 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 zonegroup default 2024-04-17T04:54:04.732 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.734+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:04.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 started sync module instance, tier type = 2024-04-17T04:54:04.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.738+0000 7fadd7474980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:04.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=0 2024-04-17T04:54:04.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=1 2024-04-17T04:54:04.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=2 2024-04-17T04:54:04.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=3 2024-04-17T04:54:04.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=4 2024-04-17T04:54:04.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=5 2024-04-17T04:54:04.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=6 2024-04-17T04:54:04.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 20 add_watcher() i=7 2024-04-17T04:54:04.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.762+0000 7fadd7474980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:04.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.766+0000 7fac8b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:04.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.766+0000 7fadd7474980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:04.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.766+0000 7fadd7474980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:04.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.766+0000 7fadd7474980 5 note: GC not initialized 2024-04-17T04:54:04.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.766+0000 7fac8a7e4640 20 reqs_thread_entry: start 2024-04-17T04:54:04.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:04.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 Filter name: none 2024-04-17T04:54:04.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fac837fe640 20 reqs_thread_entry: start 2024-04-17T04:54:04.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:04.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:04.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:04.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:04.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.770+0000 7fadd7474980 20 rados->read ofs=0 len=0 2024-04-17T04:54:04.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.774+0000 7fadd7474980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:04.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.774+0000 7fadd7474980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:04.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.774+0000 7fac8b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:04.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.778+0000 7fadd7474980 20 remove_watcher() i=0 2024-04-17T04:54:04.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.778+0000 7fadd7474980 2 removed watcher, disabling cache 2024-04-17T04:54:04.779 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.782+0000 7fadd7474980 20 remove_watcher() i=1 2024-04-17T04:54:04.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.786+0000 7fadd7474980 20 remove_watcher() i=2 2024-04-17T04:54:04.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.786+0000 7fadd7474980 20 remove_watcher() i=3 2024-04-17T04:54:04.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.790+0000 7fadd7474980 20 remove_watcher() i=4 2024-04-17T04:54:04.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.790+0000 7fadd7474980 20 remove_watcher() i=5 2024-04-17T04:54:04.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.794+0000 7fadd7474980 20 remove_watcher() i=6 2024-04-17T04:54:04.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:04.794+0000 7fadd7474980 20 remove_watcher() i=7 2024-04-17T04:54:04.808 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:04.808 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:04.808 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21 num_objects: 20 size_kb: 1 num_shards: 2 2024-04-17T04:54:04.808 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:05.692 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with abort at set_target_layout 2024-04-17T04:54:05.692 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:07.927 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-04-17T04:54:07.959 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:07.959 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:08.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.006+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 realm 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:08.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.010+0000 7fe71c853980 20 searching for the correct realm 2024-04-17T04:54:08.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.018+0000 7fe71ab6f640 0 -- 172.21.15.88:0/1962906710 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fe710087040 msgr2=0x7fe710060310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.234+0000 7fe71a36e640 0 -- 172.21.15.88:0/1962906710 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55f216c5d6f0 msgr2=0x55f216c5fae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.239 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.242+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:08.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 zone default found 2024-04-17T04:54:08.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 4 Realm: () 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 zonegroup default 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 started sync module instance, tier type = 2024-04-17T04:54:08.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.246+0000 7fe71c853980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:08.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=0 2024-04-17T04:54:08.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=1 2024-04-17T04:54:08.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=2 2024-04-17T04:54:08.265 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=3 2024-04-17T04:54:08.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=4 2024-04-17T04:54:08.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=5 2024-04-17T04:54:08.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=6 2024-04-17T04:54:08.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 20 add_watcher() i=7 2024-04-17T04:54:08.267 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.270+0000 7fe71c853980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:08.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe5d0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:08.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe71c853980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:08.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe71c853980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:08.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe71c853980 5 note: GC not initialized 2024-04-17T04:54:08.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe54b7fe640 20 reqs_thread_entry: start 2024-04-17T04:54:08.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe71c853980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:08.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe71c853980 20 Filter name: none 2024-04-17T04:54:08.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe5497fa640 20 reqs_thread_entry: start 2024-04-17T04:54:08.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.274+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:08.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:08.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:08.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:08.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.278+0000 7fe71c853980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.282+0000 7fe71c853980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:08.276 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.282+0000 7fe71c853980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:08.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.282+0000 7fe5d0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:08.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.286+0000 7fe71c853980 20 remove_watcher() i=0 2024-04-17T04:54:08.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.286+0000 7fe71c853980 2 removed watcher, disabling cache 2024-04-17T04:54:08.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.290+0000 7fe71c853980 20 remove_watcher() i=1 2024-04-17T04:54:08.288 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.290+0000 7fe71c853980 20 remove_watcher() i=2 2024-04-17T04:54:08.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.294+0000 7fe71c853980 20 remove_watcher() i=3 2024-04-17T04:54:08.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.298+0000 7fe71c853980 20 remove_watcher() i=4 2024-04-17T04:54:08.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.298+0000 7fe71c853980 20 remove_watcher() i=5 2024-04-17T04:54:08.295 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.298+0000 7fe71c853980 20 remove_watcher() i=6 2024-04-17T04:54:08.297 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.302+0000 7fe71c853980 20 remove_watcher() i=7 2024-04-17T04:54:08.311 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.311 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:08.312 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:08.312 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.312 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-04-17T04:54:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:08.348 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:08.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.394+0000 7f10cb892980 20 realm 2024-04-17T04:54:08.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:08.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.398+0000 7f10cb892980 20 searching for the correct realm 2024-04-17T04:54:08.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.406+0000 7f10c9bae640 0 -- 172.21.15.88:0/1676808354 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f10b403d0b0 msgr2=0x7f10b40601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.622+0000 7f10c93ad640 0 -- 172.21.15.88:0/1676808354 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5651441cc320 msgr2=0x5651441ce710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.627 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 zone default found 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 4 Realm: () 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 zonegroup default 2024-04-17T04:54:08.629 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.631 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 started sync module instance, tier type = 2024-04-17T04:54:08.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.634+0000 7f10cb892980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:08.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=0 2024-04-17T04:54:08.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=1 2024-04-17T04:54:08.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=2 2024-04-17T04:54:08.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=3 2024-04-17T04:54:08.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=4 2024-04-17T04:54:08.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=5 2024-04-17T04:54:08.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=6 2024-04-17T04:54:08.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.658+0000 7f10cb892980 20 add_watcher() i=7 2024-04-17T04:54:08.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.658+0000 7f10cb892980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:08.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f0f7ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:08.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:08.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:08.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 5 note: GC not initialized 2024-04-17T04:54:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f0f7efe5640 20 reqs_thread_entry: start 2024-04-17T04:54:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 20 Filter name: none 2024-04-17T04:54:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f0f7cfe1640 20 reqs_thread_entry: start 2024-04-17T04:54:08.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-17T04:54:08.660 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.662+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:08.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-17T04:54:08.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-17T04:54:08.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 : miss 2024-04-17T04:54:08.661 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:08.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:08.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:08.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:08.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:08.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.666+0000 7f10cb892980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 2024-04-17T04:54:08.667 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f10cb892980 time 2024-04-17T04:54:08.675466+0000 2024-04-17T04:54:08.667 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:08.669 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.669 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f10cf169e1f] 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 4: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 6: main() 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 8: __libc_start_main() 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 9: _start() 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.674+0000 7f10cb892980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f10cb892980 time 2024-04-17T04:54:08.675466+0000 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.670 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f10cf169e1f] 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 4: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 6: main() 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 8: __libc_start_main() 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 9: _start() 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr:*** Caught signal (Aborted) ** 2024-04-17T04:54:08.671 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f10cb892980 thread_name:radosgw-admin 2024-04-17T04:54:08.703 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.703 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f10cdfeb520] 2024-04-17T04:54:08.703 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:08.703 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:08.703 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:08.703 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f10cf169e79] 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 8: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 10: main() 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 12: __libc_start_main() 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 13: _start() 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.674+0000 7f10cb892980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f10cb892980 thread_name:radosgw-admin 2024-04-17T04:54:08.704 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f10cdfeb520] 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f10cf169e79] 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 8: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 10: main() 2024-04-17T04:54:08.705 INFO:tasks.workunit.client.0.smithi088.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: 12: __libc_start_main() 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: 13: _start() 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.024509s 2024-04-17T04:54:08.706 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): get_auth_request con 0x565143f8e640 auth_method 0 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x565143f9ad10 auth_method 0 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.707 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x56514410ccf0 auth_method 0 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_done global_id 6035 payload 274 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:54:08.374+0000 7f10c8bac640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: dump: 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.708 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 monclient: authenticate success, global_id 6035 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.709 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:54:08.374+0000 7f10cb892980 4 set_mon_vals no callback set 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: dump: 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.710 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: shutdown 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 asok(0x565143ed0f40) unregister_commands rotate-key 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.711 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding auth protocol: cephx 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding auth protocol: cephx 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding auth protocol: cephx 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.712 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:54:08.374+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: starting msgr at 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: starting objecter 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command objecter_requests hook 0x565143f5be50 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: setting wanted keys 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: calling monclient init 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding auth protocol: cephx 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding auth protocol: cephx 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding auth protocol: cephx 2024-04-17T04:54:08.713 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:54:08.374+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:54:08.374+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.714 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key hook 0x5651440de078 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x565143f8e640 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x565143f9ad10 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x56514410ccf0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.027480s 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x565143f9ad10 auth_method 0 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.715 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x56514410ccf0 auth_method 0 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_done global_id 6038 payload 824 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:54:08.374+0000 7f10c9bae640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.716 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: dump: 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.717 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:54:08.378+0000 7f10cb892980 5 monclient: authenticate success, global_id 6038 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: dump: 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.718 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:54:08.378+0000 7f0fb7606640 4 set_mon_vals no callback set 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:54:08.378+0000 7f10cb892980 1 librados: init done 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:54:08.378+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f0fb0065e00 auth_method 0 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:54:08.378+0000 7f10c8bac640 10 monclient: get_auth_request con 0x56514411c320 auth_method 0 2024-04-17T04:54:08.719 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:54:08.378+0000 7f10c93ad640 10 monclient: get_auth_request con 0x565144129030 auth_method 0 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:54:08.382+0000 7f10c8bac640 10 monclient: get_auth_request con 0x56514412e0c0 auth_method 0 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:54:08.386+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.720 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding auth protocol: cephx 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding auth protocol: cephx 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding auth protocol: cephx 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.721 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:54:08.386+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command objecter_requests cmddesc objecter_requests hook 0x5651441376f0 EEXIST 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding auth protocol: cephx 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding auth protocol: cephx 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding auth protocol: cephx 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.722 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:54:08.386+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:54:08.386+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key cmddesc rotate-key hook 0x565144131210 EEXIST 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x56514415c330 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x56514415c710 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x5651441577f0 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.723 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.037250s 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x56514415c710 auth_method 0 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.724 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): get_auth_request con 0x56514415c330 auth_method 0 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_done global_id 6041 payload 824 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.725 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:54:08.386+0000 7f10c8bac640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: dump: 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.726 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 monclient: authenticate success, global_id 6041 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: dump: 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.727 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:54:08.386+0000 7f10c3fff640 4 set_mon_vals no callback set 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:54:08.386+0000 7f0faeffd640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:54:08.386+0000 7f0faeffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:54:08.386+0000 7f0faeffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.728 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f0fa0065d40 auth_method 0 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: get_monmap_and_config 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.729 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.730 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding auth protocol: cephx 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding auth protocol: cephx 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding auth protocol: cephx 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.731 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key cmddesc rotate-key hook 0x7ffcc995bc48 EEXIST 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x5651441ae590 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x5651441a9a80 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.732 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x5651441b36b0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.041279s 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x5651441b36b0 auth_method 0 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_done global_id 6351 payload 274 2024-04-17T04:54:08.733 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:54:08.390+0000 7f10c9bae640 1 monclient: found mon.noname-a 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: dump: 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.734 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 monclient: authenticate success, global_id 6351 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: dump: 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.735 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:54:08.390+0000 7f10cb892980 4 set_mon_vals no callback set 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: shutdown 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.736 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.737 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: starting msgr at 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: starting objecter 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command objecter_requests cmddesc objecter_requests hook 0x565144171f00 EEXIST 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: setting wanted keys 2024-04-17T04:54:08.738 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: calling monclient init 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding auth protocol: cephx 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding auth protocol: cephx 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding auth protocol: cephx 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.739 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key cmddesc rotate-key hook 0x5651441791b8 EEXIST 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x5651441a9a80 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x5651441ae590 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x5651441b36b0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.740 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.043843s 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): get_auth_request con 0x5651441a9a80 auth_method 0 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x5651441b36b0 auth_method 0 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.741 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x5651441ae590 auth_method 0 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_done global_id 6044 payload 824 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:54:08.394+0000 7f10c93ad640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: dump: 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.742 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: dump: 2024-04-17T04:54:08.743 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:54:08.394+0000 7f10cb892980 5 monclient: authenticate success, global_id 6044 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:54:08.394+0000 7f0facff9640 4 set_mon_vals no callback set 2024-04-17T04:54:08.744 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:54:08.394+0000 7f10cb892980 1 librados: init done 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:54:08.394+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cr dump hook 0x5651441a4fe8 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:54:08.394+0000 7f0f9effd640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:54:08.394+0000 7f0f9effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:54:08.394+0000 7f0f9effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient: get_auth_request con 0x7f0f94065e00 auth_method 0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient: get_auth_request con 0x5651441c7f50 auth_method 0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 realm 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.745 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:54:08.398+0000 7f10cb892980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:54:08.398+0000 7f10c93ad640 10 monclient: get_auth_request con 0x5651441cc320 auth_method 0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 searching for the correct realm 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:54:08.398+0000 7f10c8bac640 10 monclient: get_auth_request con 0x5651441d1470 auth_method 0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:54:08.398+0000 7f10c9bae640 10 monclient: get_auth_request con 0x7f10b4015070 auth_method 0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:54:08.402+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f10c40f8b90 auth_method 0 2024-04-17T04:54:08.746 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:54:08.402+0000 7f10c8bac640 10 monclient: get_auth_request con 0x7f10bc02d070 auth_method 0 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:54:08.406+0000 7f10c9bae640 0 -- 172.21.15.88:0/1676808354 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f10b403d0b0 msgr2=0x7f10b40601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:54:08.406+0000 7f10c9bae640 10 monclient: get_auth_request con 0x7f10b403d0b0 auth_method 0 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:54:08.610+0000 7f10c9bae640 10 monclient: get_auth_request con 0x7f10b403d0b0 auth_method 0 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:54:08.618+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f10b40606e0 auth_method 0 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:54:08.622+0000 7f10c93ad640 0 -- 172.21.15.88:0/1676808354 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5651441cc320 msgr2=0x5651441ce710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:54:08.622+0000 7f0f9effd640 1 client.6044.objecter ms_handle_reset 0x5651441cc320 session 0x5651441cb1a0 osd.5 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:54:08.622+0000 7f0f9effd640 10 monclient: _renew_subs 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:54:08.622+0000 7f0f9effd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:54:08.622+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f0f940763a0 auth_method 0 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:08.747 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 zone default found 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:54:08.634+0000 7f10cb892980 4 Realm: () 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:54:08.634+0000 7f10cb892980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:54:08.634+0000 7f10cb892980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:54:08.634+0000 7f10cb892980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 zonegroup default 2024-04-17T04:54:08.748 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 started sync module instance, tier type = 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=0 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=1 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=2 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=3 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=4 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=5 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=6 2024-04-17T04:54:08.749 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:54:08.658+0000 7f10cb892980 20 add_watcher() i=7 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:54:08.658+0000 7f10cb892980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding auth protocol: cephx 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding auth protocol: cephx 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:54:08.662+0000 7f0f7ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding auth protocol: cephx 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.750 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:54:08.662+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache list hook 0x5651441a5e50 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache inspect hook 0x5651441a5e50 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache erase hook 0x5651441a5e50 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache zap hook 0x5651441a5e50 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 note: GC not initialized 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:54:08.662+0000 7f0f7efe5640 20 reqs_thread_entry: start 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace show hook 0x7f10c40043b0 2024-04-17T04:54:08.751 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace history hook 0x7f10c40043b0 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace active hook 0x7f10c40043b0 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace active_short hook 0x7f10c40043b0 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 Filter name: none 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:54:08.662+0000 7f0f7cfe1640 20 reqs_thread_entry: start 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:54:08.662+0000 7f10cb892980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 : miss 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.752 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:54:08.674+0000 7f10cb892980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f10cb892980 time 2024-04-17T04:54:08.675466+0000 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f10cf169e1f] 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: 4: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.753 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 6: main() 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 8: __libc_start_main() 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 9: _start() 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:54:08.674+0000 7f10cb892980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f10cb892980 thread_name:radosgw-admin 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f10cdfeb520] 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:08.754 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f10cf169e79] 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 8: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 10: main() 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 12: __libc_start_main() 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 13: _start() 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:54:08.755 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:54:08.756 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:54:08.757 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:54:08.758 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:54:08.759 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:54:08.760 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:54:08.761 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f7cfe1640 / http_manager 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f7efe5640 / http_manager 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f7ffe7640 / rgw_dt_lg_renew 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f9effd640 / ms_dispatch 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0facff9640 / io_context_pool 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0faeffd640 / ms_dispatch 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0fb5e03640 / ms_dispatch 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0fb7606640 / io_context_pool 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c3fff640 / io_context_pool 2024-04-17T04:54:08.762 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c8bac640 / msgr-worker-2 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c93ad640 / msgr-worker-1 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c9bae640 / msgr-worker-0 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10cb892980 / radosgw-admin 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/log/ceph/ceph-client.admin.82322.log 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.763 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.024509s 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): get_auth_request con 0x565143f8e640 auth_method 0 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x565143f9ad10 auth_method 0 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:54:08.374+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.764 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x56514410ccf0 auth_method 0 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_done global_id 6035 payload 274 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:54:08.374+0000 7f10c8bac640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: dump: 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.765 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 monclient: authenticate success, global_id 6035 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.766 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:54:08.374+0000 7f10cb892980 4 set_mon_vals no callback set 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:54:08.374+0000 7f0fb7606640 10 monclient: dump: 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: shutdown 2024-04-17T04:54:08.767 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 asok(0x565143ed0f40) unregister_commands rotate-key 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.768 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding auth protocol: cephx 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding auth protocol: cephx 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding auth protocol: cephx 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: crc 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440dfd70) adding con mode: secure 2024-04-17T04:54:08.769 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:54:08.374+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: starting msgr at 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: starting objecter 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command objecter_requests hook 0x565143f5be50 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: setting wanted keys 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:54:08.374+0000 7f10cb892980 1 librados: calling monclient init 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding auth protocol: cephx 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding auth protocol: cephx 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding auth protocol: cephx 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.770 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: crc 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 AuthRegistry(0x5651440ddf30) adding con mode: secure 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:54:08.374+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:54:08.374+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:54:08.374+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key hook 0x5651440de078 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:54:08.771 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x565143f8e640 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x565143f9ad10 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x56514410ccf0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:54:08.374+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.027480s 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x565143f9ad10 auth_method 0 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x56514410ccf0 auth_method 0 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.772 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:54:08.374+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient(hunting): handle_auth_done global_id 6038 payload 824 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:54:08.374+0000 7f10c9bae640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:54:08.374+0000 7f10c9bae640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.773 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: dump: 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.774 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:54:08.378+0000 7f10cb892980 5 monclient: authenticate success, global_id 6038 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 10 monclient: dump: 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.775 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:54:08.378+0000 7f0fb7606640 4 set_mon_vals no callback set 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:54:08.378+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:54:08.378+0000 7f10cb892980 1 librados: init done 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:54:08.378+0000 7f0fb5e03640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:54:08.378+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f0fb0065e00 auth_method 0 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:54:08.378+0000 7f10c8bac640 10 monclient: get_auth_request con 0x56514411c320 auth_method 0 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:54:08.378+0000 7f10c93ad640 10 monclient: get_auth_request con 0x565144129030 auth_method 0 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:54:08.382+0000 7f10c8bac640 10 monclient: get_auth_request con 0x56514412e0c0 auth_method 0 2024-04-17T04:54:08.776 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:54:08.386+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding auth protocol: cephx 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding auth protocol: cephx 2024-04-17T04:54:08.777 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding auth protocol: cephx 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: crc 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441321a0) adding con mode: secure 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:54:08.386+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.778 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command objecter_requests cmddesc objecter_requests hook 0x5651441376f0 EEXIST 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding auth protocol: cephx 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding auth protocol: cephx 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding auth protocol: cephx 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.779 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: crc 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 AuthRegistry(0x5651441310c8) adding con mode: secure 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:54:08.386+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:54:08.386+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key cmddesc rotate-key hook 0x565144131210 EEXIST 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x56514415c330 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x56514415c710 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x5651441577f0 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.780 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.037250s 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x56514415c710 auth_method 0 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): get_auth_request con 0x56514415c330 auth_method 0 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.781 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient(hunting): handle_auth_done global_id 6041 payload 824 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:54:08.386+0000 7f10c8bac640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:54:08.386+0000 7f10c8bac640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: dump: 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.782 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:54:08.386+0000 7f10cb892980 5 monclient: authenticate success, global_id 6041 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:54:08.386+0000 7f0faeffd640 10 monclient: dump: 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.783 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:54:08.386+0000 7f10c3fff640 4 set_mon_vals no callback set 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:54:08.386+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:54:08.386+0000 7f0faeffd640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:54:08.386+0000 7f0faeffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.784 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:54:08.386+0000 7f0faeffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:54:08.386+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f0fa0065d40 auth_method 0 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: get_monmap_and_config 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.785 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.786 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding auth protocol: cephx 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding auth protocol: cephx 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding auth protocol: cephx 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.787 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: crc 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bb00) adding con mode: secure 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key cmddesc rotate-key hook 0x7ffcc995bc48 EEXIST 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x5651441ae590 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x5651441a9a80 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x5651441b36b0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.788 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.041279s 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x5651441b36b0 auth_method 0 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient(hunting): handle_auth_done global_id 6351 payload 274 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:54:08.390+0000 7f10c9bae640 1 monclient: found mon.noname-a 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:54:08.390+0000 7f10c9bae640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:08.789 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: dump: 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.790 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 monclient: authenticate success, global_id 6351 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:54:08.390+0000 7f0facff9640 10 monclient: dump: 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.791 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:54:08.390+0000 7f10cb892980 4 set_mon_vals no callback set 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: shutdown 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: build_initial_monmap 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 build_initial for_mkfs: 0 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: monmap: 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:08.792 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding auth protocol: cephx 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.793 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: crc 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179ea0) adding con mode: secure 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: starting msgr at 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: starting objecter 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command objecter_requests cmddesc objecter_requests hook 0x565144171f00 EEXIST 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: setting wanted keys 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:54:08.390+0000 7f10cb892980 1 librados: calling monclient init 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: init 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding auth protocol: cephx 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding auth protocol: cephx 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding auth protocol: cephx 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.794 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: crc 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 AuthRegistry(0x565144179070) adding con mode: secure 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:54:08.390+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:54:08.390+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command rotate-key cmddesc rotate-key hook 0x5651441791b8 EEXIST 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:08.795 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-c con 0x5651441a9a80 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-b con 0x5651441ae590 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): picked mon.noname-a con 0x5651441b36b0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:54:08.390+0000 7f10cb892980 10 monclient(hunting): authenticate will time out at 2050.043843s 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): get_auth_request con 0x5651441a9a80 auth_method 0 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:54:08.390+0000 7f10c93ad640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): get_auth_request con 0x5651441b36b0 auth_method 0 2024-04-17T04:54:08.796 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): get_auth_request con 0x5651441ae590 auth_method 0 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient(hunting): handle_auth_done global_id 6044 payload 824 2024-04-17T04:54:08.797 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient: _finish_hunting 0 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:54:08.394+0000 7f10c93ad640 1 monclient: found mon.noname-c 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:54:08.394+0000 7f10c93ad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: dump: 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.798 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: _finish_auth 0 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: _check_auth_tickets 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:54:08.394+0000 7f0f9effd640 10 monclient: dump: 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:08.799 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:54:08.394+0000 7f10cb892980 5 monclient: authenticate success, global_id 6044 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:54:08.394+0000 7f0facff9640 4 set_mon_vals no callback set 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _renew_subs 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:54:08.394+0000 7f10cb892980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:54:08.394+0000 7f10cb892980 1 librados: init done 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:54:08.394+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cr dump hook 0x5651441a4fe8 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.800 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:54:08.394+0000 7f0f9effd640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:54:08.394+0000 7f0f9effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:54:08.394+0000 7f0f9effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:54:08.394+0000 7f10c8bac640 10 monclient: get_auth_request con 0x7f0f94065e00 auth_method 0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:54:08.394+0000 7f10c9bae640 10 monclient: get_auth_request con 0x5651441c7f50 auth_method 0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 realm 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:54:08.394+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:54:08.398+0000 7f10cb892980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:54:08.398+0000 7f10c93ad640 10 monclient: get_auth_request con 0x5651441cc320 auth_method 0 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.801 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:54:08.398+0000 7f10cb892980 20 searching for the correct realm 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:54:08.398+0000 7f10c8bac640 10 monclient: get_auth_request con 0x5651441d1470 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:54:08.398+0000 7f10c9bae640 10 monclient: get_auth_request con 0x7f10b4015070 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:54:08.402+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f10c40f8b90 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:54:08.402+0000 7f10c8bac640 10 monclient: get_auth_request con 0x7f10bc02d070 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:54:08.406+0000 7f10c9bae640 0 -- 172.21.15.88:0/1676808354 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f10b403d0b0 msgr2=0x7f10b40601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:54:08.406+0000 7f10c9bae640 10 monclient: get_auth_request con 0x7f10b403d0b0 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:54:08.610+0000 7f10c9bae640 10 monclient: get_auth_request con 0x7f10b403d0b0 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:54:08.618+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f10b40606e0 auth_method 0 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:54:08.622+0000 7f10c93ad640 0 -- 172.21.15.88:0/1676808354 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5651441cc320 msgr2=0x5651441ce710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:54:08.622+0000 7f0f9effd640 1 client.6044.objecter ms_handle_reset 0x5651441cc320 session 0x5651441cb1a0 osd.5 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:54:08.622+0000 7f0f9effd640 10 monclient: _renew_subs 2024-04-17T04:54:08.802 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:54:08.622+0000 7f0f9effd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:54:08.622+0000 7f10c93ad640 10 monclient: get_auth_request con 0x7f0f940763a0 auth_method 0 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:54:08.630+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:08.803 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 zone default found 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:54:08.634+0000 7f10cb892980 4 Realm: () 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:54:08.634+0000 7f10cb892980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:54:08.634+0000 7f10cb892980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:54:08.634+0000 7f10cb892980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 zonegroup default 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 started sync module instance, tier type = 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:54:08.634+0000 7f10cb892980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:08.804 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=0 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=1 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=2 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=3 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=4 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=5 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:54:08.654+0000 7f10cb892980 20 add_watcher() i=6 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:54:08.658+0000 7f10cb892980 20 add_watcher() i=7 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:54:08.658+0000 7f10cb892980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding auth protocol: cephx 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding auth protocol: cephx 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:54:08.662+0000 7f0f7ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding auth protocol: cephx 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.805 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: crc 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 AuthRegistry(0x7ffcc995bf90) adding con mode: secure 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:54:08.662+0000 7f10cb892980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:08.806 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache list hook 0x5651441a5e50 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache inspect hook 0x5651441a5e50 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache erase hook 0x5651441a5e50 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command cache zap hook 0x5651441a5e50 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 note: GC not initialized 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:54:08.662+0000 7f0f7efe5640 20 reqs_thread_entry: start 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace show hook 0x7f10c40043b0 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace history hook 0x7f10c40043b0 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace active hook 0x7f10c40043b0 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:54:08.662+0000 7f10cb892980 5 asok(0x565143ed0f40) register_command sync trace active_short hook 0x7f10c40043b0 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 Filter name: none 2024-04-17T04:54:08.807 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:54:08.662+0000 7f0f7cfe1640 20 reqs_thread_entry: start 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:54:08.662+0000 7f10cb892980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:54:08.662+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 : miss 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:54:08.666+0000 7f10cb892980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:54:08.666+0000 7f10cb892980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 2024-04-17T04:54:08.808 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:54:08.674+0000 7f10cb892980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f10cb892980 time 2024-04-17T04:54:08.675466+0000 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f10cf169e1f] 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 4: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 6: main() 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 8: __libc_start_main() 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 9: _start() 2024-04-17T04:54:08.809 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:54:08.674+0000 7f10cb892980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f10cb892980 thread_name:radosgw-admin 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f10cdfeb520] 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f10cf169e79] 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f10cf169fd3] 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56514266c191] 2024-04-17T04:54:08.810 INFO:tasks.workunit.client.0.smithi088.stderr: 8: radosgw-admin(+0x8f597a) [0x56514267597a] 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56514267a93b] 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 10: main() 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f10cdfd2d90] 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 12: __libc_start_main() 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 13: _start() 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:54:08.811 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:54:08.812 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:54:08.813 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:54:08.814 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:54:08.815 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:54:08.816 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f7cfe1640 / http_manager 2024-04-17T04:54:08.817 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f7efe5640 / http_manager 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f7ffe7640 / rgw_dt_lg_renew 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0f9effd640 / ms_dispatch 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0facff9640 / io_context_pool 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0faeffd640 / ms_dispatch 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0fb5e03640 / ms_dispatch 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f0fb7606640 / io_context_pool 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c3fff640 / io_context_pool 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c8bac640 / msgr-worker-2 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c93ad640 / msgr-worker-1 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10c9bae640 / msgr-worker-0 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: 7f10cb892980 / radosgw-admin 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:54:08.818 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/lib/ceph/crash/2024-04-17T04:54:08.680259Z_8e23145a-9aca-4538-82e4-37bc6f601f1b/log 2024-04-17T04:54:08.819 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:54:08.819 INFO:tasks.workunit.client.0.smithi088.stderr:Aborted 2024-04-17T04:54:08.819 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-04-17T04:54:08.820 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:08.820 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.758+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 realm 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.762+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.766+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.766+0000 7f6c67fdf980 20 searching for the correct realm 2024-04-17T04:54:08.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.770+0000 7f6c662fb640 0 -- 172.21.15.88:0/1483421673 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f6c5810e0d0 msgr2=0x7f6c580fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.986+0000 7f6c65afa640 0 -- 172.21.15.88:0/1483421673 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55ede41b4400 msgr2=0x55ede41b67f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:08.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:08.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:08.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:08.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:08.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 zone default found 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 4 Realm: () 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 zonegroup default 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.994+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:08.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:08.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 started sync module instance, tier type = 2024-04-17T04:54:08.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:08.998+0000 7f6c67fdf980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:09.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=0 2024-04-17T04:54:09.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=1 2024-04-17T04:54:09.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=2 2024-04-17T04:54:09.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=3 2024-04-17T04:54:09.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=4 2024-04-17T04:54:09.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=5 2024-04-17T04:54:09.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.022+0000 7f6c67fdf980 20 add_watcher() i=6 2024-04-17T04:54:09.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.026+0000 7f6c67fdf980 20 add_watcher() i=7 2024-04-17T04:54:09.021 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.026+0000 7f6c67fdf980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:09.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6b177fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:09.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6c67fdf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:09.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6c67fdf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:09.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6c67fdf980 5 note: GC not initialized 2024-04-17T04:54:09.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6b167fc640 20 reqs_thread_entry: start 2024-04-17T04:54:09.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6c67fdf980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:09.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6c67fdf980 20 Filter name: none 2024-04-17T04:54:09.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6a87fff640 20 reqs_thread_entry: start 2024-04-17T04:54:09.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.030+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:09.028 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:09.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:09.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:09.030 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:09.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.034+0000 7f6c67fdf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:09.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.038+0000 7f6b177fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:09.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.042+0000 7f6c67fdf980 20 remove_watcher() i=0 2024-04-17T04:54:09.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.042+0000 7f6c67fdf980 2 removed watcher, disabling cache 2024-04-17T04:54:09.041 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.046+0000 7f6c67fdf980 20 remove_watcher() i=1 2024-04-17T04:54:09.043 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.046+0000 7f6c67fdf980 20 remove_watcher() i=2 2024-04-17T04:54:09.045 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.050+0000 7f6c67fdf980 20 remove_watcher() i=3 2024-04-17T04:54:09.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.050+0000 7f6c67fdf980 20 remove_watcher() i=4 2024-04-17T04:54:09.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.054+0000 7f6c67fdf980 20 remove_watcher() i=5 2024-04-17T04:54:09.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.054+0000 7f6c67fdf980 20 remove_watcher() i=6 2024-04-17T04:54:09.052 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.058+0000 7f6c67fdf980 20 remove_watcher() i=7 2024-04-17T04:54:09.069 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:09.069 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:09.070 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:09.070 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:09.190 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:54:09.225 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:09.225 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:09.270 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 20 realm 2024-04-17T04:54:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.274+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.278+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.278+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.278+0000 7fb8c31b2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:09.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.278+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.278+0000 7fb8c31b2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:09.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.278+0000 7fb8c31b2980 20 searching for the correct realm 2024-04-17T04:54:09.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.286+0000 7fb8c14ce640 0 -- 172.21.15.88:0/801995835 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fb8a8085c20 msgr2=0x7fb8a807c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:54:09.498 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.502+0000 7fb8c0ccd640 0 -- 172.21.15.88:0/801995835 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55a10cfd3a60 msgr2=0x55a10cfd5e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:09.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.506 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:09.507 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 zone default found 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 4 Realm: () 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:09.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 zonegroup default 2024-04-17T04:54:09.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.510+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:09.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 started sync module instance, tier type = 2024-04-17T04:54:09.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.514+0000 7fb8c31b2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=0 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=1 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=2 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=3 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=4 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=5 2024-04-17T04:54:09.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.538+0000 7fb8c31b2980 20 add_watcher() i=6 2024-04-17T04:54:09.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.542+0000 7fb8c31b2980 20 add_watcher() i=7 2024-04-17T04:54:09.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.542+0000 7fb8c31b2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:09.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.546+0000 7fb776fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:09.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.546+0000 7fb8c31b2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:09.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.546+0000 7fb8c31b2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:09.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.546+0000 7fb8c31b2980 5 note: GC not initialized 2024-04-17T04:54:09.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.546+0000 7fb775fe3640 20 reqs_thread_entry: start 2024-04-17T04:54:09.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:09.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 20 Filter name: none 2024-04-17T04:54:09.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb6e77fe640 20 reqs_thread_entry: start 2024-04-17T04:54:09.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-17T04:54:09.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:09.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-17T04:54:09.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-17T04:54:09.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 : miss 2024-04-17T04:54:09.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:09.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:09.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:09.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:09.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:09.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:09.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.550+0000 7fb8c31b2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 2024-04-17T04:54:09.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.554+0000 7fb8c31b2980 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to be held by another RGW process; skipping for now 2024-04-17T04:54:09.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.554+0000 7fb776fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:09.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.558+0000 7fb8c31b2980 20 remove_watcher() i=0 2024-04-17T04:54:09.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.558+0000 7fb8c31b2980 2 removed watcher, disabling cache 2024-04-17T04:54:09.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.562+0000 7fb8c31b2980 20 remove_watcher() i=1 2024-04-17T04:54:09.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.566+0000 7fb8c31b2980 20 remove_watcher() i=2 2024-04-17T04:54:09.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.566+0000 7fb8c31b2980 20 remove_watcher() i=3 2024-04-17T04:54:09.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.570+0000 7fb8c31b2980 20 remove_watcher() i=4 2024-04-17T04:54:09.566 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.570+0000 7fb8c31b2980 20 remove_watcher() i=5 2024-04-17T04:54:09.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.570+0000 7fb8c31b2980 20 remove_watcher() i=6 2024-04-17T04:54:09.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:09.574+0000 7fb8c31b2980 20 remove_watcher() i=7 2024-04-17T04:54:09.584 INFO:tasks.workunit.client.0.smithi088.stderr:waiting 30 seconds for reshard lock to expire... 2024-04-17T04:54:39.598 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:54:39.634 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:39.635 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:39.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.682+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 realm 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:39.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:39.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.686+0000 7f83792de980 20 searching for the correct realm 2024-04-17T04:54:39.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.694+0000 7f83765f8640 0 -- 172.21.15.88:0/3663022293 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x56293298a3d0 msgr2=0x56293298c890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:39.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.698+0000 7f8376df9640 0 -- 172.21.15.88:0/3663022293 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f8360082bb0 msgr2=0x7f8360088640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.706+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:39.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 zone default found 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 4 Realm: () 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 zonegroup default 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 started sync module instance, tier type = 2024-04-17T04:54:39.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.710+0000 7f83792de980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:39.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=0 2024-04-17T04:54:39.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=1 2024-04-17T04:54:39.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=2 2024-04-17T04:54:39.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=3 2024-04-17T04:54:39.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=4 2024-04-17T04:54:39.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=5 2024-04-17T04:54:39.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=6 2024-04-17T04:54:39.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 20 add_watcher() i=7 2024-04-17T04:54:39.723 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.726+0000 7f83792de980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:39.726 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f822dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 5 note: GC not initialized 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f822cfe1640 20 reqs_thread_entry: start 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 Filter name: none 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f8225ffb640 20 reqs_thread_entry: start 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-17T04:54:39.727 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:39.728 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 : miss 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.730+0000 7f83792de980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.734+0000 7f83792de980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.734+0000 7f83792de980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.734+0000 7f83792de980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.734+0000 7f83792de980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.734+0000 7f83792de980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:39.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.734+0000 7f83792de980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f83792de980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f83792de980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f83792de980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f83792de980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22, ofs0, ns] 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f825a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464265 cookie 94734942533776 notifier 6411 bl.length()=1001 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f825a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f825a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:39.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.738+0000 7f825a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:39.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.742+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): orig_sources= new_sources= 2024-04-17T04:54:39.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.742+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): potential sources added=[] removed=[] 2024-04-17T04:54:39.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.742+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): orig_dests= new_dests= 2024-04-17T04:54:39.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.742+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): potential dests added=[] removed=[] 2024-04-17T04:54:39.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.746+0000 7f83792de980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22.1.0 2024-04-17T04:54:39.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.746+0000 7f83792de980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22.1.0 2024-04-17T04:54:39.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.746+0000 7f83792de980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22.1.1 2024-04-17T04:54:39.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.746+0000 7f83792de980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22.1.1 2024-04-17T04:54:39.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.746+0000 7f83792de980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22.0 2024-04-17T04:54:39.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.750+0000 7f83792de980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 : hit (requested=0x17, cached=0x17) 2024-04-17T04:54:39.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.750+0000 7f83792de980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 2024-04-17T04:54:39.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22, ofs0, ns] 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f8259ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464266 cookie 94734942533776 notifier 6411 bl.length()=969 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f8259ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 info.flags=0x17 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f8259ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 to cache LRU end 2024-04-17T04:54:39.749 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f8259ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:39.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): orig_sources= new_sources= 2024-04-17T04:54:39.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): potential sources added=[] removed=[] 2024-04-17T04:54:39.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): orig_dests= new_dests= 2024-04-17T04:54:39.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.754+0000 7f83792de980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22]): potential dests added=[] removed=[] 2024-04-17T04:54:39.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.758+0000 7f83792de980 1 execute INFO: reshard of bucket "abort-at-set-target-layout" completed successfully 2024-04-17T04:54:39.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.762+0000 7f822dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:39.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.766+0000 7f83792de980 20 remove_watcher() i=0 2024-04-17T04:54:39.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.766+0000 7f83792de980 2 removed watcher, disabling cache 2024-04-17T04:54:39.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.766+0000 7f83792de980 20 remove_watcher() i=1 2024-04-17T04:54:39.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.770+0000 7f83792de980 20 remove_watcher() i=2 2024-04-17T04:54:39.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.774+0000 7f83792de980 20 remove_watcher() i=3 2024-04-17T04:54:39.770 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.774+0000 7f83792de980 20 remove_watcher() i=4 2024-04-17T04:54:39.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.774+0000 7f83792de980 20 remove_watcher() i=5 2024-04-17T04:54:39.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.778+0000 7f83792de980 20 remove_watcher() i=6 2024-04-17T04:54:39.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.778+0000 7f83792de980 20 remove_watcher() i=7 2024-04-17T04:54:39.788 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-04-17T04:54:39.814 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:39.814 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:39.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 realm 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.862+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.866+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:39.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.866+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.866+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:39.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.866+0000 7f9c98784980 20 searching for the correct realm 2024-04-17T04:54:39.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.870+0000 7f9c96aa0640 0 -- 172.21.15.88:0/3547612024 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x558eb6eb1f40 msgr2=0x558eb6eb4360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:39.873 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.878+0000 7f9c95a9e640 0 -- 172.21.15.88:0/3547612024 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f9c8804b040 msgr2=0x7f9c8804b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.882+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:39.881 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 zone default found 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 4 Realm: () 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 zonegroup default 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.882 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:39.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 started sync module instance, tier type = 2024-04-17T04:54:39.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.886+0000 7f9c98784980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:39.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=0 2024-04-17T04:54:39.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=1 2024-04-17T04:54:39.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=2 2024-04-17T04:54:39.898 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=3 2024-04-17T04:54:39.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=4 2024-04-17T04:54:39.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=5 2024-04-17T04:54:39.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=6 2024-04-17T04:54:39.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 20 add_watcher() i=7 2024-04-17T04:54:39.899 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.902+0000 7f9c98784980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:39.902 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9b4cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 5 note: GC not initialized 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9ac77fe640 20 reqs_thread_entry: start 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 Filter name: none 2024-04-17T04:54:39.903 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9ac57fa640 20 reqs_thread_entry: start 2024-04-17T04:54:39.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:39.904 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.906+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:39.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:39.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.905 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=218 2024-04-17T04:54:39.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 rados->read ofs=0 len=0 2024-04-17T04:54:39.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 rados_obj.operate() r=0 bl.length=432 2024-04-17T04:54:39.906 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.910+0000 7f9c98784980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:39.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.914+0000 7f9b4cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:39.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.918+0000 7f9c98784980 20 remove_watcher() i=0 2024-04-17T04:54:39.913 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.918+0000 7f9c98784980 2 removed watcher, disabling cache 2024-04-17T04:54:39.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.918+0000 7f9c98784980 20 remove_watcher() i=1 2024-04-17T04:54:39.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.922+0000 7f9c98784980 20 remove_watcher() i=2 2024-04-17T04:54:39.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.922+0000 7f9c98784980 20 remove_watcher() i=3 2024-04-17T04:54:39.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.926+0000 7f9c98784980 20 remove_watcher() i=4 2024-04-17T04:54:39.922 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.926+0000 7f9c98784980 20 remove_watcher() i=5 2024-04-17T04:54:39.923 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.926+0000 7f9c98784980 20 remove_watcher() i=6 2024-04-17T04:54:39.924 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:39.926+0000 7f9c98784980 20 remove_watcher() i=7 2024-04-17T04:54:39.937 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:39.937 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:39.937 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-set-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:54:39.938 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:40.743 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-04-17T04:54:40.743 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:42.996 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-04-17T04:54:43.027 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:43.027 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:43.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 realm 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.076 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.078+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.082+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:43.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.082+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.082+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:43.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.082+0000 7f1a818d8980 20 searching for the correct realm 2024-04-17T04:54:43.085 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.090+0000 7f1a7fbf4640 0 -- 172.21.15.88:0/358710988 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f1a7410a740 msgr2=0x7f1a740cc390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:43.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.306+0000 7f1a7f3f3640 0 -- 172.21.15.88:0/358710988 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55b24b8728d0 msgr2=0x55b24b874cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:43.311 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.312 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.314+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:43.313 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 zone default found 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 4 Realm: () 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 zonegroup default 2024-04-17T04:54:43.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.315 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 started sync module instance, tier type = 2024-04-17T04:54:43.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.318+0000 7f1a818d8980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:43.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.342+0000 7f1a818d8980 20 add_watcher() i=0 2024-04-17T04:54:43.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.342+0000 7f1a818d8980 20 add_watcher() i=1 2024-04-17T04:54:43.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.342+0000 7f1a818d8980 20 add_watcher() i=2 2024-04-17T04:54:43.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.342+0000 7f1a818d8980 20 add_watcher() i=3 2024-04-17T04:54:43.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.342+0000 7f1a818d8980 20 add_watcher() i=4 2024-04-17T04:54:43.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.342+0000 7f1a818d8980 20 add_watcher() i=5 2024-04-17T04:54:43.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.346+0000 7f1a818d8980 20 add_watcher() i=6 2024-04-17T04:54:43.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.346+0000 7f1a818d8980 20 add_watcher() i=7 2024-04-17T04:54:43.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.346+0000 7f1a818d8980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:43.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.350+0000 7f1935fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:43.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.350+0000 7f1a818d8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:43.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.350+0000 7f1a818d8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:43.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.350+0000 7f1a818d8980 5 note: GC not initialized 2024-04-17T04:54:43.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.350+0000 7f1934fe1640 20 reqs_thread_entry: start 2024-04-17T04:54:43.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:43.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 Filter name: none 2024-04-17T04:54:43.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f18ae7fc640 20 reqs_thread_entry: start 2024-04-17T04:54:43.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:43.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:43.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:43.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.354+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.358+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:43.352 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.358+0000 7f1a818d8980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.358+0000 7f1a818d8980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:43.353 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.358+0000 7f1a818d8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:43.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.358+0000 7f1935fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:43.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.366+0000 7f1a818d8980 20 remove_watcher() i=0 2024-04-17T04:54:43.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.366+0000 7f1a818d8980 2 removed watcher, disabling cache 2024-04-17T04:54:43.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.366+0000 7f1a818d8980 20 remove_watcher() i=1 2024-04-17T04:54:43.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.370+0000 7f1a818d8980 20 remove_watcher() i=2 2024-04-17T04:54:43.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.370+0000 7f1a818d8980 20 remove_watcher() i=3 2024-04-17T04:54:43.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.374+0000 7f1a818d8980 20 remove_watcher() i=4 2024-04-17T04:54:43.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.374+0000 7f1a818d8980 20 remove_watcher() i=5 2024-04-17T04:54:43.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.374+0000 7f1a818d8980 20 remove_watcher() i=6 2024-04-17T04:54:43.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.378+0000 7f1a818d8980 20 remove_watcher() i=7 2024-04-17T04:54:43.390 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:43.390 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:43.390 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-block-writes id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:43.390 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:43.390 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-04-17T04:54:43.423 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:43.423 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:43.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.470+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 realm 2024-04-17T04:54:43.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:43.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.471 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:43.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.474+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.478+0000 7f198bc26980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:43.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.478+0000 7f198bc26980 20 searching for the correct realm 2024-04-17T04:54:43.480 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.486+0000 7f1989f42640 0 -- 172.21.15.88:0/2515128376 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f1980008040 msgr2=0x7f19800678d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:43.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.702+0000 7f1989741640 0 -- 172.21.15.88:0/2515128376 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55762dd442e0 msgr2=0x55762dd46730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:43.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:43.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:43.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:43.705 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:43.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:43.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:43.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:43.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 zone default found 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 4 Realm: () 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 zonegroup default 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.710+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.714+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.714+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.714+0000 7f198bc26980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.714+0000 7f198bc26980 20 started sync module instance, tier type = 2024-04-17T04:54:43.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.714+0000 7f198bc26980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:43.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=0 2024-04-17T04:54:43.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=1 2024-04-17T04:54:43.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=2 2024-04-17T04:54:43.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=3 2024-04-17T04:54:43.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=4 2024-04-17T04:54:43.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=5 2024-04-17T04:54:43.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.738+0000 7f198bc26980 20 add_watcher() i=6 2024-04-17T04:54:43.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.742+0000 7f198bc26980 20 add_watcher() i=7 2024-04-17T04:54:43.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.742+0000 7f198bc26980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:43.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f183ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:43.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f198bc26980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:43.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f198bc26980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:43.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f198bc26980 5 note: GC not initialized 2024-04-17T04:54:43.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f183efe5640 20 reqs_thread_entry: start 2024-04-17T04:54:43.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f198bc26980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:43.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f198bc26980 20 Filter name: none 2024-04-17T04:54:43.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f183cfe1640 20 reqs_thread_entry: start 2024-04-17T04:54:43.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.746+0000 7f198bc26980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-04-17T04:54:43.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:43.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-04-17T04:54:43.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-04-17T04:54:43.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 : miss 2024-04-17T04:54:43.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:43.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:43.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:43.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:43.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:43.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.750+0000 7f198bc26980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 2024-04-17T04:54:43.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f198bc26980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:43.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f198bc26980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:43.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f198bc26980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:43.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f198bc26980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23, ofs0, ns] 2024-04-17T04:54:43.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f186cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464297 cookie 93966063394320 notifier 6462 bl.length()=991 2024-04-17T04:54:43.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f186cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:43.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f186cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:43.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.762+0000 7f186cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:43.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_sources= new_sources= 2024-04-17T04:54:43.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential sources added=[] removed=[] 2024-04-17T04:54:43.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_dests= new_dests= 2024-04-17T04:54:43.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential dests added=[] removed=[] 2024-04-17T04:54:43.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 -1 Injecting error=-5 at location=block_writes 2024-04-17T04:54:43.762 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-04-17T04:54:43.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 : hit (requested=0x17, cached=0x17) 2024-04-17T04:54:43.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.766+0000 7f198bc26980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 2024-04-17T04:54:43.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.770+0000 7f198bc26980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:43.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.770+0000 7f198bc26980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:43.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.770+0000 7f198bc26980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:43.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.770+0000 7f198bc26980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23, ofs0, ns] 2024-04-17T04:54:43.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f186cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464298 cookie 93966063394320 notifier 6462 bl.length()=959 2024-04-17T04:54:43.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f186cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:43.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f186cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:43.769 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f186cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:43.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_sources= new_sources= 2024-04-17T04:54:43.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential sources added=[] removed=[] 2024-04-17T04:54:43.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_dests= new_dests= 2024-04-17T04:54:43.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.774+0000 7f198bc26980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential dests added=[] removed=[] 2024-04-17T04:54:43.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.778+0000 7f183ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:43.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.786+0000 7f198bc26980 20 remove_watcher() i=0 2024-04-17T04:54:43.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.786+0000 7f198bc26980 2 removed watcher, disabling cache 2024-04-17T04:54:43.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.786+0000 7f198bc26980 20 remove_watcher() i=1 2024-04-17T04:54:43.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.790+0000 7f198bc26980 20 remove_watcher() i=2 2024-04-17T04:54:43.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.794+0000 7f198bc26980 20 remove_watcher() i=3 2024-04-17T04:54:43.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.794+0000 7f198bc26980 20 remove_watcher() i=4 2024-04-17T04:54:43.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.794+0000 7f198bc26980 20 remove_watcher() i=5 2024-04-17T04:54:43.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.798+0000 7f198bc26980 20 remove_watcher() i=6 2024-04-17T04:54:43.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.798+0000 7f198bc26980 20 remove_watcher() i=7 2024-04-17T04:54:43.809 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-04-17T04:54:43.841 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:43.841 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:43.884 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 realm 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.887 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.890+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.894+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:43.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.894+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.894+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:43.889 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.894+0000 7f5264259980 20 searching for the correct realm 2024-04-17T04:54:43.895 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.898+0000 7f5261573640 0 -- 172.21.15.88:0/738510174 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5254047040 msgr2=0x7f5254047420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:43.901 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.906+0000 7f5261573640 0 -- 172.21.15.88:0/738510174 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f512c076420 msgr2=0x7f512c078850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:43.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:43.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:43.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:43.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:43.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:43.908 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:43.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.910+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.909 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 zone default found 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 4 Realm: () 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:43.910 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:43.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:43.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 zonegroup default 2024-04-17T04:54:43.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.911 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:43.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 started sync module instance, tier type = 2024-04-17T04:54:43.912 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.914+0000 7f5264259980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:43.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=0 2024-04-17T04:54:43.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=1 2024-04-17T04:54:43.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=2 2024-04-17T04:54:43.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=3 2024-04-17T04:54:43.926 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=4 2024-04-17T04:54:43.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=5 2024-04-17T04:54:43.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=6 2024-04-17T04:54:43.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 20 add_watcher() i=7 2024-04-17T04:54:43.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.930+0000 7f5264259980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:43.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f51137fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:43.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:43.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:43.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 5 note: GC not initialized 2024-04-17T04:54:43.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f51127fc640 20 reqs_thread_entry: start 2024-04-17T04:54:43.931 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:43.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 Filter name: none 2024-04-17T04:54:43.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5087fff640 20 reqs_thread_entry: start 2024-04-17T04:54:43.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:43.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.934+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:43.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:43.933 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:43.934 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 rados->read ofs=0 len=0 2024-04-17T04:54:43.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:43.935 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.938+0000 7f5264259980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:43.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.942+0000 7f51137fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:43.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.946+0000 7f5264259980 20 remove_watcher() i=0 2024-04-17T04:54:43.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.946+0000 7f5264259980 2 removed watcher, disabling cache 2024-04-17T04:54:43.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.950+0000 7f5264259980 20 remove_watcher() i=1 2024-04-17T04:54:43.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.950+0000 7f5264259980 20 remove_watcher() i=2 2024-04-17T04:54:43.948 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.954+0000 7f5264259980 20 remove_watcher() i=3 2024-04-17T04:54:43.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.954+0000 7f5264259980 20 remove_watcher() i=4 2024-04-17T04:54:43.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.958+0000 7f5264259980 20 remove_watcher() i=5 2024-04-17T04:54:43.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.958+0000 7f5264259980 20 remove_watcher() i=6 2024-04-17T04:54:43.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:43.958+0000 7f5264259980 20 remove_watcher() i=7 2024-04-17T04:54:43.968 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:43.968 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:43.968 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-block-writes id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:43.968 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:44.098 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:54:44.134 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:44.134 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:44.183 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 20 realm 2024-04-17T04:54:44.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:44.184 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.186+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.190+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.190+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.186 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.190+0000 7f24d39ee980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:44.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.190+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.190+0000 7f24d39ee980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:44.187 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.190+0000 7f24d39ee980 20 searching for the correct realm 2024-04-17T04:54:44.193 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.198+0000 7f24d0d08640 0 -- 172.21.15.88:0/485683243 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f24c4122c20 msgr2=0x7f24c40e03a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:54:44.201 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.206+0000 7f24d0d08640 0 -- 172.21.15.88:0/485683243 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f239c0763a0 msgr2=0x7f239c0787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.208 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.210+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:44.209 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:44.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 zone default found 2024-04-17T04:54:44.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 4 Realm: () 2024-04-17T04:54:44.210 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 zonegroup default 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.211 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 started sync module instance, tier type = 2024-04-17T04:54:44.212 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.214+0000 7f24d39ee980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=0 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=1 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=2 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=3 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=4 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=5 2024-04-17T04:54:44.227 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=6 2024-04-17T04:54:44.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 20 add_watcher() i=7 2024-04-17T04:54:44.228 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.230+0000 7f24d39ee980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:44.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f2387fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:44.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f24d39ee980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:44.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f24d39ee980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:44.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f24d39ee980 5 note: GC not initialized 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f2386fe5640 20 reqs_thread_entry: start 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f24d39ee980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f24d39ee980 20 Filter name: none 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.234+0000 7f2384fe1640 20 reqs_thread_entry: start 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-04-17T04:54:44.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 : miss 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:44.234 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:44.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.238+0000 7f24d39ee980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 2024-04-17T04:54:44.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f24d39ee980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:44.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f24d39ee980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:44.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f24d39ee980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:44.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f24d39ee980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23, ofs0, ns] 2024-04-17T04:54:44.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f23b4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464299 cookie 93835162434000 notifier 6176 bl.length()=991 2024-04-17T04:54:44.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f23b4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:44.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f23b4ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:44.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.246+0000 7f23b4ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:44.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.250+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_sources= new_sources= 2024-04-17T04:54:44.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.250+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential sources added=[] removed=[] 2024-04-17T04:54:44.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.250+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_dests= new_dests= 2024-04-17T04:54:44.247 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.250+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential dests added=[] removed=[] 2024-04-17T04:54:44.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.254+0000 7f24d39ee980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23.1.0 2024-04-17T04:54:44.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.254+0000 7f24d39ee980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23.1.0 2024-04-17T04:54:44.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.254+0000 7f24d39ee980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23.1.1 2024-04-17T04:54:44.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.254+0000 7f24d39ee980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23.1.1 2024-04-17T04:54:44.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.254+0000 7f24d39ee980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23.0 2024-04-17T04:54:44.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.258+0000 7f24d39ee980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 : hit (requested=0x17, cached=0x17) 2024-04-17T04:54:44.255 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.258+0000 7f24d39ee980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 2024-04-17T04:54:44.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f24d39ee980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:44.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f24d39ee980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:44.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f24d39ee980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:44.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f24d39ee980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23, ofs0, ns] 2024-04-17T04:54:44.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f23a3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464300 cookie 93835162434000 notifier 6176 bl.length()=959 2024-04-17T04:54:44.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f23a3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 info.flags=0x17 2024-04-17T04:54:44.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f23a3fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 to cache LRU end 2024-04-17T04:54:44.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.262+0000 7f23a3fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:44.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.266+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_sources= new_sources= 2024-04-17T04:54:44.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.266+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential sources added=[] removed=[] 2024-04-17T04:54:44.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.266+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): orig_dests= new_dests= 2024-04-17T04:54:44.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.266+0000 7f24d39ee980 20 handle_bi_update(): bucket=:error-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23]): potential dests added=[] removed=[] 2024-04-17T04:54:44.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.266+0000 7f24d39ee980 1 execute INFO: reshard of bucket "error-at-block-writes" completed successfully 2024-04-17T04:54:44.268 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.270+0000 7f2387fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:44.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.278+0000 7f24d39ee980 20 remove_watcher() i=0 2024-04-17T04:54:44.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.278+0000 7f24d39ee980 2 removed watcher, disabling cache 2024-04-17T04:54:44.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.278+0000 7f24d39ee980 20 remove_watcher() i=1 2024-04-17T04:54:44.278 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.282+0000 7f24d39ee980 20 remove_watcher() i=2 2024-04-17T04:54:44.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.282+0000 7f24d39ee980 20 remove_watcher() i=3 2024-04-17T04:54:44.281 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.286+0000 7f24d39ee980 20 remove_watcher() i=4 2024-04-17T04:54:44.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.286+0000 7f24d39ee980 20 remove_watcher() i=5 2024-04-17T04:54:44.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.290+0000 7f24d39ee980 20 remove_watcher() i=6 2024-04-17T04:54:44.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.290+0000 7f24d39ee980 20 remove_watcher() i=7 2024-04-17T04:54:44.299 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-04-17T04:54:44.326 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:44.326 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:44.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 realm 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.374+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.378+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:44.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.378+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.378+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:44.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.378+0000 7fa293182980 20 searching for the correct realm 2024-04-17T04:54:44.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.386+0000 7fa29149e640 0 -- 172.21.15.88:0/1995200181 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fa2840541a0 msgr2=0x7fa2840480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:44.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.602+0000 7fa290c9d640 0 -- 172.21.15.88:0/1995200181 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55c85512c350 msgr2=0x55c85512e7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:44.607 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:44.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.610+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:44.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 zone default found 2024-04-17T04:54:44.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 4 Realm: () 2024-04-17T04:54:44.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:44.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:44.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:44.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 zonegroup default 2024-04-17T04:54:44.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:44.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 started sync module instance, tier type = 2024-04-17T04:54:44.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.614+0000 7fa293182980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=0 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=1 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=2 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=3 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=4 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=5 2024-04-17T04:54:44.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.638+0000 7fa293182980 20 add_watcher() i=6 2024-04-17T04:54:44.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.642+0000 7fa293182980 20 add_watcher() i=7 2024-04-17T04:54:44.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.642+0000 7fa293182980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:44.641 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa146fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:44.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa293182980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:44.642 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa293182980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:44.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa293182980 5 note: GC not initialized 2024-04-17T04:54:44.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa145fe3640 20 reqs_thread_entry: start 2024-04-17T04:54:44.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa293182980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:44.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa293182980 20 Filter name: none 2024-04-17T04:54:44.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa0b77fe640 20 reqs_thread_entry: start 2024-04-17T04:54:44.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.646+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:44.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:44.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:44.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:44.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados->read ofs=0 len=0 2024-04-17T04:54:44.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:44.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.650+0000 7fa293182980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:44.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.654+0000 7fa146fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:44.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.658+0000 7fa293182980 20 remove_watcher() i=0 2024-04-17T04:54:44.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.658+0000 7fa293182980 2 removed watcher, disabling cache 2024-04-17T04:54:44.658 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.662+0000 7fa293182980 20 remove_watcher() i=1 2024-04-17T04:54:44.660 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.662+0000 7fa293182980 20 remove_watcher() i=2 2024-04-17T04:54:44.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.666+0000 7fa293182980 20 remove_watcher() i=3 2024-04-17T04:54:44.664 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.670+0000 7fa293182980 20 remove_watcher() i=4 2024-04-17T04:54:44.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.670+0000 7fa293182980 20 remove_watcher() i=5 2024-04-17T04:54:44.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.670+0000 7fa293182980 20 remove_watcher() i=6 2024-04-17T04:54:44.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:44.674+0000 7fa293182980 20 remove_watcher() i=7 2024-04-17T04:54:44.683 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:44.683 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:44.683 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-block-writes id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:54:44.683 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:45.481 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with abort at block_writes 2024-04-17T04:54:45.481 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:47.721 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-04-17T04:54:47.755 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:47.756 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:47.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.802+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:47.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:47.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 realm 2024-04-17T04:54:47.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:47.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:47.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:47.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:47.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:47.802 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:47.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:47.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:47.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:47.804 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.806+0000 7fcaeb5ae980 20 searching for the correct realm 2024-04-17T04:54:47.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:47.814+0000 7fcae98ca640 0 -- 172.21.15.88:0/1019122064 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fcad4053920 msgr2=0x7fcad4018410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.029+0000 7fcae90c9640 0 -- 172.21.15.88:0/1019122064 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55d1392f2940 msgr2=0x55d1392f4d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:48.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.035 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 zone default found 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 4 Realm: () 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 zonegroup default 2024-04-17T04:54:48.036 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.037+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.037 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 started sync module instance, tier type = 2024-04-17T04:54:48.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.041+0000 7fcaeb5ae980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=0 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=1 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=2 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=3 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=4 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=5 2024-04-17T04:54:48.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=6 2024-04-17T04:54:48.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 20 add_watcher() i=7 2024-04-17T04:54:48.064 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.065+0000 7fcaeb5ae980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:48.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.069+0000 7fc99f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:48.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:48.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 5 note: GC not initialized 2024-04-17T04:54:48.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fc99e7e4640 20 reqs_thread_entry: start 2024-04-17T04:54:48.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:48.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 Filter name: none 2024-04-17T04:54:48.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fc90ffff640 20 reqs_thread_entry: start 2024-04-17T04:54:48.070 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.073+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:48.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:48.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:48.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.077+0000 7fcaeb5ae980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:48.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.081+0000 7fc99f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.085+0000 7fcaeb5ae980 20 remove_watcher() i=0 2024-04-17T04:54:48.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.085+0000 7fcaeb5ae980 2 removed watcher, disabling cache 2024-04-17T04:54:48.084 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.085+0000 7fcaeb5ae980 20 remove_watcher() i=1 2024-04-17T04:54:48.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.089+0000 7fcaeb5ae980 20 remove_watcher() i=2 2024-04-17T04:54:48.089 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.093+0000 7fcaeb5ae980 20 remove_watcher() i=3 2024-04-17T04:54:48.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.093+0000 7fcaeb5ae980 20 remove_watcher() i=4 2024-04-17T04:54:48.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.097+0000 7fcaeb5ae980 20 remove_watcher() i=5 2024-04-17T04:54:48.095 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.097+0000 7fcaeb5ae980 20 remove_watcher() i=6 2024-04-17T04:54:48.096 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.101+0000 7fcaeb5ae980 20 remove_watcher() i=7 2024-04-17T04:54:48.111 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.111 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:48.111 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-block-writes id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:48.111 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.112 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-04-17T04:54:48.143 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:48.143 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:48.185 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 realm 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.188 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.190 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:48.191 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.193+0000 7f22986f2980 20 searching for the correct realm 2024-04-17T04:54:48.197 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.201+0000 7f2296a0e640 0 -- 172.21.15.88:0/1272581658 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f228c017030 msgr2=0x7f228c0592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.417+0000 7f229620d640 0 -- 172.21.15.88:0/1272581658 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x56072d895760 msgr2=0x56072d897b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:48.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 zone default found 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 4 Realm: () 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:48.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 zonegroup default 2024-04-17T04:54:48.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 started sync module instance, tier type = 2024-04-17T04:54:48.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.429+0000 7f22986f2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:48.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=0 2024-04-17T04:54:48.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=1 2024-04-17T04:54:48.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=2 2024-04-17T04:54:48.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=3 2024-04-17T04:54:48.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=4 2024-04-17T04:54:48.450 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=5 2024-04-17T04:54:48.450 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=6 2024-04-17T04:54:48.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=7 2024-04-17T04:54:48.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.453+0000 7f22986f2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:48.454 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.457+0000 7f214cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.457+0000 7f22986f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:48.455 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.457+0000 7f22986f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:48.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.457+0000 7f22986f2980 5 note: GC not initialized 2024-04-17T04:54:48.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.457+0000 7f20c77fe640 20 reqs_thread_entry: start 2024-04-17T04:54:48.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.457+0000 7f22986f2980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:48.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 20 Filter name: none 2024-04-17T04:54:48.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f20c57fa640 20 reqs_thread_entry: start 2024-04-17T04:54:48.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-17T04:54:48.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.457 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-17T04:54:48.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-17T04:54:48.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 : miss 2024-04-17T04:54:48.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:48.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:48.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.461+0000 7f22986f2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 2024-04-17T04:54:48.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f22986f2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f22986f2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f22986f2980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24, ofs0, ns] 2024-04-17T04:54:48.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464269 cookie 94588828783040 notifier 6552 bl.length()=991 2024-04-17T04:54:48.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_sources= new_sources= 2024-04-17T04:54:48.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential sources added=[] removed=[] 2024-04-17T04:54:48.469 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f22986f2980 time 2024-04-17T04:54:48.477460+0000 2024-04-17T04:54:48.469 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:48.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_dests= new_dests= 2024-04-17T04:54:48.469 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential dests added=[] removed=[] 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f229bfc9e1f] 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:54:48.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.473+0000 7f22986f2980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f22986f2980 time 2024-04-17T04:54:48.477460+0000 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f229bfc9e1f] 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:54:48.471 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.472 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:54:48.472 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:54:48.472 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.472 INFO:tasks.workunit.client.0.smithi088.stderr:*** Caught signal (Aborted) ** 2024-04-17T04:54:48.472 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f22986f2980 thread_name:radosgw-admin 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f229ae4b520] 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f229bfc9e79] 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:54:48.491 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.473+0000 7f22986f2980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f22986f2980 thread_name:radosgw-admin 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f229ae4b520] 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f229bfc9e79] 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.492 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:54:48.165+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): get_auth_request con 0x56072d662b80 auth_method 0 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:54:48.165+0000 7f2296a0e640 10 monclient(hunting): handle_auth_done global_id 6215 payload 274 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:54:48.165+0000 7f2296a0e640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.493 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:54:48.165+0000 7f2296a0e640 1 monclient: found mon.noname-c 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:54:48.165+0000 7f2296a0e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: dump: 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.494 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 monclient: authenticate success, global_id 6215 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:54:48.169+0000 7f22986f2980 4 set_mon_vals no callback set 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: dump: 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.495 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: shutdown 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 asok(0x56072d599f40) unregister_commands rotate-key 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.496 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding auth protocol: cephx 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding auth protocol: cephx 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding auth protocol: cephx 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.497 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:54:48.169+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: starting msgr at 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: starting objecter 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 asok(0x56072d599f40) register_command objecter_requests hook 0x56072d54b4d0 2024-04-17T04:54:48.498 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: setting wanted keys 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: calling monclient init 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding auth protocol: cephx 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding auth protocol: cephx 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding auth protocol: cephx 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.499 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:54:48.169+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:54:48.169+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key hook 0x56072d7a70b8 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _add_conns ranks=[1,2,0] 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d6585c0 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d662b80 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d7ac700 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.500 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.822278s 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): get_auth_request con 0x56072d662b80 auth_method 0 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d7ac700 auth_method 0 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.501 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_done global_id 6218 payload 824 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:54:48.169+0000 7f2295a0c640 1 monclient: found mon.noname-c 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: dump: 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.502 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: dump: 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.503 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 monclient: authenticate success, global_id 6218 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:54:48.169+0000 7f2184608640 4 set_mon_vals no callback set 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: init done 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:54:48.173+0000 7f2182e05640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:48.504 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:54:48.173+0000 7f2182e05640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:54:48.173+0000 7f2182e05640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:54:48.173+0000 7f229620d640 10 monclient: get_auth_request con 0x7f217c065e30 auth_method 0 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:54:48.173+0000 7f2296a0e640 10 monclient: get_auth_request con 0x56072d7d4660 auth_method 0 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:54:48.173+0000 7f229620d640 10 monclient: get_auth_request con 0x56072d7f1cd0 auth_method 0 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:54:48.177+0000 7f2295a0c640 10 monclient: get_auth_request con 0x56072d7f55b0 auth_method 0 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:54:48.177+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.505 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding auth protocol: cephx 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding auth protocol: cephx 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding auth protocol: cephx 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.506 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:54:48.177+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 asok(0x56072d599f40) register_command objecter_requests cmddesc objecter_requests hook 0x56072d800600 EEXIST 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding auth protocol: cephx 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding auth protocol: cephx 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding auth protocol: cephx 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.507 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:54:48.177+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:54:48.177+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key cmddesc rotate-key hook 0x56072d7fa100 EEXIST 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:48.508 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d825260 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d825640 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d820800 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.831497s 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): get_auth_request con 0x56072d825260 auth_method 0 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d820800 auth_method 0 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.509 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_done global_id 6546 payload 824 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:54:48.177+0000 7f2296a0e640 1 monclient: found mon.noname-a 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: dump: 2024-04-17T04:54:48.510 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.511 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: dump: 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 monclient: authenticate success, global_id 6546 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:54:48.181+0000 7f229720f640 4 set_mon_vals no callback set 2024-04-17T04:54:48.512 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:54:48.181+0000 7f217bfff640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:54:48.181+0000 7f217bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:54:48.181+0000 7f217bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient: get_auth_request con 0x7f2170065d40 auth_method 0 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: get_monmap_and_config 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:54:48.181+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.513 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.514 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:54:48.181+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding auth protocol: cephx 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding auth protocol: cephx 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding auth protocol: cephx 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.515 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:54:48.181+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:54:48.181+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key cmddesc rotate-key hook 0x7ffc92a1d858 EEXIST 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:48.516 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d871c70 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d844f40 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d869eb0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.835224s 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d869eb0 auth_method 0 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): get_auth_request con 0x56072d871c70 auth_method 0 2024-04-17T04:54:48.517 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_done global_id 6549 payload 274 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:54:48.181+0000 7f2296a0e640 1 monclient: found mon.noname-a 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.518 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: dump: 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.519 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: dump: 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 monclient: authenticate success, global_id 6549 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:48.520 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:54:48.181+0000 7f22986f2980 4 set_mon_vals no callback set 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: shutdown 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.521 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.522 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:54:48.185+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: starting msgr at 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: starting objecter 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 asok(0x56072d599f40) register_command objecter_requests cmddesc objecter_requests hook 0x56072d845ab0 EEXIST 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: setting wanted keys 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: calling monclient init 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding auth protocol: cephx 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding auth protocol: cephx 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding auth protocol: cephx 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.523 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:54:48.185+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:54:48.185+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key cmddesc rotate-key hook 0x56072d8420c8 EEXIST 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d844f40 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.524 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d871c70 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d884c40 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.837695s 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): get_auth_request con 0x56072d871c70 auth_method 0 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): get_auth_request con 0x56072d844f40 auth_method 0 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.525 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d884c40 auth_method 0 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_done global_id 6552 payload 824 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:54:48.185+0000 7f229620d640 1 monclient: found mon.noname-a 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.526 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: dump: 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.527 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: dump: 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.528 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:54:48.185+0000 7f2179ffb640 4 set_mon_vals no callback set 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 monclient: authenticate success, global_id 6552 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: init done 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cr dump hook 0x56072d849578 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:54:48.189+0000 7f2167fff640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:54:48.189+0000 7f2167fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.529 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:54:48.189+0000 7f2167fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:54:48.189+0000 7f2295a0c640 10 monclient: get_auth_request con 0x7f2160065e00 auth_method 0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:54:48.189+0000 7f2296a0e640 10 monclient: get_auth_request con 0x56072d8913d0 auth_method 0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 realm 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:54:48.189+0000 7f22986f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:54:48.193+0000 7f229620d640 10 monclient: get_auth_request con 0x56072d895760 auth_method 0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:48.530 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 searching for the correct realm 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:54:48.193+0000 7f2295a0c640 10 monclient: get_auth_request con 0x56072d89a850 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:54:48.193+0000 7f2296a0e640 10 monclient: get_auth_request con 0x7f228c069c20 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:54:48.197+0000 7f229620d640 10 monclient: get_auth_request con 0x7f2290109690 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:54:48.197+0000 7f2295a0c640 10 monclient: get_auth_request con 0x7f228805fbc0 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:54:48.201+0000 7f2296a0e640 0 -- 172.21.15.88:0/1272581658 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f228c017030 msgr2=0x7f228c0592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:54:48.405+0000 7f2296a0e640 10 monclient: get_auth_request con 0x7f228c017030 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:54:48.409+0000 7f229620d640 10 monclient: get_auth_request con 0x7f228c008050 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:54:48.417+0000 7f229620d640 0 -- 172.21.15.88:0/1272581658 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x56072d895760 msgr2=0x56072d897b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:54:48.417+0000 7f2167fff640 1 client.6552.objecter ms_handle_reset 0x56072d895760 session 0x56072d8945e0 osd.5 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:54:48.417+0000 7f2167fff640 10 monclient: _renew_subs 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:54:48.417+0000 7f2167fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:54:48.417+0000 7f229620d640 10 monclient: get_auth_request con 0x7f2160076310 auth_method 0 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:48.531 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 zone default found 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:54:48.429+0000 7f22986f2980 4 Realm: () 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:54:48.429+0000 7f22986f2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:54:48.429+0000 7f22986f2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:48.532 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:54:48.429+0000 7f22986f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 zonegroup default 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 started sync module instance, tier type = 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=0 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=1 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=2 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=3 2024-04-17T04:54:48.533 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=4 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=5 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=6 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=7 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:54:48.453+0000 7f22986f2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding auth protocol: cephx 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding auth protocol: cephx 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding auth protocol: cephx 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.534 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:54:48.457+0000 7f214cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:54:48.457+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:54:48.457+0000 7f22986f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:54:48.457+0000 7f22986f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache list hook 0x56072d86a620 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache inspect hook 0x56072d86a620 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache erase hook 0x56072d86a620 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache zap hook 0x56072d86a620 2024-04-17T04:54:48.535 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 note: GC not initialized 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:54:48.457+0000 7f20c77fe640 20 reqs_thread_entry: start 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace show hook 0x7f22900047f0 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace history hook 0x7f22900047f0 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace active hook 0x7f22900047f0 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace active_short hook 0x7f22900047f0 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:54:48.457+0000 7f22986f2980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 Filter name: none 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:54:48.461+0000 7f20c57fa640 20 reqs_thread_entry: start 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-17T04:54:48.536 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 : miss 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24, ofs0, ns] 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464269 cookie 94588828783040 notifier 6552 bl.length()=991 2024-04-17T04:54:48.537 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_sources= new_sources= 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential sources added=[] removed=[] 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_dests= new_dests= 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential dests added=[] removed=[] 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:54:48.473+0000 7f22986f2980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f22986f2980 time 2024-04-17T04:54:48.477460+0000 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f229bfc9e1f] 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.538 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:54:48.473+0000 7f22986f2980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f22986f2980 thread_name:radosgw-admin 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f229ae4b520] 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f229bfc9e79] 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.539 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:54:48.540 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:54:48.541 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:54:48.542 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:54:48.543 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:54:48.544 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:54:48.545 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 7f20c57fa640 / http_manager 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 7f20c77fe640 / http_manager 2024-04-17T04:54:48.546 INFO:tasks.workunit.client.0.smithi088.stderr: 7f214cfe1640 / rgw_dt_lg_renew 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2167fff640 / ms_dispatch 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f21797fa640 / io_context_pool 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2179ffb640 / io_context_pool 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f217bfff640 / ms_dispatch 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2182e05640 / ms_dispatch 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2184608640 / io_context_pool 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2295a0c640 / msgr-worker-2 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f229620d640 / msgr-worker-1 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2296a0e640 / msgr-worker-0 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f229720f640 / io_context_pool 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: 7f22986f2980 / radosgw-admin 2024-04-17T04:54:48.547 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/log/ceph/ceph-client.admin.88813.log 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:54:48.165+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): get_auth_request con 0x56072d662b80 auth_method 0 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:54:48.165+0000 7f2295a0c640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:54:48.165+0000 7f2296a0e640 10 monclient(hunting): handle_auth_done global_id 6215 payload 274 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:54:48.165+0000 7f2296a0e640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:54:48.165+0000 7f2296a0e640 1 monclient: found mon.noname-c 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:54:48.165+0000 7f2296a0e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.548 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: dump: 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.549 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 monclient: authenticate success, global_id 6215 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:54:48.169+0000 7f22986f2980 4 set_mon_vals no callback set 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:54:48.169+0000 7f2184608640 10 monclient: dump: 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.550 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: shutdown 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 asok(0x56072d599f40) unregister_commands rotate-key 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.551 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding auth protocol: cephx 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding auth protocol: cephx 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding auth protocol: cephx 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.552 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: crc 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a8db0) adding con mode: secure 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:54:48.169+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: starting msgr at 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: starting objecter 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 asok(0x56072d599f40) register_command objecter_requests hook 0x56072d54b4d0 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: setting wanted keys 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: calling monclient init 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding auth protocol: cephx 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding auth protocol: cephx 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding auth protocol: cephx 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.553 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: crc 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 AuthRegistry(0x56072d7a6f70) adding con mode: secure 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:54:48.169+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:54:48.169+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key hook 0x56072d7a70b8 2024-04-17T04:54:48.554 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _add_conns ranks=[1,2,0] 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d6585c0 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d662b80 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d7ac700 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.822278s 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): get_auth_request con 0x56072d662b80 auth_method 0 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d7ac700 auth_method 0 2024-04-17T04:54:48.555 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:54:48.169+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient(hunting): handle_auth_done global_id 6218 payload 824 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:54:48.169+0000 7f2295a0c640 1 monclient: found mon.noname-c 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:54:48.169+0000 7f2295a0c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.556 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: dump: 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.557 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:54:48.169+0000 7f2182e05640 10 monclient: dump: 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:54:48.169+0000 7f22986f2980 5 monclient: authenticate success, global_id 6218 2024-04-17T04:54:48.558 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:54:48.169+0000 7f2184608640 4 set_mon_vals no callback set 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:54:48.169+0000 7f22986f2980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:54:48.169+0000 7f22986f2980 1 librados: init done 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:54:48.173+0000 7f2182e05640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:54:48.173+0000 7f2182e05640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:54:48.173+0000 7f2182e05640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:54:48.173+0000 7f229620d640 10 monclient: get_auth_request con 0x7f217c065e30 auth_method 0 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:54:48.173+0000 7f2296a0e640 10 monclient: get_auth_request con 0x56072d7d4660 auth_method 0 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:54:48.173+0000 7f229620d640 10 monclient: get_auth_request con 0x56072d7f1cd0 auth_method 0 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:54:48.177+0000 7f2295a0c640 10 monclient: get_auth_request con 0x56072d7f55b0 auth_method 0 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.559 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:54:48.177+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding auth protocol: cephx 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding auth protocol: cephx 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding auth protocol: cephx 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.560 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: crc 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7fb070) adding con mode: secure 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:54:48.177+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 asok(0x56072d599f40) register_command objecter_requests cmddesc objecter_requests hook 0x56072d800600 EEXIST 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.561 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding auth protocol: cephx 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding auth protocol: cephx 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding auth protocol: cephx 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: crc 2024-04-17T04:54:48.562 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 AuthRegistry(0x56072d7f9fb8) adding con mode: secure 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:54:48.177+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:54:48.177+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:54:48.177+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key cmddesc rotate-key hook 0x56072d7fa100 EEXIST 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d825260 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d825640 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d820800 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:54:48.177+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.831497s 2024-04-17T04:54:48.563 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): get_auth_request con 0x56072d825260 auth_method 0 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d820800 auth_method 0 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:54:48.177+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient(hunting): handle_auth_done global_id 6546 payload 824 2024-04-17T04:54:48.564 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:54:48.177+0000 7f2296a0e640 1 monclient: found mon.noname-a 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:54:48.177+0000 7f2296a0e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: dump: 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.565 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:54:48.181+0000 7f217bfff640 10 monclient: dump: 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.566 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 monclient: authenticate success, global_id 6546 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:54:48.181+0000 7f229720f640 4 set_mon_vals no callback set 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:54:48.181+0000 7f217bfff640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:54:48.181+0000 7f217bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:54:48.181+0000 7f217bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient: get_auth_request con 0x7f2170065d40 auth_method 0 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: get_monmap_and_config 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.567 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:54:48.181+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.568 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:54:48.181+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.569 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding auth protocol: cephx 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding auth protocol: cephx 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding auth protocol: cephx 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: crc 2024-04-17T04:54:48.570 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1d710) adding con mode: secure 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:54:48.181+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:54:48.181+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key cmddesc rotate-key hook 0x7ffc92a1d858 EEXIST 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: _add_conns ranks=[2,1,0] 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d871c70 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d844f40 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d869eb0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.835224s 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d869eb0 auth_method 0 2024-04-17T04:54:48.571 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): get_auth_request con 0x56072d871c70 auth_method 0 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:54:48.181+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient(hunting): handle_auth_done global_id 6549 payload 274 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.572 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:54:48.181+0000 7f2296a0e640 1 monclient: found mon.noname-a 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:54:48.181+0000 7f2296a0e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: dump: 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.573 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:54:48.181+0000 7f2179ffb640 10 monclient: dump: 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.574 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:54:48.181+0000 7f22986f2980 5 monclient: authenticate success, global_id 6549 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:54:48.181+0000 7f22986f2980 10 monclient: get_monmap_and_config success 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:54:48.181+0000 7f22986f2980 4 set_mon_vals no callback set 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: shutdown 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: build_initial_monmap 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 build_initial for_mkfs: 0 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: monmap: 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.575 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding auth protocol: cephx 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.577 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: crc 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d842db0) adding con mode: secure 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:54:48.185+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: starting msgr at 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: starting objecter 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 asok(0x56072d599f40) register_command objecter_requests cmddesc objecter_requests hook 0x56072d845ab0 EEXIST 2024-04-17T04:54:48.578 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: setting wanted keys 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: calling monclient init 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: init 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding auth protocol: cephx 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding auth protocol: cephx 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding auth protocol: cephx 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.579 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: crc 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 AuthRegistry(0x56072d841f80) adding con mode: secure 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:54:48.185+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:54:48.185+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 asok(0x56072d599f40) register_command rotate-key cmddesc rotate-key hook 0x56072d8420c8 EEXIST 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _reopen_session rank -1 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-a con 0x56072d844f40 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-c con 0x56072d871c70 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:54:48.580 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): picked mon.noname-b con 0x56072d884c40 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): start opening mon connection 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): _renew_subs 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient(hunting): authenticate will time out at 2089.837695s 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): get_auth_request con 0x56072d871c70 auth_method 0 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): get_auth_request con 0x56072d844f40 auth_method 0 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.581 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:54:48.185+0000 7f2295a0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): get_auth_request con 0x56072d884c40 auth_method 0 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:54:48.185+0000 7f2296a0e640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient(hunting): handle_auth_done global_id 6552 payload 824 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient: _finish_hunting 0 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:54:48.185+0000 7f229620d640 1 monclient: found mon.noname-a 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:54:48.185+0000 7f229620d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.582 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: dump: 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: _finish_auth 0 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: _check_auth_tickets 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: handle_config config(0 keys) 2024-04-17T04:54:48.583 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:54:48.185+0000 7f2167fff640 10 monclient: dump: 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:54:48.185+0000 7f2179ffb640 4 set_mon_vals no callback set 2024-04-17T04:54:48.584 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 monclient: authenticate success, global_id 6552 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _renew_subs 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:54:48.185+0000 7f22986f2980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:54:48.185+0000 7f22986f2980 1 librados: init done 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:54:48.185+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cr dump hook 0x56072d849578 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:54:48.189+0000 7f2167fff640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:54:48.189+0000 7f2167fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:54:48.189+0000 7f2167fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:54:48.189+0000 7f2295a0c640 10 monclient: get_auth_request con 0x7f2160065e00 auth_method 0 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:54:48.189+0000 7f2296a0e640 10 monclient: get_auth_request con 0x56072d8913d0 auth_method 0 2024-04-17T04:54:48.585 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 realm 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:54:48.189+0000 7f22986f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:54:48.189+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:54:48.193+0000 7f229620d640 10 monclient: get_auth_request con 0x56072d895760 auth_method 0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:54:48.193+0000 7f22986f2980 20 searching for the correct realm 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:54:48.193+0000 7f2295a0c640 10 monclient: get_auth_request con 0x56072d89a850 auth_method 0 2024-04-17T04:54:48.586 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:54:48.193+0000 7f2296a0e640 10 monclient: get_auth_request con 0x7f228c069c20 auth_method 0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:54:48.197+0000 7f229620d640 10 monclient: get_auth_request con 0x7f2290109690 auth_method 0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:54:48.197+0000 7f2295a0c640 10 monclient: get_auth_request con 0x7f228805fbc0 auth_method 0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:54:48.201+0000 7f2296a0e640 0 -- 172.21.15.88:0/1272581658 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f228c017030 msgr2=0x7f228c0592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:54:48.405+0000 7f2296a0e640 10 monclient: get_auth_request con 0x7f228c017030 auth_method 0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:54:48.409+0000 7f229620d640 10 monclient: get_auth_request con 0x7f228c008050 auth_method 0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:54:48.417+0000 7f229620d640 0 -- 172.21.15.88:0/1272581658 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x56072d895760 msgr2=0x56072d897b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:54:48.417+0000 7f2167fff640 1 client.6552.objecter ms_handle_reset 0x56072d895760 session 0x56072d8945e0 osd.5 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:54:48.417+0000 7f2167fff640 10 monclient: _renew_subs 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:54:48.417+0000 7f2167fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:54:48.417+0000 7f229620d640 10 monclient: get_auth_request con 0x7f2160076310 auth_method 0 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:48.587 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:54:48.425+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 zone default found 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:54:48.429+0000 7f22986f2980 4 Realm: () 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:54:48.429+0000 7f22986f2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:54:48.429+0000 7f22986f2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:54:48.429+0000 7f22986f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 zonegroup default 2024-04-17T04:54:48.588 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 started sync module instance, tier type = 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:54:48.429+0000 7f22986f2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=0 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=1 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=2 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=3 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=4 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=5 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=6 2024-04-17T04:54:48.589 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:54:48.453+0000 7f22986f2980 20 add_watcher() i=7 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:54:48.453+0000 7f22986f2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding auth protocol: cephx 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding auth protocol: cephx 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding auth protocol: cephx 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.590 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: crc 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 AuthRegistry(0x7ffc92a1dba0) adding con mode: secure 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:54:48.457+0000 7f214cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:54:48.457+0000 7f22986f2980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:54:48.457+0000 7f22986f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:54:48.457+0000 7f22986f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache list hook 0x56072d86a620 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache inspect hook 0x56072d86a620 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache erase hook 0x56072d86a620 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command cache zap hook 0x56072d86a620 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 note: GC not initialized 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:54:48.457+0000 7f20c77fe640 20 reqs_thread_entry: start 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace show hook 0x7f22900047f0 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace history hook 0x7f22900047f0 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace active hook 0x7f22900047f0 2024-04-17T04:54:48.591 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:54:48.457+0000 7f22986f2980 5 asok(0x56072d599f40) register_command sync trace active_short hook 0x7f22900047f0 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:54:48.457+0000 7f22986f2980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 Filter name: none 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:54:48.461+0000 7f20c57fa640 20 reqs_thread_entry: start 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 : miss 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:54:48.461+0000 7f22986f2980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:48.592 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:54:48.461+0000 7f22986f2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:54:48.469+0000 7f22986f2980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24, ofs0, ns] 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464269 cookie 94588828783040 notifier 6552 bl.length()=991 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:54:48.469+0000 7f21797fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_sources= new_sources= 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential sources added=[] removed=[] 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_dests= new_dests= 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:54:48.473+0000 7f22986f2980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential dests added=[] removed=[] 2024-04-17T04:54:48.593 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:54:48.473+0000 7f22986f2980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f22986f2980 time 2024-04-17T04:54:48.477460+0000 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f229bfc9e1f] 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:54:48.594 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:54:48.473+0000 7f22986f2980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f22986f2980 thread_name:radosgw-admin 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f229ae4b520] 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f229bfc9e79] 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f229bfc9fd3] 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56072c673191] 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2a5) [0x56072c681995] 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f229ae32d90] 2024-04-17T04:54:48.595 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:54:48.596 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:54:48.597 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:54:48.598 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:54:48.599 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:54:48.600 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:54:48.601 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f20c57fa640 / http_manager 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f20c77fe640 / http_manager 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f214cfe1640 / rgw_dt_lg_renew 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2167fff640 / ms_dispatch 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f21797fa640 / io_context_pool 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2179ffb640 / io_context_pool 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f217bfff640 / ms_dispatch 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2182e05640 / ms_dispatch 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2184608640 / io_context_pool 2024-04-17T04:54:48.602 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2295a0c640 / msgr-worker-2 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: 7f229620d640 / msgr-worker-1 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: 7f2296a0e640 / msgr-worker-0 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: 7f229720f640 / io_context_pool 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: 7f22986f2980 / radosgw-admin 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/lib/ceph/crash/2024-04-17T04:54:48.479405Z_bf107c41-4ae7-4995-8d9e-cf3e61581af1/log 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr:Aborted 2024-04-17T04:54:48.603 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.557+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.557+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.557+0000 7f28cfb97980 20 realm 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.557+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.557+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.557+0000 7f28cfb97980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:48.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.561+0000 7f28cfb97980 20 searching for the correct realm 2024-04-17T04:54:48.606 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.569+0000 7f28cdeb3640 0 -- 172.21.15.88:0/3363828344 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f28c40cd790 msgr2=0x7f28c411b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:54:48.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.785+0000 7f28cd6b2640 0 -- 172.21.15.88:0/3363828344 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5608aa874620 msgr2=0x5608aa876a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:48.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:48.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.793+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:48.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 zone default found 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 4 Realm: () 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 zonegroup default 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:48.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 started sync module instance, tier type = 2024-04-17T04:54:48.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.797+0000 7f28cfb97980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=0 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=1 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=2 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=3 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=4 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=5 2024-04-17T04:54:48.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.821+0000 7f28cfb97980 20 add_watcher() i=6 2024-04-17T04:54:48.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.825+0000 7f28cfb97980 20 add_watcher() i=7 2024-04-17T04:54:48.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.825+0000 7f28cfb97980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:48.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f27837e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:48.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:48.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 5 note: GC not initialized 2024-04-17T04:54:48.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f27827e4640 20 reqs_thread_entry: start 2024-04-17T04:54:48.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:48.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 Filter name: none 2024-04-17T04:54:48.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f26f3fff640 20 reqs_thread_entry: start 2024-04-17T04:54:48.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.828 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.829+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=459 2024-04-17T04:54:48.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:48.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:48.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 rados->read ofs=0 len=0 2024-04-17T04:54:48.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 rados_obj.operate() r=0 bl.length=459 2024-04-17T04:54:48.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.833+0000 7f28cfb97980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:54:48.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.837+0000 7f27837e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:48.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.841+0000 7f28cfb97980 20 remove_watcher() i=0 2024-04-17T04:54:48.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.841+0000 7f28cfb97980 2 removed watcher, disabling cache 2024-04-17T04:54:48.841 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.845+0000 7f28cfb97980 20 remove_watcher() i=1 2024-04-17T04:54:48.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.845+0000 7f28cfb97980 20 remove_watcher() i=2 2024-04-17T04:54:48.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.849+0000 7f28cfb97980 20 remove_watcher() i=3 2024-04-17T04:54:48.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.849+0000 7f28cfb97980 20 remove_watcher() i=4 2024-04-17T04:54:48.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.853+0000 7f28cfb97980 20 remove_watcher() i=5 2024-04-17T04:54:48.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.853+0000 7f28cfb97980 20 remove_watcher() i=6 2024-04-17T04:54:48.852 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:48.857+0000 7f28cfb97980 20 remove_watcher() i=7 2024-04-17T04:54:48.876 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:48.876 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:54:48.876 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-block-writes id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:54:48.876 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:54:49.014 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:54:49.051 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:54:49.051 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:54:49.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 realm 2024-04-17T04:54:49.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:54:49.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.100 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.101+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.105+0000 7f075ca27980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:49.101 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.105+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.105+0000 7f075ca27980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:54:49.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.105+0000 7f075ca27980 20 searching for the correct realm 2024-04-17T04:54:49.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.109+0000 7f0759d41640 0 -- 172.21.15.88:0/2716153680 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f074c064c40 msgr2=0x7f074c05b610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:49.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.117+0000 7f075a542640 0 -- 172.21.15.88:0/2716153680 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f07440093d0 msgr2=0x7f07440478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:54:49.332 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.333+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:54:49.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 zone default found 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 4 Realm: () 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 zonegroup default 2024-04-17T04:54:49.335 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.337+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.341+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.341+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.341+0000 7f075ca27980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:54:49.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.341+0000 7f075ca27980 20 started sync module instance, tier type = 2024-04-17T04:54:49.337 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.341+0000 7f075ca27980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:54:49.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=0 2024-04-17T04:54:49.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=1 2024-04-17T04:54:49.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=2 2024-04-17T04:54:49.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=3 2024-04-17T04:54:49.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=4 2024-04-17T04:54:49.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=5 2024-04-17T04:54:49.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=6 2024-04-17T04:54:49.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 20 add_watcher() i=7 2024-04-17T04:54:49.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.365+0000 7f075ca27980 2 all 8 watchers are set, enabling cache 2024-04-17T04:54:49.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.369+0000 7f06117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:49.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:54:49.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:54:49.369 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 5 note: GC not initialized 2024-04-17T04:54:49.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f060b7fe640 20 reqs_thread_entry: start 2024-04-17T04:54:49.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 init_complete bucket index max shards: 1 2024-04-17T04:54:49.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 Filter name: none 2024-04-17T04:54:49.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f06097fa640 20 reqs_thread_entry: start 2024-04-17T04:54:49.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-17T04:54:49.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:54:49.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-17T04:54:49.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-17T04:54:49.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 : miss 2024-04-17T04:54:49.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.373+0000 7f075ca27980 20 rados->read ofs=0 len=0 2024-04-17T04:54:49.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.377+0000 7f075ca27980 20 rados_obj.operate() r=0 bl.length=459 2024-04-17T04:54:49.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.377+0000 7f075ca27980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:54:49.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.377+0000 7f075ca27980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:54:49.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.377+0000 7f075ca27980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:54:49.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.377+0000 7f075ca27980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:54:49.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.377+0000 7f075ca27980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 2024-04-17T04:54:49.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.381+0000 7f075ca27980 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to be held by another RGW process; skipping for now 2024-04-17T04:54:49.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.381+0000 7f06117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:54:49.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.385+0000 7f075ca27980 20 remove_watcher() i=0 2024-04-17T04:54:49.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.385+0000 7f075ca27980 2 removed watcher, disabling cache 2024-04-17T04:54:49.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.389+0000 7f075ca27980 20 remove_watcher() i=1 2024-04-17T04:54:49.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.393+0000 7f075ca27980 20 remove_watcher() i=2 2024-04-17T04:54:49.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.393+0000 7f075ca27980 20 remove_watcher() i=3 2024-04-17T04:54:49.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.397+0000 7f075ca27980 20 remove_watcher() i=4 2024-04-17T04:54:49.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.397+0000 7f075ca27980 20 remove_watcher() i=5 2024-04-17T04:54:49.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.397+0000 7f075ca27980 20 remove_watcher() i=6 2024-04-17T04:54:49.398 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:54:49.401+0000 7f075ca27980 20 remove_watcher() i=7 2024-04-17T04:54:49.411 INFO:tasks.workunit.client.0.smithi088.stderr:waiting 30 seconds for reshard lock to expire... 2024-04-17T04:55:19.442 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:55:19.478 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:19.478 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:19.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 realm 2024-04-17T04:55:19.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:19.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.525+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.529+0000 7f7f863df980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:19.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.529+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.529+0000 7f7f863df980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:19.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.529+0000 7f7f863df980 20 searching for the correct realm 2024-04-17T04:55:19.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.533+0000 7f7f836f9640 0 -- 172.21.15.88:0/172799859 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f7f74040070 msgr2=0x7f7f74037410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:19.538 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.541+0000 7f7f83efa640 0 -- 172.21.15.88:0/172799859 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f7f780f0920 msgr2=0x7f7f780df440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:19.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:19.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.757+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.757 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 zone default found 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 4 Realm: () 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 zonegroup default 2024-04-17T04:55:19.758 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 started sync module instance, tier type = 2024-04-17T04:55:19.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.761+0000 7f7f863df980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:19.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=0 2024-04-17T04:55:19.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=1 2024-04-17T04:55:19.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=2 2024-04-17T04:55:19.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=3 2024-04-17T04:55:19.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=4 2024-04-17T04:55:19.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=5 2024-04-17T04:55:19.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.785+0000 7f7f863df980 20 add_watcher() i=6 2024-04-17T04:55:19.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.789+0000 7f7f863df980 20 add_watcher() i=7 2024-04-17T04:55:19.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.789+0000 7f7f863df980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:19.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7e3a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:19.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:19.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:19.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 5 note: GC not initialized 2024-04-17T04:55:19.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7e397e2640 20 reqs_thread_entry: start 2024-04-17T04:55:19.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:19.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 20 Filter name: none 2024-04-17T04:55:19.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7e327fc640 20 reqs_thread_entry: start 2024-04-17T04:55:19.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-17T04:55:19.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.793+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:55:19.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-17T04:55:19.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-17T04:55:19.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 : miss 2024-04-17T04:55:19.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 20 rados_obj.operate() r=0 bl.length=459 2024-04-17T04:55:19.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:55:19.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:55:19.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:19.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:19.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.797+0000 7f7f863df980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 2024-04-17T04:55:19.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.801+0000 7f7f863df980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-04-17T04:55:19.805 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7f863df980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:55:19.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7f863df980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:55:19.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7f863df980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:19.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7f863df980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24, ofs0, ns] 2024-04-17T04:55:19.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7e62ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464270 cookie 94712197293200 notifier 6275 bl.length()=991 2024-04-17T04:55:19.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7e62ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:55:19.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7e62ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:55:19.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.809+0000 7f7e62ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:19.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_sources= new_sources= 2024-04-17T04:55:19.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential sources added=[] removed=[] 2024-04-17T04:55:19.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_dests= new_dests= 2024-04-17T04:55:19.809 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential dests added=[] removed=[] 2024-04-17T04:55:19.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24.2.0 2024-04-17T04:55:19.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24.2.0 2024-04-17T04:55:19.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24.2.1 2024-04-17T04:55:19.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24.2.1 2024-04-17T04:55:19.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.813+0000 7f7f863df980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24.0 2024-04-17T04:55:19.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.817+0000 7f7f863df980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 : hit (requested=0x17, cached=0x17) 2024-04-17T04:55:19.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.817+0000 7f7f863df980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 2024-04-17T04:55:19.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7f863df980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:55:19.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7f863df980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:55:19.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7f863df980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:19.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7f863df980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24, ofs0, ns] 2024-04-17T04:55:19.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7e627fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464271 cookie 94712197293200 notifier 6275 bl.length()=959 2024-04-17T04:55:19.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7e627fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 info.flags=0x17 2024-04-17T04:55:19.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7e627fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 to cache LRU end 2024-04-17T04:55:19.820 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.821+0000 7f7e627fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:19.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.825+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_sources= new_sources= 2024-04-17T04:55:19.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.825+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential sources added=[] removed=[] 2024-04-17T04:55:19.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.825+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): orig_dests= new_dests= 2024-04-17T04:55:19.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.825+0000 7f7f863df980 20 handle_bi_update(): bucket=:abort-at-block-writes[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24]): potential dests added=[] removed=[] 2024-04-17T04:55:19.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.829+0000 7f7f863df980 1 execute INFO: reshard of bucket "abort-at-block-writes" completed successfully 2024-04-17T04:55:19.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.833+0000 7f7e3a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:19.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.837+0000 7f7f863df980 20 remove_watcher() i=0 2024-04-17T04:55:19.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.837+0000 7f7f863df980 2 removed watcher, disabling cache 2024-04-17T04:55:19.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.841+0000 7f7f863df980 20 remove_watcher() i=1 2024-04-17T04:55:19.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.841+0000 7f7f863df980 20 remove_watcher() i=2 2024-04-17T04:55:19.842 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.845+0000 7f7f863df980 20 remove_watcher() i=3 2024-04-17T04:55:19.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.845+0000 7f7f863df980 20 remove_watcher() i=4 2024-04-17T04:55:19.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.849+0000 7f7f863df980 20 remove_watcher() i=5 2024-04-17T04:55:19.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.849+0000 7f7f863df980 20 remove_watcher() i=6 2024-04-17T04:55:19.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.853+0000 7f7f863df980 20 remove_watcher() i=7 2024-04-17T04:55:19.866 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-04-17T04:55:19.893 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:19.893 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:19.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 realm 2024-04-17T04:55:19.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.939 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:19.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:19.940 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.941+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.945+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:19.941 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.945+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:19.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.945+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:19.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.945+0000 7f7713343980 20 searching for the correct realm 2024-04-17T04:55:19.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:19.949+0000 7f770bfff640 0 -- 172.21.15.88:0/1988664752 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f77040e2940 msgr2=0x7f77040f2400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:55:20.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.165+0000 7f7710e5e640 0 -- 172.21.15.88:0/1988664752 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f76fc03b120 msgr2=0x7f76fc041870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:20.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:20.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.381+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:20.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.382 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 zone default found 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 4 Realm: () 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 zonegroup default 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:20.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:20.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.385+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.389+0000 7f7713343980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:20.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.389+0000 7f7713343980 20 started sync module instance, tier type = 2024-04-17T04:55:20.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.389+0000 7f7713343980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.409+0000 7f7713343980 20 add_watcher() i=0 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.409+0000 7f7713343980 20 add_watcher() i=1 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.409+0000 7f7713343980 20 add_watcher() i=2 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.409+0000 7f7713343980 20 add_watcher() i=3 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.409+0000 7f7713343980 20 add_watcher() i=4 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.409+0000 7f7713343980 20 add_watcher() i=5 2024-04-17T04:55:20.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.413+0000 7f7713343980 20 add_watcher() i=6 2024-04-17T04:55:20.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.413+0000 7f7713343980 20 add_watcher() i=7 2024-04-17T04:55:20.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.413+0000 7f7713343980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:20.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.417+0000 7f75c6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:20.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:20.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:20.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 5 note: GC not initialized 2024-04-17T04:55:20.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f75c5fe3640 20 reqs_thread_entry: start 2024-04-17T04:55:20.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:20.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 Filter name: none 2024-04-17T04:55:20.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f75377fe640 20 reqs_thread_entry: start 2024-04-17T04:55:20.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:55:20.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:55:20.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:20.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=213 2024-04-17T04:55:20.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.421+0000 7f7713343980 20 rados->read ofs=0 len=0 2024-04-17T04:55:20.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.425+0000 7f7713343980 20 rados_obj.operate() r=0 bl.length=427 2024-04-17T04:55:20.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.425+0000 7f7713343980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:20.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.425+0000 7f75c6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:20.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.429+0000 7f7713343980 20 remove_watcher() i=0 2024-04-17T04:55:20.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.433+0000 7f7713343980 2 removed watcher, disabling cache 2024-04-17T04:55:20.430 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.433+0000 7f7713343980 20 remove_watcher() i=1 2024-04-17T04:55:20.433 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.437+0000 7f7713343980 20 remove_watcher() i=2 2024-04-17T04:55:20.435 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.437+0000 7f7713343980 20 remove_watcher() i=3 2024-04-17T04:55:20.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.441+0000 7f7713343980 20 remove_watcher() i=4 2024-04-17T04:55:20.438 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.441+0000 7f7713343980 20 remove_watcher() i=5 2024-04-17T04:55:20.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.441+0000 7f7713343980 20 remove_watcher() i=6 2024-04-17T04:55:20.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:20.445+0000 7f7713343980 20 remove_watcher() i=7 2024-04-17T04:55:20.458 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:20.458 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:20.458 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-block-writes id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:55:20.458 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:21.205 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-04-17T04:55:21.205 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:23.454 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-17T04:55:23.486 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:23.487 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:23.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 20 realm 2024-04-17T04:55:23.532 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:23.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.532+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.536+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.536+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.536+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:23.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.536+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.536+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:23.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.536+0000 7f589934b980 20 searching for the correct realm 2024-04-17T04:55:23.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.544+0000 7f5896665640 0 -- 172.21.15.88:0/1537207478 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5888008040 msgr2=0x7f588805d910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:23.548 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.548+0000 7f5896665640 0 -- 172.21.15.88:0/1537207478 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5760076350 msgr2=0x7f5760078780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:23.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:23.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:23.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:23.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:23.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:23.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:23.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:23.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.556+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 zone default found 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 4 Realm: () 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:23.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 zonegroup default 2024-04-17T04:55:23.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 started sync module instance, tier type = 2024-04-17T04:55:23.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.560+0000 7f589934b980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:23.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=0 2024-04-17T04:55:23.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=1 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=2 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=3 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=4 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=5 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=6 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 20 add_watcher() i=7 2024-04-17T04:55:23.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.576+0000 7f589934b980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:23.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f574d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 5 note: GC not initialized 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f56c7fff640 20 reqs_thread_entry: start 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 Filter name: none 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f56c5ffb640 20 reqs_thread_entry: start 2024-04-17T04:55:23.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:23.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.580+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.584+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:23.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.584+0000 7f589934b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:23.582 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.584+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.584+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:23.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.584+0000 7f589934b980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.588+0000 7f589934b980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:23.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.588+0000 7f589934b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:23.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.588+0000 7f574d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:23.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.592+0000 7f589934b980 20 remove_watcher() i=0 2024-04-17T04:55:23.592 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.592+0000 7f589934b980 2 removed watcher, disabling cache 2024-04-17T04:55:23.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.596+0000 7f589934b980 20 remove_watcher() i=1 2024-04-17T04:55:23.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.596+0000 7f589934b980 20 remove_watcher() i=2 2024-04-17T04:55:23.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.600+0000 7f589934b980 20 remove_watcher() i=3 2024-04-17T04:55:23.600 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.600+0000 7f589934b980 20 remove_watcher() i=4 2024-04-17T04:55:23.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.604+0000 7f589934b980 20 remove_watcher() i=5 2024-04-17T04:55:23.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.604+0000 7f589934b980 20 remove_watcher() i=6 2024-04-17T04:55:23.604 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.604+0000 7f589934b980 20 remove_watcher() i=7 2024-04-17T04:55:23.618 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:23.618 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:23.618 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:55:23.618 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:23.618 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-04-17T04:55:23.650 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:23.650 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:23.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.692+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 realm 2024-04-17T04:55:23.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:23.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.695 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:23.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.696+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.700+0000 7fd2eb2be980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:23.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.700+0000 7fd2eb2be980 20 searching for the correct realm 2024-04-17T04:55:23.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.704+0000 7fd2e85d8640 0 -- 172.21.15.88:0/680677364 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x5647a1f61ed0 msgr2=0x5647a1f64370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:23.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.712+0000 7fd2e8dd9640 0 -- 172.21.15.88:0/680677364 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fd2d0062200 msgr2=0x7fd2d0058300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:23.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:23.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:23.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:23.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:23.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:23.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:23.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.716+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:23.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:23.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 zone default found 2024-04-17T04:55:23.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 4 Realm: () 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 zonegroup default 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 started sync module instance, tier type = 2024-04-17T04:55:23.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.720+0000 7fd2eb2be980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:23.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=0 2024-04-17T04:55:23.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=1 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=2 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=3 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=4 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=5 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=6 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 20 add_watcher() i=7 2024-04-17T04:55:23.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.736+0000 7fd2eb2be980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:23.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd19efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 5 note: GC not initialized 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd19dfe3640 20 reqs_thread_entry: start 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 20 Filter name: none 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd10f7fe640 20 reqs_thread_entry: start 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-04-17T04:55:23.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.740+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:23.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-04-17T04:55:23.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-04-17T04:55:23.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 : miss 2024-04-17T04:55:23.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:23.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:23.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:23.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:23.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:23.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.744+0000 7fd2eb2be980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 2024-04-17T04:55:23.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:23.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:23.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:23.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25, ofs0, ns] 2024-04-17T04:55:23.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd1bffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464289 cookie 94865659936400 notifier 6660 bl.length()=1007 2024-04-17T04:55:23.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd1bffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:23.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd1bffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:23.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd1bffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:23.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): orig_sources= new_sources= 2024-04-17T04:55:23.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): potential sources added=[] removed=[] 2024-04-17T04:55:23.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): orig_dests= new_dests= 2024-04-17T04:55:23.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.752+0000 7fd2eb2be980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): potential dests added=[] removed=[] 2024-04-17T04:55:23.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.756+0000 7fd2eb2be980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.1.0 2024-04-17T04:55:23.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.756+0000 7fd2eb2be980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.1.0 2024-04-17T04:55:23.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.756+0000 7fd2eb2be980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.1.1 2024-04-17T04:55:23.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.756+0000 7fd2eb2be980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.1.1 2024-04-17T04:55:23.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.756+0000 7fd2eb2be980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.0 2024-04-17T04:55:23.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.760+0000 7fd2eb2be980 -1 Injecting error=-5 at location=commit_target_layout 2024-04-17T04:55:23.760 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.760+0000 7fd2eb2be980 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-04-17T04:55:23.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.768+0000 7fd19efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:23.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.772+0000 7fd2eb2be980 20 remove_watcher() i=0 2024-04-17T04:55:23.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.772+0000 7fd2eb2be980 2 removed watcher, disabling cache 2024-04-17T04:55:23.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.776+0000 7fd2eb2be980 20 remove_watcher() i=1 2024-04-17T04:55:23.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.776+0000 7fd2eb2be980 20 remove_watcher() i=2 2024-04-17T04:55:23.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.780+0000 7fd2eb2be980 20 remove_watcher() i=3 2024-04-17T04:55:23.780 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.780+0000 7fd2eb2be980 20 remove_watcher() i=4 2024-04-17T04:55:23.781 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.784+0000 7fd2eb2be980 20 remove_watcher() i=5 2024-04-17T04:55:23.782 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.784+0000 7fd2eb2be980 20 remove_watcher() i=6 2024-04-17T04:55:23.784 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.784+0000 7fd2eb2be980 20 remove_watcher() i=7 2024-04-17T04:55:23.797 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-17T04:55:23.823 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:23.824 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:23.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 realm 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:23.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.868+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.872+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:23.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.872+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:23.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.872+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:23.870 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.872+0000 7fdb4c972980 20 searching for the correct realm 2024-04-17T04:55:23.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:23.880+0000 7fdb4ac8e640 0 -- 172.21.15.88:0/4043884513 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fdb40053710 msgr2=0x7fdb40068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:55:24.094 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.096+0000 7fdb4a48d640 0 -- 172.21.15.88:0/4043884513 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55c95eea4580 msgr2=0x55c95eea6970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:24.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:24.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 zone default found 2024-04-17T04:55:24.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 4 Realm: () 2024-04-17T04:55:24.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:24.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:24.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:24.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 zonegroup default 2024-04-17T04:55:24.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.104+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.106 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 started sync module instance, tier type = 2024-04-17T04:55:24.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.108+0000 7fdb4c972980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:24.130 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=0 2024-04-17T04:55:24.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=1 2024-04-17T04:55:24.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=2 2024-04-17T04:55:24.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=3 2024-04-17T04:55:24.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=4 2024-04-17T04:55:24.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=5 2024-04-17T04:55:24.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.132+0000 7fdb4c972980 20 add_watcher() i=6 2024-04-17T04:55:24.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.136+0000 7fdb4c972980 20 add_watcher() i=7 2024-04-17T04:55:24.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.136+0000 7fdb4c972980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:24.137 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fda00fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:24.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:24.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:24.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 5 note: GC not initialized 2024-04-17T04:55:24.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fd97b7fe640 20 reqs_thread_entry: start 2024-04-17T04:55:24.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:24.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 20 Filter name: none 2024-04-17T04:55:24.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fd9797fa640 20 reqs_thread_entry: start 2024-04-17T04:55:24.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.140+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:24.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.144+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.144+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=467 2024-04-17T04:55:24.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.144+0000 7fdb4c972980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:24.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.144+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.144+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:24.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.144+0000 7fdb4c972980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.148+0000 7fdb4c972980 20 rados_obj.operate() r=0 bl.length=467 2024-04-17T04:55:24.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.148+0000 7fdb4c972980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:24.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.148+0000 7fda00fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:24.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.156+0000 7fdb4c972980 20 remove_watcher() i=0 2024-04-17T04:55:24.153 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.156+0000 7fdb4c972980 2 removed watcher, disabling cache 2024-04-17T04:55:24.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.156+0000 7fdb4c972980 20 remove_watcher() i=1 2024-04-17T04:55:24.157 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.160+0000 7fdb4c972980 20 remove_watcher() i=2 2024-04-17T04:55:24.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.160+0000 7fdb4c972980 20 remove_watcher() i=3 2024-04-17T04:55:24.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.164+0000 7fdb4c972980 20 remove_watcher() i=4 2024-04-17T04:55:24.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.164+0000 7fdb4c972980 20 remove_watcher() i=5 2024-04-17T04:55:24.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.168+0000 7fdb4c972980 20 remove_watcher() i=6 2024-04-17T04:55:24.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.168+0000 7fdb4c972980 20 remove_watcher() i=7 2024-04-17T04:55:24.185 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:24.185 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:24.185 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:55:24.185 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:24.294 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:55:24.330 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:24.330 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:24.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 realm 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.376+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.380+0000 7fa2cff21980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:24.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.380+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.380+0000 7fa2cff21980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:24.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.380+0000 7fa2cff21980 20 searching for the correct realm 2024-04-17T04:55:24.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.384+0000 7fa2cd23b640 0 -- 172.21.15.88:0/2352089910 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x55ee009de1c0 msgr2=0x55ee009e0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:24.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.392+0000 7fa2cda3c640 0 -- 172.21.15.88:0/2352089910 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fa2c4045430 msgr2=0x7fa2c4045890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.608 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.608+0000 7fa2cff21980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:24.609 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 zone default found 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 4 Realm: () 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 zonegroup default 2024-04-17T04:55:24.610 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.611 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 started sync module instance, tier type = 2024-04-17T04:55:24.612 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.612+0000 7fa2cff21980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.636+0000 7fa2cff21980 20 add_watcher() i=0 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=1 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=2 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=3 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=4 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=5 2024-04-17T04:55:24.637 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=6 2024-04-17T04:55:24.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 20 add_watcher() i=7 2024-04-17T04:55:24.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.640+0000 7fa2cff21980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:24.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.644+0000 7fa183fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:24.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.644+0000 7fa2cff21980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:24.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.644+0000 7fa2cff21980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:24.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 5 note: GC not initialized 2024-04-17T04:55:24.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa182fe5640 20 reqs_thread_entry: start 2024-04-17T04:55:24.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:24.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 20 Filter name: none 2024-04-17T04:55:24.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa180fe1640 20 reqs_thread_entry: start 2024-04-17T04:55:24.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-04-17T04:55:24.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:24.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-04-17T04:55:24.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-04-17T04:55:24.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 : miss 2024-04-17T04:55:24.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 20 rados_obj.operate() r=0 bl.length=467 2024-04-17T04:55:24.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:24.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:24.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:24.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:24.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.648+0000 7fa2cff21980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 2024-04-17T04:55:24.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.652+0000 7fa2cff21980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa2cff21980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa2cff21980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa2cff21980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa2cff21980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25, ofs0, ns] 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa1b17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464290 cookie 94480700932848 notifier 6699 bl.length()=1007 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa1b17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:24.662 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa1b17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:24.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.660+0000 7fa1b17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:24.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.664+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): orig_sources= new_sources= 2024-04-17T04:55:24.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.664+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): potential sources added=[] removed=[] 2024-04-17T04:55:24.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.664+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): orig_dests= new_dests= 2024-04-17T04:55:24.663 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.664+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): potential dests added=[] removed=[] 2024-04-17T04:55:24.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.668+0000 7fa2cff21980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.2.0 2024-04-17T04:55:24.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.668+0000 7fa2cff21980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.2.0 2024-04-17T04:55:24.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.668+0000 7fa2cff21980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.2.1 2024-04-17T04:55:24.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.668+0000 7fa2cff21980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.2.1 2024-04-17T04:55:24.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.668+0000 7fa2cff21980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25.0 2024-04-17T04:55:24.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.672+0000 7fa2cff21980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 : hit (requested=0x17, cached=0x17) 2024-04-17T04:55:24.672 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.672+0000 7fa2cff21980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 2024-04-17T04:55:24.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa2cff21980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:24.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa2cff21980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:24.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa2cff21980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:24.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa2cff21980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25, ofs0, ns] 2024-04-17T04:55:24.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa1b0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464291 cookie 94480700932848 notifier 6699 bl.length()=975 2024-04-17T04:55:24.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa1b0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 info.flags=0x17 2024-04-17T04:55:24.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa1b0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 to cache LRU end 2024-04-17T04:55:24.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.676+0000 7fa1b0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:24.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.680+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): orig_sources= new_sources= 2024-04-17T04:55:24.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.680+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): potential sources added=[] removed=[] 2024-04-17T04:55:24.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.680+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): orig_dests= new_dests= 2024-04-17T04:55:24.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.680+0000 7fa2cff21980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25]): potential dests added=[] removed=[] 2024-04-17T04:55:24.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.684+0000 7fa2cff21980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-04-17T04:55:24.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.688+0000 7fa183fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:24.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.692+0000 7fa2cff21980 20 remove_watcher() i=0 2024-04-17T04:55:24.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.692+0000 7fa2cff21980 2 removed watcher, disabling cache 2024-04-17T04:55:24.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.696+0000 7fa2cff21980 20 remove_watcher() i=1 2024-04-17T04:55:24.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.696+0000 7fa2cff21980 20 remove_watcher() i=2 2024-04-17T04:55:24.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.700+0000 7fa2cff21980 20 remove_watcher() i=3 2024-04-17T04:55:24.700 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.704+0000 7fa2cff21980 20 remove_watcher() i=4 2024-04-17T04:55:24.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.704+0000 7fa2cff21980 20 remove_watcher() i=5 2024-04-17T04:55:24.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.704+0000 7fa2cff21980 20 remove_watcher() i=6 2024-04-17T04:55:24.706 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.708+0000 7fa2cff21980 20 remove_watcher() i=7 2024-04-17T04:55:24.723 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-17T04:55:24.753 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:24.753 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:24.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.796+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 realm 2024-04-17T04:55:24.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:24.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:24.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.800+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.804+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:24.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.804+0000 7f374b3bc980 20 searching for the correct realm 2024-04-17T04:55:24.807 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.808+0000 7f37486d6640 0 -- 172.21.15.88:0/2287821508 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f3738008070 msgr2=0x7f3738085000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:24.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.816+0000 7f37486d6640 0 -- 172.21.15.88:0/2287821508 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f36140763d0 msgr2=0x7f3614078800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.820+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.821 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:24.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:24.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 zone default found 2024-04-17T04:55:24.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 4 Realm: () 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 zonegroup default 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:24.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 started sync module instance, tier type = 2024-04-17T04:55:24.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.824+0000 7f374b3bc980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:24.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=0 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=1 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=2 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=3 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=4 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=5 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=6 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 20 add_watcher() i=7 2024-04-17T04:55:24.840 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.840+0000 7f374b3bc980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:24.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f35ff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:24.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f374b3bc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:24.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f374b3bc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:24.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f374b3bc980 5 note: GC not initialized 2024-04-17T04:55:24.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f35fe7e4640 20 reqs_thread_entry: start 2024-04-17T04:55:24.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f374b3bc980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:24.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f374b3bc980 20 Filter name: none 2024-04-17T04:55:24.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f35f77fe640 20 reqs_thread_entry: start 2024-04-17T04:55:24.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.844+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:24.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:24.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:24.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:24.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados->read ofs=0 len=0 2024-04-17T04:55:24.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:24.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.848+0000 7f374b3bc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:24.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.852+0000 7f35ff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:24.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.856+0000 7f374b3bc980 20 remove_watcher() i=0 2024-04-17T04:55:24.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.856+0000 7f374b3bc980 2 removed watcher, disabling cache 2024-04-17T04:55:24.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.860+0000 7f374b3bc980 20 remove_watcher() i=1 2024-04-17T04:55:24.859 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.860+0000 7f374b3bc980 20 remove_watcher() i=2 2024-04-17T04:55:24.861 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.864+0000 7f374b3bc980 20 remove_watcher() i=3 2024-04-17T04:55:24.864 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.864+0000 7f374b3bc980 20 remove_watcher() i=4 2024-04-17T04:55:24.865 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.868+0000 7f374b3bc980 20 remove_watcher() i=5 2024-04-17T04:55:24.866 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.868+0000 7f374b3bc980 20 remove_watcher() i=6 2024-04-17T04:55:24.867 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:24.868+0000 7f374b3bc980 20 remove_watcher() i=7 2024-04-17T04:55:24.880 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:24.880 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:24.881 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:55:24.881 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:25.613 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-04-17T04:55:25.613 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:27.850 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-17T04:55:27.883 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:27.883 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:27.927 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.928+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 realm 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.930 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:27.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:27.932 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.932+0000 7f00e28db980 20 searching for the correct realm 2024-04-17T04:55:27.937 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.940+0000 7f00e0bf7640 0 -- 172.21.15.88:0/2753405909 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x564287820850 msgr2=0x564287822ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:55:27.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.944+0000 7f00e03f6640 0 -- 172.21.15.88:0/2753405909 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f00d40617b0 msgr2=0x7f00d405df30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:27.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:27.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:27.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:27.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:27.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:27.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:27.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.952+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:27.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 zone default found 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 4 Realm: () 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 zonegroup default 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:27.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 started sync module instance, tier type = 2024-04-17T04:55:27.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.956+0000 7f00e28db980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=0 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=1 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=2 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=3 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=4 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=5 2024-04-17T04:55:27.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.972+0000 7f00e28db980 20 add_watcher() i=6 2024-04-17T04:55:27.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 20 add_watcher() i=7 2024-04-17T04:55:27.973 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:27.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7eff96fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:27.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:27.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 5 note: GC not initialized 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7eff95fdb640 20 reqs_thread_entry: start 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 20 Filter name: none 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7eff077fe640 20 reqs_thread_entry: start 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.976+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:27.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:27.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:27.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:27.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados->read ofs=0 len=0 2024-04-17T04:55:27.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:27.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.980+0000 7f00e28db980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:27.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.984+0000 7eff96fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:27.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.988+0000 7f00e28db980 20 remove_watcher() i=0 2024-04-17T04:55:27.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.988+0000 7f00e28db980 2 removed watcher, disabling cache 2024-04-17T04:55:27.989 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.992+0000 7f00e28db980 20 remove_watcher() i=1 2024-04-17T04:55:27.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.992+0000 7f00e28db980 20 remove_watcher() i=2 2024-04-17T04:55:27.993 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.996+0000 7f00e28db980 20 remove_watcher() i=3 2024-04-17T04:55:27.995 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.996+0000 7f00e28db980 20 remove_watcher() i=4 2024-04-17T04:55:27.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:27.996+0000 7f00e28db980 20 remove_watcher() i=5 2024-04-17T04:55:27.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.000+0000 7f00e28db980 20 remove_watcher() i=6 2024-04-17T04:55:27.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.000+0000 7f00e28db980 20 remove_watcher() i=7 2024-04-17T04:55:28.012 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:28.012 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:28.012 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:55:28.012 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:28.013 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-04-17T04:55:28.046 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:28.046 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:28.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.088+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 realm 2024-04-17T04:55:28.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:28.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:28.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:28.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.092+0000 7f848acba980 20 searching for the correct realm 2024-04-17T04:55:28.099 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.100+0000 7f8488fd6640 0 -- 172.21.15.88:0/2250834742 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f8470034040 msgr2=0x7f847005d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:55:28.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.316+0000 7f84887d5640 0 -- 172.21.15.88:0/2250834742 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55ae064f5b00 msgr2=0x55ae064f7f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:28.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:28.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 zone default found 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 4 Realm: () 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 zonegroup default 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.324+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 started sync module instance, tier type = 2024-04-17T04:55:28.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.328+0000 7f848acba980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:28.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=0 2024-04-17T04:55:28.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=1 2024-04-17T04:55:28.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=2 2024-04-17T04:55:28.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=3 2024-04-17T04:55:28.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=4 2024-04-17T04:55:28.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=5 2024-04-17T04:55:28.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.348+0000 7f848acba980 20 add_watcher() i=6 2024-04-17T04:55:28.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.352+0000 7f848acba980 20 add_watcher() i=7 2024-04-17T04:55:28.349 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.352+0000 7f848acba980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:28.355 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.356+0000 7f833dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:28.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.356+0000 7f848acba980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:28.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.356+0000 7f848acba980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:28.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.356+0000 7f848acba980 5 note: GC not initialized 2024-04-17T04:55:28.357 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f833cfe1640 20 reqs_thread_entry: start 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 20 Filter name: none 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f82b67fc640 20 reqs_thread_entry: start 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:28.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-04-17T04:55:28.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-04-17T04:55:28.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 : miss 2024-04-17T04:55:28.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:28.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 info.flags=0x17 2024-04-17T04:55:28.359 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 to cache LRU end 2024-04-17T04:55:28.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:28.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:28.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.360+0000 7f848acba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 2024-04-17T04:55:28.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f848acba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 info.flags=0x17 2024-04-17T04:55:28.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f848acba980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 to cache LRU end 2024-04-17T04:55:28.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f848acba980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:28.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f848acba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26, ofs0, ns] 2024-04-17T04:55:28.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f836affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464294 cookie 94205918578464 notifier 6410 bl.length()=1007 2024-04-17T04:55:28.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f836affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 info.flags=0x17 2024-04-17T04:55:28.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f836affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 to cache LRU end 2024-04-17T04:55:28.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.368+0000 7f836affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:28.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.372+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): orig_sources= new_sources= 2024-04-17T04:55:28.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.372+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): potential sources added=[] removed=[] 2024-04-17T04:55:28.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.372+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): orig_dests= new_dests= 2024-04-17T04:55:28.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.372+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): potential dests added=[] removed=[] 2024-04-17T04:55:28.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.376+0000 7f848acba980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26.1.0 2024-04-17T04:55:28.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.376+0000 7f848acba980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26.1.0 2024-04-17T04:55:28.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.376+0000 7f848acba980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26.1.1 2024-04-17T04:55:28.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.376+0000 7f848acba980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26.1.1 2024-04-17T04:55:28.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.376+0000 7f848acba980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26.0 2024-04-17T04:55:28.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 -1 Injecting error=-125 at location=commit_target_layout 2024-04-17T04:55:28.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 : hit (requested=0x17, cached=0x17) 2024-04-17T04:55:28.378 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 2024-04-17T04:55:28.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 info.flags=0x17 2024-04-17T04:55:28.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 to cache LRU end 2024-04-17T04:55:28.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:28.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f848acba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26, ofs0, ns] 2024-04-17T04:55:28.380 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f836a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464295 cookie 94205918578464 notifier 6410 bl.length()=975 2024-04-17T04:55:28.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f836a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 info.flags=0x17 2024-04-17T04:55:28.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f836a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 to cache LRU end 2024-04-17T04:55:28.381 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.380+0000 7f836a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:28.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.384+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): orig_sources= new_sources= 2024-04-17T04:55:28.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.384+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): potential sources added=[] removed=[] 2024-04-17T04:55:28.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.384+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): orig_dests= new_dests= 2024-04-17T04:55:28.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.384+0000 7f848acba980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26]): potential dests added=[] removed=[] 2024-04-17T04:55:28.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.388+0000 7f848acba980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-04-17T04:55:28.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.392+0000 7f833dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:28.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.396+0000 7f848acba980 20 remove_watcher() i=0 2024-04-17T04:55:28.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.396+0000 7f848acba980 2 removed watcher, disabling cache 2024-04-17T04:55:28.397 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.400+0000 7f848acba980 20 remove_watcher() i=1 2024-04-17T04:55:28.399 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.400+0000 7f848acba980 20 remove_watcher() i=2 2024-04-17T04:55:28.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.404+0000 7f848acba980 20 remove_watcher() i=3 2024-04-17T04:55:28.404 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.404+0000 7f848acba980 20 remove_watcher() i=4 2024-04-17T04:55:28.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.408+0000 7f848acba980 20 remove_watcher() i=5 2024-04-17T04:55:28.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.408+0000 7f848acba980 20 remove_watcher() i=6 2024-04-17T04:55:28.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.412+0000 7f848acba980 20 remove_watcher() i=7 2024-04-17T04:55:28.432 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-17T04:55:28.463 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:28.463 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:28.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 realm 2024-04-17T04:55:28.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:28.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.512+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.516+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:28.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.516+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.516+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:28.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.516+0000 7f373e078980 20 searching for the correct realm 2024-04-17T04:55:28.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.520+0000 7f373bb93640 0 -- 172.21.15.88:0/1099918275 >> [v2:172.21.15.88:6804/508820735,v1:172.21.15.88:6806/508820735] conn(0x7f3724003ac0 msgr2=0x7f3734076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:55:28.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.524+0000 7f373c394640 0 -- 172.21.15.88:0/1099918275 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f37300a3f40 msgr2=0x7f37300c9450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:28.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:28.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 zone default found 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 4 Realm: () 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 zonegroup default 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.536+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.540+0000 7f373e078980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:28.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.540+0000 7f373e078980 20 started sync module instance, tier type = 2024-04-17T04:55:28.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.540+0000 7f373e078980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:28.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=0 2024-04-17T04:55:28.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=1 2024-04-17T04:55:28.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=2 2024-04-17T04:55:28.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=3 2024-04-17T04:55:28.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=4 2024-04-17T04:55:28.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=5 2024-04-17T04:55:28.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.552+0000 7f373e078980 20 add_watcher() i=6 2024-04-17T04:55:28.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.556+0000 7f373e078980 20 add_watcher() i=7 2024-04-17T04:55:28.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.556+0000 7f373e078980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:28.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.556+0000 7f35f1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.556+0000 7f373e078980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.556+0000 7f373e078980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.556+0000 7f373e078980 5 note: GC not initialized 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f35f0fe1640 20 reqs_thread_entry: start 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 Filter name: none 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f356a7fc640 20 reqs_thread_entry: start 2024-04-17T04:55:28.557 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:28.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:28.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:28.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:28.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados->read ofs=0 len=0 2024-04-17T04:55:28.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:28.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.560+0000 7f373e078980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:28.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.564+0000 7f35f1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:28.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.568+0000 7f373e078980 20 remove_watcher() i=0 2024-04-17T04:55:28.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.568+0000 7f373e078980 2 removed watcher, disabling cache 2024-04-17T04:55:28.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.572+0000 7f373e078980 20 remove_watcher() i=1 2024-04-17T04:55:28.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.572+0000 7f373e078980 20 remove_watcher() i=2 2024-04-17T04:55:28.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.576+0000 7f373e078980 20 remove_watcher() i=3 2024-04-17T04:55:28.576 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.576+0000 7f373e078980 20 remove_watcher() i=4 2024-04-17T04:55:28.577 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.580+0000 7f373e078980 20 remove_watcher() i=5 2024-04-17T04:55:28.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.580+0000 7f373e078980 20 remove_watcher() i=6 2024-04-17T04:55:28.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:28.580+0000 7f373e078980 20 remove_watcher() i=7 2024-04-17T04:55:28.594 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:28.594 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:28.594 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26 num_objects: 20 size_kb: 1 num_shards: 2 2024-04-17T04:55:28.595 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:29.369 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-04-17T04:55:29.370 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:31.653 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-04-17T04:55:31.686 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:31.687 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:31.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.736+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.736+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.736+0000 7f22ea323980 20 realm 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.736+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:31.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:31.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.740+0000 7f22ea323980 20 searching for the correct realm 2024-04-17T04:55:31.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.748+0000 7f22e863f640 0 -- 172.21.15.88:0/2282540888 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f22dc010950 msgr2=0x7f22dc021470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:31.747 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.752+0000 7f22e7e3e640 0 -- 172.21.15.88:0/2282540888 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f22dc03e910 msgr2=0x7f22dc02f420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:31.963 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.964 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.968+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:31.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 zone default found 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 4 Realm: () 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:31.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 zonegroup default 2024-04-17T04:55:31.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:31.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:31.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 started sync module instance, tier type = 2024-04-17T04:55:31.968 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.972+0000 7f22ea323980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:31.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=0 2024-04-17T04:55:31.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=1 2024-04-17T04:55:31.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=2 2024-04-17T04:55:31.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=3 2024-04-17T04:55:31.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=4 2024-04-17T04:55:31.990 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=5 2024-04-17T04:55:31.991 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=6 2024-04-17T04:55:31.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 20 add_watcher() i=7 2024-04-17T04:55:31.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:31.996+0000 7f22ea323980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:31.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f219dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:31.997 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:31.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:31.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 5 note: GC not initialized 2024-04-17T04:55:31.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f219cfe1640 20 reqs_thread_entry: start 2024-04-17T04:55:31.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:31.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 Filter name: none 2024-04-17T04:55:31.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f21167fc640 20 reqs_thread_entry: start 2024-04-17T04:55:31.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.000 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:32.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:32.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.004+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.008+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.008+0000 7f22ea323980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.008+0000 7f22ea323980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:32.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.008+0000 7f22ea323980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:32.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.008+0000 7f219dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:32.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.016+0000 7f22ea323980 20 remove_watcher() i=0 2024-04-17T04:55:32.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.016+0000 7f22ea323980 2 removed watcher, disabling cache 2024-04-17T04:55:32.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.016+0000 7f22ea323980 20 remove_watcher() i=1 2024-04-17T04:55:32.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.020+0000 7f22ea323980 20 remove_watcher() i=2 2024-04-17T04:55:32.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.020+0000 7f22ea323980 20 remove_watcher() i=3 2024-04-17T04:55:32.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.024+0000 7f22ea323980 20 remove_watcher() i=4 2024-04-17T04:55:32.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.024+0000 7f22ea323980 20 remove_watcher() i=5 2024-04-17T04:55:32.022 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.028+0000 7f22ea323980 20 remove_watcher() i=6 2024-04-17T04:55:32.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.028+0000 7f22ea323980 20 remove_watcher() i=7 2024-04-17T04:55:32.042 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.042 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:32.042 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:55:32.042 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.042 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-04-17T04:55:32.074 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:32.074 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:32.116 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.120+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 realm 2024-04-17T04:55:32.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:32.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.118 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:32.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 searching for the correct realm 2024-04-17T04:55:32.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.132+0000 7fbd1bac2640 0 -- 172.21.15.88:0/3542194109 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbd0c093410 msgr2=0x7fbd0c009410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.140+0000 7fbd1bac2640 0 -- 172.21.15.88:0/3542194109 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbbe40763a0 msgr2=0x7fbbe40787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:32.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:32.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:32.140 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.148+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.148+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:32.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 zone default found 2024-04-17T04:55:32.339 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 Realm: () 2024-04-17T04:55:32.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:32.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:32.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:32.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 zonegroup default 2024-04-17T04:55:32.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 started sync module instance, tier type = 2024-04-17T04:55:32.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:32.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=0 2024-04-17T04:55:32.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=1 2024-04-17T04:55:32.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=2 2024-04-17T04:55:32.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=3 2024-04-17T04:55:32.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=4 2024-04-17T04:55:32.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=5 2024-04-17T04:55:32.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=6 2024-04-17T04:55:32.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=7 2024-04-17T04:55:32.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.372+0000 7fbd1e7a8980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:32.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.376+0000 7fbbd2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:32.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:32.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:32.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 note: GC not initialized 2024-04-17T04:55:32.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbbd1fdb640 20 reqs_thread_entry: start 2024-04-17T04:55:32.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:32.374 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 Filter name: none 2024-04-17T04:55:32.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbb437fe640 20 reqs_thread_entry: start 2024-04-17T04:55:32.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-17T04:55:32.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-17T04:55:32.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-17T04:55:32.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 : miss 2024-04-17T04:55:32.376 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:32.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:32.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.377 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 2024-04-17T04:55:32.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27, ofs0, ns] 2024-04-17T04:55:32.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464304 cookie 94262600597344 notifier 6813 bl.length()=1007 2024-04-17T04:55:32.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_sources= new_sources= 2024-04-17T04:55:32.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential sources added=[] removed=[] 2024-04-17T04:55:32.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_dests= new_dests= 2024-04-17T04:55:32.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential dests added=[] removed=[] 2024-04-17T04:55:32.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.400+0000 7fbd1e7a8980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:55:32.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:55:32.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.400+0000 7fbd1e7a8980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:55:32.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:55:32.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.0 2024-04-17T04:55:32.401 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd1e7a8980 time 2024-04-17T04:55:32.409467+0000 2024-04-17T04:55:32.401 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:55:32.404 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.404 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd2207fe1f] 2024-04-17T04:55:32.404 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.404 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.404 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.408+0000 7fbd1e7a8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd1e7a8980 time 2024-04-17T04:55:32.409467+0000 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd2207fe1f] 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:55:32.405 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.406 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:55:32.406 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:55:32.406 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.406 INFO:tasks.workunit.client.0.smithi088.stderr:*** Caught signal (Aborted) ** 2024-04-17T04:55:32.406 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7fbd1e7a8980 thread_name:radosgw-admin 2024-04-17T04:55:32.407 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.407 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd20f01520] 2024-04-17T04:55:32.407 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:55:32.407 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd2207fe79] 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.412+0000 7fbd1e7a8980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7fbd1e7a8980 thread_name:radosgw-admin 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.408 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd20f01520] 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd2207fe79] 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:55:32.409 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:55:32.410 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.410 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6446 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: dump: 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.463 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: get_monmap_and_config success 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 4 set_mon_vals no callback set 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: shutdown 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) unregister_commands rotate-key 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.464 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding auth protocol: cephx 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding auth protocol: cephx 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding auth protocol: cephx 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.465 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: starting msgr at 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: starting objecter 2024-04-17T04:55:32.466 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command objecter_requests hook 0x55bb389d9c70 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: setting wanted keys 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: calling monclient init 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding auth protocol: cephx 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding auth protocol: cephx 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding auth protocol: cephx 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.467 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key hook 0x55bb38c35e48 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38ae6640 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38af2d10 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38c64bd0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.468 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.753028s 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38af2d10 auth_method 0 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request con 0x55bb38c64bd0 auth_method 0 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_done global_id 6804 payload 824 2024-04-17T04:55:32.469 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 1 monclient: found mon.noname-a 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: dump: 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.470 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: dump: 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.471 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6804 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:55:32.104+0000 7fbc0a60c640 4 set_mon_vals no callback set 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: init done 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:55:32.104+0000 7fbc08e09640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:55:32.104+0000 7fbc08e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:55:32.104+0000 7fbc08e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:55:32.104+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x7fbc04065e30 auth_method 0 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:55:32.108+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x55bb38c630f0 auth_method 0 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:55:32.108+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x55bb38c80c80 auth_method 0 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:55:32.108+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x55bb38c84560 auth_method 0 2024-04-17T04:55:32.472 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding auth protocol: cephx 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding auth protocol: cephx 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding auth protocol: cephx 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.473 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command objecter_requests cmddesc objecter_requests hook 0x55bb38c8f590 EEXIST 2024-04-17T04:55:32.474 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding auth protocol: cephx 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding auth protocol: cephx 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding auth protocol: cephx 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.475 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key cmddesc rotate-key hook 0x55bb38c89050 EEXIST 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38cb41f0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38cb45d0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38caf790 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.762423s 2024-04-17T04:55:32.476 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request con 0x55bb38cb41f0 auth_method 0 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38cb45d0 auth_method 0 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.477 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request con 0x55bb38caf790 auth_method 0 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_done global_id 6807 payload 824 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 1 monclient: found mon.noname-a 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: dump: 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.478 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: dump: 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.479 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6807 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:55:32.112+0000 7fbd1d2c5640 4 set_mon_vals no callback set 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:55:32.116+0000 7fbc01ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:55:32.116+0000 7fbc01ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.480 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:55:32.116+0000 7fbc01ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbf4065d40 auth_method 0 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: get_monmap_and_config 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.481 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.482 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding auth protocol: cephx 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding auth protocol: cephx 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding auth protocol: cephx 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.483 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key cmddesc rotate-key hook 0x7fffb2a60d08 EEXIST 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38d064a0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38d019b0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38d0b690 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.484 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.766325s 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38d019b0 auth_method 0 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request con 0x55bb38d064a0 auth_method 0 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.485 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_done global_id 6455 payload 274 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 1 monclient: found mon.noname-c 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: dump: 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.486 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6455 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: dump: 2024-04-17T04:55:32.487 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: get_monmap_and_config success 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 4 set_mon_vals no callback set 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: shutdown 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.488 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.489 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: starting msgr at 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: starting objecter 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command objecter_requests cmddesc objecter_requests hook 0x55bb38cc66c0 EEXIST 2024-04-17T04:55:32.490 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: setting wanted keys 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: calling monclient init 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding auth protocol: cephx 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding auth protocol: cephx 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding auth protocol: cephx 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.491 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key cmddesc rotate-key hook 0x55bb38cd1058 EEXIST 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[0,1,2] 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38d019b0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38d064a0 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38d0b690 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.492 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.768753s 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request con 0x55bb38d0b690 auth_method 0 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request con 0x55bb38d019b0 auth_method 0 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.493 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38d064a0 auth_method 0 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_done global_id 6813 payload 824 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 1 monclient: found mon.noname-a 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: dump: 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.494 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: dump: 2024-04-17T04:55:32.495 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6813 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:55:32.120+0000 7fbbf37fe640 4 set_mon_vals no callback set 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.496 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: init done 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cr dump hook 0x55bb38cf9b38 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe4065e00 auth_method 0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x55bb38d208b0 auth_method 0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 realm 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:32.497 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:55:32.124+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x55bb38d24ad0 auth_method 0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 searching for the correct realm 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:55:32.124+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x55bb38d29d40 auth_method 0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:55:32.128+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x7fbd04018070 auth_method 0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:55:32.128+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x7fbd140c1920 auth_method 0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:55:32.128+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbd0c093410 auth_method 0 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:55:32.132+0000 7fbd1bac2640 0 -- 172.21.15.88:0/3542194109 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbd0c093410 msgr2=0x7fbd0c009410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.498 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:55:32.132+0000 7fbbf1ffb640 1 client.6813.objecter ms_handle_reset 0x7fbd0c093410 session 0x7fbd0c064650 osd.4 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:55:32.132+0000 7fbbf1ffb640 10 monclient: _renew_subs 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:55:32.132+0000 7fbbf1ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:55:32.132+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe40763a0 auth_method 0 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:55:32.132+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x7fbd040826e0 auth_method 0 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:55:32.136+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x7fbd040415c0 auth_method 0 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:55:32.140+0000 7fbd1bac2640 0 -- 172.21.15.88:0/3542194109 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbbe40763a0 msgr2=0x7fbbe40787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:55:32.140+0000 7fbbf1ffb640 1 client.6813.objecter ms_handle_reset 0x7fbbe40763a0 session 0x7fbd0c064650 osd.4 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:55:32.140+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe4079120 auth_method 0 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:32.499 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:55:32.148+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:55:32.148+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:55:32.344+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe4079120 auth_method 0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 zone default found 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 Realm: () 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 zonegroup default 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.500 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 started sync module instance, tier type = 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=0 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=1 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=2 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=3 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=4 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=5 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=6 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=7 2024-04-17T04:55:32.501 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding auth protocol: cephx 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding auth protocol: cephx 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding auth protocol: cephx 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:55:32.376+0000 7fbbd2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.502 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache list hook 0x55bb38cf5d80 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache inspect hook 0x55bb38cf5d80 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache erase hook 0x55bb38cf5d80 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache zap hook 0x55bb38cf5d80 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 note: GC not initialized 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:55:32.380+0000 7fbbd1fdb640 20 reqs_thread_entry: start 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace show hook 0x7fbd14004730 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace history hook 0x7fbd14004730 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace active hook 0x7fbd14004730 2024-04-17T04:55:32.503 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace active_short hook 0x7fbd14004730 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 Filter name: none 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:55:32.380+0000 7fbb437fe640 20 reqs_thread_entry: start 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 : miss 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.504 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27, ofs0, ns] 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464304 cookie 94262600597344 notifier 6813 bl.length()=1007 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_sources= new_sources= 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential sources added=[] removed=[] 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_dests= new_dests= 2024-04-17T04:55:32.505 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential dests added=[] removed=[] 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.0 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:55:32.408+0000 7fbd1e7a8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd1e7a8980 time 2024-04-17T04:55:32.409467+0000 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd2207fe1f] 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.506 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:55:32.412+0000 7fbd1e7a8980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7fbd1e7a8980 thread_name:radosgw-admin 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd20f01520] 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:55:32.507 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd2207fe79] 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:55:32.508 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:55:32.509 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:55:32.510 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:55:32.511 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:55:32.512 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:55:32.513 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbb437fe640 / http_manager 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbd1fdb640 / http_manager 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbd2fdd640 / rgw_dt_lg_renew 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbf1ffb640 / ms_dispatch 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbf37fe640 / io_context_pool 2024-04-17T04:55:32.514 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbc01ffb640 / ms_dispatch 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbc08e09640 / ms_dispatch 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbc0a60c640 / io_context_pool 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1bac2640 / msgr-worker-2 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1c2c3640 / msgr-worker-1 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1cac4640 / msgr-worker-0 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1d2c5640 / io_context_pool 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1e7a8980 / radosgw-admin 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/log/ceph/ceph-client.admin.97076.log 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.515 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6446 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:55:32.100+0000 7fbc0a60c640 10 monclient: dump: 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.516 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: get_monmap_and_config success 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 4 set_mon_vals no callback set 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: shutdown 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) unregister_commands rotate-key 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.517 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding auth protocol: cephx 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding auth protocol: cephx 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding auth protocol: cephx 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.518 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: crc 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:55:32.100+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c37b40) adding con mode: secure 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: starting msgr at 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: starting objecter 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command objecter_requests hook 0x55bb389d9c70 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: setting wanted keys 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: calling monclient init 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding auth protocol: cephx 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding auth protocol: cephx 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding auth protocol: cephx 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.519 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: crc 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c35d00) adding con mode: secure 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key hook 0x55bb38c35e48 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.520 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38ae6640 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38af2d10 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38c64bd0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.753028s 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38af2d10 auth_method 0 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request con 0x55bb38c64bd0 auth_method 0 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.521 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:55:32.104+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_done global_id 6804 payload 824 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 1 monclient: found mon.noname-a 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:55:32.104+0000 7fbd1bac2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: dump: 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.522 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:55:32.104+0000 7fbc08e09640 10 monclient: dump: 2024-04-17T04:55:32.523 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6804 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:55:32.104+0000 7fbc0a60c640 4 set_mon_vals no callback set 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.524 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:55:32.104+0000 7fbd1e7a8980 1 librados: init done 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:55:32.104+0000 7fbc08e09640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:55:32.104+0000 7fbc08e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:55:32.104+0000 7fbc08e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:55:32.104+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x7fbc04065e30 auth_method 0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:55:32.108+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x55bb38c630f0 auth_method 0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:55:32.108+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x55bb38c80c80 auth_method 0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:55:32.108+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x55bb38c84560 auth_method 0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.525 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding auth protocol: cephx 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding auth protocol: cephx 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding auth protocol: cephx 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.526 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: crc 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c89fc0) adding con mode: secure 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command objecter_requests cmddesc objecter_requests hook 0x55bb38c8f590 EEXIST 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding auth protocol: cephx 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding auth protocol: cephx 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding auth protocol: cephx 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.527 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: crc 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38c88f08) adding con mode: secure 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key cmddesc rotate-key hook 0x55bb38c89050 EEXIST 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38cb41f0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38cb45d0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.528 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38caf790 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.762423s 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request con 0x55bb38cb41f0 auth_method 0 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38cb45d0 auth_method 0 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.529 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:55:32.112+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request con 0x55bb38caf790 auth_method 0 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:55:32.112+0000 7fbd1cac4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_done global_id 6807 payload 824 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 1 monclient: found mon.noname-a 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:55:32.112+0000 7fbd1c2c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:55:32.530 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: dump: 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.531 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:55:32.112+0000 7fbc01ffb640 10 monclient: dump: 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6807 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:55:32.112+0000 7fbd1d2c5640 4 set_mon_vals no callback set 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.532 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:55:32.112+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:55:32.116+0000 7fbc01ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:55:32.116+0000 7fbc01ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:55:32.116+0000 7fbc01ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbf4065d40 auth_method 0 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: get_monmap_and_config 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.533 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.534 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding auth protocol: cephx 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding auth protocol: cephx 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding auth protocol: cephx 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.535 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: crc 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a60bc0) adding con mode: secure 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key cmddesc rotate-key hook 0x7fffb2a60d08 EEXIST 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[0,2,1] 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38d064a0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38d019b0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38d0b690 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.536 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.766325s 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38d019b0 auth_method 0 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request con 0x55bb38d064a0 auth_method 0 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.537 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:55:32.116+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient(hunting): handle_auth_done global_id 6455 payload 274 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 1 monclient: found mon.noname-c 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:55:32.116+0000 7fbd1bac2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: dump: 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.538 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6455 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:55:32.539 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:55:32.116+0000 7fbbf37fe640 10 monclient: dump: 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: get_monmap_and_config success 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 4 set_mon_vals no callback set 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: shutdown 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: build_initial_monmap 2024-04-17T04:55:32.540 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 1 build_initial for_mkfs: 0 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 10 monclient: monmap: 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding auth protocol: cephx 2024-04-17T04:55:32.541 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: crc 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd1d40) adding con mode: secure 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:55:32.116+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: starting msgr at 2024-04-17T04:55:32.542 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: starting objecter 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command objecter_requests cmddesc objecter_requests hook 0x55bb38cc66c0 EEXIST 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: setting wanted keys 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: calling monclient init 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: init 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding auth protocol: cephx 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding auth protocol: cephx 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding auth protocol: cephx 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.543 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: crc 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 AuthRegistry(0x55bb38cd0f10) adding con mode: secure 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command rotate-key cmddesc rotate-key hook 0x55bb38cd1058 EEXIST 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _reopen_session rank -1 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _add_conns ranks=[0,1,2] 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-a con 0x55bb38d019b0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-b con 0x55bb38d064a0 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): picked mon.noname-c con 0x55bb38d0b690 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.544 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): start opening mon connection 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): _renew_subs 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient(hunting): authenticate will time out at 2133.768753s 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request con 0x55bb38d0b690 auth_method 0 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request con 0x55bb38d019b0 auth_method 0 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.545 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request con 0x55bb38d064a0 auth_method 0 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient(hunting): handle_auth_done global_id 6813 payload 824 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient: _finish_hunting 0 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 1 monclient: found mon.noname-a 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:55:32.120+0000 7fbd1c2c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: dump: 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.546 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: _finish_auth 0 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: _check_auth_tickets 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 10 monclient: dump: 2024-04-17T04:55:32.547 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 monclient: authenticate success, global_id 6813 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:55:32.120+0000 7fbbf37fe640 4 set_mon_vals no callback set 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.548 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _renew_subs 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 1 librados: init done 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cr dump hook 0x55bb38cf9b38 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:55:32.120+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:55:32.120+0000 7fbbf1ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:55:32.120+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe4065e00 auth_method 0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:55:32.120+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x55bb38d208b0 auth_method 0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 realm 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.549 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:55:32.124+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x55bb38d24ad0 auth_method 0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:55:32.124+0000 7fbd1e7a8980 20 searching for the correct realm 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:55:32.124+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x55bb38d29d40 auth_method 0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:55:32.128+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x7fbd04018070 auth_method 0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:55:32.128+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x7fbd140c1920 auth_method 0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:55:32.128+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbd0c093410 auth_method 0 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:55:32.132+0000 7fbd1bac2640 0 -- 172.21.15.88:0/3542194109 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbd0c093410 msgr2=0x7fbd0c009410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:55:32.132+0000 7fbbf1ffb640 1 client.6813.objecter ms_handle_reset 0x7fbd0c093410 session 0x7fbd0c064650 osd.4 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:55:32.132+0000 7fbbf1ffb640 10 monclient: _renew_subs 2024-04-17T04:55:32.550 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:55:32.132+0000 7fbbf1ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:55:32.132+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe40763a0 auth_method 0 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:55:32.132+0000 7fbd1cac4640 10 monclient: get_auth_request con 0x7fbd040826e0 auth_method 0 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:55:32.136+0000 7fbd1c2c3640 10 monclient: get_auth_request con 0x7fbd040415c0 auth_method 0 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:55:32.140+0000 7fbd1bac2640 0 -- 172.21.15.88:0/3542194109 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbbe40763a0 msgr2=0x7fbbe40787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:55:32.140+0000 7fbbf1ffb640 1 client.6813.objecter ms_handle_reset 0x7fbbe40763a0 session 0x7fbd0c064650 osd.4 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:55:32.140+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe4079120 auth_method 0 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.551 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:55:32.144+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:55:32.148+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:55:32.148+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:55:32.344+0000 7fbd1bac2640 10 monclient: get_auth_request con 0x7fbbe4079120 auth_method 0 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 zone default found 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 Realm: () 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 zonegroup default 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.552 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:55:32.344+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 started sync module instance, tier type = 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:55:32.348+0000 7fbd1e7a8980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=0 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=1 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=2 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=3 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=4 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=5 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=6 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 20 add_watcher() i=7 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:55:32.372+0000 7fbd1e7a8980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:32.553 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding auth protocol: cephx 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding auth protocol: cephx 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding auth protocol: cephx 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:55:32.376+0000 7fbbd2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.554 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: crc 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:55:32.376+0000 7fbd1e7a8980 5 AuthRegistry(0x7fffb2a61050) adding con mode: secure 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache list hook 0x55bb38cf5d80 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache inspect hook 0x55bb38cf5d80 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache erase hook 0x55bb38cf5d80 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command cache zap hook 0x55bb38cf5d80 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 note: GC not initialized 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:55:32.380+0000 7fbbd1fdb640 20 reqs_thread_entry: start 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace show hook 0x7fbd14004730 2024-04-17T04:55:32.555 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace history hook 0x7fbd14004730 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace active hook 0x7fbd14004730 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 5 asok(0x55bb38a28f40) register_command sync trace active_short hook 0x7fbd14004730 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 Filter name: none 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:55:32.380+0000 7fbb437fe640 20 reqs_thread_entry: start 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 : miss 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.556 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:55:32.380+0000 7fbd1e7a8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:55:32.384+0000 7fbd1e7a8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:55:32.392+0000 7fbd1e7a8980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27, ofs0, ns] 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464304 cookie 94262600597344 notifier 6813 bl.length()=1007 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:55:32.392+0000 7fbbf37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_sources= new_sources= 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential sources added=[] removed=[] 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_dests= new_dests= 2024-04-17T04:55:32.557 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:55:32.396+0000 7fbd1e7a8980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential dests added=[] removed=[] 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:55:32.400+0000 7fbd1e7a8980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.0 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:55:32.408+0000 7fbd1e7a8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd1e7a8980 time 2024-04-17T04:55:32.409467+0000 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd2207fe1f] 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.558 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:55:32.412+0000 7fbd1e7a8980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7fbd1e7a8980 thread_name:radosgw-admin 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd20f01520] 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd2207fe79] 2024-04-17T04:55:32.559 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7fbd2207ffd3] 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bb36ccf191] 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x55bb36cddc67] 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd20ee8d90] 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:55:32.560 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:55:32.561 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:55:32.562 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:55:32.563 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:55:32.564 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:55:32.565 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbb437fe640 / http_manager 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbd1fdb640 / http_manager 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbd2fdd640 / rgw_dt_lg_renew 2024-04-17T04:55:32.566 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbf1ffb640 / ms_dispatch 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbbf37fe640 / io_context_pool 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbc01ffb640 / ms_dispatch 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbc08e09640 / ms_dispatch 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbc0a60c640 / io_context_pool 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1bac2640 / msgr-worker-2 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1c2c3640 / msgr-worker-1 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1cac4640 / msgr-worker-0 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1d2c5640 / io_context_pool 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: 7fbd1e7a8980 / radosgw-admin 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/lib/ceph/crash/2024-04-17T04:55:32.415365Z_7a1d1b29-f39c-452e-b94c-02b803e5fde7/log 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr:Aborted 2024-04-17T04:55:32.567 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-04-17T04:55:32.568 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:55:32.568 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.508+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 realm 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:55:32.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.512+0000 7fc0fe9b3980 20 searching for the correct realm 2024-04-17T04:55:32.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.520+0000 7fc0fcccf640 0 -- 172.21.15.88:0/1052446905 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fc0f01169f0 msgr2=0x7fc0f00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:55:32.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.736+0000 7fc0fc4ce640 0 -- 172.21.15.88:0/1052446905 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x565020eeb320 msgr2=0x565020eed7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:55:32.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:55:32.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:55:32.739 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:55:32.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 zone default found 2024-04-17T04:55:32.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 4 Realm: () 2024-04-17T04:55:32.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:55:32.740 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:55:32.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:55:32.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 zonegroup default 2024-04-17T04:55:32.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.744+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.741 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.748+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.748+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.748+0000 7fc0fe9b3980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:55:32.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.748+0000 7fc0fe9b3980 20 started sync module instance, tier type = 2024-04-17T04:55:32.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.748+0000 7fc0fe9b3980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=0 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=1 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=2 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=3 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=4 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=5 2024-04-17T04:55:32.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=6 2024-04-17T04:55:32.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 20 add_watcher() i=7 2024-04-17T04:55:32.768 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.772+0000 7fc0fe9b3980 2 all 8 watchers are set, enabling cache 2024-04-17T04:55:32.771 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.776+0000 7fbfb2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:32.772 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.776+0000 7fc0fe9b3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:55:32.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.776+0000 7fc0fe9b3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:55:32.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.776+0000 7fc0fe9b3980 5 note: GC not initialized 2024-04-17T04:55:32.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fbfb1fdb640 20 reqs_thread_entry: start 2024-04-17T04:55:32.773 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 init_complete bucket index max shards: 1 2024-04-17T04:55:32.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 Filter name: none 2024-04-17T04:55:32.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fbf237fe640 20 reqs_thread_entry: start 2024-04-17T04:55:32.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.774 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=467 2024-04-17T04:55:32.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:32.775 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:55:32.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados->read ofs=0 len=0 2024-04-17T04:55:32.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 rados_obj.operate() r=0 bl.length=467 2024-04-17T04:55:32.776 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.780+0000 7fc0fe9b3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:55:32.778 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.784+0000 7fbfb2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:55:32.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.788+0000 7fc0fe9b3980 20 remove_watcher() i=0 2024-04-17T04:55:32.783 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.788+0000 7fc0fe9b3980 2 removed watcher, disabling cache 2024-04-17T04:55:32.786 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.792+0000 7fc0fe9b3980 20 remove_watcher() i=1 2024-04-17T04:55:32.788 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.792+0000 7fc0fe9b3980 20 remove_watcher() i=2 2024-04-17T04:55:32.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.796+0000 7fc0fe9b3980 20 remove_watcher() i=3 2024-04-17T04:55:32.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.800+0000 7fc0fe9b3980 20 remove_watcher() i=4 2024-04-17T04:55:32.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.800+0000 7fc0fe9b3980 20 remove_watcher() i=5 2024-04-17T04:55:32.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.800+0000 7fc0fe9b3980 20 remove_watcher() i=6 2024-04-17T04:55:32.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:55:32.804+0000 7fc0fe9b3980 20 remove_watcher() i=7 2024-04-17T04:55:32.814 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:55:32.814 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:55:32.814 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:55:32.814 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:03.175 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:56:03.211 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:03.211 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:03.257 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 realm 2024-04-17T04:56:03.258 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:03.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:03.260 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:03.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.263+0000 7fe9c19c5980 20 searching for the correct realm 2024-04-17T04:56:03.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.271+0000 7fe9bfce1640 0 -- 172.21.15.88:0/877427359 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7fe9b40d8070 msgr2=0x7fe9b40c3480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:03.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.275+0000 7fe9bf4e0640 0 -- 172.21.15.88:0/877427359 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fe9b40f8b50 msgr2=0x7fe9b4112980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:03.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.491+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:03.490 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 zone default found 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 4 Realm: () 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 zonegroup default 2024-04-17T04:56:03.491 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.493 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.495+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.499+0000 7fe9c19c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.499+0000 7fe9c19c5980 20 started sync module instance, tier type = 2024-04-17T04:56:03.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.499+0000 7fe9c19c5980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:03.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=0 2024-04-17T04:56:03.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=1 2024-04-17T04:56:03.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=2 2024-04-17T04:56:03.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=3 2024-04-17T04:56:03.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=4 2024-04-17T04:56:03.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=5 2024-04-17T04:56:03.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=6 2024-04-17T04:56:03.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 20 add_watcher() i=7 2024-04-17T04:56:03.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.523+0000 7fe9c19c5980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:03.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.527+0000 7fe875fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:03.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.527+0000 7fe9c19c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:03.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.527+0000 7fe9c19c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:03.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.527+0000 7fe9c19c5980 5 note: GC not initialized 2024-04-17T04:56:03.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe874fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:03.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:03.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 20 Filter name: none 2024-04-17T04:56:03.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe7ee7fc640 20 reqs_thread_entry: start 2024-04-17T04:56:03.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-17T04:56:03.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:56:03.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-17T04:56:03.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-17T04:56:03.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 : miss 2024-04-17T04:56:03.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:56:03.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:56:03.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:56:03.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:03.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:03.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.531+0000 7fe9c19c5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 2024-04-17T04:56:03.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.539+0000 7fe9c19c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:56:03.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.539+0000 7fe9c19c5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:56:03.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.539+0000 7fe9c19c5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:03.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.539+0000 7fe9c19c5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27, ofs0, ns] 2024-04-17T04:56:03.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe8a2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464306 cookie 94035909623648 notifier 6855 bl.length()=1007 2024-04-17T04:56:03.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe8a2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:56:03.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe8a2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:56:03.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe8a2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:03.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_sources= new_sources= 2024-04-17T04:56:03.539 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential sources added=[] removed=[] 2024-04-17T04:56:03.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_dests= new_dests= 2024-04-17T04:56:03.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.543+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential dests added=[] removed=[] 2024-04-17T04:56:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.547+0000 7fe9c19c5980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:56:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.547+0000 7fe9c19c5980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.0 2024-04-17T04:56:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.547+0000 7fe9c19c5980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:56:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.547+0000 7fe9c19c5980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.1.1 2024-04-17T04:56:03.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.547+0000 7fe9c19c5980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27.0 2024-04-17T04:56:03.550 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe9c19c5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:03.550 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe9c19c5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 2024-04-17T04:56:03.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe9c19c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:56:03.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe9c19c5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:56:03.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe9c19c5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:03.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe9c19c5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27, ofs0, ns] 2024-04-17T04:56:03.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe8a2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464307 cookie 94035909623648 notifier 6855 bl.length()=975 2024-04-17T04:56:03.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe8a2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 info.flags=0x17 2024-04-17T04:56:03.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe8a2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 to cache LRU end 2024-04-17T04:56:03.553 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.555+0000 7fe8a2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:03.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.559+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_sources= new_sources= 2024-04-17T04:56:03.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.559+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential sources added=[] removed=[] 2024-04-17T04:56:03.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.559+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): orig_dests= new_dests= 2024-04-17T04:56:03.555 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.559+0000 7fe9c19c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27]): potential dests added=[] removed=[] 2024-04-17T04:56:03.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.563+0000 7fe9c19c5980 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" completed successfully 2024-04-17T04:56:03.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.567+0000 7fe875fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:03.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.571+0000 7fe9c19c5980 20 remove_watcher() i=0 2024-04-17T04:56:03.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.571+0000 7fe9c19c5980 2 removed watcher, disabling cache 2024-04-17T04:56:03.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.575+0000 7fe9c19c5980 20 remove_watcher() i=1 2024-04-17T04:56:03.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.579+0000 7fe9c19c5980 20 remove_watcher() i=2 2024-04-17T04:56:03.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.579+0000 7fe9c19c5980 20 remove_watcher() i=3 2024-04-17T04:56:03.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.583+0000 7fe9c19c5980 20 remove_watcher() i=4 2024-04-17T04:56:03.580 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.583+0000 7fe9c19c5980 20 remove_watcher() i=5 2024-04-17T04:56:03.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.587+0000 7fe9c19c5980 20 remove_watcher() i=6 2024-04-17T04:56:03.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.587+0000 7fe9c19c5980 20 remove_watcher() i=7 2024-04-17T04:56:03.606 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-04-17T04:56:03.636 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:03.636 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:03.681 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 realm 2024-04-17T04:56:03.683 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:03.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.687+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.691+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:03.685 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.691+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.691+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:03.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.691+0000 7f63a88d2980 20 searching for the correct realm 2024-04-17T04:56:03.692 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.695+0000 7f63a6bee640 0 -- 172.21.15.88:0/3751099742 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55f96c888880 msgr2=0x55f96c88acd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:03.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.703+0000 7f63a5bec640 0 -- 172.21.15.88:0/3751099742 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f63980e7050 msgr2=0x7f63980f4400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.707 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:03.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.708 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 zone default found 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 4 Realm: () 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 zonegroup default 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.711+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.715+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.715+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.715+0000 7f63a88d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:03.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.715+0000 7f63a88d2980 20 started sync module instance, tier type = 2024-04-17T04:56:03.710 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.715+0000 7f63a88d2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=0 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=1 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=2 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=3 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=4 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=5 2024-04-17T04:56:03.729 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=6 2024-04-17T04:56:03.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 20 add_watcher() i=7 2024-04-17T04:56:03.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.735+0000 7f63a88d2980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:03.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f625cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 5 note: GC not initialized 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f61d77fe640 20 reqs_thread_entry: start 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 Filter name: none 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f61d57fa640 20 reqs_thread_entry: start 2024-04-17T04:56:03.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:56:03.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=221 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados->read ofs=0 len=0 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 rados_obj.operate() r=0 bl.length=435 2024-04-17T04:56:03.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.739+0000 7f63a88d2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:03.738 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.743+0000 7f625cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:03.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.747+0000 7f63a88d2980 20 remove_watcher() i=0 2024-04-17T04:56:03.743 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.747+0000 7f63a88d2980 2 removed watcher, disabling cache 2024-04-17T04:56:03.745 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.751+0000 7f63a88d2980 20 remove_watcher() i=1 2024-04-17T04:56:03.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.751+0000 7f63a88d2980 20 remove_watcher() i=2 2024-04-17T04:56:03.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.755+0000 7f63a88d2980 20 remove_watcher() i=3 2024-04-17T04:56:03.754 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.759+0000 7f63a88d2980 20 remove_watcher() i=4 2024-04-17T04:56:03.756 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.759+0000 7f63a88d2980 20 remove_watcher() i=5 2024-04-17T04:56:03.759 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.763+0000 7f63a88d2980 20 remove_watcher() i=6 2024-04-17T04:56:03.761 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:03.767+0000 7f63a88d2980 20 remove_watcher() i=7 2024-04-17T04:56:03.774 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:03.774 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:03.774 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-commit-target-layout id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:56:03.775 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:04.507 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-04-17T04:56:04.507 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:06.713 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-04-17T04:56:06.746 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:06.746 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:06.791 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:06.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:06.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 realm 2024-04-17T04:56:06.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:06.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:06.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:06.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:06.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:06.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.795+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:06.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.799+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:06.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.799+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:06.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.799+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:06.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.799+0000 7f6e03f5b980 20 searching for the correct realm 2024-04-17T04:56:06.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:06.803+0000 7f6e01275640 0 -- 172.21.15.88:0/1558739545 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f6df405f960 msgr2=0x7f6df4053430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:07.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.019+0000 7f6e01a76640 0 -- 172.21.15.88:0/1558739545 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f6df8042740 msgr2=0x7f6df8043050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:07.231 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.232 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:07.233 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.235+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:07.235 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 zone default found 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 4 Realm: () 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 zonegroup default 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.236 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.239+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.237 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.243+0000 7f6e03f5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.243+0000 7f6e03f5b980 20 started sync module instance, tier type = 2024-04-17T04:56:07.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.243+0000 7f6e03f5b980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=0 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=1 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=2 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=3 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=4 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=5 2024-04-17T04:56:07.266 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.267+0000 7f6e03f5b980 20 add_watcher() i=6 2024-04-17T04:56:07.268 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.271+0000 7f6e03f5b980 20 add_watcher() i=7 2024-04-17T04:56:07.268 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.271+0000 7f6e03f5b980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:07.271 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.275+0000 7f6cb7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:07.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.275+0000 7f6e03f5b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:07.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.275+0000 7f6e03f5b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:07.272 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.275+0000 7f6e03f5b980 5 note: GC not initialized 2024-04-17T04:56:07.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.275+0000 7f6cb6fe5640 20 reqs_thread_entry: start 2024-04-17T04:56:07.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:07.273 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 Filter name: none 2024-04-17T04:56:07.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6cb4fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:07.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:07.274 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.279+0000 7f6e03f5b980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.283+0000 7f6e03f5b980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:07.279 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.283+0000 7f6e03f5b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:07.280 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.283+0000 7f6cb7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:07.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.287+0000 7f6e03f5b980 20 remove_watcher() i=0 2024-04-17T04:56:07.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.287+0000 7f6e03f5b980 2 removed watcher, disabling cache 2024-04-17T04:56:07.287 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.291+0000 7f6e03f5b980 20 remove_watcher() i=1 2024-04-17T04:56:07.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.295+0000 7f6e03f5b980 20 remove_watcher() i=2 2024-04-17T04:56:07.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.295+0000 7f6e03f5b980 20 remove_watcher() i=3 2024-04-17T04:56:07.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.299+0000 7f6e03f5b980 20 remove_watcher() i=4 2024-04-17T04:56:07.296 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.299+0000 7f6e03f5b980 20 remove_watcher() i=5 2024-04-17T04:56:07.298 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.303+0000 7f6e03f5b980 20 remove_watcher() i=6 2024-04-17T04:56:07.299 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.303+0000 7f6e03f5b980 20 remove_watcher() i=7 2024-04-17T04:56:07.316 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:07.316 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:07.316 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-do-reshard id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:56:07.316 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:07.316 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-04-17T04:56:07.349 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:07.349 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:07.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 realm 2024-04-17T04:56:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.393 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.395+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.399+0000 7fc8023c6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:07.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.399+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.399+0000 7fc8023c6980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:07.395 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.399+0000 7fc8023c6980 20 searching for the correct realm 2024-04-17T04:56:07.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.407+0000 7fc7ff6e0640 0 -- 172.21.15.88:0/4267042434 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x556dc5605970 msgr2=0x556dc5607e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:07.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.411+0000 7fc7ff6e0640 0 -- 172.21.15.88:0/4267042434 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fc7f00193a0 msgr2=0x7fc7f00197c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:07.416 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.419+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:07.417 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:07.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 zone default found 2024-04-17T04:56:07.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 4 Realm: () 2024-04-17T04:56:07.418 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:07.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:07.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:07.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 zonegroup default 2024-04-17T04:56:07.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 started sync module instance, tier type = 2024-04-17T04:56:07.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.423+0000 7fc8023c6980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=0 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=1 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=2 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=3 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=4 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=5 2024-04-17T04:56:07.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=6 2024-04-17T04:56:07.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.439+0000 7fc8023c6980 20 add_watcher() i=7 2024-04-17T04:56:07.437 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:07.439 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc6b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:07.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:07.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:07.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 5 note: GC not initialized 2024-04-17T04:56:07.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc6b4fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:07.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:07.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 20 Filter name: none 2024-04-17T04:56:07.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc62e7fc640 20 reqs_thread_entry: start 2024-04-17T04:56:07.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-04-17T04:56:07.441 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.443+0000 7fc8023c6980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 : miss 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:07.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.447+0000 7fc8023c6980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 2024-04-17T04:56:07.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc8023c6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:07.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc8023c6980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:07.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc8023c6980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:07.447 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc8023c6980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28, ofs0, ns] 2024-04-17T04:56:07.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc6deffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464274 cookie 93929951291120 notifier 6536 bl.length()=987 2024-04-17T04:56:07.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc6deffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:07.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc6deffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:07.448 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.451+0000 7fc6deffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:07.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.455+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_sources= new_sources= 2024-04-17T04:56:07.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.455+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential sources added=[] removed=[] 2024-04-17T04:56:07.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.455+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_dests= new_dests= 2024-04-17T04:56:07.449 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.455+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential dests added=[] removed=[] 2024-04-17T04:56:07.451 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.455+0000 7fc8023c6980 -1 Injecting error=-5 at location=do_reshard 2024-04-17T04:56:07.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.459+0000 7fc8023c6980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:07.456 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.459+0000 7fc8023c6980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28, ofs0, ns] 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc6deffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464275 cookie 93929951291120 notifier 6536 bl.length()=955 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc6deffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc6deffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:07.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc6deffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:07.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_sources= new_sources= 2024-04-17T04:56:07.459 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential sources added=[] removed=[] 2024-04-17T04:56:07.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_dests= new_dests= 2024-04-17T04:56:07.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential dests added=[] removed=[] 2024-04-17T04:56:07.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.463+0000 7fc8023c6980 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-04-17T04:56:07.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.467+0000 7fc6b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:07.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.471+0000 7fc8023c6980 20 remove_watcher() i=0 2024-04-17T04:56:07.466 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.471+0000 7fc8023c6980 2 removed watcher, disabling cache 2024-04-17T04:56:07.468 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.471+0000 7fc8023c6980 20 remove_watcher() i=1 2024-04-17T04:56:07.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.475+0000 7fc8023c6980 20 remove_watcher() i=2 2024-04-17T04:56:07.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.475+0000 7fc8023c6980 20 remove_watcher() i=3 2024-04-17T04:56:07.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.479+0000 7fc8023c6980 20 remove_watcher() i=4 2024-04-17T04:56:07.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.479+0000 7fc8023c6980 20 remove_watcher() i=5 2024-04-17T04:56:07.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.479+0000 7fc8023c6980 20 remove_watcher() i=6 2024-04-17T04:56:07.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.483+0000 7fc8023c6980 20 remove_watcher() i=7 2024-04-17T04:56:07.491 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-04-17T04:56:07.517 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:07.517 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:07.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.563+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.563+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.563+0000 7ff720818980 20 realm 2024-04-17T04:56:07.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.563+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.561 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:07.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:07.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:07.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.567+0000 7ff720818980 20 searching for the correct realm 2024-04-17T04:56:07.571 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.575+0000 7ff71eb34640 0 -- 172.21.15.88:0/2389360035 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7ff714008040 msgr2=0x7ff71403b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:07.787 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.791+0000 7ff71e333640 0 -- 172.21.15.88:0/2389360035 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55dfaf1ab4c0 msgr2=0x55dfaf1ad980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:07.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:07.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:07.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:07.795 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:07.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:07.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:07.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.799+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:07.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 zone default found 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 4 Realm: () 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 zonegroup default 2024-04-17T04:56:07.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 started sync module instance, tier type = 2024-04-17T04:56:07.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.803+0000 7ff720818980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=0 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=1 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=2 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=3 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=4 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=5 2024-04-17T04:56:07.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.827+0000 7ff720818980 20 add_watcher() i=6 2024-04-17T04:56:07.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.831+0000 7ff720818980 20 add_watcher() i=7 2024-04-17T04:56:07.827 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.831+0000 7ff720818980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:07.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff5d4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:07.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff720818980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:07.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff720818980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:07.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff720818980 5 note: GC not initialized 2024-04-17T04:56:07.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff54f7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff720818980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff720818980 20 Filter name: none 2024-04-17T04:56:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff54d7fa640 20 reqs_thread_entry: start 2024-04-17T04:56:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.835+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:07.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:07.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:07.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:07.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados->read ofs=0 len=0 2024-04-17T04:56:07.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:07.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.839+0000 7ff720818980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:07.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.843+0000 7ff5d4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:07.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.847+0000 7ff720818980 20 remove_watcher() i=0 2024-04-17T04:56:07.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.847+0000 7ff720818980 2 removed watcher, disabling cache 2024-04-17T04:56:07.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.851+0000 7ff720818980 20 remove_watcher() i=1 2024-04-17T04:56:07.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.851+0000 7ff720818980 20 remove_watcher() i=2 2024-04-17T04:56:07.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.855+0000 7ff720818980 20 remove_watcher() i=3 2024-04-17T04:56:07.853 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.855+0000 7ff720818980 20 remove_watcher() i=4 2024-04-17T04:56:07.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.859+0000 7ff720818980 20 remove_watcher() i=5 2024-04-17T04:56:07.856 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.859+0000 7ff720818980 20 remove_watcher() i=6 2024-04-17T04:56:07.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:07.863+0000 7ff720818980 20 remove_watcher() i=7 2024-04-17T04:56:07.873 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:07.873 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:07.873 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-do-reshard id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:56:07.873 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:07.987 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:56:08.021 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:08.021 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:08.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 realm 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.067 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.068 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:08.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.071+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.075+0000 7f1f32a98980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:08.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.075+0000 7f1f32a98980 20 searching for the correct realm 2024-04-17T04:56:08.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.079+0000 7f1f2fdb2640 0 -- 172.21.15.88:0/103039151 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x5571229de940 msgr2=0x5571229e0d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:08.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.087+0000 7f1f305b3640 0 -- 172.21.15.88:0/103039151 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f1f2407a8f0 msgr2=0x7f1f240858e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.090 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:08.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 zone default found 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 4 Realm: () 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 zonegroup default 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 started sync module instance, tier type = 2024-04-17T04:56:08.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.095+0000 7f1f32a98980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=0 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=1 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=2 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=3 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=4 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=5 2024-04-17T04:56:08.109 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.111+0000 7f1f32a98980 20 add_watcher() i=6 2024-04-17T04:56:08.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.115+0000 7f1f32a98980 20 add_watcher() i=7 2024-04-17T04:56:08.110 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.115+0000 7f1f32a98980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:08.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.115+0000 7f1de6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:08.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.115+0000 7f1f32a98980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:08.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.115+0000 7f1f32a98980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:08.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 5 note: GC not initialized 2024-04-17T04:56:08.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1de5fe3640 20 reqs_thread_entry: start 2024-04-17T04:56:08.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:08.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 20 Filter name: none 2024-04-17T04:56:08.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1ddeffd640 20 reqs_thread_entry: start 2024-04-17T04:56:08.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-04-17T04:56:08.114 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 : miss 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:08.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.119+0000 7f1f32a98980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 2024-04-17T04:56:08.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1f32a98980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:08.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1f32a98980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:08.120 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1f32a98980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:08.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1f32a98980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28, ofs0, ns] 2024-04-17T04:56:08.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1e0f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464276 cookie 93944400467440 notifier 6572 bl.length()=987 2024-04-17T04:56:08.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1e0f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:08.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1e0f7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:08.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.123+0000 7f1e0f7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:08.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_sources= new_sources= 2024-04-17T04:56:08.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential sources added=[] removed=[] 2024-04-17T04:56:08.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_dests= new_dests= 2024-04-17T04:56:08.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential dests added=[] removed=[] 2024-04-17T04:56:08.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28.1.0 2024-04-17T04:56:08.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28.1.0 2024-04-17T04:56:08.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28.1.1 2024-04-17T04:56:08.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28.1.1 2024-04-17T04:56:08.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.127+0000 7f1f32a98980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28.0 2024-04-17T04:56:08.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1f32a98980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:08.131 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1f32a98980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 2024-04-17T04:56:08.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1f32a98980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:08.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1f32a98980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:08.132 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1f32a98980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:08.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1f32a98980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28, ofs0, ns] 2024-04-17T04:56:08.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1e0effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464277 cookie 93944400467440 notifier 6572 bl.length()=955 2024-04-17T04:56:08.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1e0effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 info.flags=0x17 2024-04-17T04:56:08.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1e0effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 to cache LRU end 2024-04-17T04:56:08.133 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.135+0000 7f1e0effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:08.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.139+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_sources= new_sources= 2024-04-17T04:56:08.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.139+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential sources added=[] removed=[] 2024-04-17T04:56:08.134 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.139+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): orig_dests= new_dests= 2024-04-17T04:56:08.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.139+0000 7f1f32a98980 20 handle_bi_update(): bucket=:error-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28]): potential dests added=[] removed=[] 2024-04-17T04:56:08.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.139+0000 7f1f32a98980 1 execute INFO: reshard of bucket "error-at-do-reshard" completed successfully 2024-04-17T04:56:08.139 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.143+0000 7f1de6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:08.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.147+0000 7f1f32a98980 20 remove_watcher() i=0 2024-04-17T04:56:08.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.147+0000 7f1f32a98980 2 removed watcher, disabling cache 2024-04-17T04:56:08.145 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.151+0000 7f1f32a98980 20 remove_watcher() i=1 2024-04-17T04:56:08.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.151+0000 7f1f32a98980 20 remove_watcher() i=2 2024-04-17T04:56:08.150 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.155+0000 7f1f32a98980 20 remove_watcher() i=3 2024-04-17T04:56:08.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.155+0000 7f1f32a98980 20 remove_watcher() i=4 2024-04-17T04:56:08.154 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.159+0000 7f1f32a98980 20 remove_watcher() i=5 2024-04-17T04:56:08.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.159+0000 7f1f32a98980 20 remove_watcher() i=6 2024-04-17T04:56:08.156 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.159+0000 7f1f32a98980 20 remove_watcher() i=7 2024-04-17T04:56:08.171 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-04-17T04:56:08.197 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:08.197 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:08.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.243+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 realm 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:08.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:08.244 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.247+0000 7fce72ce6980 20 searching for the correct realm 2024-04-17T04:56:08.251 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.255+0000 7fce71002640 0 -- 172.21.15.88:0/1773022329 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fce6401b040 msgr2=0x7fce6401a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:08.467 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.471+0000 7fce70801640 0 -- 172.21.15.88:0/1773022329 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55690fb15900 msgr2=0x55690fb17cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:08.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:08.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 zone default found 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 4 Realm: () 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 zonegroup default 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.479+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.477 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.478 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:08.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 started sync module instance, tier type = 2024-04-17T04:56:08.479 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.483+0000 7fce72ce6980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=0 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=1 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=2 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=3 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=4 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=5 2024-04-17T04:56:08.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.507+0000 7fce72ce6980 20 add_watcher() i=6 2024-04-17T04:56:08.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.511+0000 7fce72ce6980 20 add_watcher() i=7 2024-04-17T04:56:08.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.511+0000 7fce72ce6980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:08.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fcd277de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:08.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 5 note: GC not initialized 2024-04-17T04:56:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fcd267dc640 20 reqs_thread_entry: start 2024-04-17T04:56:08.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:08.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 Filter name: none 2024-04-17T04:56:08.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fcd1f7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:08.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:08.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.515+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:08.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:08.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:08.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 rados->read ofs=0 len=0 2024-04-17T04:56:08.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:08.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.519+0000 7fce72ce6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:08.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.523+0000 7fcd277de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:08.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.527+0000 7fce72ce6980 20 remove_watcher() i=0 2024-04-17T04:56:08.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.527+0000 7fce72ce6980 2 removed watcher, disabling cache 2024-04-17T04:56:08.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.527+0000 7fce72ce6980 20 remove_watcher() i=1 2024-04-17T04:56:08.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.531+0000 7fce72ce6980 20 remove_watcher() i=2 2024-04-17T04:56:08.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.531+0000 7fce72ce6980 20 remove_watcher() i=3 2024-04-17T04:56:08.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.535+0000 7fce72ce6980 20 remove_watcher() i=4 2024-04-17T04:56:08.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.539+0000 7fce72ce6980 20 remove_watcher() i=5 2024-04-17T04:56:08.535 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.539+0000 7fce72ce6980 20 remove_watcher() i=6 2024-04-17T04:56:08.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:08.539+0000 7fce72ce6980 20 remove_watcher() i=7 2024-04-17T04:56:08.552 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:08.552 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:08.552 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: error-at-do-reshard id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:56:08.552 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:09.353 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: reshard bucket with abort at do_reshard 2024-04-17T04:56:09.353 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:11.571 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-04-17T04:56:11.603 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:11.603 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:11.649 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.651+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 realm 2024-04-17T04:56:11.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:11.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:11.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.655+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.659+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:11.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.659+0000 7f49a2f5e980 20 searching for the correct realm 2024-04-17T04:56:11.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.663+0000 7f499bfff640 0 -- 172.21.15.88:0/2369172996 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x564ae7453240 msgr2=0x564ae7455630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:11.666 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.671+0000 7f49a0a79640 0 -- 172.21.15.88:0/2369172996 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f498c008040 msgr2=0x7f498c05d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:11.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:11.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 zone default found 2024-04-17T04:56:11.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 4 Realm: () 2024-04-17T04:56:11.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 zonegroup default 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.679+0000 7f49a2f5e980 20 started sync module instance, tier type = 2024-04-17T04:56:11.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.683+0000 7f49a2f5e980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=0 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=1 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=2 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=3 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=4 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=5 2024-04-17T04:56:11.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.695+0000 7f49a2f5e980 20 add_watcher() i=6 2024-04-17T04:56:11.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.699+0000 7f49a2f5e980 20 add_watcher() i=7 2024-04-17T04:56:11.694 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.699+0000 7f49a2f5e980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:11.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.699+0000 7f4856fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 5 note: GC not initialized 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f4855fe3640 20 reqs_thread_entry: start 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 Filter name: none 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f47c77fe640 20 reqs_thread_entry: start 2024-04-17T04:56:11.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:11.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:11.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:11.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.703+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.707+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:11.702 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.707+0000 7f49a2f5e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.707+0000 7f49a2f5e980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:11.703 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.707+0000 7f49a2f5e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:11.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.707+0000 7f4856fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:11.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.715+0000 7f49a2f5e980 20 remove_watcher() i=0 2024-04-17T04:56:11.709 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.715+0000 7f49a2f5e980 2 removed watcher, disabling cache 2024-04-17T04:56:11.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.715+0000 7f49a2f5e980 20 remove_watcher() i=1 2024-04-17T04:56:11.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.719+0000 7f49a2f5e980 20 remove_watcher() i=2 2024-04-17T04:56:11.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.719+0000 7f49a2f5e980 20 remove_watcher() i=3 2024-04-17T04:56:11.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.723+0000 7f49a2f5e980 20 remove_watcher() i=4 2024-04-17T04:56:11.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.723+0000 7f49a2f5e980 20 remove_watcher() i=5 2024-04-17T04:56:11.720 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.723+0000 7f49a2f5e980 20 remove_watcher() i=6 2024-04-17T04:56:11.722 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.727+0000 7f49a2f5e980 20 remove_watcher() i=7 2024-04-17T04:56:11.735 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:11.736 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:11.736 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-do-reshard id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:56:11.736 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:11.736 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-04-17T04:56:11.768 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:11.768 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:11.811 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.815+0000 7f4fdac99980 20 realm 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:11.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:11.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:11.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:11.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.819+0000 7f4fdac99980 20 searching for the correct realm 2024-04-17T04:56:11.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:11.827+0000 7f4fd8fb5640 0 -- 172.21.15.88:0/621037237 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f4fc00ca7f0 msgr2=0x7f4fc012c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:12.039 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.043+0000 7f4fd87b4640 0 -- 172.21.15.88:0/621037237 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55e3764fd0c0 msgr2=0x55e3764ff4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:12.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:12.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:12.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:12.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:12.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:12.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:12.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:12.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 zone default found 2024-04-17T04:56:12.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 4 Realm: () 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 zonegroup default 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.050 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 started sync module instance, tier type = 2024-04-17T04:56:12.051 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.055+0000 7f4fdac99980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:12.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=0 2024-04-17T04:56:12.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=1 2024-04-17T04:56:12.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=2 2024-04-17T04:56:12.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=3 2024-04-17T04:56:12.073 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=4 2024-04-17T04:56:12.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=5 2024-04-17T04:56:12.074 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=6 2024-04-17T04:56:12.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.079+0000 7f4fdac99980 20 add_watcher() i=7 2024-04-17T04:56:12.075 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.079+0000 7f4fdac99980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:12.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4e8efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 5 note: GC not initialized 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4e8dfdb640 20 reqs_thread_entry: start 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 20 Filter name: none 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4dff7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-17T04:56:12.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:12.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-17T04:56:12.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-17T04:56:12.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 : miss 2024-04-17T04:56:12.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.087+0000 7f4fdac99980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 2024-04-17T04:56:12.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4fdac99980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4fdac99980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4fdac99980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29, ofs0, ns] 2024-04-17T04:56:12.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464282 cookie 94435430929664 notifier 7011 bl.length()=987 2024-04-17T04:56:12.091 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_sources= new_sources= 2024-04-17T04:56:12.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential sources added=[] removed=[] 2024-04-17T04:56:12.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_dests= new_dests= 2024-04-17T04:56:12.093 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential dests added=[] removed=[] 2024-04-17T04:56:12.095 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4fdac99980 time 2024-04-17T04:56:12.103059+0000 2024-04-17T04:56:12.095 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:56:12.096 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.096 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4fde570e1f] 2024-04-17T04:56:12.096 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.096 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.099+0000 7f4fdac99980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4fdac99980 time 2024-04-17T04:56:12.103059+0000 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.097 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4fde570e1f] 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr:*** Caught signal (Aborted) ** 2024-04-17T04:56:12.098 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f4fdac99980 thread_name:radosgw-admin 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4fdd3f2520] 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4fde570e79] 2024-04-17T04:56:12.111 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.103+0000 7f4fdac99980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f4fdac99980 thread_name:radosgw-admin 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4fdd3f2520] 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:56:12.112 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4fde570e79] 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.113 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_done global_id 6999 payload 274 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: dump: 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.114 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 monclient: authenticate success, global_id 6999 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.115 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: dump: 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: get_monmap_and_config success 2024-04-17T04:56:12.116 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:56:11.795+0000 7f4fdac99980 4 set_mon_vals no callback set 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: shutdown 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 asok(0x55e376201f40) unregister_commands rotate-key 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.117 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.118 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding auth protocol: cephx 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding auth protocol: cephx 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding auth protocol: cephx 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.119 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:56:11.795+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: starting msgr at 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: starting objecter 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command objecter_requests hook 0x55e3762365c0 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: setting wanted keys 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: calling monclient init 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding auth protocol: cephx 2024-04-17T04:56:12.120 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding auth protocol: cephx 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding auth protocol: cephx 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.121 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:56:11.795+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:56:11.795+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key hook 0x55e37640dac8 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: _add_conns ranks=[2,0,1] 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3762cab80 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.122 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e376357370 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e376410c40 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.446733s 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request con 0x55e3762cab80 auth_method 0 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request con 0x55e376357370 auth_method 0 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.123 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_done global_id 6611 payload 824 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 1 monclient: found mon.noname-c 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.124 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: dump: 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.125 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: dump: 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.126 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:56:11.795+0000 7f4ec660c640 4 set_mon_vals no callback set 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 monclient: authenticate success, global_id 6611 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:56:11.799+0000 7f4fdac99980 1 librados: init done 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:56:11.799+0000 7f4ec4e09640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:56:12.127 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:56:11.799+0000 7f4ec4e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:56:11.799+0000 7f4ec4e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:56:11.799+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x7f4ec0065e30 auth_method 0 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:56:11.799+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x55e376456e70 auth_method 0 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:56:11.799+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x55e37645a050 auth_method 0 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:56:11.803+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x55e37645d740 auth_method 0 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:56:11.807+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.128 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding auth protocol: cephx 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding auth protocol: cephx 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding auth protocol: cephx 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.129 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:56:11.807+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command objecter_requests cmddesc objecter_requests hook 0x55e3764665e0 EEXIST 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding auth protocol: cephx 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding auth protocol: cephx 2024-04-17T04:56:12.130 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding auth protocol: cephx 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:56:11.807+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.131 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:56:11.807+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key cmddesc rotate-key hook 0x55e376461c10 EEXIST 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e37648cd50 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e37648d130 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3764882f0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.132 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.457804s 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request con 0x55e37648d130 auth_method 0 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request con 0x55e3764882f0 auth_method 0 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.133 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request con 0x55e37648cd50 auth_method 0 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_done global_id 7005 payload 824 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.134 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: dump: 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.135 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: dump: 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.136 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 monclient: authenticate success, global_id 7005 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:56:11.807+0000 7f4fd77b2640 4 set_mon_vals no callback set 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:56:11.811+0000 7f4ebdffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:56:11.811+0000 7f4ebdffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:56:11.811+0000 7f4ebdffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4eb0065d40 auth_method 0 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: get_monmap_and_config 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.137 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.138 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding auth protocol: cephx 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding auth protocol: cephx 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding auth protocol: cephx 2024-04-17T04:56:12.139 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key cmddesc rotate-key hook 0x7fff16b50b48 EEXIST 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.140 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: _add_conns ranks=[0,1,2] 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e3764d9080 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e3764d9460 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3764e42e0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.461748s 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request con 0x55e3764d9080 auth_method 0 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.141 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request con 0x55e3764e42e0 auth_method 0 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_done global_id 7008 payload 274 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.142 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: dump: 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.143 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: dump: 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 monclient: authenticate success, global_id 7008 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: get_monmap_and_config success 2024-04-17T04:56:12.144 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:56:11.811+0000 7f4fdac99980 4 set_mon_vals no callback set 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: shutdown 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.145 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.146 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: starting msgr at 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: starting objecter 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command objecter_requests cmddesc objecter_requests hook 0x55e3764b0430 EEXIST 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: setting wanted keys 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: calling monclient init 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding auth protocol: cephx 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding auth protocol: cephx 2024-04-17T04:56:12.147 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding auth protocol: cephx 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.148 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:56:11.815+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:56:11.815+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key cmddesc rotate-key hook 0x55e3764a9bb8 EEXIST 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e3764d9080 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e3764e42e0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3764d7a70 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.149 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.464199s 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request con 0x55e3764e42e0 auth_method 0 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request con 0x55e3764d7a70 auth_method 0 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.150 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_done global_id 7011 payload 824 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: dump: 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.151 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.152 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: dump: 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.153 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:56:11.815+0000 7f4fdac99980 5 monclient: authenticate success, global_id 7011 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:56:11.815+0000 7f4eaf7fe640 4 set_mon_vals no callback set 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:56:11.815+0000 7f4fdac99980 1 librados: init done 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:56:11.815+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cr dump hook 0x55e3764dd208 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:56:11.815+0000 7f4eadffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:56:12.154 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:56:11.815+0000 7f4eadffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:56:11.815+0000 7f4eadffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:56:11.815+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4ea0065e00 auth_method 0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x55e3764f8d70 auth_method 0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 realm 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:56:11.819+0000 7f4fdac99980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.155 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:56:11.819+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x55e3764fd0c0 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 searching for the correct realm 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:56:11.819+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x55e376502230 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:56:11.819+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x7f4fc00db060 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:56:11.823+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4fd001e040 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:56:11.823+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x7f4fc8079040 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:56:11.827+0000 7f4fd8fb5640 0 -- 172.21.15.88:0/621037237 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f4fc00ca7f0 msgr2=0x7f4fc012c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:56:12.031+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x7f4fc00ca7f0 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:56:12.035+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4fc00d2020 auth_method 0 2024-04-17T04:56:12.156 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:56:12.043+0000 7f4fd87b4640 0 -- 172.21.15.88:0/621037237 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55e3764fd0c0 msgr2=0x55e3764ff4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:56:12.043+0000 7f4eadffb640 1 client.7011.objecter ms_handle_reset 0x55e3764fd0c0 session 0x55e3764fbf20 osd.5 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:56:12.043+0000 7f4eadffb640 10 monclient: _renew_subs 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:56:12.043+0000 7f4eadffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:56:12.043+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4ea00763a0 auth_method 0 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:12.157 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 zone default found 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:56:12.055+0000 7f4fdac99980 4 Realm: () 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:56:12.055+0000 7f4fdac99980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:56:12.055+0000 7f4fdac99980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:56:12.055+0000 7f4fdac99980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 zonegroup default 2024-04-17T04:56:12.158 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 started sync module instance, tier type = 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=0 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=1 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=2 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=3 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=4 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=5 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=6 2024-04-17T04:56:12.159 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:56:12.079+0000 7f4fdac99980 20 add_watcher() i=7 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:56:12.079+0000 7f4fdac99980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding auth protocol: cephx 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding auth protocol: cephx 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding auth protocol: cephx 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:56:12.083+0000 7f4e8efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.160 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:56:12.083+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache list hook 0x55e3764d9cf0 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache inspect hook 0x55e3764d9cf0 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache erase hook 0x55e3764d9cf0 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache zap hook 0x55e3764d9cf0 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 note: GC not initialized 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:56:12.083+0000 7f4e8dfdb640 20 reqs_thread_entry: start 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace show hook 0x7f4fc00043b0 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace history hook 0x7f4fc00043b0 2024-04-17T04:56:12.161 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace active hook 0x7f4fc00043b0 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace active_short hook 0x7f4fc00043b0 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 Filter name: none 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:56:12.083+0000 7f4dff7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:56:12.083+0000 7f4fdac99980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-17T04:56:12.162 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 : miss 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29, ofs0, ns] 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464282 cookie 94435430929664 notifier 7011 bl.length()=987 2024-04-17T04:56:12.163 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_sources= new_sources= 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential sources added=[] removed=[] 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_dests= new_dests= 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential dests added=[] removed=[] 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:56:12.099+0000 7f4fdac99980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4fdac99980 time 2024-04-17T04:56:12.103059+0000 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4fde570e1f] 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.164 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:56:12.103+0000 7f4fdac99980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f4fdac99980 thread_name:radosgw-admin 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.165 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.166 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4fdd3f2520] 2024-04-17T04:56:12.166 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:56:12.166 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:56:12.166 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:56:12.166 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4fde570e79] 2024-04-17T04:56:12.166 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:56:12.167 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:56:12.168 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:56:12.169 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:56:12.169 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:56:12.169 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:56:12.169 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:56:12.169 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:56:12.169 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:56:12.170 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:56:12.171 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:56:12.172 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:56:12.173 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:56:12.174 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:56:12.175 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4dff7fe640 / http_manager 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4e8dfdb640 / http_manager 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4e8efdd640 / rgw_dt_lg_renew 2024-04-17T04:56:12.176 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4eadffb640 / ms_dispatch 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4eaf7fe640 / io_context_pool 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4ebdffb640 / ms_dispatch 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4ec4e09640 / ms_dispatch 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4ec660c640 / io_context_pool 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd77b2640 / io_context_pool 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd7fb3640 / msgr-worker-2 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd87b4640 / msgr-worker-1 2024-04-17T04:56:12.177 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd8fb5640 / msgr-worker-0 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fdac99980 / radosgw-admin 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/log/ceph/ceph-client.admin.102976.log 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr:--- begin dump of recent events --- 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -499> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -498> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -497> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -496> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -495> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -494> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_done global_id 6999 payload 274 2024-04-17T04:56:12.178 INFO:tasks.workunit.client.0.smithi088.stderr: -493> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr: -492> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr: -491> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr: -490> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr: -489> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr: -488> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: dump: 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.179 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -487> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -486> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -485> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 monclient: authenticate success, global_id 6999 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -484> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -483> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -482> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr: -481> 2024-04-17T04:56:11.795+0000 7f4ec660c640 10 monclient: dump: 2024-04-17T04:56:12.180 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr: -480> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: get_monmap_and_config success 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr: -479> 2024-04-17T04:56:11.795+0000 7f4fdac99980 4 set_mon_vals no callback set 2024-04-17T04:56:12.181 INFO:tasks.workunit.client.0.smithi088.stderr: -478> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: shutdown 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr: -477> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 asok(0x55e376201f40) unregister_commands rotate-key 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr: -476> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr: -475> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr: -474> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.182 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: -473> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding auth protocol: cephx 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: -472> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding auth protocol: cephx 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: -471> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding auth protocol: cephx 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: -470> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: -469> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.183 INFO:tasks.workunit.client.0.smithi088.stderr: -468> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.185 INFO:tasks.workunit.client.0.smithi088.stderr: -467> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.185 INFO:tasks.workunit.client.0.smithi088.stderr: -466> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -465> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -464> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -463> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -462> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -461> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -460> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: crc 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -459> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640f7c0) adding con mode: secure 2024-04-17T04:56:12.186 INFO:tasks.workunit.client.0.smithi088.stderr: -458> 2024-04-17T04:56:11.795+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -457> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: starting msgr at 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -456> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: starting objecter 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -455> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command objecter_requests hook 0x55e3762365c0 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -454> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: setting wanted keys 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -453> 2024-04-17T04:56:11.795+0000 7f4fdac99980 1 librados: calling monclient init 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -452> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -451> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding auth protocol: cephx 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -450> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding auth protocol: cephx 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -449> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding auth protocol: cephx 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -448> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -447> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -446> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -445> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.187 INFO:tasks.workunit.client.0.smithi088.stderr: -444> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -443> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -442> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -441> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -440> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -439> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -438> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: crc 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -437> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 AuthRegistry(0x55e37640d980) adding con mode: secure 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -436> 2024-04-17T04:56:11.795+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -435> 2024-04-17T04:56:11.795+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -434> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key hook 0x55e37640dac8 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -433> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.188 INFO:tasks.workunit.client.0.smithi088.stderr: -432> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient: _add_conns ranks=[2,0,1] 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -431> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3762cab80 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -430> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e376357370 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -429> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e376410c40 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -428> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -427> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -426> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -425> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -424> 2024-04-17T04:56:11.795+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.446733s 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -423> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request con 0x55e3762cab80 auth_method 0 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -422> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -421> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -420> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -419> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request con 0x55e376357370 auth_method 0 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -418> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.189 INFO:tasks.workunit.client.0.smithi088.stderr: -417> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -416> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -415> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -414> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -413> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -412> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -411> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -410> 2024-04-17T04:56:11.795+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -409> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_done global_id 6611 payload 824 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -408> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -407> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 1 monclient: found mon.noname-c 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -406> 2024-04-17T04:56:11.795+0000 7f4fd87b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.88:3301/0 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -405> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.190 INFO:tasks.workunit.client.0.smithi088.stderr: -404> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr: -403> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: dump: 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.191 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr: -402> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr: -401> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr: -400> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr: -399> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr: -398> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr: -397> 2024-04-17T04:56:11.795+0000 7f4ec4e09640 10 monclient: dump: 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.192 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -396> 2024-04-17T04:56:11.795+0000 7f4ec660c640 4 set_mon_vals no callback set 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -395> 2024-04-17T04:56:11.795+0000 7f4fdac99980 5 monclient: authenticate success, global_id 6611 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -394> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -393> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -392> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -391> 2024-04-17T04:56:11.799+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.88:3301/0 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -390> 2024-04-17T04:56:11.799+0000 7f4fdac99980 1 librados: init done 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -389> 2024-04-17T04:56:11.799+0000 7f4ec4e09640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:56:12.193 INFO:tasks.workunit.client.0.smithi088.stderr: -388> 2024-04-17T04:56:11.799+0000 7f4ec4e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -387> 2024-04-17T04:56:11.799+0000 7f4ec4e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -386> 2024-04-17T04:56:11.799+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x7f4ec0065e30 auth_method 0 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -385> 2024-04-17T04:56:11.799+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x55e376456e70 auth_method 0 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -384> 2024-04-17T04:56:11.799+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x55e37645a050 auth_method 0 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -383> 2024-04-17T04:56:11.803+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x55e37645d740 auth_method 0 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -382> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -381> 2024-04-17T04:56:11.807+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr: -380> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.194 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -379> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding auth protocol: cephx 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -378> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding auth protocol: cephx 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -377> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding auth protocol: cephx 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -376> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -375> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -374> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -373> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -372> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.195 INFO:tasks.workunit.client.0.smithi088.stderr: -371> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -370> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -369> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -368> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -367> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -366> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: crc 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -365> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376462b80) adding con mode: secure 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -364> 2024-04-17T04:56:11.807+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -363> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command objecter_requests cmddesc objecter_requests hook 0x55e3764665e0 EEXIST 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -362> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.196 INFO:tasks.workunit.client.0.smithi088.stderr: -361> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding auth protocol: cephx 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -360> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding auth protocol: cephx 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -359> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding auth protocol: cephx 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -358> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -357> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -356> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -355> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -354> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -353> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -352> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -351> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -350> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -349> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.197 INFO:tasks.workunit.client.0.smithi088.stderr: -348> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: crc 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -347> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 AuthRegistry(0x55e376461ac8) adding con mode: secure 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -346> 2024-04-17T04:56:11.807+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -345> 2024-04-17T04:56:11.807+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -344> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key cmddesc rotate-key hook 0x55e376461c10 EEXIST 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -343> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -342> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -341> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e37648cd50 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -340> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e37648d130 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -339> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3764882f0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -338> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -337> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -336> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.198 INFO:tasks.workunit.client.0.smithi088.stderr: -335> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -334> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.457804s 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -333> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request con 0x55e37648d130 auth_method 0 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -332> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -331> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -330> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -329> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request con 0x55e3764882f0 auth_method 0 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -328> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -327> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -326> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -325> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -324> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -323> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.199 INFO:tasks.workunit.client.0.smithi088.stderr: -322> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -321> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -320> 2024-04-17T04:56:11.807+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -319> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request con 0x55e37648cd50 auth_method 0 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -318> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -317> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -316> 2024-04-17T04:56:11.807+0000 7f4fd87b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -315> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_done global_id 7005 payload 824 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -314> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -313> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -312> 2024-04-17T04:56:11.807+0000 7f4fd7fb3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -311> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -310> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr: -309> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: dump: 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.200 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr: -308> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr: -307> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr: -306> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.201 INFO:tasks.workunit.client.0.smithi088.stderr: -305> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr: -304> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr: -303> 2024-04-17T04:56:11.807+0000 7f4ebdffb640 10 monclient: dump: 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.202 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -302> 2024-04-17T04:56:11.807+0000 7f4fdac99980 5 monclient: authenticate success, global_id 7005 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -301> 2024-04-17T04:56:11.807+0000 7f4fd77b2640 4 set_mon_vals no callback set 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -300> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -299> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -298> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -297> 2024-04-17T04:56:11.807+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -296> 2024-04-17T04:56:11.811+0000 7f4ebdffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -295> 2024-04-17T04:56:11.811+0000 7f4ebdffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -294> 2024-04-17T04:56:11.811+0000 7f4ebdffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -293> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4eb0065d40 auth_method 0 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -292> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: get_monmap_and_config 2024-04-17T04:56:12.203 INFO:tasks.workunit.client.0.smithi088.stderr: -291> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr: -290> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr: -289> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.204 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -288> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -287> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -286> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -285> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -284> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -283> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -282> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -281> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -280> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -279> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -278> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.205 INFO:tasks.workunit.client.0.smithi088.stderr: -277> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -276> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -275> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -274> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -273> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -272> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -271> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding auth protocol: cephx 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -270> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding auth protocol: cephx 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -269> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding auth protocol: cephx 2024-04-17T04:56:12.206 INFO:tasks.workunit.client.0.smithi088.stderr: -268> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -267> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -266> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -265> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -264> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -263> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -262> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.207 INFO:tasks.workunit.client.0.smithi088.stderr: -261> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -260> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -259> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -258> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: crc 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -257> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50a00) adding con mode: secure 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -256> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -255> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -254> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key cmddesc rotate-key hook 0x7fff16b50b48 EEXIST 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -253> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -252> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: _add_conns ranks=[0,1,2] 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -251> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e3764d9080 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -250> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e3764d9460 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -249> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3764e42e0 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -248> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.208 INFO:tasks.workunit.client.0.smithi088.stderr: -247> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -246> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -245> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -244> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.461748s 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -243> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request con 0x55e3764d9080 auth_method 0 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -242> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -241> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -240> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -239> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request con 0x55e3764e42e0 auth_method 0 2024-04-17T04:56:12.209 INFO:tasks.workunit.client.0.smithi088.stderr: -238> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -237> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -236> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -235> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -234> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -233> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -232> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -231> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.210 INFO:tasks.workunit.client.0.smithi088.stderr: -230> 2024-04-17T04:56:11.811+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -229> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient(hunting): handle_auth_done global_id 7008 payload 274 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -228> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -227> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -226> 2024-04-17T04:56:11.811+0000 7f4fd87b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -225> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -224> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr: -223> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: dump: 2024-04-17T04:56:12.211 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.212 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr: -222> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr: -221> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr: -220> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr: -219> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr: -218> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr: -217> 2024-04-17T04:56:11.811+0000 7f4eaf7fe640 10 monclient: dump: 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.213 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.214 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -216> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 monclient: authenticate success, global_id 7008 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -215> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: get_monmap_and_config success 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -214> 2024-04-17T04:56:11.811+0000 7f4fdac99980 4 set_mon_vals no callback set 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -213> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: shutdown 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -212> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: build_initial_monmap 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -211> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 build_initial for_mkfs: 0 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr: -210> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: monmap: 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 0 2024-04-17T04:56:12.215 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 0.000000 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:created 0.000000 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 0 (unknown) 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.noname-a 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.noname-b 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.noname-c 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.216 INFO:tasks.workunit.client.0.smithi088.stderr: -209> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -208> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -207> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding auth protocol: cephx 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -206> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -205> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -204> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -203> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -202> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.217 INFO:tasks.workunit.client.0.smithi088.stderr: -201> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -200> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -199> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -198> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -197> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -196> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: crc 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -195> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764aa8a0) adding con mode: secure 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -194> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.218 INFO:tasks.workunit.client.0.smithi088.stderr: -193> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: starting msgr at 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -192> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: starting objecter 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -191> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command objecter_requests cmddesc objecter_requests hook 0x55e3764b0430 EEXIST 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -190> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: setting wanted keys 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -189> 2024-04-17T04:56:11.811+0000 7f4fdac99980 1 librados: calling monclient init 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -188> 2024-04-17T04:56:11.811+0000 7f4fdac99980 10 monclient: init 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -187> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding auth protocol: cephx 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -186> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding auth protocol: cephx 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -185> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding auth protocol: cephx 2024-04-17T04:56:12.219 INFO:tasks.workunit.client.0.smithi088.stderr: -184> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -183> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -182> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -181> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -180> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -179> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -178> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -177> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -176> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.220 INFO:tasks.workunit.client.0.smithi088.stderr: -175> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -174> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: crc 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -173> 2024-04-17T04:56:11.811+0000 7f4fdac99980 5 AuthRegistry(0x55e3764a9a70) adding con mode: secure 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -172> 2024-04-17T04:56:11.811+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -171> 2024-04-17T04:56:11.815+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -170> 2024-04-17T04:56:11.815+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command rotate-key cmddesc rotate-key hook 0x55e3764a9bb8 EEXIST 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -169> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _reopen_session rank -1 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -168> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _add_conns ranks=[1,0,2] 2024-04-17T04:56:12.221 INFO:tasks.workunit.client.0.smithi088.stderr: -167> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-b con 0x55e3764d9080 addr [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -166> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-a con 0x55e3764e42e0 addr [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -165> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): picked mon.noname-c con 0x55e3764d7a70 addr [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -164> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -163> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -162> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): start opening mon connection 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -161> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): _renew_subs 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -160> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient(hunting): authenticate will time out at 2173.464199s 2024-04-17T04:56:12.222 INFO:tasks.workunit.client.0.smithi088.stderr: -159> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request con 0x55e3764e42e0 auth_method 0 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -158> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -157> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -156> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -155> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request con 0x55e3764d7a70 auth_method 0 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -154> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -153> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth method 2 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -152> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): _init_auth creating new auth 2024-04-17T04:56:12.223 INFO:tasks.workunit.client.0.smithi088.stderr: -151> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -150> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -149> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -148> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -147> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -146> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -145> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient(hunting): handle_auth_done global_id 7011 payload 824 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -144> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient: _finish_hunting 0 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -143> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 1 monclient: found mon.noname-a 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -142> 2024-04-17T04:56:11.815+0000 7f4fd7fb3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -141> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -140> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-17T04:56:12.224 INFO:tasks.workunit.client.0.smithi088.stderr: -139> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: dump: 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr: -138> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: _finish_auth 0 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr: -137> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: _check_auth_tickets 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr: -136> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: handle_config config(0 keys) 2024-04-17T04:56:12.225 INFO:tasks.workunit.client.0.smithi088.stderr: -135> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr: -134> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr: -133> 2024-04-17T04:56:11.815+0000 7f4eadffb640 10 monclient: dump: 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:epoch 1 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:fsid 5b040342-9999-436c-bdde-ed9b9762bed0 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:last_changed 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:created 2024-04-17T04:35:19.003237+0000 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:min_mon_release 19 (squid) 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:election_strategy: 1 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr:2: [v2:172.21.15.88:3301/0,v1:172.21.15.88:6790/0] mon.c 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.226 INFO:tasks.workunit.client.0.smithi088.stderr: -132> 2024-04-17T04:56:11.815+0000 7f4fdac99980 5 monclient: authenticate success, global_id 7011 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -131> 2024-04-17T04:56:11.815+0000 7f4eaf7fe640 4 set_mon_vals no callback set 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -130> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -129> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -128> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _renew_subs 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -127> 2024-04-17T04:56:11.815+0000 7f4fdac99980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -126> 2024-04-17T04:56:11.815+0000 7f4fdac99980 1 librados: init done 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -125> 2024-04-17T04:56:11.815+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cr dump hook 0x55e3764dd208 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -124> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -123> 2024-04-17T04:56:11.815+0000 7f4eadffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -122> 2024-04-17T04:56:11.815+0000 7f4eadffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -121> 2024-04-17T04:56:11.815+0000 7f4eadffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.88:6832/2906002900,v1:172.21.15.88:6833/2906002900] 2024-04-17T04:56:12.227 INFO:tasks.workunit.client.0.smithi088.stderr: -120> 2024-04-17T04:56:11.815+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4ea0065e00 auth_method 0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -119> 2024-04-17T04:56:11.815+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x55e3764f8d70 auth_method 0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -118> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -117> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 realm 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -116> 2024-04-17T04:56:11.815+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -115> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -114> 2024-04-17T04:56:11.819+0000 7f4fdac99980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -113> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -112> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -111> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -110> 2024-04-17T04:56:11.819+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x55e3764fd0c0 auth_method 0 2024-04-17T04:56:12.228 INFO:tasks.workunit.client.0.smithi088.stderr: -109> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -108> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -107> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -106> 2024-04-17T04:56:11.819+0000 7f4fdac99980 20 searching for the correct realm 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -105> 2024-04-17T04:56:11.819+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x55e376502230 auth_method 0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -104> 2024-04-17T04:56:11.819+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x7f4fc00db060 auth_method 0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -103> 2024-04-17T04:56:11.823+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4fd001e040 auth_method 0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -102> 2024-04-17T04:56:11.823+0000 7f4fd7fb3640 10 monclient: get_auth_request con 0x7f4fc8079040 auth_method 0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -101> 2024-04-17T04:56:11.827+0000 7f4fd8fb5640 0 -- 172.21.15.88:0/621037237 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f4fc00ca7f0 msgr2=0x7f4fc012c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -100> 2024-04-17T04:56:12.031+0000 7f4fd8fb5640 10 monclient: get_auth_request con 0x7f4fc00ca7f0 auth_method 0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -99> 2024-04-17T04:56:12.035+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4fc00d2020 auth_method 0 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -98> 2024-04-17T04:56:12.043+0000 7f4fd87b4640 0 -- 172.21.15.88:0/621037237 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55e3764fd0c0 msgr2=0x55e3764ff4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -97> 2024-04-17T04:56:12.043+0000 7f4eadffb640 1 client.7011.objecter ms_handle_reset 0x55e3764fd0c0 session 0x55e3764fbf20 osd.5 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -96> 2024-04-17T04:56:12.043+0000 7f4eadffb640 10 monclient: _renew_subs 2024-04-17T04:56:12.229 INFO:tasks.workunit.client.0.smithi088.stderr: -95> 2024-04-17T04:56:12.043+0000 7f4eadffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.88:3300/0 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -94> 2024-04-17T04:56:12.043+0000 7f4fd87b4640 10 monclient: get_auth_request con 0x7f4ea00763a0 auth_method 0 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -93> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -92> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -91> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -90> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -89> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -88> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -87> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -86> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -85> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -84> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.230 INFO:tasks.workunit.client.0.smithi088.stderr: -83> 2024-04-17T04:56:12.051+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -82> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -81> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 zone default found 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -80> 2024-04-17T04:56:12.055+0000 7f4fdac99980 4 Realm: () 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -79> 2024-04-17T04:56:12.055+0000 7f4fdac99980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -78> 2024-04-17T04:56:12.055+0000 7f4fdac99980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -77> 2024-04-17T04:56:12.055+0000 7f4fdac99980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -76> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 zonegroup default 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -75> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -74> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -73> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -72> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -71> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -70> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -69> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 started sync module instance, tier type = 2024-04-17T04:56:12.231 INFO:tasks.workunit.client.0.smithi088.stderr: -68> 2024-04-17T04:56:12.055+0000 7f4fdac99980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -67> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=0 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -66> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=1 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -65> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=2 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -64> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=3 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -63> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=4 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -62> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=5 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -61> 2024-04-17T04:56:12.075+0000 7f4fdac99980 20 add_watcher() i=6 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -60> 2024-04-17T04:56:12.079+0000 7f4fdac99980 20 add_watcher() i=7 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -59> 2024-04-17T04:56:12.079+0000 7f4fdac99980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -58> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding auth protocol: cephx 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -57> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding auth protocol: cephx 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -56> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding auth protocol: cephx 2024-04-17T04:56:12.232 INFO:tasks.workunit.client.0.smithi088.stderr: -55> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -54> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -53> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -52> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -51> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -50> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -49> 2024-04-17T04:56:12.083+0000 7f4e8efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:12.233 INFO:tasks.workunit.client.0.smithi088.stderr: -48> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -47> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -46> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -45> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -44> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: crc 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -43> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 AuthRegistry(0x7fff16b50e90) adding con mode: secure 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -42> 2024-04-17T04:56:12.083+0000 7f4fdac99980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -41> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -40> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -39> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache list hook 0x55e3764d9cf0 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -38> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache inspect hook 0x55e3764d9cf0 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -37> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache erase hook 0x55e3764d9cf0 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -36> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command cache zap hook 0x55e3764d9cf0 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -35> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 note: GC not initialized 2024-04-17T04:56:12.234 INFO:tasks.workunit.client.0.smithi088.stderr: -34> 2024-04-17T04:56:12.083+0000 7f4e8dfdb640 20 reqs_thread_entry: start 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -33> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace show hook 0x7f4fc00043b0 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -32> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace history hook 0x7f4fc00043b0 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -31> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace active hook 0x7f4fc00043b0 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -30> 2024-04-17T04:56:12.083+0000 7f4fdac99980 5 asok(0x55e376201f40) register_command sync trace active_short hook 0x7f4fc00043b0 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -29> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -28> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 Filter name: none 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -27> 2024-04-17T04:56:12.083+0000 7f4dff7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -26> 2024-04-17T04:56:12.083+0000 7f4fdac99980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -25> 2024-04-17T04:56:12.083+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -24> 2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -23> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -22> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -21> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 : miss 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -20> 2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.235 INFO:tasks.workunit.client.0.smithi088.stderr: -19> 2024-04-17T04:56:12.087+0000 7f4fdac99980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -18> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -17> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -16> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -15> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -14> 2024-04-17T04:56:12.087+0000 7f4fdac99980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -13> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -12> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -11> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -10> 2024-04-17T04:56:12.095+0000 7f4fdac99980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29, ofs0, ns] 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -9> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464282 cookie 94435430929664 notifier 7011 bl.length()=987 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -8> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -7> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -6> 2024-04-17T04:56:12.095+0000 7f4eaf7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:12.236 INFO:tasks.workunit.client.0.smithi088.stderr: -5> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_sources= new_sources= 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: -4> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential sources added=[] removed=[] 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: -3> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_dests= new_dests= 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: -2> 2024-04-17T04:56:12.099+0000 7f4fdac99980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential dests added=[] removed=[] 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: -1> 2024-04-17T04:56:12.099+0000 7f4fdac99980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4fdac99980 time 2024-04-17T04:56:12.103059+0000 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4fde570e1f] 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.237 INFO:tasks.workunit.client.0.smithi088.stderr: 5: main() 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 7: __libc_start_main() 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 8: _start() 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 0> 2024-04-17T04:56:12.103+0000 7f4fdac99980 -1 *** Caught signal (Aborted) ** 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: in thread 7f4fdac99980 thread_name:radosgw-admin 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: ceph version 19.0.0-2287-g461a84e6 (461a84e6fbbf089011eafd13b98169fb9e9b8cc9) squid (dev) 2024-04-17T04:56:12.238 INFO:tasks.workunit.client.0.smithi088.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4fdd3f2520] 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 2: pthread_kill() 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 3: raise() 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 4: abort() 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4fde570e79] 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182fd3) [0x7f4fde570fd3] 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3745a9191] 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x315) [0x55e3745b7a05] 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 9: main() 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4fdd3d9d90] 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 11: __libc_start_main() 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 12: _start() 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr:--- logging levels --- 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 none 2024-04-17T04:56:12.239 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 lockdep 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 context 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 crush 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_balancer 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_locker 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_log_expire 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mds_migrator 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 3/ 5 mds_quiesce 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 buffer 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 timer 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 filer 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 striper 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 1 objecter 2024-04-17T04:56:12.240 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rados 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_mirror 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_replay 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 rbd_pwl 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 journaler 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objectcacher 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 immutable_obj_cache 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 client 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 osd 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 optracker 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 objclass 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 filestore 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 journal 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 ms 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mon 2024-04-17T04:56:12.241 INFO:tasks.workunit.client.0.smithi088.stderr: 0/10 monc 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 paxos 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 tp 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 auth 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 crypto 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 finisher 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 reserver 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 heartbeatmap 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 perfcounter 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 20/20 rgw 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_sync 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_datacache 2024-04-17T04:56:12.242 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_access 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_dbstore 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_flight 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 rgw_lifecycle 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 javaclient 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 asok 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 1 throttle 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 0 refs 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 compressor 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluestore 2024-04-17T04:56:12.243 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 bluefs 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 3 bdev 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 kstore 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 4/ 5 rocksdb 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 fuse 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 2/ 5 mgr 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mgrc 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 dpdk 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 eventtrace 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 prioritycache 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 test 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephfs_mirror 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cephsqlite 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 crimson_interrupt 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_onode 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_odata 2024-04-17T04:56:12.244 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_omap 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_tm 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_t 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cleaner 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_epm 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_lba 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_cache 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_journal 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_device 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 seastore_backref 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 alienstore 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 mclock 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 0/ 5 cyanstore 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 ceph_exporter 2024-04-17T04:56:12.245 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 memstore 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 1/ 5 trace 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: -2/-2 (syslog threshold) 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 99/99 (stderr threshold) 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4dff7fe640 / http_manager 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4e8dfdb640 / http_manager 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4e8efdd640 / rgw_dt_lg_renew 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4eadffb640 / ms_dispatch 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4eaf7fe640 / io_context_pool 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4ebdffb640 / ms_dispatch 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4ec4e09640 / ms_dispatch 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4ec660c640 / io_context_pool 2024-04-17T04:56:12.246 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd77b2640 / io_context_pool 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd7fb3640 / msgr-worker-2 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd87b4640 / msgr-worker-1 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fd8fb5640 / msgr-worker-0 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: 7f4fdac99980 / radosgw-admin 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: max_recent 500 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: max_new 1000 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr: log_file /var/lib/ceph/crash/2024-04-17T04:56:12.105009Z_fa696eaa-b3dd-46e3-9611-e10688145c1a/log 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr:--- end dump of recent events --- 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr:Aborted 2024-04-17T04:56:12.247 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 20 realm 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.183+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.249 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.187+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.187+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.187+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.187+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.187+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:12.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.187+0000 7fee21af4980 20 searching for the correct realm 2024-04-17T04:56:12.250 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.191+0000 7fee1ee0e640 0 -- 172.21.15.88:0/524004432 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fee100e4020 msgr2=0x7fee10007c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:12.402 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.407+0000 7fee1f60f640 0 -- 172.21.15.88:0/524004432 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fee14073870 msgr2=0x7fee1401a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:12.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:12.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:12.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:12.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:12.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:12.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:12.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:12.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.623+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:12.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 zone default found 2024-04-17T04:56:12.622 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 4 Realm: () 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 zonegroup default 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.624 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:12.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 started sync module instance, tier type = 2024-04-17T04:56:12.625 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.627+0000 7fee21af4980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:12.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=0 2024-04-17T04:56:12.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=1 2024-04-17T04:56:12.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=2 2024-04-17T04:56:12.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=3 2024-04-17T04:56:12.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=4 2024-04-17T04:56:12.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=5 2024-04-17T04:56:12.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=6 2024-04-17T04:56:12.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 20 add_watcher() i=7 2024-04-17T04:56:12.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.651+0000 7fee21af4980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:12.650 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fecd5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:12.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fee21af4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:12.651 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fee21af4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:12.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fee21af4980 5 note: GC not initialized 2024-04-17T04:56:12.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fecd4fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:12.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fee21af4980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:12.652 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fee21af4980 20 Filter name: none 2024-04-17T04:56:12.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fec4e7fc640 20 reqs_thread_entry: start 2024-04-17T04:56:12.653 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.655+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:12.654 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=457 2024-04-17T04:56:12.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:12.655 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:12.656 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.659+0000 7fee21af4980 20 rados->read ofs=0 len=0 2024-04-17T04:56:12.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.663+0000 7fee21af4980 20 rados_obj.operate() r=0 bl.length=457 2024-04-17T04:56:12.657 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.663+0000 7fee21af4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:12.659 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.663+0000 7fecd5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:12.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.667+0000 7fee21af4980 20 remove_watcher() i=0 2024-04-17T04:56:12.665 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.667+0000 7fee21af4980 2 removed watcher, disabling cache 2024-04-17T04:56:12.667 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.671+0000 7fee21af4980 20 remove_watcher() i=1 2024-04-17T04:56:12.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.675+0000 7fee21af4980 20 remove_watcher() i=2 2024-04-17T04:56:12.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.675+0000 7fee21af4980 20 remove_watcher() i=3 2024-04-17T04:56:12.673 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.679+0000 7fee21af4980 20 remove_watcher() i=4 2024-04-17T04:56:12.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.679+0000 7fee21af4980 20 remove_watcher() i=5 2024-04-17T04:56:12.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.679+0000 7fee21af4980 20 remove_watcher() i=6 2024-04-17T04:56:12.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:12.683+0000 7fee21af4980 20 remove_watcher() i=7 2024-04-17T04:56:12.697 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:12.697 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:12.697 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-do-reshard id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-17T04:56:12.697 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:42.975 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-17T04:56:43.010 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:43.011 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:43.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.058+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.058+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.058+0000 7f64ba60c980 20 realm 2024-04-17T04:56:43.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.058+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:43.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.058 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:43.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:43.059 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.062+0000 7f64ba60c980 20 searching for the correct realm 2024-04-17T04:56:43.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.066+0000 7f64b8928640 0 -- 172.21.15.88:0/1901301671 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f64a8012150 msgr2=0x7f64a805b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:43.071 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.074+0000 7f64b8127640 0 -- 172.21.15.88:0/1901301671 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f64a8086810 msgr2=0x7f64a80849a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:43.289 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.290+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:43.290 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:43.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 zone default found 2024-04-17T04:56:43.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 4 Realm: () 2024-04-17T04:56:43.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:43.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:43.291 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:43.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 zonegroup default 2024-04-17T04:56:43.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.292 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.293 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.294+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.298+0000 7f64ba60c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.298+0000 7f64ba60c980 20 started sync module instance, tier type = 2024-04-17T04:56:43.294 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.298+0000 7f64ba60c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=0 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=1 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=2 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=3 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=4 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=5 2024-04-17T04:56:43.317 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.318+0000 7f64ba60c980 20 add_watcher() i=6 2024-04-17T04:56:43.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.322+0000 7f64ba60c980 20 add_watcher() i=7 2024-04-17T04:56:43.318 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.322+0000 7f64ba60c980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:43.322 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f636efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:43.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 5 note: GC not initialized 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f636dfdb640 20 reqs_thread_entry: start 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 Filter name: none 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f62df7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-17T04:56:43.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:43.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-17T04:56:43.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-17T04:56:43.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 : miss 2024-04-17T04:56:43.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.326+0000 7f64ba60c980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.330+0000 7f64ba60c980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:43.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.330+0000 7f64ba60c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:43.326 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.330+0000 7f64ba60c980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:43.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.330+0000 7f64ba60c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:43.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.330+0000 7f64ba60c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:43.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.330+0000 7f64ba60c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 2024-04-17T04:56:43.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:43.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:43.333 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:43.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29, ofs0, ns] 2024-04-17T04:56:43.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f638f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464284 cookie 93897962205312 notifier 7050 bl.length()=987 2024-04-17T04:56:43.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f638f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:43.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f638f7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:43.334 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f638f7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:43.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_sources= new_sources= 2024-04-17T04:56:43.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential sources added=[] removed=[] 2024-04-17T04:56:43.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_dests= new_dests= 2024-04-17T04:56:43.336 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.338+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential dests added=[] removed=[] 2024-04-17T04:56:43.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.342+0000 7f64ba60c980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29.1.0 2024-04-17T04:56:43.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.342+0000 7f64ba60c980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29.1.0 2024-04-17T04:56:43.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.342+0000 7f64ba60c980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29.1.1 2024-04-17T04:56:43.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.342+0000 7f64ba60c980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29.1.1 2024-04-17T04:56:43.338 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.342+0000 7f64ba60c980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29.0 2024-04-17T04:56:43.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f64ba60c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:43.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f64ba60c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 2024-04-17T04:56:43.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f64ba60c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:43.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f64ba60c980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:43.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f64ba60c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:43.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f64ba60c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29, ofs0, ns] 2024-04-17T04:56:43.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f638effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464285 cookie 93897962205312 notifier 7050 bl.length()=955 2024-04-17T04:56:43.345 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f638effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 info.flags=0x17 2024-04-17T04:56:43.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f638effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 to cache LRU end 2024-04-17T04:56:43.346 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.346+0000 7f638effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-04-17T04:56:43.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.350+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_sources= new_sources= 2024-04-17T04:56:43.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.350+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential sources added=[] removed=[] 2024-04-17T04:56:43.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.350+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): orig_dests= new_dests= 2024-04-17T04:56:43.347 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.350+0000 7f64ba60c980 20 handle_bi_update(): bucket=:abort-at-do-reshard[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29]): potential dests added=[] removed=[] 2024-04-17T04:56:43.348 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.350+0000 7f64ba60c980 1 execute INFO: reshard of bucket "abort-at-do-reshard" completed successfully 2024-04-17T04:56:43.351 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.354+0000 7f636efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:43.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.358+0000 7f64ba60c980 20 remove_watcher() i=0 2024-04-17T04:56:43.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.358+0000 7f64ba60c980 2 removed watcher, disabling cache 2024-04-17T04:56:43.358 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.362+0000 7f64ba60c980 20 remove_watcher() i=1 2024-04-17T04:56:43.360 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.362+0000 7f64ba60c980 20 remove_watcher() i=2 2024-04-17T04:56:43.362 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.366+0000 7f64ba60c980 20 remove_watcher() i=3 2024-04-17T04:56:43.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.366+0000 7f64ba60c980 20 remove_watcher() i=4 2024-04-17T04:56:43.366 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.370+0000 7f64ba60c980 20 remove_watcher() i=5 2024-04-17T04:56:43.367 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.370+0000 7f64ba60c980 20 remove_watcher() i=6 2024-04-17T04:56:43.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.370+0000 7f64ba60c980 20 remove_watcher() i=7 2024-04-17T04:56:43.384 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-04-17T04:56:43.414 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:43.414 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:43.458 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.460 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 realm 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.461 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.462+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.466+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:43.462 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.466+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.466+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:43.463 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.466+0000 7f4c3ec9d980 20 searching for the correct realm 2024-04-17T04:56:43.470 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.474+0000 7f4c3bfb7640 0 -- 172.21.15.88:0/1372581844 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f4c2c04dc20 msgr2=0x7f4c2c008950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:43.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.478+0000 7f4c3cfb9640 0 -- 172.21.15.88:0/1372581844 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f4c300740b0 msgr2=0x7f4c3005ab60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.485 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.486+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:43.486 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 zone default found 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 4 Realm: () 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:43.487 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 zonegroup default 2024-04-17T04:56:43.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.488 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:43.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 started sync module instance, tier type = 2024-04-17T04:56:43.489 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.490+0000 7f4c3ec9d980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:43.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=0 2024-04-17T04:56:43.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=1 2024-04-17T04:56:43.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=2 2024-04-17T04:56:43.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=3 2024-04-17T04:56:43.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=4 2024-04-17T04:56:43.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=5 2024-04-17T04:56:43.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.506+0000 7f4c3ec9d980 20 add_watcher() i=6 2024-04-17T04:56:43.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 20 add_watcher() i=7 2024-04-17T04:56:43.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:43.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4af2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 5 note: GC not initialized 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4af1fdb640 20 reqs_thread_entry: start 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 20 Filter name: none 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4a637fe640 20 reqs_thread_entry: start 2024-04-17T04:56:43.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.510+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:43.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:43.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:43.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=211 2024-04-17T04:56:43.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:43.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 rados_obj.operate() r=0 bl.length=425 2024-04-17T04:56:43.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.514+0000 7f4c3ec9d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:43.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.518+0000 7f4af2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:43.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.522+0000 7f4c3ec9d980 20 remove_watcher() i=0 2024-04-17T04:56:43.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.522+0000 7f4c3ec9d980 2 removed watcher, disabling cache 2024-04-17T04:56:43.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.526+0000 7f4c3ec9d980 20 remove_watcher() i=1 2024-04-17T04:56:43.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.526+0000 7f4c3ec9d980 20 remove_watcher() i=2 2024-04-17T04:56:43.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.530+0000 7f4c3ec9d980 20 remove_watcher() i=3 2024-04-17T04:56:43.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.530+0000 7f4c3ec9d980 20 remove_watcher() i=4 2024-04-17T04:56:43.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.530+0000 7f4c3ec9d980 20 remove_watcher() i=5 2024-04-17T04:56:43.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.534+0000 7f4c3ec9d980 20 remove_watcher() i=6 2024-04-17T04:56:43.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:43.534+0000 7f4c3ec9d980 20 remove_watcher() i=7 2024-04-17T04:56:43.544 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:43.544 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:43.544 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: abort-at-do-reshard id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-17T04:56:43.544 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:44.280 INFO:tasks.workunit.client.0.smithi088.stderr: test: reshard versioned bucket 2024-04-17T04:56:44.281 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket myver 2024-04-17T04:56:44.315 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:44.315 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:44.361 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.362+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 realm 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.363 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:44.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:44.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.366+0000 7f37cf6b5980 20 searching for the correct realm 2024-04-17T04:56:44.368 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.370+0000 7f37cd1d0640 0 -- 172.21.15.88:0/75856306 >> [v2:172.21.15.88:6804/508820735,v1:172.21.15.88:6806/508820735] conn(0x7f37c802f930 msgr2=0x7f37c803b3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:44.375 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.378+0000 7f37cd9d1640 0 -- 172.21.15.88:0/75856306 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f37b80d0040 msgr2=0x7f37b80f5db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:44.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 zone default found 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 4 Realm: () 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 zonegroup default 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 started sync module instance, tier type = 2024-04-17T04:56:44.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.390+0000 7f37cf6b5980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:44.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=0 2024-04-17T04:56:44.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=1 2024-04-17T04:56:44.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=2 2024-04-17T04:56:44.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=3 2024-04-17T04:56:44.406 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=4 2024-04-17T04:56:44.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=5 2024-04-17T04:56:44.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=6 2024-04-17T04:56:44.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 20 add_watcher() i=7 2024-04-17T04:56:44.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.410+0000 7f37cf6b5980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:44.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f367f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 5 note: GC not initialized 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f367e7e4640 20 reqs_thread_entry: start 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 Filter name: none 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f35effff640 20 reqs_thread_entry: start 2024-04-17T04:56:44.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:44.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:44.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:44.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:44.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:44.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.414+0000 7f37cf6b5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:44.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.418+0000 7f367f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:44.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.422+0000 7f37cf6b5980 20 remove_watcher() i=0 2024-04-17T04:56:44.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.422+0000 7f37cf6b5980 2 removed watcher, disabling cache 2024-04-17T04:56:44.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.426+0000 7f37cf6b5980 20 remove_watcher() i=1 2024-04-17T04:56:44.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.426+0000 7f37cf6b5980 20 remove_watcher() i=2 2024-04-17T04:56:44.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.430+0000 7f37cf6b5980 20 remove_watcher() i=3 2024-04-17T04:56:44.428 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.430+0000 7f37cf6b5980 20 remove_watcher() i=4 2024-04-17T04:56:44.429 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.434+0000 7f37cf6b5980 20 remove_watcher() i=5 2024-04-17T04:56:44.431 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.434+0000 7f37cf6b5980 20 remove_watcher() i=6 2024-04-17T04:56:44.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.434+0000 7f37cf6b5980 20 remove_watcher() i=7 2024-04-17T04:56:44.446 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:44.446 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:44.446 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: myver id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 num_objects: 0 size_kb: 0 num_shards: 1 2024-04-17T04:56:44.446 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:44.446 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 2 2024-04-17T04:56:44.474 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:44.475 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:44.518 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.519 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 realm 2024-04-17T04:56:44.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:44.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.520 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.522+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.526+0000 7f66fe689980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:44.521 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.526+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.526+0000 7f66fe689980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:44.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.526+0000 7f66fe689980 20 searching for the correct realm 2024-04-17T04:56:44.528 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.530+0000 7f66fb9a3640 0 -- 172.21.15.88:0/2358765117 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x56040ff750f0 msgr2=0x56040ff77500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:44.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.538+0000 7f66fb9a3640 0 -- 172.21.15.88:0/2358765117 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f66ec00e040 msgr2=0x7f66ec059fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:44.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 zone default found 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 4 Realm: () 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:44.545 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 zonegroup default 2024-04-17T04:56:44.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.546 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.546+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.550+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.550+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.550+0000 7f66fe689980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.550+0000 7f66fe689980 20 started sync module instance, tier type = 2024-04-17T04:56:44.547 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.550+0000 7f66fe689980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:44.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=0 2024-04-17T04:56:44.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=1 2024-04-17T04:56:44.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=2 2024-04-17T04:56:44.562 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=3 2024-04-17T04:56:44.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=4 2024-04-17T04:56:44.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=5 2024-04-17T04:56:44.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=6 2024-04-17T04:56:44.563 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 20 add_watcher() i=7 2024-04-17T04:56:44.564 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f66fe689980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:44.565 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.566+0000 7f65b2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:44.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:44.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:44.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 5 note: GC not initialized 2024-04-17T04:56:44.567 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f65b1fdb640 20 reqs_thread_entry: start 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 Filter name: none 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f65237fe640 20 reqs_thread_entry: start 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-17T04:56:44.568 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : miss 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 adding default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:44.569 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.570+0000 7f66fe689980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.574+0000 7f66fe689980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.574+0000 7f66fe689980 10 moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.574+0000 7f66fe689980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.574+0000 7f66fe689980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19, ofs0, ns] 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f65d37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464297 cookie 94575447742240 notifier 7104 bl.length()=897 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f65d37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f65d37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:44.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f65d37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:44.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_sources= new_sources= 2024-04-17T04:56:44.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential sources added=[] removed=[] 2024-04-17T04:56:44.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_dests= new_dests= 2024-04-17T04:56:44.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.578+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential dests added=[] removed=[] 2024-04-17T04:56:44.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.0 2024-04-17T04:56:44.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.0 2024-04-17T04:56:44.578 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.1 2024-04-17T04:56:44.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.1 2024-04-17T04:56:44.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.0 2024-04-17T04:56:44.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:44.579 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f66fe689980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19, ofs0, ns] 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f65d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464298 cookie 94575447742240 notifier 7104 bl.length()=865 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f65d2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f65d2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:44.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.582+0000 7f65d2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:44.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.586+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_sources= new_sources= 2024-04-17T04:56:44.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.586+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential sources added=[] removed=[] 2024-04-17T04:56:44.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.586+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_dests= new_dests= 2024-04-17T04:56:44.583 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.586+0000 7f66fe689980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential dests added=[] removed=[] 2024-04-17T04:56:44.585 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.590+0000 7f66fe689980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-04-17T04:56:44.588 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.590+0000 7f65b2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:44.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.598+0000 7f66fe689980 20 remove_watcher() i=0 2024-04-17T04:56:44.593 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.598+0000 7f66fe689980 2 removed watcher, disabling cache 2024-04-17T04:56:44.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.598+0000 7f66fe689980 20 remove_watcher() i=1 2024-04-17T04:56:44.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.602+0000 7f66fe689980 20 remove_watcher() i=2 2024-04-17T04:56:44.599 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.602+0000 7f66fe689980 20 remove_watcher() i=3 2024-04-17T04:56:44.601 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.606+0000 7f66fe689980 20 remove_watcher() i=4 2024-04-17T04:56:44.602 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.606+0000 7f66fe689980 20 remove_watcher() i=5 2024-04-17T04:56:44.603 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.606+0000 7f66fe689980 20 remove_watcher() i=6 2024-04-17T04:56:44.605 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.606+0000 7f66fe689980 20 remove_watcher() i=7 2024-04-17T04:56:44.619 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket myver 2024-04-17T04:56:44.645 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:44.645 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:44.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.690+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.690+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.690+0000 7fc6ed31e980 20 realm 2024-04-17T04:56:44.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.690+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.690+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.690+0000 7fc6ed31e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:44.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.690 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:44.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:44.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.694+0000 7fc6ed31e980 20 searching for the correct realm 2024-04-17T04:56:44.698 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.702+0000 7fc6ea638640 0 -- 172.21.15.88:0/669146984 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fc6dc05d830 msgr2=0x7fc6dc0821e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:44.704 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.706+0000 7fc6ea638640 0 -- 172.21.15.88:0/669146984 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fc5b40763a0 msgr2=0x7fc5b40787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:44.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:44.711 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:44.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:44.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:44.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:44.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:44.712 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.713 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.714+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 zone default found 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 4 Realm: () 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 zonegroup default 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:44.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 started sync module instance, tier type = 2024-04-17T04:56:44.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.718+0000 7fc6ed31e980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=0 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=1 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=2 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=3 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=4 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=5 2024-04-17T04:56:44.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=6 2024-04-17T04:56:44.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 20 add_watcher() i=7 2024-04-17T04:56:44.731 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.734+0000 7fc6ed31e980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:44.733 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc5a17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:44.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:44.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:44.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 5 note: GC not initialized 2024-04-17T04:56:44.734 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc51bfff640 20 reqs_thread_entry: start 2024-04-17T04:56:44.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:44.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 Filter name: none 2024-04-17T04:56:44.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc519ffb640 20 reqs_thread_entry: start 2024-04-17T04:56:44.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.735 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados->read ofs=0 len=0 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:44.736 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.738+0000 7fc6ed31e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:44.737 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.742+0000 7fc5a17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:44.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.746+0000 7fc6ed31e980 20 remove_watcher() i=0 2024-04-17T04:56:44.742 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.746+0000 7fc6ed31e980 2 removed watcher, disabling cache 2024-04-17T04:56:44.744 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.746+0000 7fc6ed31e980 20 remove_watcher() i=1 2024-04-17T04:56:44.746 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.750+0000 7fc6ed31e980 20 remove_watcher() i=2 2024-04-17T04:56:44.748 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.750+0000 7fc6ed31e980 20 remove_watcher() i=3 2024-04-17T04:56:44.750 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.754+0000 7fc6ed31e980 20 remove_watcher() i=4 2024-04-17T04:56:44.751 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.754+0000 7fc6ed31e980 20 remove_watcher() i=5 2024-04-17T04:56:44.752 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.754+0000 7fc6ed31e980 20 remove_watcher() i=6 2024-04-17T04:56:44.753 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:44.758+0000 7fc6ed31e980 20 remove_watcher() i=7 2024-04-17T04:56:44.767 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:44.767 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:44.767 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: myver id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 num_objects: 0 size_kb: 0 num_shards: 2 2024-04-17T04:56:44.767 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:44.850 INFO:tasks.workunit.client.0.smithi088.stderr: test: reshard removes olh entries with empty name 2024-04-17T04:56:45.263 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-17T04:56:45.295 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:45.295 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:45.340 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.342+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.341 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 realm 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.342 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:45.343 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:45.344 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.346+0000 7f5320f82980 20 searching for the correct realm 2024-04-17T04:56:45.350 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.354+0000 7f531f29e640 0 -- 172.21.15.88:0/4201564760 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55ad3aae3760 msgr2=0x55ad3aae5c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:45.356 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.358+0000 7f531ea9d640 0 -- 172.21.15.88:0/4201564760 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f53140d88f0 msgr2=0x7f53140b83e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:45.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:45.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:45.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:45.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:45.364 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:45.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:45.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.365 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:45.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.366+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.370 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:45.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 zone default found 2024-04-17T04:56:45.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 4 Realm: () 2024-04-17T04:56:45.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:45.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:45.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:45.371 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 zonegroup default 2024-04-17T04:56:45.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.372 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 started sync module instance, tier type = 2024-04-17T04:56:45.373 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.370+0000 7f5320f82980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:45.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=0 2024-04-17T04:56:45.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=1 2024-04-17T04:56:45.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=2 2024-04-17T04:56:45.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=3 2024-04-17T04:56:45.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=4 2024-04-17T04:56:45.383 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=5 2024-04-17T04:56:45.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=6 2024-04-17T04:56:45.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 20 add_watcher() i=7 2024-04-17T04:56:45.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.386+0000 7f5320f82980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:45.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f51d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:45.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 5 note: GC not initialized 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f51d4fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 Filter name: none 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f51cdffb640 20 reqs_thread_entry: start 2024-04-17T04:56:45.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:56:45.389 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.390+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:56:45.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:45.390 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:56:45.391 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:56:45.392 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.394+0000 7f5320f82980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:45.394 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.398+0000 7f51d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:45.399 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.402+0000 7f5320f82980 20 remove_watcher() i=0 2024-04-17T04:56:45.400 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.402+0000 7f5320f82980 2 removed watcher, disabling cache 2024-04-17T04:56:45.401 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.406+0000 7f5320f82980 20 remove_watcher() i=1 2024-04-17T04:56:45.403 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.406+0000 7f5320f82980 20 remove_watcher() i=2 2024-04-17T04:56:45.405 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.410+0000 7f5320f82980 20 remove_watcher() i=3 2024-04-17T04:56:45.407 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.410+0000 7f5320f82980 20 remove_watcher() i=4 2024-04-17T04:56:45.409 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.410+0000 7f5320f82980 20 remove_watcher() i=5 2024-04-17T04:56:45.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.414+0000 7f5320f82980 20 remove_watcher() i=6 2024-04-17T04:56:45.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.414+0000 7f5320f82980 20 remove_watcher() i=7 2024-04-17T04:56:45.425 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:45.425 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:45.425 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: a-bucket id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 num_objects: 0 size_kb: 0 num_shards: 3 2024-04-17T04:56:45.425 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:45.425 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-04-17T04:56:45.452 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:45.452 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:45.494 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 realm 2024-04-17T04:56:45.496 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:45.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.497 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.498+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.502+0000 7f5ef4709980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:45.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.502+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.502+0000 7f5ef4709980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:45.499 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.502+0000 7f5ef4709980 20 searching for the correct realm 2024-04-17T04:56:45.505 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.510+0000 7f5ef1a23640 0 -- 172.21.15.88:0/32468919 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5ee4029070 msgr2=0x7f5ee405c340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:45.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.514+0000 7f5ef1a23640 0 -- 172.21.15.88:0/32468919 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5dbc0763a0 msgr2=0x7f5dbc0787d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:45.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:45.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:45.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:45.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:45.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:45.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:45.523 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 zone default found 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 4 Realm: () 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:45.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:45.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:45.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 zonegroup default 2024-04-17T04:56:45.525 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.526 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 started sync module instance, tier type = 2024-04-17T04:56:45.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.522+0000 7f5ef4709980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:45.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=0 2024-04-17T04:56:45.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=1 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=2 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=3 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=4 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=5 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.538+0000 7f5ef4709980 20 add_watcher() i=6 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 20 add_watcher() i=7 2024-04-17T04:56:45.537 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:45.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5da8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 5 note: GC not initialized 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5d237fe640 20 reqs_thread_entry: start 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 20 Filter name: none 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5d217fa640 20 reqs_thread_entry: start 2024-04-17T04:56:45.541 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.542+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.546+0000 7f5ef4709980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:56:45.542 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.546+0000 7f5ef4709980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.546+0000 7f5ef4709980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:56:45.543 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.546+0000 7f5ef4709980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:45.544 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.546+0000 7f5da8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:45.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.550+0000 7f5ef4709980 20 remove_watcher() i=0 2024-04-17T04:56:45.549 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.550+0000 7f5ef4709980 2 removed watcher, disabling cache 2024-04-17T04:56:45.551 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.554+0000 7f5ef4709980 20 remove_watcher() i=1 2024-04-17T04:56:45.552 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.554+0000 7f5ef4709980 20 remove_watcher() i=2 2024-04-17T04:56:45.554 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.558+0000 7f5ef4709980 20 remove_watcher() i=3 2024-04-17T04:56:45.556 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.558+0000 7f5ef4709980 20 remove_watcher() i=4 2024-04-17T04:56:45.558 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.562+0000 7f5ef4709980 20 remove_watcher() i=5 2024-04-17T04:56:45.559 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.562+0000 7f5ef4709980 20 remove_watcher() i=6 2024-04-17T04:56:45.560 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.562+0000 7f5ef4709980 20 remove_watcher() i=7 2024-04-17T04:56:45.574 INFO:tasks.workunit.client.0.smithi088.stderr:rados -p default.rgw.buckets.index setomapval .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-04-17T04:56:45.599 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:45.599 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:45.618 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket a-bucket 2024-04-17T04:56:45.643 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:45.643 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:45.684 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.686+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 realm 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:45.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:45.688 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.690+0000 7fc1bde1d980 20 searching for the correct realm 2024-04-17T04:56:45.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.698+0000 7fc1bc139640 0 -- 172.21.15.88:0/1186996576 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7fc1b0047040 msgr2=0x7fc1b0049950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:45.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.702+0000 7fc1bb938640 0 -- 172.21.15.88:0/1186996576 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fc1b0047930 msgr2=0x7fc1b0019410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:45.915 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:45.916 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.918+0000 7fc1bde1d980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:45.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 zone default found 2024-04-17T04:56:45.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 4 Realm: () 2024-04-17T04:56:45.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:45.917 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:45.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:45.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 zonegroup default 2024-04-17T04:56:45.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.918 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.919 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:45.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 started sync module instance, tier type = 2024-04-17T04:56:45.920 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.922+0000 7fc1bde1d980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:45.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=0 2024-04-17T04:56:45.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=1 2024-04-17T04:56:45.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=2 2024-04-17T04:56:45.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=3 2024-04-17T04:56:45.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=4 2024-04-17T04:56:45.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=5 2024-04-17T04:56:45.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.946+0000 7fc1bde1d980 20 add_watcher() i=6 2024-04-17T04:56:45.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.950+0000 7fc1bde1d980 20 add_watcher() i=7 2024-04-17T04:56:45.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.950+0000 7fc1bde1d980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:45.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc071fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:45.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:45.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:45.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 5 note: GC not initialized 2024-04-17T04:56:45.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc070fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:45.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:45.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 Filter name: none 2024-04-17T04:56:45.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fbfea7fc640 20 reqs_thread_entry: start 2024-04-17T04:56:45.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:56:45.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.954+0000 7fc1bde1d980 20 rados->read ofs=0 len=0 2024-04-17T04:56:45.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:56:45.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:45.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-17T04:56:45.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 INFO: main: starting shard=0 2024-04-17T04:56:45.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 2024-04-17T04:56:45.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 2024-04-17T04:56:45.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:56:45.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 INFO: main: starting shard=1 2024-04-17T04:56:45.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.1 2024-04-17T04:56:45.956 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.958+0000 7fc1bde1d980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.1 2024-04-17T04:56:45.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc1bde1d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:45.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc1bde1d980 20 INFO: main: starting shard=2 2024-04-17T04:56:45.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc1bde1d980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.2 2024-04-17T04:56:45.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc1bde1d980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.2 2024-04-17T04:56:45.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc1bde1d980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:45.958 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc1bde1d980 20 INFO: main: done 2024-04-17T04:56:45.960 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.962+0000 7fc071fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:45.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.966+0000 7fc1bde1d980 20 remove_watcher() i=0 2024-04-17T04:56:45.965 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.966+0000 7fc1bde1d980 2 removed watcher, disabling cache 2024-04-17T04:56:45.967 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.970+0000 7fc1bde1d980 20 remove_watcher() i=1 2024-04-17T04:56:45.969 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.974+0000 7fc1bde1d980 20 remove_watcher() i=2 2024-04-17T04:56:45.971 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.974+0000 7fc1bde1d980 20 remove_watcher() i=3 2024-04-17T04:56:45.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.978+0000 7fc1bde1d980 20 remove_watcher() i=4 2024-04-17T04:56:45.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.978+0000 7fc1bde1d980 20 remove_watcher() i=5 2024-04-17T04:56:45.977 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.978+0000 7fc1bde1d980 20 remove_watcher() i=6 2024-04-17T04:56:45.978 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:45.982+0000 7fc1bde1d980 20 remove_watcher() i=7 2024-04-17T04:56:45.999 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-04-17T04:56:46.032 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:46.032 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:46.078 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 realm 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.082+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.086+0000 7fa7fed60980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.086+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.086+0000 7fa7fed60980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:46.082 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.086+0000 7fa7fed60980 20 searching for the correct realm 2024-04-17T04:56:46.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.090+0000 7fa7fc07a640 0 -- 172.21.15.88:0/2539805779 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5557b99d90f0 msgr2=0x5557b99db570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.095 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.098+0000 7fa7fc07a640 0 -- 172.21.15.88:0/2539805779 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fa7f40f1430 msgr2=0x7fa7f40f1830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.102 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.103 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 zone default found 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 4 Realm: () 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 zonegroup default 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.106+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.110+0000 7fa7fed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.110+0000 7fa7fed60980 20 started sync module instance, tier type = 2024-04-17T04:56:46.105 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.110+0000 7fa7fed60980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=0 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=1 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=2 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=3 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=4 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=5 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.122+0000 7fa7fed60980 20 add_watcher() i=6 2024-04-17T04:56:46.121 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa7fed60980 20 add_watcher() i=7 2024-04-17T04:56:46.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa7fed60980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:46.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa6b2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa7fed60980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:46.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa7fed60980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa7fed60980 5 note: GC not initialized 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa6b1fe3640 20 reqs_thread_entry: start 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.126+0000 7fa7fed60980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 20 Filter name: none 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa6237fe640 20 reqs_thread_entry: start 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-17T04:56:46.127 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:56:46.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-04-17T04:56:46.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-17T04:56:46.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : miss 2024-04-17T04:56:46.128 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:56:46.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:56:46.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:56:46.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:46.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.129 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.130+0000 7fa7fed60980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa7fed60980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa7fed60980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa7fed60980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa7fed60980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18, ofs0, ns] 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa6d37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464309 cookie 93835264680784 notifier 6794 bl.length()=903 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa6d37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:56:46.135 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa6d37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:56:46.136 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.138+0000 7fa6d37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.142+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_sources= new_sources= 2024-04-17T04:56:46.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.142+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential sources added=[] removed=[] 2024-04-17T04:56:46.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.142+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_dests= new_dests= 2024-04-17T04:56:46.138 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.142+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential dests added=[] removed=[] 2024-04-17T04:56:46.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.3.0 2024-04-17T04:56:46.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.3.0 2024-04-17T04:56:46.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 2024-04-17T04:56:46.141 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.0 2024-04-17T04:56:46.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-04-17T04:56:46.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.1 2024-04-17T04:56:46.142 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.1 2024-04-17T04:56:46.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.2 2024-04-17T04:56:46.143 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.2.2 2024-04-17T04:56:46.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:46.144 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.146+0000 7fa7fed60980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 2024-04-17T04:56:46.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa7fed60980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:56:46.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa7fed60980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:56:46.146 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa7fed60980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa7fed60980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18, ofs0, ns] 2024-04-17T04:56:46.147 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa6d3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464310 cookie 93835264680784 notifier 6794 bl.length()=871 2024-04-17T04:56:46.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa6d3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 info.flags=0x17 2024-04-17T04:56:46.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa6d3fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18 to cache LRU end 2024-04-17T04:56:46.148 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.150+0000 7fa6d3fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.154+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_sources= new_sources= 2024-04-17T04:56:46.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.154+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential sources added=[] removed=[] 2024-04-17T04:56:46.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.154+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): orig_dests= new_dests= 2024-04-17T04:56:46.149 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.154+0000 7fa7fed60980 20 handle_bi_update(): bucket=:a-bucket[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18]): potential dests added=[] removed=[] 2024-04-17T04:56:46.152 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.154+0000 7fa7fed60980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-04-17T04:56:46.155 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.158+0000 7fa6b2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.162+0000 7fa7fed60980 20 remove_watcher() i=0 2024-04-17T04:56:46.160 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.162+0000 7fa7fed60980 2 removed watcher, disabling cache 2024-04-17T04:56:46.162 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.166+0000 7fa7fed60980 20 remove_watcher() i=1 2024-04-17T04:56:46.164 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.166+0000 7fa7fed60980 20 remove_watcher() i=2 2024-04-17T04:56:46.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.170+0000 7fa7fed60980 20 remove_watcher() i=3 2024-04-17T04:56:46.168 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.170+0000 7fa7fed60980 20 remove_watcher() i=4 2024-04-17T04:56:46.170 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.174+0000 7fa7fed60980 20 remove_watcher() i=5 2024-04-17T04:56:46.171 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.174+0000 7fa7fed60980 20 remove_watcher() i=6 2024-04-17T04:56:46.173 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.174+0000 7fa7fed60980 20 remove_watcher() i=7 2024-04-17T04:56:46.187 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket a-bucket 2024-04-17T04:56:46.216 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:46.216 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:46.259 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.262+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.262+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.262+0000 7f8dd24b5980 20 realm 2024-04-17T04:56:46.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.262+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:46.261 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.262 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.263 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:46.264 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.266+0000 7f8dd24b5980 20 searching for the correct realm 2024-04-17T04:56:46.269 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.274+0000 7f8dd07d1640 0 -- 172.21.15.88:0/2555872072 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5559a3858ac0 msgr2=0x5559a385af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.275 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.278+0000 7f8dcffd0640 0 -- 172.21.15.88:0/2555872072 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f8dc4053190 msgr2=0x7f8dc405d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:46.283 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.284 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 zone default found 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 4 Realm: () 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 zonegroup default 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.285 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.286+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.290+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.290+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.290+0000 7f8dd24b5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.290+0000 7f8dd24b5980 20 started sync module instance, tier type = 2024-04-17T04:56:46.286 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.290+0000 7f8dd24b5980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:46.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=0 2024-04-17T04:56:46.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=1 2024-04-17T04:56:46.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=2 2024-04-17T04:56:46.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=3 2024-04-17T04:56:46.302 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=4 2024-04-17T04:56:46.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=5 2024-04-17T04:56:46.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=6 2024-04-17T04:56:46.303 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 20 add_watcher() i=7 2024-04-17T04:56:46.304 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8dd24b5980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:46.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.306+0000 7f8c85fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.308 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 5 note: GC not initialized 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8c84fe1640 20 reqs_thread_entry: start 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 Filter name: none 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8bfe7fc640 20 reqs_thread_entry: start 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 rados_obj.operate() r=0 bl.length=200 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 rados_obj.operate() r=0 bl.length=414 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-04-17T04:56:46.309 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 INFO: main: starting shard=0 2024-04-17T04:56:46.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.3.0 2024-04-17T04:56:46.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18.3.0 2024-04-17T04:56:46.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:46.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.310+0000 7f8dd24b5980 20 INFO: main: done 2024-04-17T04:56:46.310 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.314+0000 7f8c85fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.318+0000 7f8dd24b5980 20 remove_watcher() i=0 2024-04-17T04:56:46.314 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.318+0000 7f8dd24b5980 2 removed watcher, disabling cache 2024-04-17T04:56:46.316 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.318+0000 7f8dd24b5980 20 remove_watcher() i=1 2024-04-17T04:56:46.319 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.322+0000 7f8dd24b5980 20 remove_watcher() i=2 2024-04-17T04:56:46.321 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.322+0000 7f8dd24b5980 20 remove_watcher() i=3 2024-04-17T04:56:46.323 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.326+0000 7f8dd24b5980 20 remove_watcher() i=4 2024-04-17T04:56:46.324 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.326+0000 7f8dd24b5980 20 remove_watcher() i=5 2024-04-17T04:56:46.325 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.330+0000 7f8dd24b5980 20 remove_watcher() i=6 2024-04-17T04:56:46.327 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.330+0000 7f8dd24b5980 20 remove_watcher() i=7 2024-04-17T04:56:46.340 INFO:tasks.workunit.client.0.smithi088.stderr: test: PUT succeeds during reshard 2024-04-17T04:56:46.340 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket myver 2024-04-17T04:56:46.366 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:46.366 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:46.408 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.410+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.410 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 realm 2024-04-17T04:56:46.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:46.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.411 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:46.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.414+0000 7fcb5f736980 20 searching for the correct realm 2024-04-17T04:56:46.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.422+0000 7fcb5da52640 0 -- 172.21.15.88:0/1780345601 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fcb5405db90 msgr2=0x7fcb5407c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:46.635 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.638+0000 7fcb5d251640 0 -- 172.21.15.88:0/1780345601 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5573ef6da4e0 msgr2=0x5573ef6dc8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:46.643 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.644 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 zone default found 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 4 Realm: () 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:46.645 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 zonegroup default 2024-04-17T04:56:46.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.646+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.646 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.647 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 started sync module instance, tier type = 2024-04-17T04:56:46.648 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.650+0000 7fcb5f736980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=0 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=1 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=2 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=3 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=4 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=5 2024-04-17T04:56:46.669 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.670+0000 7fcb5f736980 20 add_watcher() i=6 2024-04-17T04:56:46.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.674+0000 7fcb5f736980 20 add_watcher() i=7 2024-04-17T04:56:46.671 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.674+0000 7fcb5f736980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:46.675 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fca137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fcb5f736980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:46.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fcb5f736980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:46.676 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fcb5f736980 5 note: GC not initialized 2024-04-17T04:56:46.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fca127e4640 20 reqs_thread_entry: start 2024-04-17T04:56:46.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fcb5f736980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:46.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.678+0000 7fcb5f736980 20 Filter name: none 2024-04-17T04:56:46.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fc983fff640 20 reqs_thread_entry: start 2024-04-17T04:56:46.677 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:46.678 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:46.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:46.679 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:46.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:46.680 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.682+0000 7fcb5f736980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:46.682 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.686+0000 7fca137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.686 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.690+0000 7fcb5f736980 20 remove_watcher() i=0 2024-04-17T04:56:46.687 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.690+0000 7fcb5f736980 2 removed watcher, disabling cache 2024-04-17T04:56:46.689 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.690+0000 7fcb5f736980 20 remove_watcher() i=1 2024-04-17T04:56:46.691 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.694+0000 7fcb5f736980 20 remove_watcher() i=2 2024-04-17T04:56:46.693 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.698+0000 7fcb5f736980 20 remove_watcher() i=3 2024-04-17T04:56:46.696 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.698+0000 7fcb5f736980 20 remove_watcher() i=4 2024-04-17T04:56:46.697 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.702+0000 7fcb5f736980 20 remove_watcher() i=5 2024-04-17T04:56:46.699 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.702+0000 7fcb5f736980 20 remove_watcher() i=6 2024-04-17T04:56:46.701 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.702+0000 7fcb5f736980 20 remove_watcher() i=7 2024-04-17T04:56:46.716 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:46.716 INFO:tasks.workunit.client.0.smithi088.stderr:BUCKET_STATS: 2024-04-17T04:56:46.716 INFO:tasks.workunit.client.0.smithi088.stderr:bucket: myver id: 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 num_objects: 0 size_kb: 0 num_shards: 2 2024-04-17T04:56:46.717 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:46.717 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 3 2024-04-17T04:56:46.747 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:46.747 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:46.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 realm 2024-04-17T04:56:46.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:46.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.793 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.794+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.798+0000 7f98d3caa980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.798+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.798+0000 7f98d3caa980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:46.794 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.798+0000 7f98d3caa980 20 searching for the correct realm 2024-04-17T04:56:46.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.802+0000 7f98d0fc4640 0 -- 172.21.15.88:0/1795549711 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55d19bca5170 msgr2=0x55d19bca7560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.806 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.810+0000 7f98d17c5640 0 -- 172.21.15.88:0/1795549711 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f98bc054450 msgr2=0x7f98bc0548b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:46.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:46.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:46.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:46.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:46.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:46.813 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:46.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.814+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:46.814 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:46.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 zone default found 2024-04-17T04:56:46.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 4 Realm: () 2024-04-17T04:56:46.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:46.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:46.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:46.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 zonegroup default 2024-04-17T04:56:46.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:46.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 started sync module instance, tier type = 2024-04-17T04:56:46.817 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.818+0000 7f98d3caa980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:46.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=0 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=1 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=2 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=3 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=4 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=5 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=6 2024-04-17T04:56:46.832 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 20 add_watcher() i=7 2024-04-17T04:56:46.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.834+0000 7f98d3caa980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:46.834 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f97877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 5 note: GC not initialized 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f97867e4640 20 reqs_thread_entry: start 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 Filter name: none 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f96f7fff640 20 reqs_thread_entry: start 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-17T04:56:46.836 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : miss 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 rados->read ofs=0 len=0 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 adding default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:46.837 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.838 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.838+0000 7f98d3caa980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 10 moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19, ofs0, ns] 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f97a3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464299 cookie 94358750305392 notifier 6845 bl.length()=897 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f97a3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f97a3fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:46.843 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f97a3fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:46.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_sources= new_sources= 2024-04-17T04:56:46.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential sources added=[] removed=[] 2024-04-17T04:56:46.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_dests= new_dests= 2024-04-17T04:56:46.845 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.846+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential dests added=[] removed=[] 2024-04-17T04:56:46.847 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:46.850+0000 7f98d3caa980 -1 Injecting delay=5.000000000s at location=do_reshard 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.0 2024-04-17T04:56:51.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.0 2024-04-17T04:56:51.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.1 2024-04-17T04:56:51.849 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.850+0000 7f98d3caa980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.1.1 2024-04-17T04:56:51.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.854+0000 7f98d3caa980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:51.850 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.854+0000 7f98d3caa980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:51.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f98d3caa980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:51.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f98d3caa980 10 moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:51.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f98d3caa980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:51.854 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f98d3caa980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19, ofs0, ns] 2024-04-17T04:56:51.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f97a3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464300 cookie 94358750305392 notifier 6845 bl.length()=865 2024-04-17T04:56:51.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f97a3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:51.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f97a3fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:51.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.858+0000 7f97a3fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:51.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.862+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_sources= new_sources= 2024-04-17T04:56:51.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.862+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential sources added=[] removed=[] 2024-04-17T04:56:51.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.862+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_dests= new_dests= 2024-04-17T04:56:51.860 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.862+0000 7f98d3caa980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential dests added=[] removed=[] 2024-04-17T04:56:51.863 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.866+0000 7f98d3caa980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-04-17T04:56:51.868 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.870+0000 7f97877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:51.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.878+0000 7f98d3caa980 20 remove_watcher() i=0 2024-04-17T04:56:51.874 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.878+0000 7f98d3caa980 2 removed watcher, disabling cache 2024-04-17T04:56:51.877 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.882+0000 7f98d3caa980 20 remove_watcher() i=1 2024-04-17T04:56:51.880 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.882+0000 7f98d3caa980 20 remove_watcher() i=2 2024-04-17T04:56:51.883 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.886+0000 7f98d3caa980 20 remove_watcher() i=3 2024-04-17T04:56:51.886 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.890+0000 7f98d3caa980 20 remove_watcher() i=4 2024-04-17T04:56:51.888 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.890+0000 7f98d3caa980 20 remove_watcher() i=5 2024-04-17T04:56:51.890 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.894+0000 7f98d3caa980 20 remove_watcher() i=6 2024-04-17T04:56:51.892 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:51.894+0000 7f98d3caa980 20 remove_watcher() i=7 2024-04-17T04:56:52.923 INFO:tasks.workunit.client.0.smithi088.stderr:put object successful 2024-04-17T04:56:52.924 INFO:tasks.workunit.client.0.smithi088.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-04-17T04:56:52.924 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-04-17T04:56:53.871 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-04-17T04:56:54.291 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-04-17T04:56:55.863 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:56:55.863 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket myver --object x 2024-04-17T04:56:55.897 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:55.897 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:55.942 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:55.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:55.943 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 realm 2024-04-17T04:56:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:55.944 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.946+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:55.945 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.950+0000 7fd194b92980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:55.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.950+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:55.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.950+0000 7fd194b92980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:55.946 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.950+0000 7fd194b92980 20 searching for the correct realm 2024-04-17T04:56:55.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:55.954+0000 7fd191eac640 0 -- 172.21.15.88:0/2476038788 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fd17c0022b0 msgr2=0x7fd184055560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:56.166 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.170+0000 7fd1926ad640 0 -- 172.21.15.88:0/2476038788 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fd1880d8c00 msgr2=0x7fd1880f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:56.384 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:56.385 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.386+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:56.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 zone default found 2024-04-17T04:56:56.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 4 Realm: () 2024-04-17T04:56:56.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:56.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:56.386 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:56.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 zonegroup default 2024-04-17T04:56:56.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.387 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 started sync module instance, tier type = 2024-04-17T04:56:56.388 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.390+0000 7fd194b92980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:56.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=0 2024-04-17T04:56:56.412 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=1 2024-04-17T04:56:56.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=2 2024-04-17T04:56:56.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=3 2024-04-17T04:56:56.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=4 2024-04-17T04:56:56.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=5 2024-04-17T04:56:56.413 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.414+0000 7fd194b92980 20 add_watcher() i=6 2024-04-17T04:56:56.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.418+0000 7fd194b92980 20 add_watcher() i=7 2024-04-17T04:56:56.415 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.418+0000 7fd194b92980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:56.419 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd048fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:56.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd194b92980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:56.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd194b92980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:56.420 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd194b92980 5 note: GC not initialized 2024-04-17T04:56:56.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fcfc37fe640 20 reqs_thread_entry: start 2024-04-17T04:56:56.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd194b92980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:56.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd194b92980 20 Filter name: none 2024-04-17T04:56:56.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fcfc17fa640 20 reqs_thread_entry: start 2024-04-17T04:56:56.421 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.422+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:56.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:56.422 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:56.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-17T04:56:56.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 INFO: main: starting shard=0 2024-04-17T04:56:56.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:56.423 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:56.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:56:56.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 INFO: main: starting shard=1 2024-04-17T04:56:56.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:56.424 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:56.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:56.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 INFO: main: starting shard=2 2024-04-17T04:56:56.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:56.425 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.426+0000 7fd194b92980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:56.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.430+0000 7fd194b92980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:56.426 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.430+0000 7fd194b92980 20 INFO: main: done 2024-04-17T04:56:56.427 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.430+0000 7fd048fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:56.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.434+0000 7fd194b92980 20 remove_watcher() i=0 2024-04-17T04:56:56.432 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.434+0000 7fd194b92980 2 removed watcher, disabling cache 2024-04-17T04:56:56.434 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.438+0000 7fd194b92980 20 remove_watcher() i=1 2024-04-17T04:56:56.436 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.438+0000 7fd194b92980 20 remove_watcher() i=2 2024-04-17T04:56:56.438 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.442+0000 7fd194b92980 20 remove_watcher() i=3 2024-04-17T04:56:56.440 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.442+0000 7fd194b92980 20 remove_watcher() i=4 2024-04-17T04:56:56.442 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.446+0000 7fd194b92980 20 remove_watcher() i=5 2024-04-17T04:56:56.443 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.446+0000 7fd194b92980 20 remove_watcher() i=6 2024-04-17T04:56:56.444 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.446+0000 7fd194b92980 20 remove_watcher() i=7 2024-04-17T04:56:56.634 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:56:56.635 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket myver --object y 2024-04-17T04:56:56.670 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:56.670 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:56.714 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.715 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 realm 2024-04-17T04:56:56.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:56.716 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.717 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.718+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.722+0000 7f00e3cc9980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:56.718 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.722+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.722+0000 7f00e3cc9980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:56.719 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.722+0000 7f00e3cc9980 20 searching for the correct realm 2024-04-17T04:56:56.724 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.726+0000 7f00e1fe5640 0 -- 172.21.15.88:0/380539322 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f00d801a040 msgr2=0x7f00d801d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:56.730 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.734+0000 7f00e17e4640 0 -- 172.21.15.88:0/380539322 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f00d805fa40 msgr2=0x7f00d8062120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:56.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:56.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:56.949 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.950+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.950 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:56.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.951 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 zone default found 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 4 Realm: () 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 zonegroup default 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.952 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.954+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.958+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.953 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.958+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.958+0000 7f00e3cc9980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:56.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.958+0000 7f00e3cc9980 20 started sync module instance, tier type = 2024-04-17T04:56:56.954 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.958+0000 7f00e3cc9980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:56.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=0 2024-04-17T04:56:56.974 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=1 2024-04-17T04:56:56.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=2 2024-04-17T04:56:56.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=3 2024-04-17T04:56:56.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=4 2024-04-17T04:56:56.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=5 2024-04-17T04:56:56.975 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=6 2024-04-17T04:56:56.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 20 add_watcher() i=7 2024-04-17T04:56:56.976 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.978+0000 7f00e3cc9980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:56.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.982+0000 7eff977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:56.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.982+0000 7f00e3cc9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:56.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.982+0000 7f00e3cc9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:56.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.982+0000 7f00e3cc9980 5 note: GC not initialized 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.982+0000 7eff967e4640 20 reqs_thread_entry: start 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 Filter name: none 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7eff07fff640 20 reqs_thread_entry: start 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:56.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 rados->read ofs=0 len=0 2024-04-17T04:56:56.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:56.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:56.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-17T04:56:56.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 INFO: main: starting shard=0 2024-04-17T04:56:56.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:56.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:56.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:56.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 INFO: main: starting shard=1 2024-04-17T04:56:56.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:56.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:56.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:56.984 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 INFO: main: starting shard=2 2024-04-17T04:56:56.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:56.985 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.986+0000 7f00e3cc9980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:56.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.990+0000 7f00e3cc9980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:56:56.986 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.990+0000 7f00e3cc9980 20 INFO: main: done 2024-04-17T04:56:56.987 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.990+0000 7eff977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:56.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.994+0000 7f00e3cc9980 20 remove_watcher() i=0 2024-04-17T04:56:56.992 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.994+0000 7f00e3cc9980 2 removed watcher, disabling cache 2024-04-17T04:56:56.994 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.998+0000 7f00e3cc9980 20 remove_watcher() i=1 2024-04-17T04:56:56.996 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:56.998+0000 7f00e3cc9980 20 remove_watcher() i=2 2024-04-17T04:56:56.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.002+0000 7f00e3cc9980 20 remove_watcher() i=3 2024-04-17T04:56:57.001 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.002+0000 7f00e3cc9980 20 remove_watcher() i=4 2024-04-17T04:56:57.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.006+0000 7f00e3cc9980 20 remove_watcher() i=5 2024-04-17T04:56:57.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.006+0000 7f00e3cc9980 20 remove_watcher() i=6 2024-04-17T04:56:57.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.006+0000 7f00e3cc9980 20 remove_watcher() i=7 2024-04-17T04:56:57.158 INFO:tasks.workunit.client.0.smithi088.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-17T04:56:57.159 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bi list --bucket myver --object z 2024-04-17T04:56:57.192 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:57.193 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:57.238 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 realm 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.240 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.241 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.242+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.246+0000 7fcdb4e97980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:57.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.246+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.242 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.246+0000 7fcdb4e97980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:57.243 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.246+0000 7fcdb4e97980 20 searching for the correct realm 2024-04-17T04:56:57.248 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.250+0000 7fcdb31b3640 0 -- 172.21.15.88:0/4106855282 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7fcda80c2030 msgr2=0x7fcda80ed400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:57.254 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.258+0000 7fcdb29b2640 0 -- 172.21.15.88:0/4106855282 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fcda811c6f0 msgr2=0x7fcda80ec4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T04:56:57.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:57.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:57.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:57.472 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.474+0000 7fcdb4e97980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:57.473 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:57.474 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 zone default found 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 4 Realm: () 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 zonegroup default 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.475 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:57.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 started sync module instance, tier type = 2024-04-17T04:56:57.476 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.478+0000 7fcdb4e97980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:57.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=0 2024-04-17T04:56:57.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=1 2024-04-17T04:56:57.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=2 2024-04-17T04:56:57.501 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=3 2024-04-17T04:56:57.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=4 2024-04-17T04:56:57.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=5 2024-04-17T04:56:57.502 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.502+0000 7fcdb4e97980 20 add_watcher() i=6 2024-04-17T04:56:57.503 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.506+0000 7fcdb4e97980 20 add_watcher() i=7 2024-04-17T04:56:57.504 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.506+0000 7fcdb4e97980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:57.508 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.510+0000 7fcc697e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:57.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.510+0000 7fcdb4e97980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:57.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.510+0000 7fcdb4e97980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:57.509 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.510+0000 7fcdb4e97980 5 note: GC not initialized 2024-04-17T04:56:57.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcbe3fff640 20 reqs_thread_entry: start 2024-04-17T04:56:57.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:57.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 Filter name: none 2024-04-17T04:56:57.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcbe1ffb640 20 reqs_thread_entry: start 2024-04-17T04:56:57.510 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:57.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 rados->read ofs=0 len=0 2024-04-17T04:56:57.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:57.511 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:57.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-17T04:56:57.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 INFO: main: starting shard=0 2024-04-17T04:56:57.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:57.512 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.514+0000 7fcdb4e97980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:57.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:57.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 INFO: main: starting shard=1 2024-04-17T04:56:57.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:57.513 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:57.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-17T04:56:57.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 INFO: main: starting shard=2 2024-04-17T04:56:57.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:57.514 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:57.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-17T04:56:57.515 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcdb4e97980 20 INFO: main: done 2024-04-17T04:56:57.517 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.518+0000 7fcc697e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:57.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.526+0000 7fcdb4e97980 20 remove_watcher() i=0 2024-04-17T04:56:57.522 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.526+0000 7fcdb4e97980 2 removed watcher, disabling cache 2024-04-17T04:56:57.524 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.526+0000 7fcdb4e97980 20 remove_watcher() i=1 2024-04-17T04:56:57.527 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.530+0000 7fcdb4e97980 20 remove_watcher() i=2 2024-04-17T04:56:57.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.530+0000 7fcdb4e97980 20 remove_watcher() i=3 2024-04-17T04:56:57.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.534+0000 7fcdb4e97980 20 remove_watcher() i=4 2024-04-17T04:56:57.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.534+0000 7fcdb4e97980 20 remove_watcher() i=5 2024-04-17T04:56:57.534 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.538+0000 7fcdb4e97980 20 remove_watcher() i=6 2024-04-17T04:56:57.536 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:57.538+0000 7fcdb4e97980 20 remove_watcher() i=7 2024-04-17T04:56:57.595 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-04-17T04:56:58.028 INFO:tasks.workunit.client.0.smithi088.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-04-17T04:56:58.454 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-04-17T04:56:58.486 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:58.486 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:58.529 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.530+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 realm 2024-04-17T04:56:58.530 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:58.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.531 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:58.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:58.533 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.534+0000 7f56f8cb1980 20 searching for the correct realm 2024-04-17T04:56:58.540 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.542+0000 7f56f6fcd640 0 -- 172.21.15.88:0/16472862 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f56ec0ff4e0 msgr2=0x7f56ec0fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:58.755 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.758+0000 7f56f67cc640 0 -- 172.21.15.88:0/16472862 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55a491a61210 msgr2=0x55a491a63640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:58.763 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:58.764 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 zone default found 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 4 Realm: () 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:58.765 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 zonegroup default 2024-04-17T04:56:58.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.766+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.766 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 started sync module instance, tier type = 2024-04-17T04:56:58.767 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.770+0000 7f56f8cb1980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:58.789 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=0 2024-04-17T04:56:58.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=1 2024-04-17T04:56:58.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=2 2024-04-17T04:56:58.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=3 2024-04-17T04:56:58.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=4 2024-04-17T04:56:58.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=5 2024-04-17T04:56:58.790 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=6 2024-04-17T04:56:58.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 20 add_watcher() i=7 2024-04-17T04:56:58.792 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.794+0000 7f56f8cb1980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:58.796 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.798+0000 7f55acfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:58.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:58.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:58.797 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 5 note: GC not initialized 2024-04-17T04:56:58.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f55277fe640 20 reqs_thread_entry: start 2024-04-17T04:56:58.798 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:58.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 Filter name: none 2024-04-17T04:56:58.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f55257fa640 20 reqs_thread_entry: start 2024-04-17T04:56:58.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-17T04:56:58.799 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : miss 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:58.800 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 adding default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:58.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:58.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:58.801 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.802+0000 7f56f8cb1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:58.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f56f8cb1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:58.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f56f8cb1980 10 moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:58.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f56f8cb1980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:58.815 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f56f8cb1980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19, ofs0, ns] 2024-04-17T04:56:58.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f55d97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464301 cookie 94165306611872 notifier 7365 bl.length()=897 2024-04-17T04:56:58.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f55d97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:58.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f55d97fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:58.816 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.818+0000 7f55d97fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:58.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.822+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_sources= new_sources= 2024-04-17T04:56:58.818 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.822+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential sources added=[] removed=[] 2024-04-17T04:56:58.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.822+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_dests= new_dests= 2024-04-17T04:56:58.819 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.822+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential dests added=[] removed=[] 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.0 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.0 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.1 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.1 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.2 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.2 2024-04-17T04:56:58.822 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.3 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.3 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.4 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.4 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.5 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.5 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.6 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.6 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.7 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.7 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.8 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.8 2024-04-17T04:56:58.823 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.10 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.10 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.11 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.11 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.12 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.12 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.13 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.13 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.14 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.14 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.15 2024-04-17T04:56:58.824 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.15 2024-04-17T04:56:58.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.16 2024-04-17T04:56:58.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.16 2024-04-17T04:56:58.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:58.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.0 2024-04-17T04:56:58.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:58.825 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.1 2024-04-17T04:56:58.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:58.826 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.826+0000 7f56f8cb1980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.2.2 2024-04-17T04:56:58.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.830+0000 7f56f8cb1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : hit (requested=0x17, cached=0x17) 2024-04-17T04:56:58.829 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.830+0000 7f56f8cb1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:58.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:58.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 10 moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:58.830 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:58.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19, ofs0, ns] 2024-04-17T04:56:58.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f55d9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464302 cookie 94165306611872 notifier 7365 bl.length()=865 2024-04-17T04:56:58.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f55d9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:58.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f55d9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:58.831 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f55d9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:58.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_sources= new_sources= 2024-04-17T04:56:58.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential sources added=[] removed=[] 2024-04-17T04:56:58.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): orig_dests= new_dests= 2024-04-17T04:56:58.833 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.834+0000 7f56f8cb1980 20 handle_bi_update(): bucket=:myver[52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19]): potential dests added=[] removed=[] 2024-04-17T04:56:58.835 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.838+0000 7f56f8cb1980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-04-17T04:56:58.839 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.842+0000 7f55acfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:58.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.846+0000 7f56f8cb1980 20 remove_watcher() i=0 2024-04-17T04:56:58.844 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.846+0000 7f56f8cb1980 2 removed watcher, disabling cache 2024-04-17T04:56:58.846 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.850+0000 7f56f8cb1980 20 remove_watcher() i=1 2024-04-17T04:56:58.848 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.850+0000 7f56f8cb1980 20 remove_watcher() i=2 2024-04-17T04:56:58.851 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.854+0000 7f56f8cb1980 20 remove_watcher() i=3 2024-04-17T04:56:58.853 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.858+0000 7f56f8cb1980 20 remove_watcher() i=4 2024-04-17T04:56:58.855 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.858+0000 7f56f8cb1980 20 remove_watcher() i=5 2024-04-17T04:56:58.857 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.858+0000 7f56f8cb1980 20 remove_watcher() i=6 2024-04-17T04:56:58.858 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.862+0000 7f56f8cb1980 20 remove_watcher() i=7 2024-04-17T04:56:58.876 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-17T04:56:58.912 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:58.912 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:58.955 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 realm 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.957 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.958+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.962+0000 7f59e9984980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:58.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.962+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.962+0000 7f59e9984980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:58.959 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.962+0000 7f59e9984980 20 searching for the correct realm 2024-04-17T04:56:58.966 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.970+0000 7f59e6c9e640 0 -- 172.21.15.88:0/2130534585 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f59e0040050 msgr2=0x7f59e003e420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:58.972 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.974+0000 7f59e6c9e640 0 -- 172.21.15.88:0/2130534585 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f58b0076320 msgr2=0x7f58b0078750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:58.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:58.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:58.979 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.980 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:58.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:58.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 zone default found 2024-04-17T04:56:58.981 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 4 Realm: () 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.982+0000 7f59e9984980 20 zonegroup default 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.982 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:58.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:58.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 started sync module instance, tier type = 2024-04-17T04:56:58.983 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:58.986+0000 7f59e9984980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=0 2024-04-17T04:56:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=1 2024-04-17T04:56:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=2 2024-04-17T04:56:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=3 2024-04-17T04:56:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=4 2024-04-17T04:56:58.998 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=5 2024-04-17T04:56:58.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=6 2024-04-17T04:56:58.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 20 add_watcher() i=7 2024-04-17T04:56:58.999 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.002+0000 7f59e9984980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:59.002 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f589dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:59.003 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 5 note: GC not initialized 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f589cfe1640 20 reqs_thread_entry: start 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 Filter name: none 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f58167fc640 20 reqs_thread_entry: start 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-17T04:56:59.004 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-17T04:56:59.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 : miss 2024-04-17T04:56:59.005 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 info.flags=0x17 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 adding default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 to cache LRU end 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.0 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.0 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 1 NOTICE: finished shard 0 (0 entries removed) 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.1 2024-04-17T04:56:59.006 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.006+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.1 2024-04-17T04:56:59.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 1 NOTICE: finished shard 1 (0 entries removed) 2024-04-17T04:56:59.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.2 2024-04-17T04:56:59.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.2 2024-04-17T04:56:59.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 1 NOTICE: finished shard 2 (0 entries removed) 2024-04-17T04:56:59.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.3 2024-04-17T04:56:59.007 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.3 2024-04-17T04:56:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 1 NOTICE: finished shard 3 (0 entries removed) 2024-04-17T04:56:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.4 2024-04-17T04:56:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.4 2024-04-17T04:56:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 1 NOTICE: finished shard 4 (0 entries removed) 2024-04-17T04:56:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.5 2024-04-17T04:56:59.008 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.5 2024-04-17T04:56:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 1 NOTICE: finished shard 5 (0 entries removed) 2024-04-17T04:56:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.6 2024-04-17T04:56:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.010+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.6 2024-04-17T04:56:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 1 NOTICE: finished shard 6 (0 entries removed) 2024-04-17T04:56:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.7 2024-04-17T04:56:59.009 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.7 2024-04-17T04:56:59.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 1 NOTICE: finished shard 7 (0 entries removed) 2024-04-17T04:56:59.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.8 2024-04-17T04:56:59.011 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.8 2024-04-17T04:56:59.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 1 NOTICE: finished shard 8 (0 entries removed) 2024-04-17T04:56:59.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.10 2024-04-17T04:56:59.012 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.10 2024-04-17T04:56:59.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 1 NOTICE: finished shard 10 (0 entries removed) 2024-04-17T04:56:59.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.11 2024-04-17T04:56:59.013 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.014+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.11 2024-04-17T04:56:59.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 1 NOTICE: finished shard 11 (0 entries removed) 2024-04-17T04:56:59.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.12 2024-04-17T04:56:59.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.12 2024-04-17T04:56:59.014 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 1 NOTICE: finished shard 12 (0 entries removed) 2024-04-17T04:56:59.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.13 2024-04-17T04:56:59.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.13 2024-04-17T04:56:59.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 1 NOTICE: finished shard 13 (0 entries removed) 2024-04-17T04:56:59.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.14 2024-04-17T04:56:59.015 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.14 2024-04-17T04:56:59.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 1 NOTICE: finished shard 14 (0 entries removed) 2024-04-17T04:56:59.016 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.15 2024-04-17T04:56:59.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.018+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.15 2024-04-17T04:56:59.017 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 1 NOTICE: finished shard 15 (0 entries removed) 2024-04-17T04:56:59.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 10 bucket_obj is .dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.16 2024-04-17T04:56:59.018 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.16 2024-04-17T04:56:59.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 1 NOTICE: finished shard 16 (0 entries removed) 2024-04-17T04:56:59.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.019 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:x state=0x555c72e9d908 s->prefetch_data=0 2024-04-17T04:56:59.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 20 get_obj_state: setting s->obj_tag to vltyzfyr2vmnl13r0stu51bh1md7wzv6 2024-04-17T04:56:59.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 20 get_obj_state_impl: setting s->olh_tag to ipwcet899xdzkscfaald626zb8se6s5y 2024-04-17T04:56:59.020 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.022+0000 7f59e9984980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=ipwcet899xdzkscfaald626zb8se6s5y 2024-04-17T04:56:59.023 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.026+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.025 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-04-17T04:56:59.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 olh_log_entry: epoch=2 op=3 key=x[0fnHeHWsaLVY4G9vO.t45Lh-emr10nd] 2024-04-17T04:56:59.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:_:0fnHeHWsaLVY4G9vO.t45Lh-emr10nd_x state=0x555c72ed5028 s->prefetch_data=0 2024-04-17T04:56:59.026 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 10 manifest: total_size = 22 2024-04-17T04:56:59.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13451483080043948188 2024-04-17T04:56:59.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:_:0fnHeHWsaLVY4G9vO.t45Lh-emr10nd_x state=0x555c72ed5028 s->prefetch_data=0 2024-04-17T04:56:59.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 prepare_atomic_modification: state is not atomic. state=0x555c72ed5028 2024-04-17T04:56:59.027 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.030+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.034+0000 7f59e9984980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:56:59.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.034+0000 7f59e9984980 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-17T04:56:59.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.034+0000 7f59e9984980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:56:59.031 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.034+0000 7f59e9984980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:0fnHeHWsaLVY4G9vO.t45Lh-emr10nd_.ss3OLGy9MIOwKWwv4U6Yxgn5N6OVIah_0 2024-04-17T04:56:59.033 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.038+0000 7f58caffd640 20 handle_completion(): completion ok for obj=x[0fnHeHWsaLVY4G9vO.t45Lh-emr10nd] 2024-04-17T04:56:59.034 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.038+0000 7f59e9984980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:56:59.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.042+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:x state=0x555c72e9d908 s->prefetch_data=0 2024-04-17T04:56:59.038 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.042+0000 7f59e9984980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=ipwcet899xdzkscfaald626zb8se6s5y 2024-04-17T04:56:59.044 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.046+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.050+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.047 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.050+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.050+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:y state=0x555c72e9d908 s->prefetch_data=0 2024-04-17T04:56:59.048 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.050+0000 7f59e9984980 20 get_obj_state: setting s->obj_tag to 9x51yw6xmjjdxhtotdjua08k8mdgcke2 2024-04-17T04:56:59.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.050+0000 7f59e9984980 20 get_obj_state_impl: setting s->olh_tag to q9iqz6hooxa74qitodrltd93hfenbuir 2024-04-17T04:56:59.049 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.050+0000 7f59e9984980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=q9iqz6hooxa74qitodrltd93hfenbuir 2024-04-17T04:56:59.053 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.054+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.055 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-04-17T04:56:59.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 olh_log_entry: epoch=2 op=3 key=y[hWU40DifhA0iwCsl2x2hZoRJYPsnJ72] 2024-04-17T04:56:59.056 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:_:hWU40DifhA0iwCsl2x2hZoRJYPsnJ72_y state=0x555c72edb5f8 s->prefetch_data=0 2024-04-17T04:56:59.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 10 manifest: total_size = 11 2024-04-17T04:56:59.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2026462783482090568 2024-04-17T04:56:59.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:_:hWU40DifhA0iwCsl2x2hZoRJYPsnJ72_y state=0x555c72edb5f8 s->prefetch_data=0 2024-04-17T04:56:59.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 prepare_atomic_modification: state is not atomic. state=0x555c72edb5f8 2024-04-17T04:56:59.057 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.058+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.066+0000 7f59e9984980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:56:59.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.066+0000 7f59e9984980 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-04-17T04:56:59.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.066+0000 7f59e9984980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:56:59.062 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.066+0000 7f59e9984980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:hWU40DifhA0iwCsl2x2hZoRJYPsnJ72_.v69Z1yz7vC7KguwnYte47E1AxgfdllJ_0 2024-04-17T04:56:59.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.066+0000 7f58ca7fc640 20 handle_completion(): completion ok for obj=y[hWU40DifhA0iwCsl2x2hZoRJYPsnJ72] 2024-04-17T04:56:59.065 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.070+0000 7f59e9984980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:56:59.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.074+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:y state=0x555c72e9d908 s->prefetch_data=0 2024-04-17T04:56:59.069 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.074+0000 7f59e9984980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=q9iqz6hooxa74qitodrltd93hfenbuir 2024-04-17T04:56:59.077 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.078+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.079 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.082+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.082+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.082+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:z state=0x555c72e9d908 s->prefetch_data=0 2024-04-17T04:56:59.080 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.082+0000 7f59e9984980 20 get_obj_state: setting s->obj_tag to un2ncxp3w29bxe9qrm3qbm5wi5po501l 2024-04-17T04:56:59.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.082+0000 7f59e9984980 20 get_obj_state_impl: setting s->olh_tag to mjvercxvo5vhirgc95qtbveeyvxrogc0 2024-04-17T04:56:59.081 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.082+0000 7f59e9984980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=mjvercxvo5vhirgc95qtbveeyvxrogc0 2024-04-17T04:56:59.083 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.086+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-04-17T04:56:59.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 olh_log_entry: epoch=2 op=3 key=z[WVy.Y9yAiLFGbQHBNC0tLMEXGkJU4jr] 2024-04-17T04:56:59.086 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:_:WVy.Y9yAiLFGbQHBNC0tLMEXGkJU4jr_z state=0x555c72edb5f8 s->prefetch_data=0 2024-04-17T04:56:59.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 10 manifest: total_size = 9 2024-04-17T04:56:59.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 get_obj_state: setting s->obj_tag to 52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13253264916276580691 2024-04-17T04:56:59.087 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:_:WVy.Y9yAiLFGbQHBNC0tLMEXGkJU4jr_z state=0x555c72edb5f8 s->prefetch_data=0 2024-04-17T04:56:59.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 prepare_atomic_modification: state is not atomic. state=0x555c72edb5f8 2024-04-17T04:56:59.088 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.090+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.094+0000 7f59e9984980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-17T04:56:59.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.094+0000 7f59e9984980 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-04-17T04:56:59.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.094+0000 7f59e9984980 0 deleting objects inline since gc isn't initialized 2024-04-17T04:56:59.092 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.094+0000 7f59e9984980 5 delete_objs_inline: removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:WVy.Y9yAiLFGbQHBNC0tLMEXGkJU4jr_.IT4u9YLoz2VGmLonR6Ss0RuiZCEEapg_0 2024-04-17T04:56:59.095 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.098+0000 7f58caffd640 20 handle_completion(): completion ok for obj=z[WVy.Y9yAiLFGbQHBNC0tLMEXGkJU4jr] 2024-04-17T04:56:59.095 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.098+0000 7f59e9984980 5 delete_objs_inline: refcount put returned error -2 2024-04-17T04:56:59.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.102+0000 7f59e9984980 20 get_obj_state: octx=0x555c72bf5160 obj=myver:z state=0x555c72e9d908 s->prefetch_data=0 2024-04-17T04:56:59.098 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.102+0000 7f59e9984980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=mjvercxvo5vhirgc95qtbveeyvxrogc0 2024-04-17T04:56:59.104 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.106+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.110+0000 7f59e9984980 20 bucket index object: :.dir.52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19.3.9 2024-04-17T04:56:59.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.110+0000 7f59e9984980 1 NOTICE: finished shard 9 (3 entries removed) 2024-04-17T04:56:59.107 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.110+0000 7f59e9984980 1 NOTICE: finished all shards (3 entries removed) 2024-04-17T04:56:59.108 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.110+0000 7f589dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:59.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.114+0000 7f59e9984980 20 remove_watcher() i=0 2024-04-17T04:56:59.113 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.114+0000 7f59e9984980 2 removed watcher, disabling cache 2024-04-17T04:56:59.115 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.118+0000 7f59e9984980 20 remove_watcher() i=1 2024-04-17T04:56:59.117 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.122+0000 7f59e9984980 20 remove_watcher() i=2 2024-04-17T04:56:59.119 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.122+0000 7f59e9984980 20 remove_watcher() i=3 2024-04-17T04:56:59.122 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.126+0000 7f59e9984980 20 remove_watcher() i=4 2024-04-17T04:56:59.123 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.126+0000 7f59e9984980 20 remove_watcher() i=5 2024-04-17T04:56:59.125 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.126+0000 7f59e9984980 20 remove_watcher() i=6 2024-04-17T04:56:59.126 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.130+0000 7f59e9984980 20 remove_watcher() i=7 2024-04-17T04:56:59.494 INFO:tasks.workunit.client.0.smithi088.stderr:radosgw-admin bucket stats --bucket myver 2024-04-17T04:56:59.526 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:56:59.526 INFO:tasks.workunit.client.0.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:56:59.570 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 realm 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.573 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.574+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.578+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:59.574 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.578+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.578+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:56:59.575 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.578+0000 7f5ee2dfc980 20 searching for the correct realm 2024-04-17T04:56:59.581 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.582+0000 7f5ee0116640 0 -- 172.21.15.88:0/2222094958 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55e54236cd70 msgr2=0x55e54236f160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:59.587 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.590+0000 7f5ee0116640 0 -- 172.21.15.88:0/2222094958 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f5ed0055900 msgr2=0x7f5ed00598b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:56:59.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:56:59.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:56:59.594 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:56:59.595 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:56:59.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 zone default found 2024-04-17T04:56:59.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 4 Realm: () 2024-04-17T04:56:59.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:56:59.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:56:59.596 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:56:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 zonegroup default 2024-04-17T04:56:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.597 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.598+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.602+0000 7f5ee2dfc980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:56:59.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.602+0000 7f5ee2dfc980 20 started sync module instance, tier type = 2024-04-17T04:56:59.598 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.602+0000 7f5ee2dfc980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=0 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=1 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=2 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=3 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=4 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=5 2024-04-17T04:56:59.614 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=6 2024-04-17T04:56:59.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 20 add_watcher() i=7 2024-04-17T04:56:59.615 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.618+0000 7f5ee2dfc980 2 all 8 watchers are set, enabling cache 2024-04-17T04:56:59.618 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5d977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 5 note: GC not initialized 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5d967e4640 20 reqs_thread_entry: start 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 init_complete bucket index max shards: 1 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 Filter name: none 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5d8f7fe640 20 reqs_thread_entry: start 2024-04-17T04:56:59.619 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.620 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=197 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados->read ofs=0 len=0 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 rados_obj.operate() r=0 bl.length=411 2024-04-17T04:56:59.621 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.622+0000 7f5ee2dfc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-17T04:56:59.623 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.626+0000 7f5d977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:56:59.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.630+0000 7f5ee2dfc980 20 remove_watcher() i=0 2024-04-17T04:56:59.628 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.630+0000 7f5ee2dfc980 2 removed watcher, disabling cache 2024-04-17T04:56:59.630 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.634+0000 7f5ee2dfc980 20 remove_watcher() i=1 2024-04-17T04:56:59.632 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.634+0000 7f5ee2dfc980 20 remove_watcher() i=2 2024-04-17T04:56:59.634 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.638+0000 7f5ee2dfc980 20 remove_watcher() i=3 2024-04-17T04:56:59.636 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.638+0000 7f5ee2dfc980 20 remove_watcher() i=4 2024-04-17T04:56:59.638 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.642+0000 7f5ee2dfc980 20 remove_watcher() i=5 2024-04-17T04:56:59.639 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.642+0000 7f5ee2dfc980 20 remove_watcher() i=6 2024-04-17T04:56:59.640 INFO:tasks.workunit.client.0.smithi088.stderr:2024-04-17T04:56:59.642+0000 7f5ee2dfc980 20 remove_watcher() i=7 2024-04-17T04:56:59.654 INFO:tasks.workunit.client.0.smithi088.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-04-17T04:56:59.654 INFO:tasks.workunit.client.0.smithi088.stderr:Deleting bucket a-bucket 2024-04-17T04:56:59.772 INFO:tasks.workunit.client.0.smithi088.stderr:Deleting bucket myver 2024-04-17T04:57:00.368 INFO:tasks.workunit.client.0.smithi088.stderr:Completed resharding tests 2024-04-17T04:57:00.368 INFO:tasks.workunit.client.0.smithi088.stdout:connected to http://localhost:80 2024-04-17T04:57:00.470 INFO:tasks.workunit.client.0.smithi088.stderr:+ deactivate 2024-04-17T04:57:00.470 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-04-17T04:57:00.470 INFO:tasks.workunit.client.0.smithi088.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-04-17T04:57:00.470 INFO:tasks.workunit.client.0.smithi088.stderr:+ export PATH 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset _OLD_VIRTUAL_PATH 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n '' ']' 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ hash -r 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' -n '' ']' 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset VIRTUAL_ENV 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ '[' '!' '' = nondestructive ']' 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ unset -f deactivate 2024-04-17T04:57:00.471 INFO:tasks.workunit.client.0.smithi088.stderr:+ echo OK. 2024-04-17T04:57:00.472 INFO:tasks.workunit.client.0.smithi088.stdout:OK. 2024-04-17T04:57:00.473 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-17T04:57:00.473 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-17T04:57:00.525 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-04-17T04:57:00.526 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-17T04:57:00.972 DEBUG:teuthology.parallel:result is None 2024-04-17T04:57:00.973 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:57:00.986 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:57:00.986 DEBUG:teuthology.orchestra.run.smithi088:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-17T04:57:01.033 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-17T04:57:01.033 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-04-17T04:57:01.050 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-04-17T04:57:01.051 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-04-17T04:57:01.051 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-04-17T04:57:01.051 DEBUG:teuthology.orchestra.run.smithi088:> git clone -b ceph-squid https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-04-17T04:57:01.081 INFO:teuthology.orchestra.run.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-04-17T04:57:05.905 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-04-17T04:57:05.905 DEBUG:teuthology.orchestra.run.smithi088:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-04-17T04:57:05.998 INFO:teuthology.orchestra.run.smithi088.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required DEB packages. Installing via sudo. 2024-04-17T04:57:06.058 INFO:teuthology.orchestra.run.smithi088.stdout:Reading package lists... 2024-04-17T04:57:06.259 INFO:teuthology.orchestra.run.smithi088.stdout:Building dependency tree... 2024-04-17T04:57:06.260 INFO:teuthology.orchestra.run.smithi088.stdout:Reading state information... 2024-04-17T04:57:06.455 INFO:teuthology.orchestra.run.smithi088.stdout:The following packages were automatically installed and are no longer required: 2024-04-17T04:57:06.456 INFO:teuthology.orchestra.run.smithi088.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-17T04:57:06.456 INFO:teuthology.orchestra.run.smithi088.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-17T04:57:06.456 INFO:teuthology.orchestra.run.smithi088.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-17T04:57:06.456 INFO:teuthology.orchestra.run.smithi088.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-17T04:57:06.456 INFO:teuthology.orchestra.run.smithi088.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-17T04:57:06.456 INFO:teuthology.orchestra.run.smithi088.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-17T04:57:06.457 INFO:teuthology.orchestra.run.smithi088.stdout:The following additional packages will be installed: 2024-04-17T04:57:06.458 INFO:teuthology.orchestra.run.smithi088.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-04-17T04:57:06.458 INFO:teuthology.orchestra.run.smithi088.stdout: openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-04-17T04:57:06.458 INFO:teuthology.orchestra.run.smithi088.stdout:Suggested packages: 2024-04-17T04:57:06.459 INFO:teuthology.orchestra.run.smithi088.stdout: gvfs openjdk-8-demo openjdk-8-source visualvm libnss-mdns fonts-nanum 2024-04-17T04:57:06.459 INFO:teuthology.orchestra.run.smithi088.stdout: fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei 2024-04-17T04:57:06.459 INFO:teuthology.orchestra.run.smithi088.stdout: fonts-wqy-zenhei fonts-indic 2024-04-17T04:57:06.490 INFO:teuthology.orchestra.run.smithi088.stdout:The following NEW packages will be installed: 2024-04-17T04:57:06.491 INFO:teuthology.orchestra.run.smithi088.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-04-17T04:57:06.491 INFO:teuthology.orchestra.run.smithi088.stdout: openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-04-17T04:57:06.552 INFO:teuthology.orchestra.run.smithi088.stdout:0 upgraded, 9 newly installed, 0 to remove and 302 not upgraded. 2024-04-17T04:57:06.553 INFO:teuthology.orchestra.run.smithi088.stdout:Need to get 46.1 MB of archives. 2024-04-17T04:57:06.553 INFO:teuthology.orchestra.run.smithi088.stdout:After this operation, 155 MB of additional disk space will be used. 2024-04-17T04:57:06.553 INFO:teuthology.orchestra.run.smithi088.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-common all 2.24.33-2ubuntu2 [125 kB] 2024-04-17T04:57:06.642 INFO:teuthology.orchestra.run.smithi088.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-0 amd64 2.24.33-2ubuntu2 [2,037 kB] 2024-04-17T04:57:06.738 INFO:teuthology.orchestra.run.smithi088.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail18 amd64 2.24.33-2ubuntu2 [15.9 kB] 2024-04-17T04:57:06.738 INFO:teuthology.orchestra.run.smithi088.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail-common amd64 2.24.33-2ubuntu2 [132 kB] 2024-04-17T04:57:06.741 INFO:teuthology.orchestra.run.smithi088.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-bin amd64 2.24.33-2ubuntu2 [7,932 B] 2024-04-17T04:57:06.742 INFO:teuthology.orchestra.run.smithi088.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre-headless amd64 8u402-ga-2ubuntu1~22.04 [30.8 MB] 2024-04-17T04:57:07.202 INFO:teuthology.orchestra.run.smithi088.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre amd64 8u402-ga-2ubuntu1~22.04 [75.5 kB] 2024-04-17T04:57:07.203 INFO:teuthology.orchestra.run.smithi088.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk-headless amd64 8u402-ga-2ubuntu1~22.04 [8,873 kB] 2024-04-17T04:57:07.295 INFO:teuthology.orchestra.run.smithi088.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk amd64 8u402-ga-2ubuntu1~22.04 [4,068 kB] 2024-04-17T04:57:07.581 INFO:teuthology.orchestra.run.smithi088.stderr:debconf: unable to initialize frontend: Dialog 2024-04-17T04:57:07.581 INFO:teuthology.orchestra.run.smithi088.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-17T04:57:07.581 INFO:teuthology.orchestra.run.smithi088.stderr:debconf: falling back to frontend: Readline 2024-04-17T04:57:07.596 INFO:teuthology.orchestra.run.smithi088.stderr:debconf: unable to initialize frontend: Readline 2024-04-17T04:57:07.596 INFO:teuthology.orchestra.run.smithi088.stderr:debconf: (This frontend requires a controlling tty.) 2024-04-17T04:57:07.596 INFO:teuthology.orchestra.run.smithi088.stderr:debconf: falling back to frontend: Teletype 2024-04-17T04:57:07.600 INFO:teuthology.orchestra.run.smithi088.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-04-17T04:57:07.632 INFO:teuthology.orchestra.run.smithi088.stdout:Fetched 46.1 MB in 1s (55.1 MB/s) 2024-04-17T04:57:07.771 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgtk2.0-common. 2024-04-17T04:57:07.871 INFO:teuthology.orchestra.run.smithi088.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174885 files and directories currently installed.) 2024-04-17T04:57:07.877 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../0-libgtk2.0-common_2.24.33-2ubuntu2_all.deb ... 2024-04-17T04:57:07.919 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-04-17T04:57:08.281 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgtk2.0-0:amd64. 2024-04-17T04:57:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../1-libgtk2.0-0_2.24.33-2ubuntu2_amd64.deb ... 2024-04-17T04:57:08.389 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-04-17T04:57:08.827 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgail18:amd64. 2024-04-17T04:57:08.856 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../2-libgail18_2.24.33-2ubuntu2_amd64.deb ... 2024-04-17T04:57:08.894 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-04-17T04:57:09.188 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgail-common:amd64. 2024-04-17T04:57:09.217 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../3-libgail-common_2.24.33-2ubuntu2_amd64.deb ... 2024-04-17T04:57:09.263 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-04-17T04:57:09.526 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package libgtk2.0-bin. 2024-04-17T04:57:09.540 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../4-libgtk2.0-bin_2.24.33-2ubuntu2_amd64.deb ... 2024-04-17T04:57:09.582 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-04-17T04:57:09.980 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package openjdk-8-jre-headless:amd64. 2024-04-17T04:57:10.008 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../5-openjdk-8-jre-headless_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-04-17T04:57:10.049 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:11.752 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package openjdk-8-jre:amd64. 2024-04-17T04:57:11.780 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../6-openjdk-8-jre_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-04-17T04:57:11.822 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:12.265 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package openjdk-8-jdk-headless:amd64. 2024-04-17T04:57:12.293 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../7-openjdk-8-jdk-headless_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-04-17T04:57:12.334 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:13.068 INFO:teuthology.orchestra.run.smithi088.stdout:Selecting previously unselected package openjdk-8-jdk:amd64. 2024-04-17T04:57:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing to unpack .../8-openjdk-8-jdk_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-04-17T04:57:13.138 INFO:teuthology.orchestra.run.smithi088.stdout:Unpacking openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:13.497 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:14.584 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode 2024-04-17T04:57:14.611 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode 2024-04-17T04:57:14.626 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode 2024-04-17T04:57:14.976 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-04-17T04:57:15.144 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-04-17T04:57:15.546 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:15.636 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode 2024-04-17T04:57:15.706 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:15.782 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode 2024-04-17T04:57:15.811 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode 2024-04-17T04:57:15.835 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode 2024-04-17T04:57:15.861 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode 2024-04-17T04:57:15.988 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode 2024-04-17T04:57:16.123 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode 2024-04-17T04:57:16.235 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode 2024-04-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode 2024-04-17T04:57:16.370 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode 2024-04-17T04:57:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode 2024-04-17T04:57:16.447 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode 2024-04-17T04:57:16.472 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode 2024-04-17T04:57:16.535 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-04-17T04:57:16.661 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-04-17T04:57:16.787 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-04-17T04:57:16.880 INFO:teuthology.orchestra.run.smithi088.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode 2024-04-17T04:57:16.989 INFO:teuthology.orchestra.run.smithi088.stdout:Setting up libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-04-17T04:57:17.100 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-17T04:57:18.134 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-17T04:57:18.226 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-17T04:57:18.495 INFO:teuthology.orchestra.run.smithi088.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-17T04:57:18.923 INFO:teuthology.orchestra.run.smithi088.stdout: Gradle will be installed under /opt/gradle 2024-04-17T04:57:18.958 INFO:teuthology.orchestra.run.smithi088.stderr:--2024-04-17 04:57:18-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-04-17T04:57:18.961 INFO:teuthology.orchestra.run.smithi088.stderr:Resolving services.gradle.org (services.gradle.org)... 104.16.73.101, 104.16.72.101, 2606:4700::6810:4865, ... 2024-04-17T04:57:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:Connecting to services.gradle.org (services.gradle.org)|104.16.73.101|:443... connected. 2024-04-17T04:57:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-04-17T04:57:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-04-17T04:57:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:--2024-04-17 04:57:19-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-04-17T04:57:19.023 INFO:teuthology.orchestra.run.smithi088.stderr:Resolving github.com (github.com)... 140.82.112.4 2024-04-17T04:57:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:Connecting to github.com (github.com)|140.82.112.4|:443... connected. 2024-04-17T04:57:19.121 INFO:teuthology.orchestra.run.smithi088.stderr:HTTP request sent, awaiting response... 302 Found 2024-04-17T04:57:19.121 INFO:teuthology.orchestra.run.smithi088.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240417%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240417T045719Z&X-Amz-Expires=300&X-Amz-Signature=9ff180fff6fd0464870cef43d4e5f9aadee57b32602288903bf00f9f1bd55dee&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-04-17T04:57:19.121 INFO:teuthology.orchestra.run.smithi088.stderr:--2024-04-17 04:57:19-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240417%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240417T045719Z&X-Amz-Expires=300&X-Amz-Signature=9ff180fff6fd0464870cef43d4e5f9aadee57b32602288903bf00f9f1bd55dee&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-04-17T04:57:19.123 INFO:teuthology.orchestra.run.smithi088.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ... 2024-04-17T04:57:19.139 INFO:teuthology.orchestra.run.smithi088.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected. 2024-04-17T04:57:19.205 INFO:teuthology.orchestra.run.smithi088.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-17T04:57:19.205 INFO:teuthology.orchestra.run.smithi088.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-04-17T04:57:19.205 INFO:teuthology.orchestra.run.smithi088.stderr:Saving to: 'gradle-6.0.1-bin.zip' 2024-04-17T04:57:19.205 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-17T04:57:19.222 INFO:teuthology.orchestra.run.smithi088.stderr: 0K .......... .......... .......... .......... .......... 0% 2.77M 33s 2024-04-17T04:57:19.234 INFO:teuthology.orchestra.run.smithi088.stderr: 50K .......... .......... .......... .......... .......... 0% 3.94M 28s 2024-04-17T04:57:19.237 INFO:teuthology.orchestra.run.smithi088.stderr: 100K .......... .......... .......... .......... .......... 0% 14.3M 21s 2024-04-17T04:57:19.241 INFO:teuthology.orchestra.run.smithi088.stderr: 150K .......... .......... .......... .......... .......... 0% 15.0M 17s 2024-04-17T04:57:19.250 INFO:teuthology.orchestra.run.smithi088.stderr: 200K .......... .......... .......... .......... .......... 0% 5.76M 17s 2024-04-17T04:57:19.253 INFO:teuthology.orchestra.run.smithi088.stderr: 250K .......... .......... .......... .......... .......... 0% 18.4M 15s 2024-04-17T04:57:19.254 INFO:teuthology.orchestra.run.smithi088.stderr: 300K .......... .......... .......... .......... .......... 0% 41.6M 13s 2024-04-17T04:57:19.257 INFO:teuthology.orchestra.run.smithi088.stderr: 350K .......... .......... .......... .......... .......... 0% 20.4M 12s 2024-04-17T04:57:19.257 INFO:teuthology.orchestra.run.smithi088.stderr: 400K .......... .......... .......... .......... .......... 0% 41.6M 11s 2024-04-17T04:57:19.277 INFO:teuthology.orchestra.run.smithi088.stderr: 450K .......... .......... .......... .......... .......... 0% 6.67M 11s 2024-04-17T04:57:19.277 INFO:teuthology.orchestra.run.smithi088.stderr: 500K .......... .......... .......... .......... .......... 0% 33.5M 10s 2024-04-17T04:57:19.277 INFO:teuthology.orchestra.run.smithi088.stderr: 550K .......... .......... .......... .......... .......... 0% 74.8M 10s 2024-04-17T04:57:19.277 INFO:teuthology.orchestra.run.smithi088.stderr: 600K .......... .......... .......... .......... .......... 0% 34.4M 9s 2024-04-17T04:57:19.277 INFO:teuthology.orchestra.run.smithi088.stderr: 650K .......... .......... .......... .......... .......... 0% 57.6M 8s 2024-04-17T04:57:19.277 INFO:teuthology.orchestra.run.smithi088.stderr: 700K .......... .......... .......... .......... .......... 0% 42.0M 8s 2024-04-17T04:57:19.278 INFO:teuthology.orchestra.run.smithi088.stderr: 750K .......... .......... .......... .......... .......... 0% 79.9M 8s 2024-04-17T04:57:19.278 INFO:teuthology.orchestra.run.smithi088.stderr: 800K .......... .......... .......... .......... .......... 0% 60.7M 7s 2024-04-17T04:57:19.278 INFO:teuthology.orchestra.run.smithi088.stderr: 850K .......... .......... .......... .......... .......... 0% 67.7M 7s 2024-04-17T04:57:19.278 INFO:teuthology.orchestra.run.smithi088.stderr: 900K .......... .......... .......... .......... .......... 1% 58.8M 7s 2024-04-17T04:57:20.643 INFO:teuthology.orchestra.run.smithi088.stderr: 950K .......... .......... .......... .......... .......... 1% 7.83M 7s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1000K .......... .......... .......... .......... .......... 1% 90.8M 7s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1050K .......... .......... .......... .......... .......... 1% 68.6M 6s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1100K .......... .......... .......... .......... .......... 1% 66.8M 6s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1150K .......... .......... .......... .......... .......... 1% 55.1M 6s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1200K .......... .......... .......... .......... .......... 1% 53.9M 6s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1250K .......... .......... .......... .......... .......... 1% 44.1M 6s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1300K .......... .......... .......... .......... .......... 1% 57.8M 5s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1350K .......... .......... .......... .......... .......... 1% 47.9M 5s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1400K .......... .......... .......... .......... .......... 1% 58.7M 5s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1450K .......... .......... .......... .......... .......... 1% 58.0M 5s 2024-04-17T04:57:20.644 INFO:teuthology.orchestra.run.smithi088.stderr: 1500K .......... .......... .......... .......... .......... 1% 69.7M 5s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1550K .......... .......... .......... .......... .......... 1% 72.2M 5s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1600K .......... .......... .......... .......... .......... 1% 99.3M 5s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1650K .......... .......... .......... .......... .......... 1% 106M 5s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1700K .......... .......... .......... .......... .......... 1% 125M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1750K .......... .......... .......... .......... .......... 1% 129M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1800K .......... .......... .......... .......... .......... 1% 135M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1850K .......... .......... .......... .......... .......... 2% 108M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1900K .......... .......... .......... .......... .......... 2% 130M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 1950K .......... .......... .......... .......... .......... 2% 128M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 2000K .......... .......... .......... .......... .......... 2% 21.1M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 2050K .......... .......... .......... .......... .......... 2% 59.0M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 2100K .......... .......... .......... .......... .......... 2% 73.7M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 2150K .......... .......... .......... .......... .......... 2% 75.0M 4s 2024-04-17T04:57:20.645 INFO:teuthology.orchestra.run.smithi088.stderr: 2200K .......... .......... .......... .......... .......... 2% 84.6M 4s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2250K .......... .......... .......... .......... .......... 2% 56.7M 4s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2300K .......... .......... .......... .......... .......... 2% 75.4M 4s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2350K .......... .......... .......... .......... .......... 2% 64.4M 4s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2400K .......... .......... .......... .......... .......... 2% 69.7M 4s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2450K .......... .......... .......... .......... .......... 2% 67.2M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2500K .......... .......... .......... .......... .......... 2% 68.4M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2550K .......... .......... .......... .......... .......... 2% 55.6M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2600K .......... .......... .......... .......... .......... 2% 71.2M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2650K .......... .......... .......... .......... .......... 2% 73.6M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2700K .......... .......... .......... .......... .......... 2% 70.2M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2750K .......... .......... .......... .......... .......... 3% 62.2M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2800K .......... .......... .......... .......... .......... 3% 60.4M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2850K .......... .......... .......... .......... .......... 3% 78.3M 3s 2024-04-17T04:57:20.646 INFO:teuthology.orchestra.run.smithi088.stderr: 2900K .......... .......... .......... .......... .......... 3% 73.9M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 2950K .......... .......... .......... .......... .......... 3% 67.0M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3000K .......... .......... .......... .......... .......... 3% 62.8M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3050K .......... .......... .......... .......... .......... 3% 63.5M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3100K .......... .......... .......... .......... .......... 3% 75.3M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3150K .......... .......... .......... .......... .......... 3% 90.5M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3200K .......... .......... .......... .......... .......... 3% 65.3M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3250K .......... .......... .......... .......... .......... 3% 67.2M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3300K .......... .......... .......... .......... .......... 3% 66.9M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3350K .......... .......... .......... .......... .......... 3% 67.6M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3400K .......... .......... .......... .......... .......... 3% 58.6M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3450K .......... .......... .......... .......... .......... 3% 62.5M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3500K .......... .......... .......... .......... .......... 3% 48.1M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3550K .......... .......... .......... .......... .......... 3% 61.4M 3s 2024-04-17T04:57:20.647 INFO:teuthology.orchestra.run.smithi088.stderr: 3600K .......... .......... .......... .......... .......... 3% 63.4M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3650K .......... .......... .......... .......... .......... 3% 81.8M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3700K .......... .......... .......... .......... .......... 4% 64.5M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3750K .......... .......... .......... .......... .......... 4% 77.6M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3800K .......... .......... .......... .......... .......... 4% 76.9M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3850K .......... .......... .......... .......... .......... 4% 89.6M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3900K .......... .......... .......... .......... .......... 4% 82.3M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 3950K .......... .......... .......... .......... .......... 4% 88.2M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4000K .......... .......... .......... .......... .......... 4% 82.3M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4050K .......... .......... .......... .......... .......... 4% 91.1M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4100K .......... .......... .......... .......... .......... 4% 102M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4150K .......... .......... .......... .......... .......... 4% 106M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4200K .......... .......... .......... .......... .......... 4% 75.9M 3s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4250K .......... .......... .......... .......... .......... 4% 94.3M 2s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4300K .......... .......... .......... .......... .......... 4% 90.1M 2s 2024-04-17T04:57:20.648 INFO:teuthology.orchestra.run.smithi088.stderr: 4350K .......... .......... .......... .......... .......... 4% 101M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4400K .......... .......... .......... .......... .......... 4% 85.9M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4450K .......... .......... .......... .......... .......... 4% 109M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4500K .......... .......... .......... .......... .......... 4% 99.3M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4550K .......... .......... .......... .......... .......... 4% 98.3M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4600K .......... .......... .......... .......... .......... 5% 96.3M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4650K .......... .......... .......... .......... .......... 5% 80.1M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4700K .......... .......... .......... .......... .......... 5% 102M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4750K .......... .......... .......... .......... .......... 5% 102M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4800K .......... .......... .......... .......... .......... 5% 87.8M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4850K .......... .......... .......... .......... .......... 5% 110M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4900K .......... .......... .......... .......... .......... 5% 104M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 4950K .......... .......... .......... .......... .......... 5% 111M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 5000K .......... .......... .......... .......... .......... 5% 112M 2s 2024-04-17T04:57:20.649 INFO:teuthology.orchestra.run.smithi088.stderr: 5050K .......... .......... .......... .......... .......... 5% 101M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5100K .......... .......... .......... .......... .......... 5% 111M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5150K .......... .......... .......... .......... .......... 5% 101M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5200K .......... .......... .......... .......... .......... 5% 119M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5250K .......... .......... .......... .......... .......... 5% 122M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5300K .......... .......... .......... .......... .......... 5% 119M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5350K .......... .......... .......... .......... .......... 5% 103M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5400K .......... .......... .......... .......... .......... 5% 125M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5450K .......... .......... .......... .......... .......... 5% 126M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5500K .......... .......... .......... .......... .......... 5% 115M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5550K .......... .......... .......... .......... .......... 6% 90.3M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5600K .......... .......... .......... .......... .......... 6% 164M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5650K .......... .......... .......... .......... .......... 6% 177M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5700K .......... .......... .......... .......... .......... 6% 187M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5750K .......... .......... .......... .......... .......... 6% 194M 2s 2024-04-17T04:57:20.650 INFO:teuthology.orchestra.run.smithi088.stderr: 5800K .......... .......... .......... .......... .......... 6% 193M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 5850K .......... .......... .......... .......... .......... 6% 175M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 5900K .......... .......... .......... .......... .......... 6% 191M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 5950K .......... .......... .......... .......... .......... 6% 202M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6000K .......... .......... .......... .......... .......... 6% 178M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6050K .......... .......... .......... .......... .......... 6% 167M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6100K .......... .......... .......... .......... .......... 6% 190M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6150K .......... .......... .......... .......... .......... 6% 198M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6200K .......... .......... .......... .......... .......... 6% 191M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6250K .......... .......... .......... .......... .......... 6% 162M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6300K .......... .......... .......... .......... .......... 6% 193M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6350K .......... .......... .......... .......... .......... 6% 199M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6400K .......... .......... .......... .......... .......... 6% 191M 2s 2024-04-17T04:57:20.651 INFO:teuthology.orchestra.run.smithi088.stderr: 6450K .......... .......... .......... .......... .......... 6% 186M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6500K .......... .......... .......... .......... .......... 7% 207M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6550K .......... .......... .......... .......... .......... 7% 215M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6600K .......... .......... .......... .......... .......... 7% 191M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6650K .......... .......... .......... .......... .......... 7% 202M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6700K .......... .......... .......... .......... .......... 7% 181M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6750K .......... .......... .......... .......... .......... 7% 202M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6800K .......... .......... .......... .......... .......... 7% 213M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6850K .......... .......... .......... .......... .......... 7% 183M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6900K .......... .......... .......... .......... .......... 7% 194M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 6950K .......... .......... .......... .......... .......... 7% 175M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 7000K .......... .......... .......... .......... .......... 7% 197M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 7050K .......... .......... .......... .......... .......... 7% 184M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 7100K .......... .......... .......... .......... .......... 7% 182M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 7150K .......... .......... .......... .......... .......... 7% 200M 2s 2024-04-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi088.stderr: 7200K .......... .......... .......... .......... .......... 7% 191M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7250K .......... .......... .......... .......... .......... 7% 212M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7300K .......... .......... .......... .......... .......... 7% 218M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7350K .......... .......... .......... .......... .......... 7% 193M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7400K .......... .......... .......... .......... .......... 8% 211M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7450K .......... .......... .......... .......... .......... 8% 185M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7500K .......... .......... .......... .......... .......... 8% 210M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7550K .......... .......... .......... .......... .......... 8% 229M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7600K .......... .......... .......... .......... .......... 8% 195M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7650K .......... .......... .......... .......... .......... 8% 223M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7700K .......... .......... .......... .......... .......... 8% 204M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7750K .......... .......... .......... .......... .......... 8% 211M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7800K .......... .......... .......... .......... .......... 8% 184M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7850K .......... .......... .......... .......... .......... 8% 236M 2s 2024-04-17T04:57:20.653 INFO:teuthology.orchestra.run.smithi088.stderr: 7900K .......... .......... .......... .......... .......... 8% 198M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 7950K .......... .......... .......... .......... .......... 8% 204M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8000K .......... .......... .......... .......... .......... 8% 219M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8050K .......... .......... .......... .......... .......... 8% 186M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8100K .......... .......... .......... .......... .......... 8% 227M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8150K .......... .......... .......... .......... .......... 8% 230M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8200K .......... .......... .......... .......... .......... 8% 217M 2s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8250K .......... .......... .......... .......... .......... 8% 205M 1s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8300K .......... .......... .......... .......... .......... 8% 154M 1s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8350K .......... .......... .......... .......... .......... 9% 112M 1s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8400K .......... .......... .......... .......... .......... 9% 114M 1s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8450K .......... .......... .......... .......... .......... 9% 117M 1s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8500K .......... .......... .......... .......... .......... 9% 90.3M 1s 2024-04-17T04:57:20.654 INFO:teuthology.orchestra.run.smithi088.stderr: 8550K .......... .......... .......... .......... .......... 9% 113M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8600K .......... .......... .......... .......... .......... 9% 131M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8650K .......... .......... .......... .......... .......... 9% 120M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8700K .......... .......... .......... .......... .......... 9% 86.4M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8750K .......... .......... .......... .......... .......... 9% 119M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8800K .......... .......... .......... .......... .......... 9% 110M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8850K .......... .......... .......... .......... .......... 9% 123M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8900K .......... .......... .......... .......... .......... 9% 117M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 8950K .......... .......... .......... .......... .......... 9% 73.9M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 9000K .......... .......... .......... .......... .......... 9% 130M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 9050K .......... .......... .......... .......... .......... 9% 132M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 9100K .......... .......... .......... .......... .......... 9% 100M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 9150K .......... .......... .......... .......... .......... 9% 98.5M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 9200K .......... .......... .......... .......... .......... 9% 133M 1s 2024-04-17T04:57:20.655 INFO:teuthology.orchestra.run.smithi088.stderr: 9250K .......... .......... .......... .......... .......... 10% 115M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9300K .......... .......... .......... .......... .......... 10% 108M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9350K .......... .......... .......... .......... .......... 10% 110M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9400K .......... .......... .......... .......... .......... 10% 97.6M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9450K .......... .......... .......... .......... .......... 10% 137M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9500K .......... .......... .......... .......... .......... 10% 103M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9550K .......... .......... .......... .......... .......... 10% 101M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9600K .......... .......... .......... .......... .......... 10% 137M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9650K .......... .......... .......... .......... .......... 10% 92.5M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9700K .......... .......... .......... .......... .......... 10% 117M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9750K .......... .......... .......... .......... .......... 10% 115M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9800K .......... .......... .......... .......... .......... 10% 107M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9850K .......... .......... .......... .......... .......... 10% 115M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9900K .......... .......... .......... .......... .......... 10% 109M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 9950K .......... .......... .......... .......... .......... 10% 123M 1s 2024-04-17T04:57:20.656 INFO:teuthology.orchestra.run.smithi088.stderr: 10000K .......... .......... .......... .......... .......... 10% 109M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10050K .......... .......... .......... .......... .......... 10% 90.5M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10100K .......... .......... .......... .......... .......... 10% 113M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10150K .......... .......... .......... .......... .......... 10% 125M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10200K .......... .......... .......... .......... .......... 11% 105M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10250K .......... .......... .......... .......... .......... 11% 81.4M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10300K .......... .......... .......... .......... .......... 11% 66.4M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10350K .......... .......... .......... .......... .......... 11% 60.5M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10400K .......... .......... .......... .......... .......... 11% 97.6M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10450K .......... .......... .......... .......... .......... 11% 92.3M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10500K .......... .......... .......... .......... .......... 11% 111M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10550K .......... .......... .......... .......... .......... 11% 111M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10600K .......... .......... .......... .......... .......... 11% 102M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10650K .......... .......... .......... .......... .......... 11% 119M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10700K .......... .......... .......... .......... .......... 11% 106M 1s 2024-04-17T04:57:20.657 INFO:teuthology.orchestra.run.smithi088.stderr: 10750K .......... .......... .......... .......... .......... 11% 102M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 10800K .......... .......... .......... .......... .......... 11% 121M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 10850K .......... .......... .......... .......... .......... 11% 106M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 10900K .......... .......... .......... .......... .......... 11% 116M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 10950K .......... .......... .......... .......... .......... 11% 116M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11000K .......... .......... .......... .......... .......... 11% 119M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11050K .......... .......... .......... .......... .......... 11% 107M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11100K .......... .......... .......... .......... .......... 12% 108M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11150K .......... .......... .......... .......... .......... 12% 115M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11200K .......... .......... .......... .......... .......... 12% 122M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11250K .......... .......... .......... .......... .......... 12% 136M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11300K .......... .......... .......... .......... .......... 12% 109M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11350K .......... .......... .......... .......... .......... 12% 85.3M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11400K .......... .......... .......... .......... .......... 12% 65.7M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11450K .......... .......... .......... .......... .......... 12% 116M 1s 2024-04-17T04:57:20.658 INFO:teuthology.orchestra.run.smithi088.stderr: 11500K .......... .......... .......... .......... .......... 12% 94.6M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11550K .......... .......... .......... .......... .......... 12% 34.5M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11600K .......... .......... .......... .......... .......... 12% 131M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11650K .......... .......... .......... .......... .......... 12% 68.0M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11700K .......... .......... .......... .......... .......... 12% 53.0M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11750K .......... .......... .......... .......... .......... 12% 123M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11800K .......... .......... .......... .......... .......... 12% 69.3M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11850K .......... .......... .......... .......... .......... 12% 60.8M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11900K .......... .......... .......... .......... .......... 12% 70.6M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 11950K .......... .......... .......... .......... .......... 12% 99.8M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 12000K .......... .......... .......... .......... .......... 12% 91.6M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 12050K .......... .......... .......... .......... .......... 13% 39.3M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 12100K .......... .......... .......... .......... .......... 13% 87.6M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 12150K .......... .......... .......... .......... .......... 13% 114M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 12200K .......... .......... .......... .......... .......... 13% 49.9M 1s 2024-04-17T04:57:20.659 INFO:teuthology.orchestra.run.smithi088.stderr: 12250K .......... .......... .......... .......... .......... 13% 86.7M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12300K .......... .......... .......... .......... .......... 13% 46.8M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12350K .......... .......... .......... .......... .......... 13% 74.0M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12400K .......... .......... .......... .......... .......... 13% 50.6M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12450K .......... .......... .......... .......... .......... 13% 95.2M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12500K .......... .......... .......... .......... .......... 13% 80.7M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12550K .......... .......... .......... .......... .......... 13% 72.9M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12600K .......... .......... .......... .......... .......... 13% 52.0M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12650K .......... .......... .......... .......... .......... 13% 101M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12700K .......... .......... .......... .......... .......... 13% 69.0M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12750K .......... .......... .......... .......... .......... 13% 121M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12800K .......... .......... .......... .......... .......... 13% 106M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12850K .......... .......... .......... .......... .......... 13% 115M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12900K .......... .......... .......... .......... .......... 13% 99.2M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 12950K .......... .......... .......... .......... .......... 13% 111M 1s 2024-04-17T04:57:20.660 INFO:teuthology.orchestra.run.smithi088.stderr: 13000K .......... .......... .......... .......... .......... 14% 102M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13050K .......... .......... .......... .......... .......... 14% 104M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13100K .......... .......... .......... .......... .......... 14% 127M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13150K .......... .......... .......... .......... .......... 14% 106M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13200K .......... .......... .......... .......... .......... 14% 79.4M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13250K .......... .......... .......... .......... .......... 14% 95.9M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13300K .......... .......... .......... .......... .......... 14% 128M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13350K .......... .......... .......... .......... .......... 14% 127M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13400K .......... .......... .......... .......... .......... 14% 122M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13450K .......... .......... .......... .......... .......... 14% 115M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13500K .......... .......... .......... .......... .......... 14% 94.9M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13550K .......... .......... .......... .......... .......... 14% 109M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13600K .......... .......... .......... .......... .......... 14% 110M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13650K .......... .......... .......... .......... .......... 14% 112M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13700K .......... .......... .......... .......... .......... 14% 111M 1s 2024-04-17T04:57:20.661 INFO:teuthology.orchestra.run.smithi088.stderr: 13750K .......... .......... .......... .......... .......... 14% 79.3M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 13800K .......... .......... .......... .......... .......... 14% 132M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 13850K .......... .......... .......... .......... .......... 14% 128M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 13900K .......... .......... .......... .......... .......... 15% 85.8M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 13950K .......... .......... .......... .......... .......... 15% 121M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14000K .......... .......... .......... .......... .......... 15% 123M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14050K .......... .......... .......... .......... .......... 15% 121M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14100K .......... .......... .......... .......... .......... 15% 127M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14150K .......... .......... .......... .......... .......... 15% 141M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14200K .......... .......... .......... .......... .......... 15% 91.8M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14250K .......... .......... .......... .......... .......... 15% 107M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14300K .......... .......... .......... .......... .......... 15% 118M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14350K .......... .......... .......... .......... .......... 15% 116M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14400K .......... .......... .......... .......... .......... 15% 105M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14450K .......... .......... .......... .......... .......... 15% 117M 1s 2024-04-17T04:57:20.662 INFO:teuthology.orchestra.run.smithi088.stderr: 14500K .......... .......... .......... .......... .......... 15% 115M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14550K .......... .......... .......... .......... .......... 15% 114M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14600K .......... .......... .......... .......... .......... 15% 84.9M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14650K .......... .......... .......... .......... .......... 15% 112M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14700K .......... .......... .......... .......... .......... 15% 111M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14750K .......... .......... .......... .......... .......... 15% 123M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14800K .......... .......... .......... .......... .......... 15% 114M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14850K .......... .......... .......... .......... .......... 16% 90.4M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14900K .......... .......... .......... .......... .......... 16% 115M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 14950K .......... .......... .......... .......... .......... 16% 106M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 15000K .......... .......... .......... .......... .......... 16% 114M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 15050K .......... .......... .......... .......... .......... 16% 112M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 15100K .......... .......... .......... .......... .......... 16% 109M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 15150K .......... .......... .......... .......... .......... 16% 59.1M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 15200K .......... .......... .......... .......... .......... 16% 104M 1s 2024-04-17T04:57:20.663 INFO:teuthology.orchestra.run.smithi088.stderr: 15250K .......... .......... .......... .......... .......... 16% 68.7M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15300K .......... .......... .......... .......... .......... 16% 109M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15350K .......... .......... .......... .......... .......... 16% 119M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15400K .......... .......... .......... .......... .......... 16% 120M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15450K .......... .......... .......... .......... .......... 16% 94.4M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15500K .......... .......... .......... .......... .......... 16% 69.3M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15550K .......... .......... .......... .......... .......... 16% 133M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15600K .......... .......... .......... .......... .......... 16% 134M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15650K .......... .......... .......... .......... .......... 16% 69.9M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15700K .......... .......... .......... .......... .......... 16% 171M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15750K .......... .......... .......... .......... .......... 17% 130M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15800K .......... .......... .......... .......... .......... 17% 134M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15850K .......... .......... .......... .......... .......... 17% 94.8M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15900K .......... .......... .......... .......... .......... 17% 69.3M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 15950K .......... .......... .......... .......... .......... 17% 180M 1s 2024-04-17T04:57:20.664 INFO:teuthology.orchestra.run.smithi088.stderr: 16000K .......... .......... .......... .......... .......... 17% 183M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16050K .......... .......... .......... .......... .......... 17% 88.1M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16100K .......... .......... .......... .......... .......... 17% 77.8M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16150K .......... .......... .......... .......... .......... 17% 205M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16200K .......... .......... .......... .......... .......... 17% 106M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16250K .......... .......... .......... .......... .......... 17% 54.9M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16300K .......... .......... .......... .......... .......... 17% 97.8M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16350K .......... .......... .......... .......... .......... 17% 206M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16400K .......... .......... .......... .......... .......... 17% 174M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16450K .......... .......... .......... .......... .......... 17% 198M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16500K .......... .......... .......... .......... .......... 17% 73.1M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16550K .......... .......... .......... .......... .......... 17% 123M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16600K .......... .......... .......... .......... .......... 17% 162M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16650K .......... .......... .......... .......... .......... 17% 164M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16700K .......... .......... .......... .......... .......... 18% 66.7M 1s 2024-04-17T04:57:20.665 INFO:teuthology.orchestra.run.smithi088.stderr: 16750K .......... .......... .......... .......... .......... 18% 183M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 16800K .......... .......... .......... .......... .......... 18% 177M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 16850K .......... .......... .......... .......... .......... 18% 190M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 16900K .......... .......... .......... .......... .......... 18% 66.2M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 16950K .......... .......... .......... .......... .......... 18% 204M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17000K .......... .......... .......... .......... .......... 18% 94.7M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17050K .......... .......... .......... .......... .......... 18% 130M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17100K .......... .......... .......... .......... .......... 18% 147M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17150K .......... .......... .......... .......... .......... 18% 110M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17200K .......... .......... .......... .......... .......... 18% 76.5M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17250K .......... .......... .......... .......... .......... 18% 174M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17300K .......... .......... .......... .......... .......... 18% 68.7M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17350K .......... .......... .......... .......... .......... 18% 130M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17400K .......... .......... .......... .......... .......... 18% 123M 1s 2024-04-17T04:57:20.666 INFO:teuthology.orchestra.run.smithi088.stderr: 17450K .......... .......... .......... .......... .......... 18% 92.0M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17500K .......... .......... .......... .......... .......... 18% 203M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17550K .......... .......... .......... .......... .......... 18% 104M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17600K .......... .......... .......... .......... .......... 18% 133M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17650K .......... .......... .......... .......... .......... 19% 143M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17700K .......... .......... .......... .......... .......... 19% 84.2M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17750K .......... .......... .......... .......... .......... 19% 164M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17800K .......... .......... .......... .......... .......... 19% 114M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17850K .......... .......... .......... .......... .......... 19% 95.2M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17900K .......... .......... .......... .......... .......... 19% 99.2M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 17950K .......... .......... .......... .......... .......... 19% 140M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 18000K .......... .......... .......... .......... .......... 19% 114M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 18050K .......... .......... .......... .......... .......... 19% 88.1M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 18100K .......... .......... .......... .......... .......... 19% 132M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 18150K .......... .......... .......... .......... .......... 19% 90.8M 1s 2024-04-17T04:57:20.667 INFO:teuthology.orchestra.run.smithi088.stderr: 18200K .......... .......... .......... .......... .......... 19% 141M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18250K .......... .......... .......... .......... .......... 19% 89.3M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18300K .......... .......... .......... .......... .......... 19% 112M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18350K .......... .......... .......... .......... .......... 19% 113M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18400K .......... .......... .......... .......... .......... 19% 102M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18450K .......... .......... .......... .......... .......... 19% 122M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18500K .......... .......... .......... .......... .......... 19% 113M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18550K .......... .......... .......... .......... .......... 20% 116M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18600K .......... .......... .......... .......... .......... 20% 45.9M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18650K .......... .......... .......... .......... .......... 20% 132M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18700K .......... .......... .......... .......... .......... 20% 192M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18750K .......... .......... .......... .......... .......... 20% 176M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18800K .......... .......... .......... .......... .......... 20% 172M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18850K .......... .......... .......... .......... .......... 20% 143M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18900K .......... .......... .......... .......... .......... 20% 100M 1s 2024-04-17T04:57:20.668 INFO:teuthology.orchestra.run.smithi088.stderr: 18950K .......... .......... .......... .......... .......... 20% 80.5M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19000K .......... .......... .......... .......... .......... 20% 168M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19050K .......... .......... .......... .......... .......... 20% 112M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19100K .......... .......... .......... .......... .......... 20% 106M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19150K .......... .......... .......... .......... .......... 20% 116M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19200K .......... .......... .......... .......... .......... 20% 73.5M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19250K .......... .......... .......... .......... .......... 20% 201M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19300K .......... .......... .......... .......... .......... 20% 114M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19350K .......... .......... .......... .......... .......... 20% 111M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19400K .......... .......... .......... .......... .......... 20% 115M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19450K .......... .......... .......... .......... .......... 20% 108M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19500K .......... .......... .......... .......... .......... 21% 110M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19550K .......... .......... .......... .......... .......... 21% 116M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19600K .......... .......... .......... .......... .......... 21% 90.0M 1s 2024-04-17T04:57:20.669 INFO:teuthology.orchestra.run.smithi088.stderr: 19650K .......... .......... .......... .......... .......... 21% 112M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 19700K .......... .......... .......... .......... .......... 21% 121M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 19750K .......... .......... .......... .......... .......... 21% 119M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 19800K .......... .......... .......... .......... .......... 21% 114M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 19850K .......... .......... .......... .......... .......... 21% 120M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 19900K .......... .......... .......... .......... .......... 21% 89.3M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 19950K .......... .......... .......... .......... .......... 21% 107M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20000K .......... .......... .......... .......... .......... 21% 118M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20050K .......... .......... .......... .......... .......... 21% 114M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20100K .......... .......... .......... .......... .......... 21% 109M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20150K .......... .......... .......... .......... .......... 21% 103M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20200K .......... .......... .......... .......... .......... 21% 122M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20250K .......... .......... .......... .......... .......... 21% 122M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20300K .......... .......... .......... .......... .......... 21% 88.9M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20350K .......... .......... .......... .......... .......... 21% 119M 1s 2024-04-17T04:57:20.670 INFO:teuthology.orchestra.run.smithi088.stderr: 20400K .......... .......... .......... .......... .......... 22% 111M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20450K .......... .......... .......... .......... .......... 22% 112M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20500K .......... .......... .......... .......... .......... 22% 116M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20550K .......... .......... .......... .......... .......... 22% 109M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20600K .......... .......... .......... .......... .......... 22% 107M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20650K .......... .......... .......... .......... .......... 22% 107M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20700K .......... .......... .......... .......... .......... 22% 95.4M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20750K .......... .......... .......... .......... .......... 22% 111M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20800K .......... .......... .......... .......... .......... 22% 124M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20850K .......... .......... .......... .......... .......... 22% 97.2M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20900K .......... .......... .......... .......... .......... 22% 111M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 20950K .......... .......... .......... .......... .......... 22% 110M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 21000K .......... .......... .......... .......... .......... 22% 110M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 21050K .......... .......... .......... .......... .......... 22% 120M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 21100K .......... .......... .......... .......... .......... 22% 86.6M 1s 2024-04-17T04:57:20.671 INFO:teuthology.orchestra.run.smithi088.stderr: 21150K .......... .......... .......... .......... .......... 22% 120M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21200K .......... .......... .......... .......... .......... 22% 115M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21250K .......... .......... .......... .......... .......... 22% 86.1M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21300K .......... .......... .......... .......... .......... 22% 173M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21350K .......... .......... .......... .......... .......... 23% 91.1M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21400K .......... .......... .......... .......... .......... 23% 115M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21450K .......... .......... .......... .......... .......... 23% 107M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21500K .......... .......... .......... .......... .......... 23% 133M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21550K .......... .......... .......... .......... .......... 23% 87.0M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21600K .......... .......... .......... .......... .......... 23% 128M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21650K .......... .......... .......... .......... .......... 23% 102M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21700K .......... .......... .......... .......... .......... 23% 110M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21750K .......... .......... .......... .......... .......... 23% 116M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21800K .......... .......... .......... .......... .......... 23% 89.6M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21850K .......... .......... .......... .......... .......... 23% 139M 1s 2024-04-17T04:57:20.672 INFO:teuthology.orchestra.run.smithi088.stderr: 21900K .......... .......... .......... .......... .......... 23% 10.6M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 21950K .......... .......... .......... .......... .......... 23% 214M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22000K .......... .......... .......... .......... .......... 23% 211M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22050K .......... .......... .......... .......... .......... 23% 176M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22100K .......... .......... .......... .......... .......... 23% 223M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22150K .......... .......... .......... .......... .......... 23% 198M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22200K .......... .......... .......... .......... .......... 23% 213M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22250K .......... .......... .......... .......... .......... 24% 214M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22300K .......... .......... .......... .......... .......... 24% 183M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22350K .......... .......... .......... .......... .......... 24% 217M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22400K .......... .......... .......... .......... .......... 24% 197M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22450K .......... .......... .......... .......... .......... 24% 207M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22500K .......... .......... .......... .......... .......... 24% 211M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22550K .......... .......... .......... .......... .......... 24% 175M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22600K .......... .......... .......... .......... .......... 24% 211M 1s 2024-04-17T04:57:20.673 INFO:teuthology.orchestra.run.smithi088.stderr: 22650K .......... .......... .......... .......... .......... 24% 212M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 22700K .......... .......... .......... .......... .......... 24% 123M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 22750K .......... .......... .......... .......... .......... 24% 112M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 22800K .......... .......... .......... .......... .......... 24% 90.3M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 22850K .......... .......... .......... .......... .......... 24% 126M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 22900K .......... .......... .......... .......... .......... 24% 49.6M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 22950K .......... .......... .......... .......... .......... 24% 124M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23000K .......... .......... .......... .......... .......... 24% 104M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23050K .......... .......... .......... .......... .......... 24% 93.9M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23100K .......... .......... .......... .......... .......... 24% 108M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23150K .......... .......... .......... .......... .......... 24% 112M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23200K .......... .......... .......... .......... .......... 25% 114M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23250K .......... .......... .......... .......... .......... 25% 107M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23300K .......... .......... .......... .......... .......... 25% 103M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23350K .......... .......... .......... .......... .......... 25% 107M 1s 2024-04-17T04:57:20.674 INFO:teuthology.orchestra.run.smithi088.stderr: 23400K .......... .......... .......... .......... .......... 25% 116M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23450K .......... .......... .......... .......... .......... 25% 109M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23500K .......... .......... .......... .......... .......... 25% 105M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23550K .......... .......... .......... .......... .......... 25% 106M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23600K .......... .......... .......... .......... .......... 25% 114M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23650K .......... .......... .......... .......... .......... 25% 86.0M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23700K .......... .......... .......... .......... .......... 25% 173M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23750K .......... .......... .......... .......... .......... 25% 101M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23800K .......... .......... .......... .......... .......... 25% 85.6M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23850K .......... .......... .......... .......... .......... 25% 111M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23900K .......... .......... .......... .......... .......... 25% 139M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 23950K .......... .......... .......... .......... .......... 25% 98.4M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 24000K .......... .......... .......... .......... .......... 25% 118M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 24050K .......... .......... .......... .......... .......... 25% 99.2M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 24100K .......... .......... .......... .......... .......... 25% 127M 1s 2024-04-17T04:57:20.675 INFO:teuthology.orchestra.run.smithi088.stderr: 24150K .......... .......... .......... .......... .......... 26% 108M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24200K .......... .......... .......... .......... .......... 26% 116M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24250K .......... .......... .......... .......... .......... 26% 105M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24300K .......... .......... .......... .......... .......... 26% 83.8M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24350K .......... .......... .......... .......... .......... 26% 149M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24400K .......... .......... .......... .......... .......... 26% 110M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24450K .......... .......... .......... .......... .......... 26% 101M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24500K .......... .......... .......... .......... .......... 26% 99.8M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24550K .......... .......... .......... .......... .......... 26% 119M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24600K .......... .......... .......... .......... .......... 26% 119M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24650K .......... .......... .......... .......... .......... 26% 115M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24700K .......... .......... .......... .......... .......... 26% 81.2M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24750K .......... .......... .......... .......... .......... 26% 119M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24800K .......... .......... .......... .......... .......... 26% 116M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24850K .......... .......... .......... .......... .......... 26% 108M 1s 2024-04-17T04:57:20.676 INFO:teuthology.orchestra.run.smithi088.stderr: 24900K .......... .......... .......... .......... .......... 26% 113M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 24950K .......... .......... .......... .......... .......... 26% 44.1M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25000K .......... .......... .......... .......... .......... 26% 114M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25050K .......... .......... .......... .......... .......... 27% 83.1M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25100K .......... .......... .......... .......... .......... 27% 54.3M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25150K .......... .......... .......... .......... .......... 27% 87.7M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25200K .......... .......... .......... .......... .......... 27% 151M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25250K .......... .......... .......... .......... .......... 27% 114M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25300K .......... .......... .......... .......... .......... 27% 72.2M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25350K .......... .......... .......... .......... .......... 27% 57.5M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25400K .......... .......... .......... .......... .......... 27% 116M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25450K .......... .......... .......... .......... .......... 27% 85.2M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25500K .......... .......... .......... .......... .......... 27% 119M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25550K .......... .......... .......... .......... .......... 27% 57.2M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25600K .......... .......... .......... .......... .......... 27% 110M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25650K .......... .......... .......... .......... .......... 27% 74.0M 1s 2024-04-17T04:57:20.677 INFO:teuthology.orchestra.run.smithi088.stderr: 25700K .......... .......... .......... .......... .......... 27% 59.7M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 25750K .......... .......... .......... .......... .......... 27% 113M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 25800K .......... .......... .......... .......... .......... 27% 119M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 25850K .......... .......... .......... .......... .......... 27% 105M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 25900K .......... .......... .......... .......... .......... 27% 70.0M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 25950K .......... .......... .......... .......... .......... 27% 69.1M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26000K .......... .......... .......... .......... .......... 28% 115M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26050K .......... .......... .......... .......... .......... 28% 83.0M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26100K .......... .......... .......... .......... .......... 28% 81.5M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26150K .......... .......... .......... .......... .......... 28% 58.1M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26200K .......... .......... .......... .......... .......... 28% 143M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26250K .......... .......... .......... .......... .......... 28% 108M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26300K .......... .......... .......... .......... .......... 28% 44.0M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26350K .......... .......... .......... .......... .......... 28% 104M 1s 2024-04-17T04:57:20.678 INFO:teuthology.orchestra.run.smithi088.stderr: 26400K .......... .......... .......... .......... .......... 28% 70.6M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26450K .......... .......... .......... .......... .......... 28% 76.8M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26500K .......... .......... .......... .......... .......... 28% 96.5M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26550K .......... .......... .......... .......... .......... 28% 109M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26600K .......... .......... .......... .......... .......... 28% 81.2M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26650K .......... .......... .......... .......... .......... 28% 70.4M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26700K .......... .......... .......... .......... .......... 28% 73.4M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26750K .......... .......... .......... .......... .......... 28% 87.2M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26800K .......... .......... .......... .......... .......... 28% 58.4M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26850K .......... .......... .......... .......... .......... 28% 104M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26900K .......... .......... .......... .......... .......... 29% 83.2M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 26950K .......... .......... .......... .......... .......... 29% 60.5M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 27000K .......... .......... .......... .......... .......... 29% 94.8M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 27050K .......... .......... .......... .......... .......... 29% 53.6M 1s 2024-04-17T04:57:20.679 INFO:teuthology.orchestra.run.smithi088.stderr: 27100K .......... .......... .......... .......... .......... 29% 119M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27150K .......... .......... .......... .......... .......... 29% 116M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27200K .......... .......... .......... .......... .......... 29% 109M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27250K .......... .......... .......... .......... .......... 29% 115M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27300K .......... .......... .......... .......... .......... 29% 116M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27350K .......... .......... .......... .......... .......... 29% 114M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27400K .......... .......... .......... .......... .......... 29% 91.2M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27450K .......... .......... .......... .......... .......... 29% 118M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27500K .......... .......... .......... .......... .......... 29% 109M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27550K .......... .......... .......... .......... .......... 29% 118M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27600K .......... .......... .......... .......... .......... 29% 121M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27650K .......... .......... .......... .......... .......... 29% 85.2M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27700K .......... .......... .......... .......... .......... 29% 119M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27750K .......... .......... .......... .......... .......... 29% 93.2M 1s 2024-04-17T04:57:20.680 INFO:teuthology.orchestra.run.smithi088.stderr: 27800K .......... .......... .......... .......... .......... 29% 142M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 27850K .......... .......... .......... .......... .......... 30% 119M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 27900K .......... .......... .......... .......... .......... 30% 109M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 27950K .......... .......... .......... .......... .......... 30% 126M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28000K .......... .......... .......... .......... .......... 30% 104M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28050K .......... .......... .......... .......... .......... 30% 93.7M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28100K .......... .......... .......... .......... .......... 30% 140M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28150K .......... .......... .......... .......... .......... 30% 103M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28200K .......... .......... .......... .......... .......... 30% 113M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28250K .......... .......... .......... .......... .......... 30% 117M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28300K .......... .......... .......... .......... .......... 30% 119M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28350K .......... .......... .......... .......... .......... 30% 85.1M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28400K .......... .......... .......... .......... .......... 30% 115M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28450K .......... .......... .......... .......... .......... 30% 103M 1s 2024-04-17T04:57:20.681 INFO:teuthology.orchestra.run.smithi088.stderr: 28500K .......... .......... .......... .......... .......... 30% 102M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28550K .......... .......... .......... .......... .......... 30% 112M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28600K .......... .......... .......... .......... .......... 30% 123M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28650K .......... .......... .......... .......... .......... 30% 121M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28700K .......... .......... .......... .......... .......... 30% 114M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28750K .......... .......... .......... .......... .......... 31% 70.8M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28800K .......... .......... .......... .......... .......... 31% 114M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28850K .......... .......... .......... .......... .......... 31% 76.4M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28900K .......... .......... .......... .......... .......... 31% 115M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 28950K .......... .......... .......... .......... .......... 31% 94.4M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 29000K .......... .......... .......... .......... .......... 31% 96.4M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 29050K .......... .......... .......... .......... .......... 31% 117M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 29100K .......... .......... .......... .......... .......... 31% 114M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 29150K .......... .......... .......... .......... .......... 31% 93.5M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 29200K .......... .......... .......... .......... .......... 31% 107M 1s 2024-04-17T04:57:20.682 INFO:teuthology.orchestra.run.smithi088.stderr: 29250K .......... .......... .......... .......... .......... 31% 113M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29300K .......... .......... .......... .......... .......... 31% 122M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29350K .......... .......... .......... .......... .......... 31% 109M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29400K .......... .......... .......... .......... .......... 31% 67.9M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29450K .......... .......... .......... .......... .......... 31% 158M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29500K .......... .......... .......... .......... .......... 31% 130M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29550K .......... .......... .......... .......... .......... 31% 115M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29600K .......... .......... .......... .......... .......... 31% 124M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29650K .......... .......... .......... .......... .......... 31% 111M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29700K .......... .......... .......... .......... .......... 32% 90.0M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29750K .......... .......... .......... .......... .......... 32% 117M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29800K .......... .......... .......... .......... .......... 32% 113M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29850K .......... .......... .......... .......... .......... 32% 115M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29900K .......... .......... .......... .......... .......... 32% 108M 1s 2024-04-17T04:57:20.683 INFO:teuthology.orchestra.run.smithi088.stderr: 29950K .......... .......... .......... .......... .......... 32% 114M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30000K .......... .......... .......... .......... .......... 32% 113M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30050K .......... .......... .......... .......... .......... 32% 99.3M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30100K .......... .......... .......... .......... .......... 32% 112M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30150K .......... .......... .......... .......... .......... 32% 110M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30200K .......... .......... .......... .......... .......... 32% 114M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30250K .......... .......... .......... .......... .......... 32% 116M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30300K .......... .......... .......... .......... .......... 32% 108M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30350K .......... .......... .......... .......... .......... 32% 94.0M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30400K .......... .......... .......... .......... .......... 32% 120M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30450K .......... .......... .......... .......... .......... 32% 112M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30500K .......... .......... .......... .......... .......... 32% 91.9M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30550K .......... .......... .......... .......... .......... 32% 132M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30600K .......... .......... .......... .......... .......... 32% 128M 1s 2024-04-17T04:57:20.684 INFO:teuthology.orchestra.run.smithi088.stderr: 30650K .......... .......... .......... .......... .......... 33% 93.8M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 30700K .......... .......... .......... .......... .......... 33% 115M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 30750K .......... .......... .......... .......... .......... 33% 104M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 30800K .......... .......... .......... .......... .......... 33% 119M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 30850K .......... .......... .......... .......... .......... 33% 110M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 30900K .......... .......... .......... .......... .......... 33% 119M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 30950K .......... .......... .......... .......... .......... 33% 105M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31000K .......... .......... .......... .......... .......... 33% 113M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31050K .......... .......... .......... .......... .......... 33% 119M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31100K .......... .......... .......... .......... .......... 33% 85.7M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31150K .......... .......... .......... .......... .......... 33% 114M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31200K .......... .......... .......... .......... .......... 33% 154M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31250K .......... .......... .......... .......... .......... 33% 89.7M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31300K .......... .......... .......... .......... .......... 33% 116M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31350K .......... .......... .......... .......... .......... 33% 119M 1s 2024-04-17T04:57:20.685 INFO:teuthology.orchestra.run.smithi088.stderr: 31400K .......... .......... .......... .......... .......... 33% 112M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31450K .......... .......... .......... .......... .......... 33% 121M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31500K .......... .......... .......... .......... .......... 33% 93.1M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31550K .......... .......... .......... .......... .......... 34% 114M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31600K .......... .......... .......... .......... .......... 34% 120M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31650K .......... .......... .......... .......... .......... 34% 109M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31700K .......... .......... .......... .......... .......... 34% 114M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31750K .......... .......... .......... .......... .......... 34% 63.2M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31800K .......... .......... .......... .......... .......... 34% 113M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31850K .......... .......... .......... .......... .......... 34% 197M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31900K .......... .......... .......... .......... .......... 34% 124M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 31950K .......... .......... .......... .......... .......... 34% 122M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 32000K .......... .......... .......... .......... .......... 34% 117M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 32050K .......... .......... .......... .......... .......... 34% 120M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 32100K .......... .......... .......... .......... .......... 34% 110M 1s 2024-04-17T04:57:20.686 INFO:teuthology.orchestra.run.smithi088.stderr: 32150K .......... .......... .......... .......... .......... 34% 101M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32200K .......... .......... .......... .......... .......... 34% 102M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32250K .......... .......... .......... .......... .......... 34% 115M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32300K .......... .......... .......... .......... .......... 34% 113M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32350K .......... .......... .......... .......... .......... 34% 119M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32400K .......... .......... .......... .......... .......... 34% 105M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32450K .......... .......... .......... .......... .......... 34% 112M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32500K .......... .......... .......... .......... .......... 35% 118M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32550K .......... .......... .......... .......... .......... 35% 111M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32600K .......... .......... .......... .......... .......... 35% 97.4M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32650K .......... .......... .......... .......... .......... 35% 107M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32700K .......... .......... .......... .......... .......... 35% 110M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32750K .......... .......... .......... .......... .......... 35% 115M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32800K .......... .......... .......... .......... .......... 35% 103M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32850K .......... .......... .......... .......... .......... 35% 106M 1s 2024-04-17T04:57:20.687 INFO:teuthology.orchestra.run.smithi088.stderr: 32900K .......... .......... .......... .......... .......... 35% 127M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 32950K .......... .......... .......... .......... .......... 35% 89.1M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33000K .......... .......... .......... .......... .......... 35% 119M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33050K .......... .......... .......... .......... .......... 35% 118M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33100K .......... .......... .......... .......... .......... 35% 117M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33150K .......... .......... .......... .......... .......... 35% 113M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33200K .......... .......... .......... .......... .......... 35% 92.3M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33250K .......... .......... .......... .......... .......... 35% 111M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33300K .......... .......... .......... .......... .......... 35% 113M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33350K .......... .......... .......... .......... .......... 35% 121M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33400K .......... .......... .......... .......... .......... 36% 105M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33450K .......... .......... .......... .......... .......... 36% 117M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33500K .......... .......... .......... .......... .......... 36% 113M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33550K .......... .......... .......... .......... .......... 36% 118M 1s 2024-04-17T04:57:20.688 INFO:teuthology.orchestra.run.smithi088.stderr: 33600K .......... .......... .......... .......... .......... 36% 81.3M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33650K .......... .......... .......... .......... .......... 36% 155M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33700K .......... .......... .......... .......... .......... 36% 107M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33750K .......... .......... .......... .......... .......... 36% 117M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33800K .......... .......... .......... .......... .......... 36% 117M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33850K .......... .......... .......... .......... .......... 36% 110M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33900K .......... .......... .......... .......... .......... 36% 91.4M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 33950K .......... .......... .......... .......... .......... 36% 112M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34000K .......... .......... .......... .......... .......... 36% 85.0M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34050K .......... .......... .......... .......... .......... 36% 121M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34100K .......... .......... .......... .......... .......... 36% 117M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34150K .......... .......... .......... .......... .......... 36% 108M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34200K .......... .......... .......... .......... .......... 36% 101M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34250K .......... .......... .......... .......... .......... 36% 121M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34300K .......... .......... .......... .......... .......... 36% 113M 1s 2024-04-17T04:57:20.689 INFO:teuthology.orchestra.run.smithi088.stderr: 34350K .......... .......... .......... .......... .......... 37% 103M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34400K .......... .......... .......... .......... .......... 37% 122M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34450K .......... .......... .......... .......... .......... 37% 110M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34500K .......... .......... .......... .......... .......... 37% 114M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34550K .......... .......... .......... .......... .......... 37% 90.3M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34600K .......... .......... .......... .......... .......... 37% 107M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34650K .......... .......... .......... .......... .......... 37% 110M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34700K .......... .......... .......... .......... .......... 37% 126M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34750K .......... .......... .......... .......... .......... 37% 74.3M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34800K .......... .......... .......... .......... .......... 37% 185M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34850K .......... .......... .......... .......... .......... 37% 101M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34900K .......... .......... .......... .......... .......... 37% 110M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 34950K .......... .......... .......... .......... .......... 37% 114M 1s 2024-04-17T04:57:20.690 INFO:teuthology.orchestra.run.smithi088.stderr: 35000K .......... .......... .......... .......... .......... 37% 106M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35050K .......... .......... .......... .......... .......... 37% 96.9M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35100K .......... .......... .......... .......... .......... 37% 114M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35150K .......... .......... .......... .......... .......... 37% 108M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35200K .......... .......... .......... .......... .......... 37% 97.4M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35250K .......... .......... .......... .......... .......... 37% 114M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35300K .......... .......... .......... .......... .......... 38% 118M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35350K .......... .......... .......... .......... .......... 38% 105M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35400K .......... .......... .......... .......... .......... 38% 84.1M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35450K .......... .......... .......... .......... .......... 38% 121M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35500K .......... .......... .......... .......... .......... 38% 97.3M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35550K .......... .......... .......... .......... .......... 38% 116M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35600K .......... .......... .......... .......... .......... 38% 111M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35650K .......... .......... .......... .......... .......... 38% 78.8M 1s 2024-04-17T04:57:20.691 INFO:teuthology.orchestra.run.smithi088.stderr: 35700K .......... .......... .......... .......... .......... 38% 66.1M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 35750K .......... .......... .......... .......... .......... 38% 106M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 35800K .......... .......... .......... .......... .......... 38% 117M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 35850K .......... .......... .......... .......... .......... 38% 74.9M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 35900K .......... .......... .......... .......... .......... 38% 85.0M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 35950K .......... .......... .......... .......... .......... 38% 105M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36000K .......... .......... .......... .......... .......... 38% 120M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36050K .......... .......... .......... .......... .......... 38% 99.2M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36100K .......... .......... .......... .......... .......... 38% 88.9M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36150K .......... .......... .......... .......... .......... 38% 119M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36200K .......... .......... .......... .......... .......... 39% 111M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36250K .......... .......... .......... .......... .......... 39% 117M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36300K .......... .......... .......... .......... .......... 39% 101M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36350K .......... .......... .......... .......... .......... 39% 86.7M 1s 2024-04-17T04:57:20.692 INFO:teuthology.orchestra.run.smithi088.stderr: 36400K .......... .......... .......... .......... .......... 39% 119M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36450K .......... .......... .......... .......... .......... 39% 109M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36500K .......... .......... .......... .......... .......... 39% 110M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36550K .......... .......... .......... .......... .......... 39% 104M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36600K .......... .......... .......... .......... .......... 39% 115M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36650K .......... .......... .......... .......... .......... 39% 96.9M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36700K .......... .......... .......... .......... .......... 39% 116M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36750K .......... .......... .......... .......... .......... 39% 101M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36800K .......... .......... .......... .......... .......... 39% 102M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36850K .......... .......... .......... .......... .......... 39% 116M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36900K .......... .......... .......... .......... .......... 39% 111M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 36950K .......... .......... .......... .......... .......... 39% 107M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 37000K .......... .......... .......... .......... .......... 39% 118M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 37050K .......... .......... .......... .......... .......... 39% 112M 1s 2024-04-17T04:57:20.693 INFO:teuthology.orchestra.run.smithi088.stderr: 37100K .......... .......... .......... .......... .......... 39% 114M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37150K .......... .......... .......... .......... .......... 40% 92.3M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37200K .......... .......... .......... .......... .......... 40% 134M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37250K .......... .......... .......... .......... .......... 40% 110M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37300K .......... .......... .......... .......... .......... 40% 115M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37350K .......... .......... .......... .......... .......... 40% 104M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37400K .......... .......... .......... .......... .......... 40% 79.1M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37450K .......... .......... .......... .......... .......... 40% 155M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37500K .......... .......... .......... .......... .......... 40% 128M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37550K .......... .......... .......... .......... .......... 40% 116M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37600K .......... .......... .......... .......... .......... 40% 114M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37650K .......... .......... .......... .......... .......... 40% 94.6M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37700K .......... .......... .......... .......... .......... 40% 113M 1s 2024-04-17T04:57:20.694 INFO:teuthology.orchestra.run.smithi088.stderr: 37750K .......... .......... .......... .......... .......... 40% 115M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 37800K .......... .......... .......... .......... .......... 40% 120M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 37850K .......... .......... .......... .......... .......... 40% 113M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 37900K .......... .......... .......... .......... .......... 40% 95.1M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 37950K .......... .......... .......... .......... .......... 40% 131M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38000K .......... .......... .......... .......... .......... 40% 107M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38050K .......... .......... .......... .......... .......... 41% 110M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38100K .......... .......... .......... .......... .......... 41% 119M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38150K .......... .......... .......... .......... .......... 41% 91.5M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38200K .......... .......... .......... .......... .......... 41% 105M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38250K .......... .......... .......... .......... .......... 41% 116M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38300K .......... .......... .......... .......... .......... 41% 115M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38350K .......... .......... .......... .......... .......... 41% 104M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38400K .......... .......... .......... .......... .......... 41% 119M 1s 2024-04-17T04:57:20.695 INFO:teuthology.orchestra.run.smithi088.stderr: 38450K .......... .......... .......... .......... .......... 41% 109M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38500K .......... .......... .......... .......... .......... 41% 114M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38550K .......... .......... .......... .......... .......... 41% 98.5M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38600K .......... .......... .......... .......... .......... 41% 107M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38650K .......... .......... .......... .......... .......... 41% 116M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38700K .......... .......... .......... .......... .......... 41% 88.3M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38750K .......... .......... .......... .......... .......... 41% 109M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38800K .......... .......... .......... .......... .......... 41% 107M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38850K .......... .......... .......... .......... .......... 41% 137M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38900K .......... .......... .......... .......... .......... 41% 103M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 38950K .......... .......... .......... .......... .......... 41% 118M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 39000K .......... .......... .......... .......... .......... 42% 97.3M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 39050K .......... .......... .......... .......... .......... 42% 127M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 39100K .......... .......... .......... .......... .......... 42% 114M 1s 2024-04-17T04:57:20.696 INFO:teuthology.orchestra.run.smithi088.stderr: 39150K .......... .......... .......... .......... .......... 42% 120M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39200K .......... .......... .......... .......... .......... 42% 90.7M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39250K .......... .......... .......... .......... .......... 42% 119M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39300K .......... .......... .......... .......... .......... 42% 119M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39350K .......... .......... .......... .......... .......... 42% 108M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39400K .......... .......... .......... .......... .......... 42% 67.0M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39450K .......... .......... .......... .......... .......... 42% 70.4M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39500K .......... .......... .......... .......... .......... 42% 86.9M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39550K .......... .......... .......... .......... .......... 42% 134M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39600K .......... .......... .......... .......... .......... 42% 114M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39650K .......... .......... .......... .......... .......... 42% 112M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39700K .......... .......... .......... .......... .......... 42% 109M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39750K .......... .......... .......... .......... .......... 42% 92.6M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39800K .......... .......... .......... .......... .......... 42% 147M 1s 2024-04-17T04:57:20.697 INFO:teuthology.orchestra.run.smithi088.stderr: 39850K .......... .......... .......... .......... .......... 42% 110M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 39900K .......... .......... .......... .......... .......... 43% 97.4M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 39950K .......... .......... .......... .......... .......... 43% 91.8M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40000K .......... .......... .......... .......... .......... 43% 124M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40050K .......... .......... .......... .......... .......... 43% 105M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40100K .......... .......... .......... .......... .......... 43% 126M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40150K .......... .......... .......... .......... .......... 43% 94.2M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40200K .......... .......... .......... .......... .......... 43% 95.9M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40250K .......... .......... .......... .......... .......... 43% 120M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40300K .......... .......... .......... .......... .......... 43% 114M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40350K .......... .......... .......... .......... .......... 43% 93.1M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40400K .......... .......... .......... .......... .......... 43% 133M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40450K .......... .......... .......... .......... .......... 43% 107M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40500K .......... .......... .......... .......... .......... 43% 108M 1s 2024-04-17T04:57:20.698 INFO:teuthology.orchestra.run.smithi088.stderr: 40550K .......... .......... .......... .......... .......... 43% 120M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40600K .......... .......... .......... .......... .......... 43% 51.6M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40650K .......... .......... .......... .......... .......... 43% 109M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40700K .......... .......... .......... .......... .......... 43% 80.3M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40750K .......... .......... .......... .......... .......... 43% 59.2M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40800K .......... .......... .......... .......... .......... 43% 120M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40850K .......... .......... .......... .......... .......... 44% 70.0M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40900K .......... .......... .......... .......... .......... 44% 61.2M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 40950K .......... .......... .......... .......... .......... 44% 105M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41000K .......... .......... .......... .......... .......... 44% 130M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41050K .......... .......... .......... .......... .......... 44% 46.9M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41100K .......... .......... .......... .......... .......... 44% 117M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41150K .......... .......... .......... .......... .......... 44% 114M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41200K .......... .......... .......... .......... .......... 44% 77.9M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41250K .......... .......... .......... .......... .......... 44% 58.4M 1s 2024-04-17T04:57:20.699 INFO:teuthology.orchestra.run.smithi088.stderr: 41300K .......... .......... .......... .......... .......... 44% 77.9M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41350K .......... .......... .......... .......... .......... 44% 97.4M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41400K .......... .......... .......... .......... .......... 44% 69.0M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41450K .......... .......... .......... .......... .......... 44% 79.0M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41500K .......... .......... .......... .......... .......... 44% 54.9M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41550K .......... .......... .......... .......... .......... 44% 61.5M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41600K .......... .......... .......... .......... .......... 44% 222M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41650K .......... .......... .......... .......... .......... 44% 73.8M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41700K .......... .......... .......... .......... .......... 44% 83.0M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41750K .......... .......... .......... .......... .......... 44% 56.1M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41800K .......... .......... .......... .......... .......... 45% 101M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41850K .......... .......... .......... .......... .......... 45% 115M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41900K .......... .......... .......... .......... .......... 45% 115M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 41950K .......... .......... .......... .......... .......... 45% 114M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 42000K .......... .......... .......... .......... .......... 45% 109M 1s 2024-04-17T04:57:20.700 INFO:teuthology.orchestra.run.smithi088.stderr: 42050K .......... .......... .......... .......... .......... 45% 118M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42100K .......... .......... .......... .......... .......... 45% 103M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42150K .......... .......... .......... .......... .......... 45% 117M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42200K .......... .......... .......... .......... .......... 45% 92.2M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42250K .......... .......... .......... .......... .......... 45% 109M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42300K .......... .......... .......... .......... .......... 45% 116M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42350K .......... .......... .......... .......... .......... 45% 109M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42400K .......... .......... .......... .......... .......... 45% 125M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42450K .......... .......... .......... .......... .......... 45% 72.6M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42500K .......... .......... .......... .......... .......... 45% 120M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42550K .......... .......... .......... .......... .......... 45% 144M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42600K .......... .......... .......... .......... .......... 45% 61.5M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42650K .......... .......... .......... .......... .......... 45% 71.7M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42700K .......... .......... .......... .......... .......... 46% 117M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42750K .......... .......... .......... .......... .......... 46% 116M 1s 2024-04-17T04:57:20.701 INFO:teuthology.orchestra.run.smithi088.stderr: 42800K .......... .......... .......... .......... .......... 46% 92.8M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 42850K .......... .......... .......... .......... .......... 46% 91.3M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 42900K .......... .......... .......... .......... .......... 46% 126M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 42950K .......... .......... .......... .......... .......... 46% 101M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43000K .......... .......... .......... .......... .......... 46% 118M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43050K .......... .......... .......... .......... .......... 46% 76.6M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43100K .......... .......... .......... .......... .......... 46% 70.3M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43150K .......... .......... .......... .......... .......... 46% 107M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43200K .......... .......... .......... .......... .......... 46% 108M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43250K .......... .......... .......... .......... .......... 46% 111M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43300K .......... .......... .......... .......... .......... 46% 122M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43350K .......... .......... .......... .......... .......... 46% 86.5M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43400K .......... .......... .......... .......... .......... 46% 115M 1s 2024-04-17T04:57:20.702 INFO:teuthology.orchestra.run.smithi088.stderr: 43450K .......... .......... .......... .......... .......... 46% 95.7M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43500K .......... .......... .......... .......... .......... 46% 116M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43550K .......... .......... .......... .......... .......... 46% 111M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43600K .......... .......... .......... .......... .......... 46% 98.6M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43650K .......... .......... .......... .......... .......... 47% 107M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43700K .......... .......... .......... .......... .......... 47% 108M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43750K .......... .......... .......... .......... .......... 47% 109M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43800K .......... .......... .......... .......... .......... 47% 79.5M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43850K .......... .......... .......... .......... .......... 47% 99.0M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43900K .......... .......... .......... .......... .......... 47% 110M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 43950K .......... .......... .......... .......... .......... 47% 139M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 44000K .......... .......... .......... .......... .......... 47% 104M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 44050K .......... .......... .......... .......... .......... 47% 82.6M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 44100K .......... .......... .......... .......... .......... 47% 103M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 44150K .......... .......... .......... .......... .......... 47% 103M 1s 2024-04-17T04:57:20.703 INFO:teuthology.orchestra.run.smithi088.stderr: 44200K .......... .......... .......... .......... .......... 47% 125M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44250K .......... .......... .......... .......... .......... 47% 101M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44300K .......... .......... .......... .......... .......... 47% 82.2M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44350K .......... .......... .......... .......... .......... 47% 95.9M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44400K .......... .......... .......... .......... .......... 47% 122M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44450K .......... .......... .......... .......... .......... 47% 125M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44500K .......... .......... .......... .......... .......... 47% 110M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44550K .......... .......... .......... .......... .......... 48% 78.7M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44600K .......... .......... .......... .......... .......... 48% 113M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44650K .......... .......... .......... .......... .......... 48% 113M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44700K .......... .......... .......... .......... .......... 48% 119M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44750K .......... .......... .......... .......... .......... 48% 105M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44800K .......... .......... .......... .......... .......... 48% 85.2M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44850K .......... .......... .......... .......... .......... 48% 94.3M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44900K .......... .......... .......... .......... .......... 48% 130M 1s 2024-04-17T04:57:20.704 INFO:teuthology.orchestra.run.smithi088.stderr: 44950K .......... .......... .......... .......... .......... 48% 119M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45000K .......... .......... .......... .......... .......... 48% 91.3M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45050K .......... .......... .......... .......... .......... 48% 75.2M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45100K .......... .......... .......... .......... .......... 48% 133M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45150K .......... .......... .......... .......... .......... 48% 105M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45200K .......... .......... .......... .......... .......... 48% 169M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45250K .......... .......... .......... .......... .......... 48% 104M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45300K .......... .......... .......... .......... .......... 48% 113M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45350K .......... .......... .......... .......... .......... 48% 115M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45400K .......... .......... .......... .......... .......... 48% 119M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45450K .......... .......... .......... .......... .......... 48% 69.6M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45500K .......... .......... .......... .......... .......... 49% 46.3M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45550K .......... .......... .......... .......... .......... 49% 68.4M 1s 2024-04-17T04:57:20.705 INFO:teuthology.orchestra.run.smithi088.stderr: 45600K .......... .......... .......... .......... .......... 49% 116M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45650K .......... .......... .......... .......... .......... 49% 89.8M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45700K .......... .......... .......... .......... .......... 49% 61.9M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45750K .......... .......... .......... .......... .......... 49% 62.2M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45800K .......... .......... .......... .......... .......... 49% 92.3M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45850K .......... .......... .......... .......... .......... 49% 118M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45900K .......... .......... .......... .......... .......... 49% 105M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 45950K .......... .......... .......... .......... .......... 49% 91.7M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 46000K .......... .......... .......... .......... .......... 49% 114M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 46050K .......... .......... .......... .......... .......... 49% 115M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 46100K .......... .......... .......... .......... .......... 49% 111M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 46150K .......... .......... .......... .......... .......... 49% 105M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 46200K .......... .......... .......... .......... .......... 49% 101M 1s 2024-04-17T04:57:20.706 INFO:teuthology.orchestra.run.smithi088.stderr: 46250K .......... .......... .......... .......... .......... 49% 114M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46300K .......... .......... .......... .......... .......... 49% 117M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46350K .......... .......... .......... .......... .......... 49% 111M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46400K .......... .......... .......... .......... .......... 49% 108M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46450K .......... .......... .......... .......... .......... 50% 123M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46500K .......... .......... .......... .......... .......... 50% 118M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46550K .......... .......... .......... .......... .......... 50% 109M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46600K .......... .......... .......... .......... .......... 50% 79.8M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46650K .......... .......... .......... .......... .......... 50% 118M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46700K .......... .......... .......... .......... .......... 50% 114M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46750K .......... .......... .......... .......... .......... 50% 138M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46800K .......... .......... .......... .......... .......... 50% 95.2M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46850K .......... .......... .......... .......... .......... 50% 111M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46900K .......... .......... .......... .......... .......... 50% 92.2M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 46950K .......... .......... .......... .......... .......... 50% 112M 1s 2024-04-17T04:57:20.707 INFO:teuthology.orchestra.run.smithi088.stderr: 47000K .......... .......... .......... .......... .......... 50% 115M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47050K .......... .......... .......... .......... .......... 50% 116M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47100K .......... .......... .......... .......... .......... 50% 108M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47150K .......... .......... .......... .......... .......... 50% 122M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47200K .......... .......... .......... .......... .......... 50% 118M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47250K .......... .......... .......... .......... .......... 50% 109M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47300K .......... .......... .......... .......... .......... 50% 94.1M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47350K .......... .......... .......... .......... .......... 51% 109M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47400K .......... .......... .......... .......... .......... 51% 90.0M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47450K .......... .......... .......... .......... .......... 51% 114M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47500K .......... .......... .......... .......... .......... 51% 111M 1s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47550K .......... .......... .......... .......... .......... 51% 94.6M 0s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47600K .......... .......... .......... .......... .......... 51% 92.8M 0s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47650K .......... .......... .......... .......... .......... 51% 130M 0s 2024-04-17T04:57:20.708 INFO:teuthology.orchestra.run.smithi088.stderr: 47700K .......... .......... .......... .......... .......... 51% 141M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 47750K .......... .......... .......... .......... .......... 51% 110M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 47800K .......... .......... .......... .......... .......... 51% 115M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 47850K .......... .......... .......... .......... .......... 51% 108M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 47900K .......... .......... .......... .......... .......... 51% 74.6M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 47950K .......... .......... .......... .......... .......... 51% 57.3M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48000K .......... .......... .......... .......... .......... 51% 45.2M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48050K .......... .......... .......... .......... .......... 51% 65.9M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48100K .......... .......... .......... .......... .......... 51% 66.6M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48150K .......... .......... .......... .......... .......... 51% 120M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48200K .......... .......... .......... .......... .......... 51% 114M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48250K .......... .......... .......... .......... .......... 51% 64.5M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48300K .......... .......... .......... .......... .......... 52% 63.6M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48350K .......... .......... .......... .......... .......... 52% 108M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48400K .......... .......... .......... .......... .......... 52% 117M 0s 2024-04-17T04:57:20.709 INFO:teuthology.orchestra.run.smithi088.stderr: 48450K .......... .......... .......... .......... .......... 52% 47.5M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48500K .......... .......... .......... .......... .......... 52% 51.6M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48550K .......... .......... .......... .......... .......... 52% 31.0M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48600K .......... .......... .......... .......... .......... 52% 120M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48650K .......... .......... .......... .......... .......... 52% 36.9M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48700K .......... .......... .......... .......... .......... 52% 53.1M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48750K .......... .......... .......... .......... .......... 52% 56.6M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48800K .......... .......... .......... .......... .......... 52% 55.5M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48850K .......... .......... .......... .......... .......... 52% 116M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48900K .......... .......... .......... .......... .......... 52% 50.0M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 48950K .......... .......... .......... .......... .......... 52% 108M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 49000K .......... .......... .......... .......... .......... 52% 117M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 49050K .......... .......... .......... .......... .......... 52% 111M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 49100K .......... .......... .......... .......... .......... 52% 106M 0s 2024-04-17T04:57:20.710 INFO:teuthology.orchestra.run.smithi088.stderr: 49150K .......... .......... .......... .......... .......... 52% 117M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49200K .......... .......... .......... .......... .......... 53% 86.1M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49250K .......... .......... .......... .......... .......... 53% 82.0M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49300K .......... .......... .......... .......... .......... 53% 85.7M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49350K .......... .......... .......... .......... .......... 53% 116M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49400K .......... .......... .......... .......... .......... 53% 114M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49450K .......... .......... .......... .......... .......... 53% 117M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49500K .......... .......... .......... .......... .......... 53% 113M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49550K .......... .......... .......... .......... .......... 53% 118M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49600K .......... .......... .......... .......... .......... 53% 86.6M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49650K .......... .......... .......... .......... .......... 53% 114M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49700K .......... .......... .......... .......... .......... 53% 117M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49750K .......... .......... .......... .......... .......... 53% 115M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49800K .......... .......... .......... .......... .......... 53% 116M 0s 2024-04-17T04:57:20.711 INFO:teuthology.orchestra.run.smithi088.stderr: 49850K .......... .......... .......... .......... .......... 53% 110M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 49900K .......... .......... .......... .......... .......... 53% 119M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 49950K .......... .......... .......... .......... .......... 53% 116M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50000K .......... .......... .......... .......... .......... 53% 94.1M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50050K .......... .......... .......... .......... .......... 53% 115M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50100K .......... .......... .......... .......... .......... 53% 110M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50150K .......... .......... .......... .......... .......... 54% 113M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50200K .......... .......... .......... .......... .......... 54% 88.9M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50250K .......... .......... .......... .......... .......... 54% 114M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50300K .......... .......... .......... .......... .......... 54% 92.4M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50350K .......... .......... .......... .......... .......... 54% 64.0M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50400K .......... .......... .......... .......... .......... 54% 82.9M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50450K .......... .......... .......... .......... .......... 54% 48.3M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50500K .......... .......... .......... .......... .......... 54% 109M 0s 2024-04-17T04:57:20.712 INFO:teuthology.orchestra.run.smithi088.stderr: 50550K .......... .......... .......... .......... .......... 54% 64.6M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50600K .......... .......... .......... .......... .......... 54% 97.6M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50650K .......... .......... .......... .......... .......... 54% 43.8M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50700K .......... .......... .......... .......... .......... 54% 64.5M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50750K .......... .......... .......... .......... .......... 54% 111M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50800K .......... .......... .......... .......... .......... 54% 111M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50850K .......... .......... .......... .......... .......... 54% 75.2M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50900K .......... .......... .......... .......... .......... 54% 62.5M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 50950K .......... .......... .......... .......... .......... 54% 90.3M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 51000K .......... .......... .......... .......... .......... 54% 117M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 51050K .......... .......... .......... .......... .......... 55% 106M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 51100K .......... .......... .......... .......... .......... 55% 89.9M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 51150K .......... .......... .......... .......... .......... 55% 122M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 51200K .......... .......... .......... .......... .......... 55% 56.8M 0s 2024-04-17T04:57:20.713 INFO:teuthology.orchestra.run.smithi088.stderr: 51250K .......... .......... .......... .......... .......... 55% 90.0M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51300K .......... .......... .......... .......... .......... 55% 44.6M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51350K .......... .......... .......... .......... .......... 55% 113M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51400K .......... .......... .......... .......... .......... 55% 60.4M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51450K .......... .......... .......... .......... .......... 55% 118M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51500K .......... .......... .......... .......... .......... 55% 62.1M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51550K .......... .......... .......... .......... .......... 55% 73.3M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51600K .......... .......... .......... .......... .......... 55% 44.1M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51650K .......... .......... .......... .......... .......... 55% 42.3M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51700K .......... .......... .......... .......... .......... 55% 83.4M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51750K .......... .......... .......... .......... .......... 55% 83.2M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51800K .......... .......... .......... .......... .......... 55% 62.9M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51850K .......... .......... .......... .......... .......... 55% 60.3M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51900K .......... .......... .......... .......... .......... 55% 108M 0s 2024-04-17T04:57:20.714 INFO:teuthology.orchestra.run.smithi088.stderr: 51950K .......... .......... .......... .......... .......... 55% 43.8M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52000K .......... .......... .......... .......... .......... 56% 156M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52050K .......... .......... .......... .......... .......... 56% 66.0M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52100K .......... .......... .......... .......... .......... 56% 106M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52150K .......... .......... .......... .......... .......... 56% 51.2M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52200K .......... .......... .......... .......... .......... 56% 128M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52250K .......... .......... .......... .......... .......... 56% 131M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52300K .......... .......... .......... .......... .......... 56% 119M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52350K .......... .......... .......... .......... .......... 56% 108M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52400K .......... .......... .......... .......... .......... 56% 116M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52450K .......... .......... .......... .......... .......... 56% 91.2M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52500K .......... .......... .......... .......... .......... 56% 128M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52550K .......... .......... .......... .......... .......... 56% 87.5M 0s 2024-04-17T04:57:20.715 INFO:teuthology.orchestra.run.smithi088.stderr: 52600K .......... .......... .......... .......... .......... 56% 92.3M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52650K .......... .......... .......... .......... .......... 56% 106M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52700K .......... .......... .......... .......... .......... 56% 74.5M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52750K .......... .......... .......... .......... .......... 56% 64.1M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52800K .......... .......... .......... .......... .......... 56% 75.2M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52850K .......... .......... .......... .......... .......... 56% 88.2M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52900K .......... .......... .......... .......... .......... 56% 138M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 52950K .......... .......... .......... .......... .......... 57% 63.0M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 53000K .......... .......... .......... .......... .......... 57% 37.1M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 53050K .......... .......... .......... .......... .......... 57% 64.2M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 53100K .......... .......... .......... .......... .......... 57% 138M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 53150K .......... .......... .......... .......... .......... 57% 68.2M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 53200K .......... .......... .......... .......... .......... 57% 44.0M 0s 2024-04-17T04:57:20.716 INFO:teuthology.orchestra.run.smithi088.stderr: 53250K .......... .......... .......... .......... .......... 57% 44.4M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53300K .......... .......... .......... .......... .......... 57% 123M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53350K .......... .......... .......... .......... .......... 57% 58.4M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53400K .......... .......... .......... .......... .......... 57% 43.0M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53450K .......... .......... .......... .......... .......... 57% 55.5M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53500K .......... .......... .......... .......... .......... 57% 39.0M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53550K .......... .......... .......... .......... .......... 57% 156M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53600K .......... .......... .......... .......... .......... 57% 49.6M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53650K .......... .......... .......... .......... .......... 57% 53.8M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53700K .......... .......... .......... .......... .......... 57% 95.9M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53750K .......... .......... .......... .......... .......... 57% 53.3M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53800K .......... .......... .......... .......... .......... 57% 101M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53850K .......... .......... .......... .......... .......... 58% 90.1M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53900K .......... .......... .......... .......... .......... 58% 61.5M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 53950K .......... .......... .......... .......... .......... 58% 44.9M 0s 2024-04-17T04:57:20.717 INFO:teuthology.orchestra.run.smithi088.stderr: 54000K .......... .......... .......... .......... .......... 58% 102M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54050K .......... .......... .......... .......... .......... 58% 110M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54100K .......... .......... .......... .......... .......... 58% 83.5M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54150K .......... .......... .......... .......... .......... 58% 58.8M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54200K .......... .......... .......... .......... .......... 58% 112M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54250K .......... .......... .......... .......... .......... 58% 46.5M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54300K .......... .......... .......... .......... .......... 58% 115M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54350K .......... .......... .......... .......... .......... 58% 49.5M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54400K .......... .......... .......... .......... .......... 58% 49.6M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54450K .......... .......... .......... .......... .......... 58% 89.4M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54500K .......... .......... .......... .......... .......... 58% 117M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54550K .......... .......... .......... .......... .......... 58% 60.1M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54600K .......... .......... .......... .......... .......... 58% 59.4M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54650K .......... .......... .......... .......... .......... 58% 115M 0s 2024-04-17T04:57:20.718 INFO:teuthology.orchestra.run.smithi088.stderr: 54700K .......... .......... .......... .......... .......... 58% 34.5M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 54750K .......... .......... .......... .......... .......... 58% 54.1M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 54800K .......... .......... .......... .......... .......... 59% 43.6M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 54850K .......... .......... .......... .......... .......... 59% 60.1M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 54900K .......... .......... .......... .......... .......... 59% 109M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 54950K .......... .......... .......... .......... .......... 59% 47.6M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55000K .......... .......... .......... .......... .......... 59% 40.3M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55050K .......... .......... .......... .......... .......... 59% 51.0M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55100K .......... .......... .......... .......... .......... 59% 29.6M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55150K .......... .......... .......... .......... .......... 59% 120M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55200K .......... .......... .......... .......... .......... 59% 113M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55250K .......... .......... .......... .......... .......... 59% 32.6M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55300K .......... .......... .......... .......... .......... 59% 117M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55350K .......... .......... .......... .......... .......... 59% 50.9M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55400K .......... .......... .......... .......... .......... 59% 111M 0s 2024-04-17T04:57:20.719 INFO:teuthology.orchestra.run.smithi088.stderr: 55450K .......... .......... .......... .......... .......... 59% 54.2M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55500K .......... .......... .......... .......... .......... 59% 108M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55550K .......... .......... .......... .......... .......... 59% 43.5M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55600K .......... .......... .......... .......... .......... 59% 134M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55650K .......... .......... .......... .......... .......... 59% 58.6M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55700K .......... .......... .......... .......... .......... 60% 37.4M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55750K .......... .......... .......... .......... .......... 60% 62.3M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55800K .......... .......... .......... .......... .......... 60% 73.0M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55850K .......... .......... .......... .......... .......... 60% 109M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55900K .......... .......... .......... .......... .......... 60% 26.8M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 55950K .......... .......... .......... .......... .......... 60% 66.3M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 56000K .......... .......... .......... .......... .......... 60% 61.4M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 56050K .......... .......... .......... .......... .......... 60% 23.4M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 56100K .......... .......... .......... .......... .......... 60% 130M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 56150K .......... .......... .......... .......... .......... 60% 48.6M 0s 2024-04-17T04:57:20.720 INFO:teuthology.orchestra.run.smithi088.stderr: 56200K .......... .......... .......... .......... .......... 60% 26.3M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56250K .......... .......... .......... .......... .......... 60% 62.8M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56300K .......... .......... .......... .......... .......... 60% 22.9M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56350K .......... .......... .......... .......... .......... 60% 140M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56400K .......... .......... .......... .......... .......... 60% 36.2M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56450K .......... .......... .......... .......... .......... 60% 24.2M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56500K .......... .......... .......... .......... .......... 60% 48.5M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56550K .......... .......... .......... .......... .......... 60% 126M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56600K .......... .......... .......... .......... .......... 60% 30.8M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56650K .......... .......... .......... .......... .......... 61% 48.4M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56700K .......... .......... .......... .......... .......... 61% 115M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56750K .......... .......... .......... .......... .......... 61% 114M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56800K .......... .......... .......... .......... .......... 61% 68.2M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56850K .......... .......... .......... .......... .......... 61% 59.2M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56900K .......... .......... .......... .......... .......... 61% 50.5M 0s 2024-04-17T04:57:20.721 INFO:teuthology.orchestra.run.smithi088.stderr: 56950K .......... .......... .......... .......... .......... 61% 117M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57000K .......... .......... .......... .......... .......... 61% 117M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57050K .......... .......... .......... .......... .......... 61% 49.8M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57100K .......... .......... .......... .......... .......... 61% 44.6M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57150K .......... .......... .......... .......... .......... 61% 69.4M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57200K .......... .......... .......... .......... .......... 61% 83.5M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57250K .......... .......... .......... .......... .......... 61% 108M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57300K .......... .......... .......... .......... .......... 61% 41.5M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57350K .......... .......... .......... .......... .......... 61% 99.3M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57400K .......... .......... .......... .......... .......... 61% 25.6M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57450K .......... .......... .......... .......... .......... 61% 84.9M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57500K .......... .......... .......... .......... .......... 61% 41.7M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57550K .......... .......... .......... .......... .......... 62% 116M 0s 2024-04-17T04:57:20.722 INFO:teuthology.orchestra.run.smithi088.stderr: 57600K .......... .......... .......... .......... .......... 62% 49.5M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57650K .......... .......... .......... .......... .......... 62% 31.6M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57700K .......... .......... .......... .......... .......... 62% 118M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57750K .......... .......... .......... .......... .......... 62% 38.1M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57800K .......... .......... .......... .......... .......... 62% 25.2M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57850K .......... .......... .......... .......... .......... 62% 26.8M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57900K .......... .......... .......... .......... .......... 62% 77.1M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 57950K .......... .......... .......... .......... .......... 62% 119M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58000K .......... .......... .......... .......... .......... 62% 58.4M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58050K .......... .......... .......... .......... .......... 62% 31.5M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58100K .......... .......... .......... .......... .......... 62% 43.7M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58150K .......... .......... .......... .......... .......... 62% 130M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58200K .......... .......... .......... .......... .......... 62% 43.6M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58250K .......... .......... .......... .......... .......... 62% 69.2M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58300K .......... .......... .......... .......... .......... 62% 35.0M 0s 2024-04-17T04:57:20.723 INFO:teuthology.orchestra.run.smithi088.stderr: 58350K .......... .......... .......... .......... .......... 62% 30.1M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58400K .......... .......... .......... .......... .......... 62% 144M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58450K .......... .......... .......... .......... .......... 62% 801K 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58500K .......... .......... .......... .......... .......... 63% 145M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58550K .......... .......... .......... .......... .......... 63% 193M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58600K .......... .......... .......... .......... .......... 63% 49.0M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58650K .......... .......... .......... .......... .......... 63% 81.4M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58700K .......... .......... .......... .......... .......... 63% 56.8M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58750K .......... .......... .......... .......... .......... 63% 105M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58800K .......... .......... .......... .......... .......... 63% 155M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58850K .......... .......... .......... .......... .......... 63% 33.1M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58900K .......... .......... .......... .......... .......... 63% 46.9M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 58950K .......... .......... .......... .......... .......... 63% 102M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 59000K .......... .......... .......... .......... .......... 63% 149M 0s 2024-04-17T04:57:20.724 INFO:teuthology.orchestra.run.smithi088.stderr: 59050K .......... .......... .......... .......... .......... 63% 147M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59100K .......... .......... .......... .......... .......... 63% 162M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59150K .......... .......... .......... .......... .......... 63% 127M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59200K .......... .......... .......... .......... .......... 63% 155M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59250K .......... .......... .......... .......... .......... 63% 158M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59300K .......... .......... .......... .......... .......... 63% 161M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59350K .......... .......... .......... .......... .......... 63% 33.9M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59400K .......... .......... .......... .......... .......... 63% 152M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59450K .......... .......... .......... .......... .......... 64% 161M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59500K .......... .......... .......... .......... .......... 64% 162M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59550K .......... .......... .......... .......... .......... 64% 130M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59600K .......... .......... .......... .......... .......... 64% 155M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59650K .......... .......... .......... .......... .......... 64% 159M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59700K .......... .......... .......... .......... .......... 64% 158M 0s 2024-04-17T04:57:20.725 INFO:teuthology.orchestra.run.smithi088.stderr: 59750K .......... .......... .......... .......... .......... 64% 163M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 59800K .......... .......... .......... .......... .......... 64% 136M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 59850K .......... .......... .......... .......... .......... 64% 159M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 59900K .......... .......... .......... .......... .......... 64% 160M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 59950K .......... .......... .......... .......... .......... 64% 163M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60000K .......... .......... .......... .......... .......... 64% 140M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60050K .......... .......... .......... .......... .......... 64% 146M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60100K .......... .......... .......... .......... .......... 64% 160M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60150K .......... .......... .......... .......... .......... 64% 161M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60200K .......... .......... .......... .......... .......... 64% 143M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60250K .......... .......... .......... .......... .......... 64% 161M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60300K .......... .......... .......... .......... .......... 64% 146M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60350K .......... .......... .......... .......... .......... 65% 164M 0s 2024-04-17T04:57:20.726 INFO:teuthology.orchestra.run.smithi088.stderr: 60400K .......... .......... .......... .......... .......... 65% 159M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60450K .......... .......... .......... .......... .......... 65% 164M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60500K .......... .......... .......... .......... .......... 65% 132M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60550K .......... .......... .......... .......... .......... 65% 24.9M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60600K .......... .......... .......... .......... .......... 65% 77.2M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60650K .......... .......... .......... .......... .......... 65% 55.9M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60700K .......... .......... .......... .......... .......... 65% 120M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60750K .......... .......... .......... .......... .......... 65% 143M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60800K .......... .......... .......... .......... .......... 65% 160M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60850K .......... .......... .......... .......... .......... 65% 153M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60900K .......... .......... .......... .......... .......... 65% 105M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 60950K .......... .......... .......... .......... .......... 65% 227M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 61000K .......... .......... .......... .......... .......... 65% 56.2M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 61050K .......... .......... .......... .......... .......... 65% 76.2M 0s 2024-04-17T04:57:20.727 INFO:teuthology.orchestra.run.smithi088.stderr: 61100K .......... .......... .......... .......... .......... 65% 121M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61150K .......... .......... .......... .......... .......... 65% 235M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61200K .......... .......... .......... .......... .......... 65% 293M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61250K .......... .......... .......... .......... .......... 65% 289M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61300K .......... .......... .......... .......... .......... 66% 300M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61350K .......... .......... .......... .......... .......... 66% 17.9M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61400K .......... .......... .......... .......... .......... 66% 281M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61450K .......... .......... .......... .......... .......... 66% 237M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61500K .......... .......... .......... .......... .......... 66% 291M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61550K .......... .......... .......... .......... .......... 66% 5.47M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61600K .......... .......... .......... .......... .......... 66% 95.4M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61650K .......... .......... .......... .......... .......... 66% 39.9M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61700K .......... .......... .......... .......... .......... 66% 38.0M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61750K .......... .......... .......... .......... .......... 66% 112M 0s 2024-04-17T04:57:20.728 INFO:teuthology.orchestra.run.smithi088.stderr: 61800K .......... .......... .......... .......... .......... 66% 40.3M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 61850K .......... .......... .......... .......... .......... 66% 33.4M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 61900K .......... .......... .......... .......... .......... 66% 32.5M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 61950K .......... .......... .......... .......... .......... 66% 108M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62000K .......... .......... .......... .......... .......... 66% 29.7M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62050K .......... .......... .......... .......... .......... 66% 41.4M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62100K .......... .......... .......... .......... .......... 66% 57.9M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62150K .......... .......... .......... .......... .......... 66% 90.4M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62200K .......... .......... .......... .......... .......... 67% 40.3M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62250K .......... .......... .......... .......... .......... 67% 65.3M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62300K .......... .......... .......... .......... .......... 67% 40.6M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62350K .......... .......... .......... .......... .......... 67% 39.3M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62400K .......... .......... .......... .......... .......... 67% 83.9M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62450K .......... .......... .......... .......... .......... 67% 40.1M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62500K .......... .......... .......... .......... .......... 67% 55.7M 0s 2024-04-17T04:57:20.729 INFO:teuthology.orchestra.run.smithi088.stderr: 62550K .......... .......... .......... .......... .......... 67% 80.2M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62600K .......... .......... .......... .......... .......... 67% 34.1M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62650K .......... .......... .......... .......... .......... 67% 102M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62700K .......... .......... .......... .......... .......... 67% 40.2M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62750K .......... .......... .......... .......... .......... 67% 45.7M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62800K .......... .......... .......... .......... .......... 67% 44.1M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62850K .......... .......... .......... .......... .......... 67% 97.3M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62900K .......... .......... .......... .......... .......... 67% 85.5M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 62950K .......... .......... .......... .......... .......... 67% 42.9M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 63000K .......... .......... .......... .......... .......... 67% 44.9M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 63050K .......... .......... .......... .......... .......... 67% 34.9M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 63100K .......... .......... .......... .......... .......... 67% 77.1M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 63150K .......... .......... .......... .......... .......... 68% 36.5M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 63200K .......... .......... .......... .......... .......... 68% 46.3M 0s 2024-04-17T04:57:20.730 INFO:teuthology.orchestra.run.smithi088.stderr: 63250K .......... .......... .......... .......... .......... 68% 36.7M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63300K .......... .......... .......... .......... .......... 68% 101M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63350K .......... .......... .......... .......... .......... 68% 67.7M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63400K .......... .......... .......... .......... .......... 68% 36.2M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63450K .......... .......... .......... .......... .......... 68% 35.7M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63500K .......... .......... .......... .......... .......... 68% 35.1M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63550K .......... .......... .......... .......... .......... 68% 31.1M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63600K .......... .......... .......... .......... .......... 68% 80.1M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63650K .......... .......... .......... .......... .......... 68% 48.4M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63700K .......... .......... .......... .......... .......... 68% 43.8M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63750K .......... .......... .......... .......... .......... 68% 38.6M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63800K .......... .......... .......... .......... .......... 68% 89.3M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63850K .......... .......... .......... .......... .......... 68% 28.0M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63900K .......... .......... .......... .......... .......... 68% 38.2M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 63950K .......... .......... .......... .......... .......... 68% 59.8M 0s 2024-04-17T04:57:20.731 INFO:teuthology.orchestra.run.smithi088.stderr: 64000K .......... .......... .......... .......... .......... 68% 88.3M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64050K .......... .......... .......... .......... .......... 68% 57.7M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64100K .......... .......... .......... .......... .......... 69% 35.3M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64150K .......... .......... .......... .......... .......... 69% 40.9M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64200K .......... .......... .......... .......... .......... 69% 28.4M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64250K .......... .......... .......... .......... .......... 69% 87.9M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64300K .......... .......... .......... .......... .......... 69% 40.4M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64350K .......... .......... .......... .......... .......... 69% 27.5M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64400K .......... .......... .......... .......... .......... 69% 33.6M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64450K .......... .......... .......... .......... .......... 69% 28.9M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64500K .......... .......... .......... .......... .......... 69% 94.5M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64550K .......... .......... .......... .......... .......... 69% 18.5M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64600K .......... .......... .......... .......... .......... 69% 32.2M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64650K .......... .......... .......... .......... .......... 69% 53.8M 0s 2024-04-17T04:57:20.732 INFO:teuthology.orchestra.run.smithi088.stderr: 64700K .......... .......... .......... .......... .......... 69% 81.0M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 64750K .......... .......... .......... .......... .......... 69% 92.9M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 64800K .......... .......... .......... .......... .......... 69% 68.1M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 64850K .......... .......... .......... .......... .......... 69% 93.0M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 64900K .......... .......... .......... .......... .......... 69% 80.7M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 64950K .......... .......... .......... .......... .......... 69% 85.6M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65000K .......... .......... .......... .......... .......... 70% 85.4M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65050K .......... .......... .......... .......... .......... 70% 84.0M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65100K .......... .......... .......... .......... .......... 70% 69.7M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65150K .......... .......... .......... .......... .......... 70% 82.1M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65200K .......... .......... .......... .......... .......... 70% 88.0M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65250K .......... .......... .......... .......... .......... 70% 85.3M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65300K .......... .......... .......... .......... .......... 70% 71.3M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65350K .......... .......... .......... .......... .......... 70% 88.1M 0s 2024-04-17T04:57:20.733 INFO:teuthology.orchestra.run.smithi088.stderr: 65400K .......... .......... .......... .......... .......... 70% 64.3M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65450K .......... .......... .......... .......... .......... 70% 94.3M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65500K .......... .......... .......... .......... .......... 70% 93.0M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65550K .......... .......... .......... .......... .......... 70% 101M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65600K .......... .......... .......... .......... .......... 70% 99.9M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65650K .......... .......... .......... .......... .......... 70% 113M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65700K .......... .......... .......... .......... .......... 70% 73.9M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65750K .......... .......... .......... .......... .......... 70% 62.8M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65800K .......... .......... .......... .......... .......... 70% 42.3M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65850K .......... .......... .......... .......... .......... 70% 47.0M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65900K .......... .......... .......... .......... .......... 70% 92.7M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 65950K .......... .......... .......... .......... .......... 71% 69.8M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 66000K .......... .......... .......... .......... .......... 71% 49.1M 0s 2024-04-17T04:57:20.734 INFO:teuthology.orchestra.run.smithi088.stderr: 66050K .......... .......... .......... .......... .......... 71% 69.5M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66100K .......... .......... .......... .......... .......... 71% 63.5M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66150K .......... .......... .......... .......... .......... 71% 81.5M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66200K .......... .......... .......... .......... .......... 71% 34.5M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66250K .......... .......... .......... .......... .......... 71% 50.1M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66300K .......... .......... .......... .......... .......... 71% 28.4M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66350K .......... .......... .......... .......... .......... 71% 87.2M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66400K .......... .......... .......... .......... .......... 71% 22.9M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66450K .......... .......... .......... .......... .......... 71% 32.1M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66500K .......... .......... .......... .......... .......... 71% 19.5M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66550K .......... .......... .......... .......... .......... 71% 55.0M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66600K .......... .......... .......... .......... .......... 71% 25.0M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66650K .......... .......... .......... .......... .......... 71% 89.7M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66700K .......... .......... .......... .......... .......... 71% 35.5M 0s 2024-04-17T04:57:20.735 INFO:teuthology.orchestra.run.smithi088.stderr: 66750K .......... .......... .......... .......... .......... 71% 28.9M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 66800K .......... .......... .......... .......... .......... 71% 34.5M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 66850K .......... .......... .......... .......... .......... 72% 71.8M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 66900K .......... .......... .......... .......... .......... 72% 29.4M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 66950K .......... .......... .......... .......... .......... 72% 40.9M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67000K .......... .......... .......... .......... .......... 72% 46.6M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67050K .......... .......... .......... .......... .......... 72% 69.6M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67100K .......... .......... .......... .......... .......... 72% 86.5M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67150K .......... .......... .......... .......... .......... 72% 38.7M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67200K .......... .......... .......... .......... .......... 72% 68.5M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67250K .......... .......... .......... .......... .......... 72% 42.1M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67300K .......... .......... .......... .......... .......... 72% 81.4M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67350K .......... .......... .......... .......... .......... 72% 124M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67400K .......... .......... .......... .......... .......... 72% 68.4M 0s 2024-04-17T04:57:20.736 INFO:teuthology.orchestra.run.smithi088.stderr: 67450K .......... .......... .......... .......... .......... 72% 80.4M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67500K .......... .......... .......... .......... .......... 72% 88.1M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67550K .......... .......... .......... .......... .......... 72% 76.3M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67600K .......... .......... .......... .......... .......... 72% 81.1M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67650K .......... .......... .......... .......... .......... 72% 91.1M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67700K .......... .......... .......... .......... .......... 72% 103M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67750K .......... .......... .......... .......... .......... 72% 51.7M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67800K .......... .......... .......... .......... .......... 73% 63.3M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67850K .......... .......... .......... .......... .......... 73% 53.3M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67900K .......... .......... .......... .......... .......... 73% 103M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 67950K .......... .......... .......... .......... .......... 73% 75.8M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 68000K .......... .......... .......... .......... .......... 73% 111M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 68050K .......... .......... .......... .......... .......... 73% 89.5M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 68100K .......... .......... .......... .......... .......... 73% 93.2M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 68150K .......... .......... .......... .......... .......... 73% 110M 0s 2024-04-17T04:57:20.737 INFO:teuthology.orchestra.run.smithi088.stderr: 68200K .......... .......... .......... .......... .......... 73% 108M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68250K .......... .......... .......... .......... .......... 73% 91.4M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68300K .......... .......... .......... .......... .......... 73% 66.5M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68350K .......... .......... .......... .......... .......... 73% 81.2M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68400K .......... .......... .......... .......... .......... 73% 63.9M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68450K .......... .......... .......... .......... .......... 73% 76.3M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68500K .......... .......... .......... .......... .......... 73% 86.4M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68550K .......... .......... .......... .......... .......... 73% 81.2M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68600K .......... .......... .......... .......... .......... 73% 87.1M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68650K .......... .......... .......... .......... .......... 73% 46.6M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68700K .......... .......... .......... .......... .......... 74% 100M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68750K .......... .......... .......... .......... .......... 74% 71.1M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68800K .......... .......... .......... .......... .......... 74% 72.7M 0s 2024-04-17T04:57:20.738 INFO:teuthology.orchestra.run.smithi088.stderr: 68850K .......... .......... .......... .......... .......... 74% 60.9M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 68900K .......... .......... .......... .......... .......... 74% 60.8M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 68950K .......... .......... .......... .......... .......... 74% 106M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69000K .......... .......... .......... .......... .......... 74% 48.4M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69050K .......... .......... .......... .......... .......... 74% 49.3M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69100K .......... .......... .......... .......... .......... 74% 40.2M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69150K .......... .......... .......... .......... .......... 74% 40.3M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69200K .......... .......... .......... .......... .......... 74% 83.9M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69250K .......... .......... .......... .......... .......... 74% 67.7M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69300K .......... .......... .......... .......... .......... 74% 38.7M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69350K .......... .......... .......... .......... .......... 74% 56.2M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69400K .......... .......... .......... .......... .......... 74% 77.3M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69450K .......... .......... .......... .......... .......... 74% 87.3M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69500K .......... .......... .......... .......... .......... 74% 72.4M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69550K .......... .......... .......... .......... .......... 74% 33.6M 0s 2024-04-17T04:57:20.739 INFO:teuthology.orchestra.run.smithi088.stderr: 69600K .......... .......... .......... .......... .......... 74% 76.0M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69650K .......... .......... .......... .......... .......... 75% 84.5M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69700K .......... .......... .......... .......... .......... 75% 62.1M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69750K .......... .......... .......... .......... .......... 75% 41.1M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69800K .......... .......... .......... .......... .......... 75% 56.3M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69850K .......... .......... .......... .......... .......... 75% 82.3M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69900K .......... .......... .......... .......... .......... 75% 84.8M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 69950K .......... .......... .......... .......... .......... 75% 67.2M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70000K .......... .......... .......... .......... .......... 75% 41.3M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70050K .......... .......... .......... .......... .......... 75% 86.6M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70100K .......... .......... .......... .......... .......... 75% 113M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70150K .......... .......... .......... .......... .......... 75% 43.8M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70200K .......... .......... .......... .......... .......... 75% 41.2M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70250K .......... .......... .......... .......... .......... 75% 79.3M 0s 2024-04-17T04:57:20.740 INFO:teuthology.orchestra.run.smithi088.stderr: 70300K .......... .......... .......... .......... .......... 75% 117M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70350K .......... .......... .......... .......... .......... 75% 96.4M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70400K .......... .......... .......... .......... .......... 75% 77.3M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70450K .......... .......... .......... .......... .......... 75% 66.9M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70500K .......... .......... .......... .......... .......... 75% 66.2M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70550K .......... .......... .......... .......... .......... 75% 99.9M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70600K .......... .......... .......... .......... .......... 76% 104M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70650K .......... .......... .......... .......... .......... 76% 72.8M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70700K .......... .......... .......... .......... .......... 76% 75.4M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70750K .......... .......... .......... .......... .......... 76% 86.9M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70800K .......... .......... .......... .......... .......... 76% 82.5M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70850K .......... .......... .......... .......... .......... 76% 84.4M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70900K .......... .......... .......... .......... .......... 76% 72.9M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 70950K .......... .......... .......... .......... .......... 76% 80.9M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 71000K .......... .......... .......... .......... .......... 76% 77.9M 0s 2024-04-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi088.stderr: 71050K .......... .......... .......... .......... .......... 76% 91.0M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71100K .......... .......... .......... .......... .......... 76% 78.6M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71150K .......... .......... .......... .......... .......... 76% 73.3M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71200K .......... .......... .......... .......... .......... 76% 86.2M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71250K .......... .......... .......... .......... .......... 76% 65.7M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71300K .......... .......... .......... .......... .......... 76% 102M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71350K .......... .......... .......... .......... .......... 76% 75.5M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71400K .......... .......... .......... .......... .......... 76% 97.8M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71450K .......... .......... .......... .......... .......... 76% 76.5M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71500K .......... .......... .......... .......... .......... 77% 101M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71550K .......... .......... .......... .......... .......... 77% 67.5M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71600K .......... .......... .......... .......... .......... 77% 72.7M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71650K .......... .......... .......... .......... .......... 77% 66.3M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71700K .......... .......... .......... .......... .......... 77% 68.0M 0s 2024-04-17T04:57:20.742 INFO:teuthology.orchestra.run.smithi088.stderr: 71750K .......... .......... .......... .......... .......... 77% 92.3M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 71800K .......... .......... .......... .......... .......... 77% 67.8M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 71850K .......... .......... .......... .......... .......... 77% 73.5M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 71900K .......... .......... .......... .......... .......... 77% 78.7M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 71950K .......... .......... .......... .......... .......... 77% 54.5M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72000K .......... .......... .......... .......... .......... 77% 92.6M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72050K .......... .......... .......... .......... .......... 77% 60.5M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72100K .......... .......... .......... .......... .......... 77% 69.2M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72150K .......... .......... .......... .......... .......... 77% 80.8M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72200K .......... .......... .......... .......... .......... 77% 94.3M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72250K .......... .......... .......... .......... .......... 77% 88.8M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72300K .......... .......... .......... .......... .......... 77% 112M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72350K .......... .......... .......... .......... .......... 77% 78.7M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72400K .......... .......... .......... .......... .......... 77% 60.9M 0s 2024-04-17T04:57:20.743 INFO:teuthology.orchestra.run.smithi088.stderr: 72450K .......... .......... .......... .......... .......... 78% 107M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72500K .......... .......... .......... .......... .......... 78% 65.6M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72550K .......... .......... .......... .......... .......... 78% 81.4M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72600K .......... .......... .......... .......... .......... 78% 56.2M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72650K .......... .......... .......... .......... .......... 78% 78.3M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72700K .......... .......... .......... .......... .......... 78% 99.1M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72750K .......... .......... .......... .......... .......... 78% 46.3M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72800K .......... .......... .......... .......... .......... 78% 48.3M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72850K .......... .......... .......... .......... .......... 78% 57.1M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72900K .......... .......... .......... .......... .......... 78% 89.5M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 72950K .......... .......... .......... .......... .......... 78% 105M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 73000K .......... .......... .......... .......... .......... 78% 67.9M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 73050K .......... .......... .......... .......... .......... 78% 80.3M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 73100K .......... .......... .......... .......... .......... 78% 59.0M 0s 2024-04-17T04:57:20.744 INFO:teuthology.orchestra.run.smithi088.stderr: 73150K .......... .......... .......... .......... .......... 78% 99.6M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73200K .......... .......... .......... .......... .......... 78% 81.7M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73250K .......... .......... .......... .......... .......... 78% 64.9M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73300K .......... .......... .......... .......... .......... 78% 70.1M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73350K .......... .......... .......... .......... .......... 79% 56.8M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73400K .......... .......... .......... .......... .......... 79% 99.4M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73450K .......... .......... .......... .......... .......... 79% 98.1M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73500K .......... .......... .......... .......... .......... 79% 48.1M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73550K .......... .......... .......... .......... .......... 79% 45.5M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73600K .......... .......... .......... .......... .......... 79% 86.6M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73650K .......... .......... .......... .......... .......... 79% 118M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73700K .......... .......... .......... .......... .......... 79% 63.9M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73750K .......... .......... .......... .......... .......... 79% 73.8M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73800K .......... .......... .......... .......... .......... 79% 71.5M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73850K .......... .......... .......... .......... .......... 79% 91.3M 0s 2024-04-17T04:57:20.745 INFO:teuthology.orchestra.run.smithi088.stderr: 73900K .......... .......... .......... .......... .......... 79% 98.3M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 73950K .......... .......... .......... .......... .......... 79% 50.3M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74000K .......... .......... .......... .......... .......... 79% 93.3M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74050K .......... .......... .......... .......... .......... 79% 53.5M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74100K .......... .......... .......... .......... .......... 79% 92.7M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74150K .......... .......... .......... .......... .......... 79% 55.3M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74200K .......... .......... .......... .......... .......... 79% 55.1M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74250K .......... .......... .......... .......... .......... 79% 54.9M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74300K .......... .......... .......... .......... .......... 80% 105M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74350K .......... .......... .......... .......... .......... 80% 93.2M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74400K .......... .......... .......... .......... .......... 80% 37.8M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74450K .......... .......... .......... .......... .......... 80% 48.0M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74500K .......... .......... .......... .......... .......... 80% 58.9M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74550K .......... .......... .......... .......... .......... 80% 88.8M 0s 2024-04-17T04:57:20.746 INFO:teuthology.orchestra.run.smithi088.stderr: 74600K .......... .......... .......... .......... .......... 80% 52.2M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74650K .......... .......... .......... .......... .......... 80% 50.2M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74700K .......... .......... .......... .......... .......... 80% 48.3M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74750K .......... .......... .......... .......... .......... 80% 111M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74800K .......... .......... .......... .......... .......... 80% 49.9M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74850K .......... .......... .......... .......... .......... 80% 93.1M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74900K .......... .......... .......... .......... .......... 80% 43.6M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 74950K .......... .......... .......... .......... .......... 80% 50.5M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 75000K .......... .......... .......... .......... .......... 80% 102M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 75050K .......... .......... .......... .......... .......... 80% 45.7M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 75100K .......... .......... .......... .......... .......... 80% 53.6M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 75150K .......... .......... .......... .......... .......... 80% 27.3M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 75200K .......... .......... .......... .......... .......... 81% 48.8M 0s 2024-04-17T04:57:20.747 INFO:teuthology.orchestra.run.smithi088.stderr: 75250K .......... .......... .......... .......... .......... 81% 94.1M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75300K .......... .......... .......... .......... .......... 81% 41.6M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75350K .......... .......... .......... .......... .......... 81% 48.1M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75400K .......... .......... .......... .......... .......... 81% 30.3M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75450K .......... .......... .......... .......... .......... 81% 105M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75500K .......... .......... .......... .......... .......... 81% 27.2M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75550K .......... .......... .......... .......... .......... 81% 32.2M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75600K .......... .......... .......... .......... .......... 81% 46.4M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75650K .......... .......... .......... .......... .......... 81% 40.9M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75700K .......... .......... .......... .......... .......... 81% 108M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75750K .......... .......... .......... .......... .......... 81% 48.8M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75800K .......... .......... .......... .......... .......... 81% 27.3M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75850K .......... .......... .......... .......... .......... 81% 51.5M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75900K .......... .......... .......... .......... .......... 81% 119M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 75950K .......... .......... .......... .......... .......... 81% 26.5M 0s 2024-04-17T04:57:20.748 INFO:teuthology.orchestra.run.smithi088.stderr: 76000K .......... .......... .......... .......... .......... 81% 51.0M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76050K .......... .......... .......... .......... .......... 81% 45.3M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76100K .......... .......... .......... .......... .......... 81% 49.1M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76150K .......... .......... .......... .......... .......... 82% 85.4M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76200K .......... .......... .......... .......... .......... 82% 23.9M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76250K .......... .......... .......... .......... .......... 82% 104M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76300K .......... .......... .......... .......... .......... 82% 39.7M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76350K .......... .......... .......... .......... .......... 82% 24.8M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76400K .......... .......... .......... .......... .......... 82% 49.7M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76450K .......... .......... .......... .......... .......... 82% 95.0M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76500K .......... .......... .......... .......... .......... 82% 17.0M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76550K .......... .......... .......... .......... .......... 82% 56.8M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76600K .......... .......... .......... .......... .......... 82% 29.3M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76650K .......... .......... .......... .......... .......... 82% 90.8M 0s 2024-04-17T04:57:20.749 INFO:teuthology.orchestra.run.smithi088.stderr: 76700K .......... .......... .......... .......... .......... 82% 36.0M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 76750K .......... .......... .......... .......... .......... 82% 29.9M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 76800K .......... .......... .......... .......... .......... 82% 47.0M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 76850K .......... .......... .......... .......... .......... 82% 94.7M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 76900K .......... .......... .......... .......... .......... 82% 28.1M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 76950K .......... .......... .......... .......... .......... 82% 48.2M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77000K .......... .......... .......... .......... .......... 82% 39.3M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77050K .......... .......... .......... .......... .......... 82% 20.4M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77100K .......... .......... .......... .......... .......... 83% 74.5M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77150K .......... .......... .......... .......... .......... 83% 94.8M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77200K .......... .......... .......... .......... .......... 83% 42.2M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77250K .......... .......... .......... .......... .......... 83% 59.6M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77300K .......... .......... .......... .......... .......... 83% 45.6M 0s 2024-04-17T04:57:20.750 INFO:teuthology.orchestra.run.smithi088.stderr: 77350K .......... .......... .......... .......... .......... 83% 108M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77400K .......... .......... .......... .......... .......... 83% 30.2M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77450K .......... .......... .......... .......... .......... 83% 1.21M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77500K .......... .......... .......... .......... .......... 83% 128M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77550K .......... .......... .......... .......... .......... 83% 143M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77600K .......... .......... .......... .......... .......... 83% 118M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77650K .......... .......... .......... .......... .......... 83% 135M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77700K .......... .......... .......... .......... .......... 83% 136M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77750K .......... .......... .......... .......... .......... 83% 133M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77800K .......... .......... .......... .......... .......... 83% 148M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77850K .......... .......... .......... .......... .......... 83% 135M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77900K .......... .......... .......... .......... .......... 83% 146M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 77950K .......... .......... .......... .......... .......... 83% 120M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 78000K .......... .......... .......... .......... .......... 84% 147M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 78050K .......... .......... .......... .......... .......... 84% 123M 0s 2024-04-17T04:57:20.751 INFO:teuthology.orchestra.run.smithi088.stderr: 78100K .......... .......... .......... .......... .......... 84% 141M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78150K .......... .......... .......... .......... .......... 84% 147M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78200K .......... .......... .......... .......... .......... 84% 141M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78250K .......... .......... .......... .......... .......... 84% 148M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78300K .......... .......... .......... .......... .......... 84% 126M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78350K .......... .......... .......... .......... .......... 84% 142M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78400K .......... .......... .......... .......... .......... 84% 148M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78450K .......... .......... .......... .......... .......... 84% 147M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78500K .......... .......... .......... .......... .......... 84% 147M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78550K .......... .......... .......... .......... .......... 84% 127M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78600K .......... .......... .......... .......... .......... 84% 142M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78650K .......... .......... .......... .......... .......... 84% 148M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78700K .......... .......... .......... .......... .......... 84% 146M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78750K .......... .......... .......... .......... .......... 84% 150M 0s 2024-04-17T04:57:20.752 INFO:teuthology.orchestra.run.smithi088.stderr: 78800K .......... .......... .......... .......... .......... 84% 120M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 78850K .......... .......... .......... .......... .......... 84% 147M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 78900K .......... .......... .......... .......... .......... 84% 147M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 78950K .......... .......... .......... .......... .......... 85% 145M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79000K .......... .......... .......... .......... .......... 85% 148M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79050K .......... .......... .......... .......... .......... 85% 119M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79100K .......... .......... .......... .......... .......... 85% 139M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79150K .......... .......... .......... .......... .......... 85% 151M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79200K .......... .......... .......... .......... .......... 85% 148M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79250K .......... .......... .......... .......... .......... 85% 147M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79300K .......... .......... .......... .......... .......... 85% 122M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79350K .......... .......... .......... .......... .......... 85% 142M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79400K .......... .......... .......... .......... .......... 85% 150M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79450K .......... .......... .......... .......... .......... 85% 131M 0s 2024-04-17T04:57:20.753 INFO:teuthology.orchestra.run.smithi088.stderr: 79500K .......... .......... .......... .......... .......... 85% 149M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79550K .......... .......... .......... .......... .......... 85% 151M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79600K .......... .......... .......... .......... .......... 85% 132M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79650K .......... .......... .......... .......... .......... 85% 149M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79700K .......... .......... .......... .......... .......... 85% 149M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79750K .......... .......... .......... .......... .......... 85% 149M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79800K .......... .......... .......... .......... .......... 85% 125M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79850K .......... .......... .......... .......... .......... 86% 150M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79900K .......... .......... .......... .......... .......... 86% 148M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 79950K .......... .......... .......... .......... .......... 86% 147M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 80000K .......... .......... .......... .......... .......... 86% 133M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 80050K .......... .......... .......... .......... .......... 86% 148M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 80100K .......... .......... .......... .......... .......... 86% 148M 0s 2024-04-17T04:57:20.754 INFO:teuthology.orchestra.run.smithi088.stderr: 80150K .......... .......... .......... .......... .......... 86% 148M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80200K .......... .......... .......... .......... .......... 86% 131M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80250K .......... .......... .......... .......... .......... 86% 142M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80300K .......... .......... .......... .......... .......... 86% 117M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80350K .......... .......... .......... .......... .......... 86% 128M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80400K .......... .......... .......... .......... .......... 86% 132M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80450K .......... .......... .......... .......... .......... 86% 106M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80500K .......... .......... .......... .......... .......... 86% 71.2M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80550K .......... .......... .......... .......... .......... 86% 147M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80600K .......... .......... .......... .......... .......... 86% 144M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80650K .......... .......... .......... .......... .......... 86% 134M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80700K .......... .......... .......... .......... .......... 86% 131M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80750K .......... .......... .......... .......... .......... 86% 88.4M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80800K .......... .......... .......... .......... .......... 87% 48.5M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80850K .......... .......... .......... .......... .......... 87% 41.8M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80900K .......... .......... .......... .......... .......... 87% 60.2M 0s 2024-04-17T04:57:20.755 INFO:teuthology.orchestra.run.smithi088.stderr: 80950K .......... .......... .......... .......... .......... 87% 45.1M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81000K .......... .......... .......... .......... .......... 87% 72.9M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81050K .......... .......... .......... .......... .......... 87% 119M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81100K .......... .......... .......... .......... .......... 87% 92.7M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81150K .......... .......... .......... .......... .......... 87% 86.6M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81200K .......... .......... .......... .......... .......... 87% 106M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81250K .......... .......... .......... .......... .......... 87% 110M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81300K .......... .......... .......... .......... .......... 87% 114M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81350K .......... .......... .......... .......... .......... 87% 114M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81400K .......... .......... .......... .......... .......... 87% 99.3M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81450K .......... .......... .......... .......... .......... 87% 64.1M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81500K .......... .......... .......... .......... .......... 87% 52.0M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81550K .......... .......... .......... .......... .......... 87% 52.2M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81600K .......... .......... .......... .......... .......... 87% 110M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81650K .......... .......... .......... .......... .......... 87% 51.4M 0s 2024-04-17T04:57:20.756 INFO:teuthology.orchestra.run.smithi088.stderr: 81700K .......... .......... .......... .......... .......... 87% 64.2M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 81750K .......... .......... .......... .......... .......... 88% 118M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 81800K .......... .......... .......... .......... .......... 88% 94.6M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 81850K .......... .......... .......... .......... .......... 88% 58.1M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 81900K .......... .......... .......... .......... .......... 88% 133M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 81950K .......... .......... .......... .......... .......... 88% 133M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82000K .......... .......... .......... .......... .......... 88% 115M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82050K .......... .......... .......... .......... .......... 88% 92.4M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82100K .......... .......... .......... .......... .......... 88% 137M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82150K .......... .......... .......... .......... .......... 88% 83.0M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82200K .......... .......... .......... .......... .......... 88% 133M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82250K .......... .......... .......... .......... .......... 88% 130M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82300K .......... .......... .......... .......... .......... 88% 101M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82350K .......... .......... .......... .......... .......... 88% 112M 0s 2024-04-17T04:57:20.757 INFO:teuthology.orchestra.run.smithi088.stderr: 82400K .......... .......... .......... .......... .......... 88% 103M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82450K .......... .......... .......... .......... .......... 88% 127M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82500K .......... .......... .......... .......... .......... 88% 108M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82550K .......... .......... .......... .......... .......... 88% 92.0M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82600K .......... .......... .......... .......... .......... 88% 115M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82650K .......... .......... .......... .......... .......... 89% 120M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82700K .......... .......... .......... .......... .......... 89% 74.6M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82750K .......... .......... .......... .......... .......... 89% 131M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82800K .......... .......... .......... .......... .......... 89% 98.3M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82850K .......... .......... .......... .......... .......... 89% 123M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82900K .......... .......... .......... .......... .......... 89% 94.8M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 82950K .......... .......... .......... .......... .......... 89% 128M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 83000K .......... .......... .......... .......... .......... 89% 64.6M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 83050K .......... .......... .......... .......... .......... 89% 104M 0s 2024-04-17T04:57:20.758 INFO:teuthology.orchestra.run.smithi088.stderr: 83100K .......... .......... .......... .......... .......... 89% 88.4M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83150K .......... .......... .......... .......... .......... 89% 133M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83200K .......... .......... .......... .......... .......... 89% 101M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83250K .......... .......... .......... .......... .......... 89% 125M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83300K .......... .......... .......... .......... .......... 89% 108M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83350K .......... .......... .......... .......... .......... 89% 111M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83400K .......... .......... .......... .......... .......... 89% 115M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83450K .......... .......... .......... .......... .......... 89% 111M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83500K .......... .......... .......... .......... .......... 89% 91.1M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83550K .......... .......... .......... .......... .......... 89% 112M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83600K .......... .......... .......... .......... .......... 90% 110M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83650K .......... .......... .......... .......... .......... 90% 113M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83700K .......... .......... .......... .......... .......... 90% 98.1M 0s 2024-04-17T04:57:20.759 INFO:teuthology.orchestra.run.smithi088.stderr: 83750K .......... .......... .......... .......... .......... 90% 97.1M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 83800K .......... .......... .......... .......... .......... 90% 120M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 83850K .......... .......... .......... .......... .......... 90% 112M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 83900K .......... .......... .......... .......... .......... 90% 117M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 83950K .......... .......... .......... .......... .......... 90% 93.9M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84000K .......... .......... .......... .......... .......... 90% 120M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84050K .......... .......... .......... .......... .......... 90% 97.3M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84100K .......... .......... .......... .......... .......... 90% 131M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84150K .......... .......... .......... .......... .......... 90% 74.9M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84200K .......... .......... .......... .......... .......... 90% 58.9M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84250K .......... .......... .......... .......... .......... 90% 119M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84300K .......... .......... .......... .......... .......... 90% 113M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84350K .......... .......... .......... .......... .......... 90% 103M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84400K .......... .......... .......... .......... .......... 90% 108M 0s 2024-04-17T04:57:20.760 INFO:teuthology.orchestra.run.smithi088.stderr: 84450K .......... .......... .......... .......... .......... 90% 101M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84500K .......... .......... .......... .......... .......... 91% 112M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84550K .......... .......... .......... .......... .......... 91% 123M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84600K .......... .......... .......... .......... .......... 91% 104M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84650K .......... .......... .......... .......... .......... 91% 98.2M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84700K .......... .......... .......... .......... .......... 91% 101M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84750K .......... .......... .......... .......... .......... 91% 128M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84800K .......... .......... .......... .......... .......... 91% 109M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84850K .......... .......... .......... .......... .......... 91% 101M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84900K .......... .......... .......... .......... .......... 91% 86.5M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 84950K .......... .......... .......... .......... .......... 91% 133M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 85000K .......... .......... .......... .......... .......... 91% 120M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 85050K .......... .......... .......... .......... .......... 91% 107M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 85100K .......... .......... .......... .......... .......... 91% 105M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 85150K .......... .......... .......... .......... .......... 91% 103M 0s 2024-04-17T04:57:20.761 INFO:teuthology.orchestra.run.smithi088.stderr: 85200K .......... .......... .......... .......... .......... 91% 105M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85250K .......... .......... .......... .......... .......... 91% 117M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85300K .......... .......... .......... .......... .......... 91% 101M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85350K .......... .......... .......... .......... .......... 91% 95.9M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85400K .......... .......... .......... .......... .......... 91% 111M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85450K .......... .......... .......... .......... .......... 92% 86.8M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85500K .......... .......... .......... .......... .......... 92% 128M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85550K .......... .......... .......... .......... .......... 92% 101M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85600K .......... .......... .......... .......... .......... 92% 110M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85650K .......... .......... .......... .......... .......... 92% 108M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85700K .......... .......... .......... .......... .......... 92% 107M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85750K .......... .......... .......... .......... .......... 92% 113M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85800K .......... .......... .......... .......... .......... 92% 104M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85850K .......... .......... .......... .......... .......... 92% 105M 0s 2024-04-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi088.stderr: 85900K .......... .......... .......... .......... .......... 92% 113M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 85950K .......... .......... .......... .......... .......... 92% 116M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86000K .......... .......... .......... .......... .......... 92% 83.6M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86050K .......... .......... .......... .......... .......... 92% 114M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86100K .......... .......... .......... .......... .......... 92% 103M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86150K .......... .......... .......... .......... .......... 92% 114M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86200K .......... .......... .......... .......... .......... 92% 106M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86250K .......... .......... .......... .......... .......... 92% 113M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86300K .......... .......... .......... .......... .......... 92% 115M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86350K .......... .......... .......... .......... .......... 93% 101M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86400K .......... .......... .......... .......... .......... 93% 94.7M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86450K .......... .......... .......... .......... .......... 93% 93.7M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86500K .......... .......... .......... .......... .......... 93% 123M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86550K .......... .......... .......... .......... .......... 93% 116M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86600K .......... .......... .......... .......... .......... 93% 90.3M 0s 2024-04-17T04:57:20.763 INFO:teuthology.orchestra.run.smithi088.stderr: 86650K .......... .......... .......... .......... .......... 93% 109M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 86700K .......... .......... .......... .......... .......... 93% 108M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 86750K .......... .......... .......... .......... .......... 93% 112M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 86800K .......... .......... .......... .......... .......... 93% 89.3M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 86850K .......... .......... .......... .......... .......... 93% 112M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 86900K .......... .......... .......... .......... .......... 93% 111M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 86950K .......... .......... .......... .......... .......... 93% 117M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87000K .......... .......... .......... .......... .......... 93% 81.0M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87050K .......... .......... .......... .......... .......... 93% 129M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87100K .......... .......... .......... .......... .......... 93% 100M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87150K .......... .......... .......... .......... .......... 93% 127M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87200K .......... .......... .......... .......... .......... 93% 74.3M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87250K .......... .......... .......... .......... .......... 93% 126M 0s 2024-04-17T04:57:20.764 INFO:teuthology.orchestra.run.smithi088.stderr: 87300K .......... .......... .......... .......... .......... 94% 97.6M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87350K .......... .......... .......... .......... .......... 94% 127M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87400K .......... .......... .......... .......... .......... 94% 104M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87450K .......... .......... .......... .......... .......... 94% 106M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87500K .......... .......... .......... .......... .......... 94% 119M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87550K .......... .......... .......... .......... .......... 94% 87.6M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87600K .......... .......... .......... .......... .......... 94% 121M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87650K .......... .......... .......... .......... .......... 94% 91.6M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87700K .......... .......... .......... .......... .......... 94% 127M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87750K .......... .......... .......... .......... .......... 94% 104M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87800K .......... .......... .......... .......... .......... 94% 114M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87850K .......... .......... .......... .......... .......... 94% 113M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87900K .......... .......... .......... .......... .......... 94% 103M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 87950K .......... .......... .......... .......... .......... 94% 89.3M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 88000K .......... .......... .......... .......... .......... 94% 85.7M 0s 2024-04-17T04:57:20.765 INFO:teuthology.orchestra.run.smithi088.stderr: 88050K .......... .......... .......... .......... .......... 94% 144M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88100K .......... .......... .......... .......... .......... 94% 116M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88150K .......... .......... .......... .......... .......... 94% 102M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88200K .......... .......... .......... .......... .......... 94% 106M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88250K .......... .......... .......... .......... .......... 95% 99.2M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88300K .......... .......... .......... .......... .......... 95% 123M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88350K .......... .......... .......... .......... .......... 95% 93.3M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88400K .......... .......... .......... .......... .......... 95% 137M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88450K .......... .......... .......... .......... .......... 95% 101M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88500K .......... .......... .......... .......... .......... 95% 87.8M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88550K .......... .......... .......... .......... .......... 95% 114M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88600K .......... .......... .......... .......... .......... 95% 111M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88650K .......... .......... .......... .......... .......... 95% 82.9M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88700K .......... .......... .......... .......... .......... 95% 80.3M 0s 2024-04-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi088.stderr: 88750K .......... .......... .......... .......... .......... 95% 87.5M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 88800K .......... .......... .......... .......... .......... 95% 97.1M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 88850K .......... .......... .......... .......... .......... 95% 131M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 88900K .......... .......... .......... .......... .......... 95% 109M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 88950K .......... .......... .......... .......... .......... 95% 100M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89000K .......... .......... .......... .......... .......... 95% 105M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89050K .......... .......... .......... .......... .......... 95% 131M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89100K .......... .......... .......... .......... .......... 95% 108M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89150K .......... .......... .......... .......... .......... 96% 93.8M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89200K .......... .......... .......... .......... .......... 96% 116M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89250K .......... .......... .......... .......... .......... 96% 118M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89300K .......... .......... .......... .......... .......... 96% 107M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89350K .......... .......... .......... .......... .......... 96% 118M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89400K .......... .......... .......... .......... .......... 96% 93.7M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89450K .......... .......... .......... .......... .......... 96% 109M 0s 2024-04-17T04:57:20.767 INFO:teuthology.orchestra.run.smithi088.stderr: 89500K .......... .......... .......... .......... .......... 96% 118M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89550K .......... .......... .......... .......... .......... 96% 96.9M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89600K .......... .......... .......... .......... .......... 96% 112M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89650K .......... .......... .......... .......... .......... 96% 108M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89700K .......... .......... .......... .......... .......... 96% 115M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89750K .......... .......... .......... .......... .......... 96% 110M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89800K .......... .......... .......... .......... .......... 96% 104M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89850K .......... .......... .......... .......... .......... 96% 112M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89900K .......... .......... .......... .......... .......... 96% 107M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 89950K .......... .......... .......... .......... .......... 96% 109M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 90000K .......... .......... .......... .......... .......... 96% 125M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 90050K .......... .......... .......... .......... .......... 96% 110M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 90100K .......... .......... .......... .......... .......... 97% 93.8M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 90150K .......... .......... .......... .......... .......... 97% 108M 0s 2024-04-17T04:57:20.768 INFO:teuthology.orchestra.run.smithi088.stderr: 90200K .......... .......... .......... .......... .......... 97% 110M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90250K .......... .......... .......... .......... .......... 97% 115M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90300K .......... .......... .......... .......... .......... 97% 99.7M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90350K .......... .......... .......... .......... .......... 97% 111M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90400K .......... .......... .......... .......... .......... 97% 114M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90450K .......... .......... .......... .......... .......... 97% 106M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90500K .......... .......... .......... .......... .......... 97% 78.2M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90550K .......... .......... .......... .......... .......... 97% 107M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90600K .......... .......... .......... .......... .......... 97% 89.3M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90650K .......... .......... .......... .......... .......... 97% 155M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90700K .......... .......... .......... .......... .......... 97% 84.8M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90750K .......... .......... .......... .......... .......... 97% 100M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90800K .......... .......... .......... .......... .......... 97% 113M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90850K .......... .......... .......... .......... .......... 97% 90.2M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90900K .......... .......... .......... .......... .......... 97% 132M 0s 2024-04-17T04:57:20.769 INFO:teuthology.orchestra.run.smithi088.stderr: 90950K .......... .......... .......... .......... .......... 97% 103M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91000K .......... .......... .......... .......... .......... 98% 85.0M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91050K .......... .......... .......... .......... .......... 98% 125M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91100K .......... .......... .......... .......... .......... 98% 97.8M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91150K .......... .......... .......... .......... .......... 98% 115M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91200K .......... .......... .......... .......... .......... 98% 80.9M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91250K .......... .......... .......... .......... .......... 98% 145M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91300K .......... .......... .......... .......... .......... 98% 102M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91350K .......... .......... .......... .......... .......... 98% 114M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91400K .......... .......... .......... .......... .......... 98% 95.4M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91450K .......... .......... .......... .......... .......... 98% 89.8M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91500K .......... .......... .......... .......... .......... 98% 107M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91550K .......... .......... .......... .......... .......... 98% 103M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91600K .......... .......... .......... .......... .......... 98% 117M 0s 2024-04-17T04:57:20.770 INFO:teuthology.orchestra.run.smithi088.stderr: 91650K .......... .......... .......... .......... .......... 98% 88.6M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 91700K .......... .......... .......... .......... .......... 98% 112M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 91750K .......... .......... .......... .......... .......... 98% 70.7M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 91800K .......... .......... .......... .......... .......... 98% 162M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 91850K .......... .......... .......... .......... .......... 98% 130M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 91900K .......... .......... .......... .......... .......... 98% 105M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 91950K .......... .......... .......... .......... .......... 99% 103M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92000K .......... .......... .......... .......... .......... 99% 102M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92050K .......... .......... .......... .......... .......... 99% 113M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92100K .......... .......... .......... .......... .......... 99% 80.1M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92150K .......... .......... .......... .......... .......... 99% 116M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92200K .......... .......... .......... .......... .......... 99% 102M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92250K .......... .......... .......... .......... .......... 99% 102M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92300K .......... .......... .......... .......... .......... 99% 109M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92350K .......... .......... .......... .......... .......... 99% 98.8M 0s 2024-04-17T04:57:20.771 INFO:teuthology.orchestra.run.smithi088.stderr: 92400K .......... .......... .......... .......... .......... 99% 104M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92450K .......... .......... .......... .......... .......... 99% 91.4M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92500K .......... .......... .......... .......... .......... 99% 121M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92550K .......... .......... .......... .......... .......... 99% 118M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92600K .......... .......... .......... .......... .......... 99% 102M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92650K .......... .......... .......... .......... .......... 99% 107M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92700K .......... .......... .......... .......... .......... 99% 108M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92750K .......... .......... .......... .......... .......... 99% 114M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92800K .......... .......... .......... .......... .......... 99% 95.0M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92850K .......... .......... .......... .......... .......... 99% 123M 0s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 92900K 100% 134G=1.2s 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17 04:57:20 (72.6 MB/s) - 'gradle-6.0.1-bin.zip' saved [95129672/95129672] 2024-04-17T04:57:20.772 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout:Archive: gradle-6.0.1-bin.zip 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-04-17T04:57:20.779 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-04-17T04:57:20.780 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-04-17T04:57:20.781 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-04-17T04:57:20.782 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-04-17T04:57:20.831 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-04-17T04:57:20.831 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-04-17T04:57:20.831 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-04-17T04:57:20.831 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-04-17T04:57:20.831 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-04-17T04:57:20.849 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-04-17T04:57:20.850 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-04-17T04:57:20.852 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-04-17T04:57:20.853 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-04-17T04:57:20.853 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-04-17T04:57:20.853 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-04-17T04:57:20.854 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-04-17T04:57:20.870 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-04-17T04:57:20.870 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-04-17T04:57:20.870 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-04-17T04:57:20.871 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-04-17T04:57:20.872 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-04-17T04:57:20.872 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-04-17T04:57:20.874 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-04-17T04:57:20.877 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-04-17T04:57:20.877 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-04-17T04:57:20.892 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-04-17T04:57:20.896 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-04-17T04:57:20.900 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-04-17T04:57:20.902 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-04-17T04:57:20.903 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-04-17T04:57:20.904 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-04-17T04:57:20.905 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-04-17T04:57:20.906 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-04-17T04:57:20.909 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-04-17T04:57:20.911 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-04-17T04:57:20.929 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-04-17T04:57:20.930 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-04-17T04:57:20.939 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-04-17T04:57:20.939 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-04-17T04:57:20.939 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-04-17T04:57:20.939 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-04-17T04:57:20.939 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-04-17T04:57:20.947 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-04-17T04:57:20.947 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-04-17T04:57:20.950 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-04-17T04:57:20.951 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-04-17T04:57:20.951 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-04-17T04:57:20.951 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-04-17T04:57:20.953 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-04-17T04:57:20.954 INFO:teuthology.orchestra.run.smithi088.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-04-17T04:57:20.954 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-04-17T04:57:20.954 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-04-17T04:57:20.954 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-04-17T04:57:20.955 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-04-17T04:57:20.955 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-04-17T04:57:20.956 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-04-17T04:57:20.959 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-04-17T04:57:20.960 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-04-17T04:57:20.960 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-04-17T04:57:20.962 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-04-17T04:57:20.962 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-04-17T04:57:20.965 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-04-17T04:57:20.965 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-04-17T04:57:20.966 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-04-17T04:57:20.967 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-04-17T04:57:20.968 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-04-17T04:57:20.969 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-04-17T04:57:20.969 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-04-17T04:57:20.971 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-04-17T04:57:20.972 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-04-17T04:57:20.974 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-04-17T04:57:20.974 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-04-17T04:57:20.975 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-04-17T04:57:20.975 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-04-17T04:57:20.976 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-04-17T04:57:20.976 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-04-17T04:57:20.977 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-04-17T04:57:20.979 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-04-17T04:57:20.979 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-04-17T04:57:20.980 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-04-17T04:57:20.984 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-04-17T04:57:20.986 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-04-17T04:57:20.986 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-04-17T04:57:20.988 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-04-17T04:57:20.989 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-04-17T04:57:20.990 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-04-17T04:57:20.996 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-04-17T04:57:20.996 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-04-17T04:57:21.011 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-04-17T04:57:21.011 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-04-17T04:57:21.012 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-04-17T04:57:21.012 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-04-17T04:57:21.012 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-04-17T04:57:21.012 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-04-17T04:57:21.013 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-04-17T04:57:21.013 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-04-17T04:57:21.013 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-04-17T04:57:21.014 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-04-17T04:57:21.033 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-04-17T04:57:21.033 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-04-17T04:57:21.033 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-04-17T04:57:21.034 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-04-17T04:57:21.035 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-04-17T04:57:21.036 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-04-17T04:57:21.044 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-04-17T04:57:21.046 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-04-17T04:57:21.050 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-04-17T04:57:21.050 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-04-17T04:57:21.051 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-04-17T04:57:21.051 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-04-17T04:57:21.052 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-04-17T04:57:21.052 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-04-17T04:57:21.054 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-04-17T04:57:21.063 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-04-17T04:57:21.065 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-04-17T04:57:21.067 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-04-17T04:57:21.067 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-04-17T04:57:21.067 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-04-17T04:57:21.068 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-04-17T04:57:21.068 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-04-17T04:57:21.070 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-04-17T04:57:21.074 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-04-17T04:57:21.076 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-04-17T04:57:21.076 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-04-17T04:57:21.079 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-04-17T04:57:21.082 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-04-17T04:57:21.082 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-04-17T04:57:21.089 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-04-17T04:57:21.092 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-04-17T04:57:21.099 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-04-17T04:57:21.104 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-04-17T04:57:21.107 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-04-17T04:57:21.108 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-04-17T04:57:21.117 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-04-17T04:57:21.119 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-04-17T04:57:21.126 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-04-17T04:57:21.126 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-04-17T04:57:21.134 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-04-17T04:57:21.136 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-04-17T04:57:21.137 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-04-17T04:57:21.141 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-04-17T04:57:21.146 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-04-17T04:57:21.175 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-04-17T04:57:21.177 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-04-17T04:57:21.177 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-04-17T04:57:21.178 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-04-17T04:57:21.179 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-04-17T04:57:21.179 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-04-17T04:57:21.179 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-04-17T04:57:21.180 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-04-17T04:57:21.182 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-04-17T04:57:21.182 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-04-17T04:57:21.183 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-04-17T04:57:21.183 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-04-17T04:57:21.184 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-04-17T04:57:21.186 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-04-17T04:57:21.186 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-04-17T04:57:21.186 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-04-17T04:57:21.187 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-04-17T04:57:21.187 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-04-17T04:57:21.187 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-04-17T04:57:21.187 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-04-17T04:57:21.188 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-04-17T04:57:21.188 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-04-17T04:57:21.188 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-04-17T04:57:21.190 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-04-17T04:57:21.190 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-04-17T04:57:21.194 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-04-17T04:57:21.195 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-04-17T04:57:21.196 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-04-17T04:57:21.196 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-04-17T04:57:21.196 INFO:teuthology.orchestra.run.smithi088.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-04-17T04:57:21.229 INFO:teuthology.orchestra.run.smithi088.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout:Welcome to Gradle 6.0.1! 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout:Here are the highlights of this release: 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: - Substantial improvements in dependency management, including 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: - Advanced control of transitive versions 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: - Support for optional features and dependencies 2024-04-17T04:57:21.647 INFO:teuthology.orchestra.run.smithi088.stdout: - Rules to tweak published metadata 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: - Support for Java 13 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: - Faster incremental Java and Groovy compilation 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: - New Zinc compiler for Scala 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: - VS2019 support 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: - Support for Gradle Enterprise plugin 3.0 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout:------------------------------------------------------------ 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout:Gradle 6.0.1 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout:------------------------------------------------------------ 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.648 INFO:teuthology.orchestra.run.smithi088.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-04-17T04:57:21.649 INFO:teuthology.orchestra.run.smithi088.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-04-17T04:57:21.649 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.649 INFO:teuthology.orchestra.run.smithi088.stdout:Kotlin: 1.3.50 2024-04-17T04:57:21.649 INFO:teuthology.orchestra.run.smithi088.stdout:Groovy: 2.5.8 2024-04-17T04:57:21.649 INFO:teuthology.orchestra.run.smithi088.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-04-17T04:57:21.650 INFO:teuthology.orchestra.run.smithi088.stdout:JVM: 11.0.17 (Ubuntu 11.0.17+8-post-Ubuntu-1ubuntu222.04) 2024-04-17T04:57:21.650 INFO:teuthology.orchestra.run.smithi088.stdout:OS: Linux 5.15.0-102-generic amd64 2024-04-17T04:57:21.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.653 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-04-17T04:57:21.654 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-17T04:57:21.654 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T04:57:21.654 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-17T04:57:21.654 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T04:57:21.654 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-04-17T04:57:21.855 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-04-17T04:57:21.855 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-04-17T04:57:21.855 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'ONWMPQGBYPSBLBMIEXSY', 'access_secret': 'Uxx8hRkJVMjEa7NDxuGgXjRqoi2DaQe0ENlpL2eDYSPaVxMkm8s9sg==', 'region': 'mexico', 'endpoint': 'smithi088.front.sepia.ceph.com:80', 'port': 80, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': False, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi088.front.sepia.ceph.com'} 2024-04-17T04:57:21.855 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-04-17T04:57:21.855 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'ONWMPQGBYPSBLBMIEXSY', '--secret', 'Uxx8hRkJVMjEa7NDxuGgXjRqoi2DaQe0ENlpL2eDYSPaVxMkm8s9sg==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-04-17T04:57:21.856 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key ONWMPQGBYPSBLBMIEXSY --secret Uxx8hRkJVMjEa7NDxuGgXjRqoi2DaQe0ENlpL2eDYSPaVxMkm8s9sg== --email tester.client.0_test@test.test --cluster ceph 2024-04-17T04:57:21.897 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:57:21.898 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:57:21.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.937+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 realm 2024-04-17T04:57:21.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:57:21.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:57:21.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:57:21.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.941+0000 7f415673c980 20 searching for the correct realm 2024-04-17T04:57:21.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.949+0000 7f4153a56640 0 -- 172.21.15.88:0/2216647357 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f414c0bf040 msgr2=0x7f414c0b59e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:57:21.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.953+0000 7f4153a56640 0 -- 172.21.15.88:0/2216647357 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f401c076320 msgr2=0x7f401c078750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:57:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 zone default found 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 4 Realm: () 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 zonegroup default 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.961+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.965+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.965+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.965+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.965+0000 7f415673c980 20 started sync module instance, tier type = 2024-04-17T04:57:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.965+0000 7f415673c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=0 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=1 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=2 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=3 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=4 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=5 2024-04-17T04:57:21.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.977+0000 7f415673c980 20 add_watcher() i=6 2024-04-17T04:57:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 20 add_watcher() i=7 2024-04-17T04:57:21.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 2 all 8 watchers are set, enabling cache 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f400afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 5 note: GC not initialized 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f4009fdb640 20 reqs_thread_entry: start 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 20 init_complete bucket index max shards: 1 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 20 Filter name: none 2024-04-17T04:57:21.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f3f7b7fe640 20 reqs_thread_entry: start 2024-04-17T04:57:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-04-17T04:57:21.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.981+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache get: name=default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY : miss 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 20 rados->read ofs=0 len=0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:21.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache put: name=default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY info.flags=0x0 2024-04-17T04:57:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 adding default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY to cache LRU end 2024-04-17T04:57:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache get: name=default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY : hit (negative entry) 2024-04-17T04:57:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache get: name=default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY : hit (negative entry) 2024-04-17T04:57:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-04-17T04:57:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-17T04:57:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.985+0000 7f415673c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-04-17T04:57:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f402b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464288 cookie 93838950079088 notifier 6977 bl.length()=670 2024-04-17T04:57:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f402b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-04-17T04:57:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f402b7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-17T04:57:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f415673c980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-04-17T04:57:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f415673c980 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-04-17T04:57:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f415673c980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-04-17T04:57:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f402affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464305 cookie 93838950065392 notifier 6977 bl.length()=200 2024-04-17T04:57:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f402affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-04-17T04:57:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.989+0000 7f402affd640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-04-17T04:57:21.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f415673c980 10 cache put: name=default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY info.flags=0x7 2024-04-17T04:57:21.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f415673c980 10 moving default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY to cache LRU end 2024-04-17T04:57:21.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f415673c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:ONWMPQGBYPSBLBMIEXSY, ofs0, ns] 2024-04-17T04:57:21.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f402affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464289 cookie 93838950079088 notifier 6977 bl.length()=189 2024-04-17T04:57:21.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f402affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY info.flags=0x7 2024-04-17T04:57:21.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f402affd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ONWMPQGBYPSBLBMIEXSY to cache LRU end 2024-04-17T04:57:21.992 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T04:57:21.992 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "tester.client.0", 2024-04-17T04:57:21.992 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Ms. tester.client.0", 2024-04-17T04:57:21.992 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "tester.client.0_test@test.test", 2024-04-17T04:57:21.992 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "tester.client.0", 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "ONWMPQGBYPSBLBMIEXSY", 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "Uxx8hRkJVMjEa7NDxuGgXjRqoi2DaQe0ENlpL2eDYSPaVxMkm8s9sg==", 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T04:57:21.989758Z" 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T04:57:21.993 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T04:57:21.994 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "rgw", 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "", 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T04:57:21.989751Z", 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T04:57:21.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:21.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:21.993+0000 7f400afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:57:21.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.001+0000 7f415673c980 20 remove_watcher() i=0 2024-04-17T04:57:21.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.001+0000 7f415673c980 2 removed watcher, disabling cache 2024-04-17T04:57:22.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.001+0000 7f415673c980 20 remove_watcher() i=1 2024-04-17T04:57:22.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.005+0000 7f415673c980 20 remove_watcher() i=2 2024-04-17T04:57:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.005+0000 7f415673c980 20 remove_watcher() i=3 2024-04-17T04:57:22.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.009+0000 7f415673c980 20 remove_watcher() i=4 2024-04-17T04:57:22.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.009+0000 7f415673c980 20 remove_watcher() i=5 2024-04-17T04:57:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.009+0000 7f415673c980 20 remove_watcher() i=6 2024-04-17T04:57:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.009+0000 7f415673c980 20 remove_watcher() i=7 2024-04-17T04:57:22.021 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T04:57:22.021 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-04-17T04:57:22.069 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-04-17T04:57:22.069 DEBUG:teuthology.orchestra.run.smithi088:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-04-17T04:57:22.114 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T04:57:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:57:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:57:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 realm 2024-04-17T04:57:22.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:57:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.229+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.233+0000 7f7afceaa980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:57:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.233+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.233+0000 7f7afceaa980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:57:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.233+0000 7f7afceaa980 20 searching for the correct realm 2024-04-17T04:57:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.237+0000 7f7afb1c6640 0 -- 172.21.15.88:0/4055548452 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f7af0008030 msgr2=0x7f7af0038920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:57:22.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.453+0000 7f7afa9c5640 0 -- 172.21.15.88:0/4055548452 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x56327ddfd080 msgr2=0x56327ddff4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:57:22.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:57:22.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:57:22.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:57:22.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:57:22.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:57:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:57:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:57:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 zone default found 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 4 Realm: () 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 zonegroup default 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.461+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.465+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.465+0000 7f7afceaa980 20 rados->read ofs=0 len=0 2024-04-17T04:57:22.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.465+0000 7f7afceaa980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:22.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.465+0000 7f7afceaa980 20 started sync module instance, tier type = 2024-04-17T04:57:22.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.465+0000 7f7afceaa980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=0 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=1 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=2 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=3 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=4 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=5 2024-04-17T04:57:22.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=6 2024-04-17T04:57:22.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 20 add_watcher() i=7 2024-04-17T04:57:22.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.489+0000 7f7afceaa980 2 all 8 watchers are set, enabling cache 2024-04-17T04:57:22.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.493+0000 7f79b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:57:22.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.497+0000 7f7afceaa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:57:22.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.497+0000 7f7afceaa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:57:22.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.585+0000 7f792bfff640 20 reqs_thread_entry: start 2024-04-17T04:57:22.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.585+0000 7f7afceaa980 20 init_complete bucket index max shards: 1 2024-04-17T04:57:22.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.585+0000 7f7afceaa980 20 Filter name: none 2024-04-17T04:57:22.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.585+0000 7f7929ffb640 20 reqs_thread_entry: start 2024-04-17T04:57:22.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.585+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T04:57:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.589+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8212965928922506626', time=2024-04-17T06:54:45.113267+0000, chain.objs.size()=1 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.VgPEG5oBGIM-uCU7ry-HA6t5KI2pDN8_0 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5695297547746510751', time=2024-04-17T06:55:29.169833+0000, chain.objs.size()=1 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.Wdj87p0prRPBVq2iB7YPfwoHLEa90tE_0 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15318756093818837628', time=2024-04-17T06:56:08.845158+0000, chain.objs.size()=1 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.CHkianKSpbRAfLXExNTMM_fU_KW1SWL_0 2024-04-17T04:57:22.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8704146302076107801', time=2024-04-17T06:56:45.246576+0000, chain.objs.size()=1 2024-04-17T04:57:22.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.AWqTWzlClRSMnRFBquxUZ0wlS7SiOJK_0 2024-04-17T04:57:22.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18178605734749002092', time=2024-04-17T06:56:59.424359+0000, chain.objs.size()=1 2024-04-17T04:57:22.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.593+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:kRUTVB1QzSdNznTkxD7m9ACSUrLll56_.5oyosv9IJrQNm0EMGCT9gOjJFGrUtg5_0 2024-04-17T04:57:22.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.601+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.605+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T04:57:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.609+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.609+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17990006581996301809', time=2024-04-17T06:55:20.788124+0000, chain.objs.size()=1 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.zQSzmaBrlpRzun3QuWqAOWm48Tm9QhE_0 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16738208338411665577', time=2024-04-17T06:55:20.926757+0000, chain.objs.size()=1 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.W-cS6deHTOrfnHsVBZNVH3zqTN9aFCk_0 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10991571433642355418', time=2024-04-17T06:56:04.144514+0000, chain.objs.size()=1 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.3zkCBe6-cH7CE5RUv-yUfhVcF8aAhXd_0 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4884877052289406233', time=2024-04-17T06:56:43.954907+0000, chain.objs.size()=1 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.bt9vZuQKm1xCj-OrgHnKPaH81eB9enA_0 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8824930858732148597', time=2024-04-17T06:56:45.224698+0000, chain.objs.size()=1 2024-04-17T04:57:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.613+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.JFSCnBrxEOO2aoNp8h4xdn4kbHzZEVE_0 2024-04-17T04:57:22.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.617+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.625+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T04:57:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.625+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9997103251580419038', time=2024-04-17T06:52:26.823544+0000, chain.objs.size()=1 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.xqnYdAf-rRPuD35yidAZk_0HzexJCy9_0 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11011259161594035317', time=2024-04-17T06:54:05.474995+0000, chain.objs.size()=1 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.eZskAyYIjix1zLNDmlnCt0a-Dh0h36U_0 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4226813656560279388', time=2024-04-17T06:54:40.355764+0000, chain.objs.size()=1 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.Cbl4JjYBZN8A5e-z1JfBBfqEyD1-hx9_0 2024-04-17T04:57:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4403682152512758103', time=2024-04-17T06:54:40.409603+0000, chain.objs.size()=1 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.Ubeg-jF-Mjx6ivYMIJlJwS8JlYmdDFn_0 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5907242274562722929', time=2024-04-17T06:54:45.114324+0000, chain.objs.size()=1 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.-sARVKBqnxX4LMcuLegpZKggVPip1yQ_0 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11424042286200502848', time=2024-04-17T06:55:28.962008+0000, chain.objs.size()=1 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.36rD875zcqRBiiUs4mPhtISuRaw3vWg_0 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16087746276088956708', time=2024-04-17T06:56:04.215698+0000, chain.objs.size()=1 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.h5W0F87J5nCCnD3B9sOC6VgZPk8hL5j_0 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5818164838389507162', time=2024-04-17T06:56:04.322595+0000, chain.objs.size()=1 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.hEc6RX7IPssOKYVldQDi6IXDor8Y7t-_0 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.865113646887581564', time=2024-04-17T06:56:44.010998+0000, chain.objs.size()=1 2024-04-17T04:57:22.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.629+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.PaB6nc-Ioc_S3vtrJTnmq8rErWcc6WB_0 2024-04-17T04:57:22.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.641+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.645+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T04:57:22.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-17T04:57:22.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18049114420975794800', time=2024-04-17T06:52:26.914429+0000, chain.objs.size()=4 2024-04-17T04:57:22.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.1 2024-04-17T04:57:22.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.2 2024-04-17T04:57:22.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.3 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13__multipart_foo.2~qw2JGA0MkUYrdKsSolMbAOR7DuJbnRH.4 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4481332236259749016', time=2024-04-17T06:52:27.224626+0000, chain.objs.size()=3 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.1 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.3 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16636855843335856865', time=2024-04-17T06:52:27.399538+0000, chain.objs.size()=2 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.t0IlW5XZgDMzeIbSRRyicNcJMVLi_nV_0 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.t0IlW5XZgDMzeIbSRRyicNcJMVLi_nV_1 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2251710783827857503', time=2024-04-17T06:54:01.351610+0000, chain.objs.size()=1 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.ivQwRG3vJUNDQiBuMRAYJEBi9753uLe_0 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11501823215002594598', time=2024-04-17T06:55:20.806506+0000, chain.objs.size()=1 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.pn0qQIWGGmFmpHAEHF6O589co7R-sqq_0 2024-04-17T04:57:22.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2129646080676694847', time=2024-04-17T06:55:25.234488+0000, chain.objs.size()=1 2024-04-17T04:57:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.649+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.Wt2Y0TVvbg8wJMx_UkPGZq5E4pOI76H_0 2024-04-17T04:57:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.657+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17908773148928210276', time=2024-04-17T06:55:25.414784+0000, chain.objs.size()=1 2024-04-17T04:57:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.657+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.Tj2R65DocoC8Gfz8oQu1lhUWcjaljS3_0 2024-04-17T04:57:22.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.661+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16593776730840984869', time=2024-04-17T06:56:04.151573+0000, chain.objs.size()=1 2024-04-17T04:57:22.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.661+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.UWoYWncxf_r-SZV6ZrLB9IotlvAKNJH_0 2024-04-17T04:57:22.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.665+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.669+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T04:57:22.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2580317577252748330', time=2024-04-17T06:52:27.046131+0000, chain.objs.size()=1 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_0 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8378322104445853895', time=2024-04-17T06:52:27.575298+0000, chain.objs.size()=1 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.8kBECyHRDV9-IwGBGjaZI4KOS-YrcSk_0 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1480795232108960936', time=2024-04-17T06:55:28.960732+0000, chain.objs.size()=1 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.glqGlcBtYkkqxi705js9wEYzlENZDOH_0 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8682533473113207040', time=2024-04-17T06:55:28.985838+0000, chain.objs.size()=1 2024-04-17T04:57:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.nWeRkupnbzxW-l1o4pfTAZn-FbXvfCX_0 2024-04-17T04:57:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9815795371823714878', time=2024-04-17T06:56:04.322313+0000, chain.objs.size()=1 2024-04-17T04:57:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.R-ZGF8i0__tzMMhnJVUQmhl5ZOAb4Fk_0 2024-04-17T04:57:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.366925885265401307', time=2024-04-17T06:56:45.196780+0000, chain.objs.size()=1 2024-04-17T04:57:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.3dKCc1RVhYipzrMmRbJ7tiYUoZ5vX7S_0 2024-04-17T04:57:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3782795633935632422', time=2024-04-17T06:56:59.436704+0000, chain.objs.size()=1 2024-04-17T04:57:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.673+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:LUR2ejp-eC6eEeZrmYmbQLBr11jrbi4_.4LhBDsGl_fOiwtS5t71Upvt3Ygm5p6g_0 2024-04-17T04:57:22.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.681+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.685+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T04:57:22.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2_26279c4c239ea20dd1a5af4444a52a63', time=2024-04-17T06:52:27.195772+0000, chain.objs.size()=3 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.1 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.3 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13453326829442974892', time=2024-04-17T06:54:05.251234+0000, chain.objs.size()=1 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.jz4c3ZCby6kuD6lWPi6DhBapGgjNnZp_0 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12404811062504289573', time=2024-04-17T06:54:40.344858+0000, chain.objs.size()=1 2024-04-17T04:57:22.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.QR-tQXnhp4SyRPyydR_2J-jwQ1FLMJb_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1453295007031302766', time=2024-04-17T06:54:45.135391+0000, chain.objs.size()=1 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.nvpuDiSaSWfn6bmnWhTwPwLiG0TzbVH_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9988765505908626503', time=2024-04-17T06:55:25.229030+0000, chain.objs.size()=1 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.9U726Wk5rkv99LxBuBsuoCIOgTyLQnW_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10748026214035793761', time=2024-04-17T06:55:25.230728+0000, chain.objs.size()=1 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.U8NwA2byRKJgcwsKIpQZwH0oh3OXyk6_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8791128252404661233', time=2024-04-17T06:55:25.233120+0000, chain.objs.size()=1 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.ErXPhnejtx2elncXhFpeX-tJ9ch1rja_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1698556040844904105', time=2024-04-17T06:55:28.978741+0000, chain.objs.size()=1 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.GpMGIIj3e81Quqv85_Y3LPIK02Bz4qZ_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5857728146895043553', time=2024-04-17T06:56:04.131710+0000, chain.objs.size()=1 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.HIWjLd0XTeY26sDN-w92gMj3dn4nDM9_0 2024-04-17T04:57:22.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16751340073900018794', time=2024-04-17T06:56:42.884555+0000, chain.objs.size()=1 2024-04-17T04:57:22.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.689+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.FZ_2U7S_9wsIrkfSFrN-Wriun7sqsCa_0 2024-04-17T04:57:22.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.693+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5353931086171768481', time=2024-04-17T06:56:45.219369+0000, chain.objs.size()=1 2024-04-17T04:57:22.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.693+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.sfhzijtiwl-RGtU7ITICcDpxTCJ_kri_0 2024-04-17T04:57:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.697+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.701+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T04:57:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-17T04:57:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12016678185059146316', time=2024-04-17T06:52:26.847254+0000, chain.objs.size()=1 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.q55BlF2Q4ouIRUB_Li3Jpu2XEMrVVPD_0 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_EhqgM5HZk1dyxdxKctGAafe5epntHXX', time=2024-04-17T06:52:27.064036+0000, chain.objs.size()=1 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.E7v-s0wKdtBdaDKpo5XtMgQZXMFflo8_0 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_1_4ebae0e1fc765aed257bffed04a65a77', time=2024-04-17T06:52:27.104182+0000, chain.objs.size()=1 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_0 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_EKeb5Or2IZwNkLinXXJwswxcvgMzMZ4', time=2024-04-17T06:52:27.250785+0000, chain.objs.size()=2 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gnlngG6-OauNzyfJdw9-pffaEVCjjf9_0 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gnlngG6-OauNzyfJdw9-pffaEVCjjf9_1 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10528315399725893095', time=2024-04-17T06:54:05.275613+0000, chain.objs.size()=1 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.o4OTJd7uPPnRpeil7-MKN1sgvjF977m_0 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17976357075558694868', time=2024-04-17T06:55:20.995985+0000, chain.objs.size()=1 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.HicMIDcFPSN0VZbinGK1AM_Pa31a6Vv_0 2024-04-17T04:57:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9616112677172330941', time=2024-04-17T06:55:28.979930+0000, chain.objs.size()=1 2024-04-17T04:57:22.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.30NpBVsPdtG5YlQh7W_YtBvqZ_Tfv59_0 2024-04-17T04:57:22.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4694457740430138645', time=2024-04-17T06:56:03.082934+0000, chain.objs.size()=1 2024-04-17T04:57:22.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.705+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.6Pc0WKn-8MYhbPEBoPV0H-6ntI-DPy__0 2024-04-17T04:57:22.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.713+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.717+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T04:57:22.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18187156190340681083', time=2024-04-17T06:54:00.142506+0000, chain.objs.size()=1 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.34gmJDwtcsGTJ4qiLY1XH3vFr-E7HPC_0 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4364340322171474573', time=2024-04-17T06:54:01.220514+0000, chain.objs.size()=1 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.qHqndXxZUkLCkqpthfbUqzFdXxWHKgm_0 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.960379163599015600', time=2024-04-17T06:54:01.224693+0000, chain.objs.size()=1 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.MNicbSptGmSkrgwUJ8Tqs8qk9U7BEsD_0 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.829312559458411134', time=2024-04-17T06:54:05.496392+0000, chain.objs.size()=1 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.VVSoS37V7yVMrIGbpIm1_qiO1v3hczx_0 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2940585740810830280', time=2024-04-17T06:55:24.239467+0000, chain.objs.size()=1 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.IbrT6tQQqeDt_JNaWEbBlyE3TwcBUEO_0 2024-04-17T04:57:22.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8865532137959187973', time=2024-04-17T06:55:25.283947+0000, chain.objs.size()=1 2024-04-17T04:57:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.aCmmA0kyPuf0HvTJgR0IiQqK_tSrK7u_0 2024-04-17T04:57:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8078296469714186941', time=2024-04-17T06:56:04.193019+0000, chain.objs.size()=1 2024-04-17T04:57:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.8JEtDm7tf12342KhBUsqbxnPHDpBCj4_0 2024-04-17T04:57:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7050116385301413364', time=2024-04-17T06:56:09.028537+0000, chain.objs.size()=1 2024-04-17T04:57:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.721+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.tku-BiCNuz1fehbRz3CcsMLizfr7cgZ_0 2024-04-17T04:57:22.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.729+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.733+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T04:57:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4630922249441394948', time=2024-04-17T06:52:27.587857+0000, chain.objs.size()=1 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.-XRt1dI4-DJOuR8rp5euc68yHKT0a6l_0 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7233882498089332419', time=2024-04-17T06:54:01.151353+0000, chain.objs.size()=1 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.bhGqc0WMuivIjUV8Q5O8gw6T2qARopA_0 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17276642790428570403', time=2024-04-17T06:54:01.153071+0000, chain.objs.size()=1 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.MA7-AUN2FuVG1rp7gdCg_E3KVQnkf6Y_0 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6438540490231352320', time=2024-04-17T06:54:05.254834+0000, chain.objs.size()=1 2024-04-17T04:57:22.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.7yFzSRkWqkgCHsX9WumhC52Q-bgKnwN_0 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8544683542111304506', time=2024-04-17T06:54:40.411277+0000, chain.objs.size()=1 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.YO2hFtZuKvLhLC-4epp1JpJLP6Nhi_h_0 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.692393692310460339', time=2024-04-17T06:54:45.112402+0000, chain.objs.size()=1 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.Z7VvyH6Xgx2251BUpbya0SFeIlXcuRW_0 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13814478025859876783', time=2024-04-17T06:54:45.293762+0000, chain.objs.size()=1 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.IPsARTW0X7GmYB1DlXD05udaE6a5MQD_0 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5022766626149687953', time=2024-04-17T06:56:44.008821+0000, chain.objs.size()=1 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.sbjn1HBfv6tzJUbLS0AcrUrg6pbQcXG_0 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15159775694668301734', time=2024-04-17T06:56:44.083969+0000, chain.objs.size()=1 2024-04-17T04:57:22.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.737+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.BA2PT8T5t86kV1wmoyjMJd450VUT4xD_0 2024-04-17T04:57:22.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.749+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.753+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T04:57:22.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.753+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.753+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-04-17T04:57:22.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_8bp469jdtlGeYoQgAE9gI_xK-ZXFqdO', time=2024-04-17T06:52:27.074227+0000, chain.objs.size()=1 2024-04-17T04:57:22.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.PpI8eG1cAFn1bjAqzxlH6ZM5pkSj5up_0 2024-04-17T04:57:22.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_-V1FlikXDUi-9chE4YMhYLnDOiBb7nJ', time=2024-04-17T06:52:27.095152+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.7fSDuzhJXGxYdqchNRXF_xUX_WxKU6s_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15777964244001158671', time=2024-04-17T06:54:01.200387+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.AXLQ_J2QrvKHg7JDIpNQPQVcE9sScli_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11731849498346592167', time=2024-04-17T06:54:05.404887+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.UbwIdBHKLKQSc6PNqui7bUQQBdeGuq1_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5880795340474816252', time=2024-04-17T06:54:40.543142+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.6yHKKbMxmpTEbolIM7LMOGawd3jgaLU_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4172143126899574244', time=2024-04-17T06:55:20.903931+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.1EHIplriSnLg9eHf84F5nf2EfrzpZs2_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7363256764497832593', time=2024-04-17T06:55:25.366787+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.zlu8HS-nM8oXuVl2IprepYZ6TGyIJKT_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17014389852554605362', time=2024-04-17T06:56:04.129454+0000, chain.objs.size()=1 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.KRs0vT8rtaMd6MOyOBgL50q7CQaSntD_0 2024-04-17T04:57:22.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16994531941634509766', time=2024-04-17T06:56:04.131444+0000, chain.objs.size()=1 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.NG_Ko9KRGCxhlszUDh_xeSeqLR46clz_0 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7694884893490254416', time=2024-04-17T06:56:09.020182+0000, chain.objs.size()=1 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.TUZWEBBCBwgB1NTWZofHkoHjo0hRK13_0 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13928334327291547901', time=2024-04-17T06:56:43.900175+0000, chain.objs.size()=1 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.XizlA4vOuHIMagQ9LmachK-0LWmG3Rm_0 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9789041663001062633', time=2024-04-17T06:56:43.901968+0000, chain.objs.size()=1 2024-04-17T04:57:22.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.757+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.n5AfC0luH7_qHvF4ZE6UlCSjKzdSGRD_0 2024-04-17T04:57:22.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.765+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.769+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T04:57:22.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_1_a1899668ab9f8d7c2428d9bc89c6752c', time=2024-04-17T06:52:27.056841+0000, chain.objs.size()=1 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.w7tHKVioHqKYnsN63bIwBjrK5vRqPRN_0 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_z0ZGbg98k2337uxANkID0FhY7DB1Hy5', time=2024-04-17T06:52:27.208874+0000, chain.objs.size()=2 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.ETw-d3CGi03o-JtEpg0TF4nTXh2PUX2_0 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.ETw-d3CGi03o-JtEpg0TF4nTXh2PUX2_1 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_HgWEf97JlN_kIuCSFqQENwZetHNCxuF', time=2024-04-17T06:52:27.219033+0000, chain.objs.size()=2 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.qN0M0IdOsOvNS9YhDggjYSVl3txI6rV_0 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.qN0M0IdOsOvNS9YhDggjYSVl3txI6rV_1 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1150976883353417069', time=2024-04-17T06:54:05.250299+0000, chain.objs.size()=1 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.N8HbYB88KjOl6b7NarimgR0f0Q1qxU8_0 2024-04-17T04:57:22.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10087758340371530279', time=2024-04-17T06:55:25.287939+0000, chain.objs.size()=1 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.JVW2WSbCqsxQdPXodk086Ct5xLftAMx_0 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17435858405303723387', time=2024-04-17T06:56:04.192738+0000, chain.objs.size()=1 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.j0a4mq4r4o8Hq-FglSc3Et5M2LLffKW_0 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17457994174249468243', time=2024-04-17T06:56:09.028613+0000, chain.objs.size()=1 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.zfHWKFVK_ulubGPzmbK9BMEYulcE_AR_0 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17684908349329303765', time=2024-04-17T06:56:43.909980+0000, chain.objs.size()=1 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.MQ4jjWyzSu15LcEqqz-I_tggFLeCN6s_0 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13360751467221694745', time=2024-04-17T06:56:44.092744+0000, chain.objs.size()=1 2024-04-17T04:57:22.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.773+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.MAI-Qj2gNlcZuhduZD1vseGVL2AHUpK_0 2024-04-17T04:57:22.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.781+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.785+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T04:57:22.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7354409550113537632', time=2024-04-17T06:52:27.388446+0000, chain.objs.size()=2 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.dTo_b8Djw-TIWzoOQQniTyBQuNYDB6w_0 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.dTo_b8Djw-TIWzoOQQniTyBQuNYDB6w_1 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.541548999841598287', time=2024-04-17T06:54:01.201925+0000, chain.objs.size()=1 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.HU8rCEe_D5jVGPj2cA_9p2ZpN-bWAU4_0 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5998248161847849432', time=2024-04-17T06:54:05.335438+0000, chain.objs.size()=1 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.axY6IU0VRv080VBM3sWkZYya4Hb4uGD_0 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3576636110130103936', time=2024-04-17T06:54:40.348834+0000, chain.objs.size()=1 2024-04-17T04:57:22.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.3aAXS34kyHJGKe_rJwyoorI_eWJYrQM_0 2024-04-17T04:57:22.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14657964749739118160', time=2024-04-17T06:54:45.142478+0000, chain.objs.size()=1 2024-04-17T04:57:22.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.kRjmi0jUiUDrll4FgeO3-1e0jIrI9E3_0 2024-04-17T04:57:22.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10774944187315596429', time=2024-04-17T06:55:29.152066+0000, chain.objs.size()=1 2024-04-17T04:57:22.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.KuP8X-_zXPRqfEEEYJvd6pFyhhvAFYC_0 2024-04-17T04:57:22.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6303611011668183395', time=2024-04-17T06:56:04.206762+0000, chain.objs.size()=1 2024-04-17T04:57:22.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.789+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.jUgqN6aod-YwVD-DUN7yCMFEzAecN8Q_0 2024-04-17T04:57:22.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.797+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T04:57:22.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-17T04:57:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12778080299033145602', time=2024-04-17T06:54:01.222568+0000, chain.objs.size()=1 2024-04-17T04:57:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.Fcv_iIf4ZJrH3qs_-iKVSfp0OgNCnkg_0 2024-04-17T04:57:22.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10981405993284906001', time=2024-04-17T06:54:40.224042+0000, chain.objs.size()=1 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.wTyPulhrqo7dfmdsWjcjcjjbfu0JQQ7_0 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7562705287109802792', time=2024-04-17T06:55:20.819314+0000, chain.objs.size()=1 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.8Wub9qxs9gXdytCfphJnMdtiamPA8fE_0 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12852517683447255495', time=2024-04-17T06:55:25.282517+0000, chain.objs.size()=1 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.rNP01w1TjPrnjB5AXcS_cgdoVWluFJA_0 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10615389119191321382', time=2024-04-17T06:55:29.120004+0000, chain.objs.size()=1 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.801+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.9Gx5T4i6N5yJXG1zg5GuXwXY2swNcz2_0 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.805+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8958026081995793531', time=2024-04-17T06:56:07.933654+0000, chain.objs.size()=1 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.805+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.8-5HIT-D2k_2-WefZMCe3gpxmA41gqA_0 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.805+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14760669465720235512', time=2024-04-17T06:56:09.128804+0000, chain.objs.size()=1 2024-04-17T04:57:22.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.805+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.8WMKxKWfsxeUup6q3I8lVE4CPeiV0sB_0 2024-04-17T04:57:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.809+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.813+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T04:57:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.813+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.813+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_0aVcWC8hwpu6Ufq_KhFiM42sNg3pwhW', time=2024-04-17T06:52:27.281610+0000, chain.objs.size()=2 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.f1afeEr3VOBzl8RIIJdkaFb_dOO3zj2_0 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.f1afeEr3VOBzl8RIIJdkaFb_dOO3zj2_1 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5897510588494304517', time=2024-04-17T06:54:01.120151+0000, chain.objs.size()=1 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.26DmMkjS0C9Yd8RSunabCtO14VaY8Ve_0 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14590246808342847297', time=2024-04-17T06:54:40.343922+0000, chain.objs.size()=1 2024-04-17T04:57:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.JNTjI6QiLouRWU6OrTMCofD8JdooFbK_0 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16985310248268687506', time=2024-04-17T06:54:45.044082+0000, chain.objs.size()=1 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.0YT7LomDj56PehSBQtrLpAOy1_hDj3S_0 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4939324098020469026', time=2024-04-17T06:54:45.137667+0000, chain.objs.size()=1 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.q4YneFyJdBAqsMQwk69t5v6SuqmFF4I_0 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18354957398209941850', time=2024-04-17T06:54:45.139734+0000, chain.objs.size()=1 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.IRdXyMIR62bQrwhgX8IbxZGKiXuixaU_0 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16832198506143972949', time=2024-04-17T06:55:20.903110+0000, chain.objs.size()=1 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.7G4jm8Ree9FMSnzy9Z0MY-Qz7cZ4wi9_0 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13555436838123609342', time=2024-04-17T06:55:29.182700+0000, chain.objs.size()=1 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.Q9X2iniWs9GoomJpZRLi62ucI-kf6Kb_0 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5707931900654911026', time=2024-04-17T06:56:43.909841+0000, chain.objs.size()=1 2024-04-17T04:57:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.817+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.tNSzBuFlIlASlTLqBCdkwY95MOFNJG5_0 2024-04-17T04:57:22.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.825+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T04:57:22.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16376418569850011000', time=2024-04-17T06:52:26.792154+0000, chain.objs.size()=1 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.Kaew7lqgXnaN-I0r3V1QDLlnHCluBJk_0 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13137316058190898743', time=2024-04-17T06:54:45.147959+0000, chain.objs.size()=1 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.SyRrm4Qvo_0YHPaGgcM80PhjU5zRzyR_0 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16051847571879177904', time=2024-04-17T06:54:45.150205+0000, chain.objs.size()=1 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.vMk4PYnqFveXr_cWPAad97SOq6TDO-C_0 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18157422446679243550', time=2024-04-17T06:56:04.151437+0000, chain.objs.size()=1 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.JBDoa5dQGXLB-VEFJxoxtVOMPhm-bo1_0 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4176594798701391755', time=2024-04-17T06:56:08.875679+0000, chain.objs.size()=1 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.ZP_Syb1SyavTNcuzQsa2Gx0OBIQmzdi_0 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5187734041813495473', time=2024-04-17T06:56:09.025507+0000, chain.objs.size()=1 2024-04-17T04:57:22.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.829+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.Oa4iPMJsKdO2iTJEi7tPYxS2YU6XfY7_0 2024-04-17T04:57:22.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.833+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.837+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T04:57:22.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2_cda98037d30c48c027fd08c4e7844713', time=2024-04-17T06:52:27.293024+0000, chain.objs.size()=3 2024-04-17T04:57:22.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.1 2024-04-17T04:57:22.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2 2024-04-17T04:57:22.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.3 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6068203785831278025', time=2024-04-17T06:52:27.506596+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.Kdx4g7r8XT7qpyUmrB-7sLdwojoMYHs_0 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18136444337245282793', time=2024-04-17T06:54:01.200651+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.PLnoov2rdSPaUlRXakqZiMaALaOFM-t_0 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3545500557630380734', time=2024-04-17T06:54:05.355924+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.SDw-O86coT27-cy_k2E2u_LB-mK112n_0 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.363390322124380843', time=2024-04-17T06:54:45.113330+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.1nJDKZygMedc6QySNVWvNCwHfCHqWPA_0 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4146570749486182711', time=2024-04-17T06:55:20.788303+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.5PpTrt59EKMYK8W1wE8x_fjnFy2Jdct_0 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6936410389573981782', time=2024-04-17T06:55:25.161483+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.srYiwp_i1U59n-H_stem2nNTwooEmoq_0 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13437017757511700206', time=2024-04-17T06:55:25.419191+0000, chain.objs.size()=1 2024-04-17T04:57:22.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.l-SmFElRoWeaf9UpMJGRyWxpsmF1TR2_0 2024-04-17T04:57:22.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5832045868998469292', time=2024-04-17T06:56:04.215994+0000, chain.objs.size()=1 2024-04-17T04:57:22.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.841+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.u35d6kav4s38uzxvDeiisBsM58nYUO3_0 2024-04-17T04:57:22.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.849+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.853+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T04:57:22.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=13, truncated=0, next_marker='' 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.907505000887342168', time=2024-04-17T06:54:05.183880+0000, chain.objs.size()=1 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.jifha6I61veV1DygR09JCrErfMvaZ4K_0 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8470845209391110740', time=2024-04-17T06:54:05.474279+0000, chain.objs.size()=1 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.wN8B8GeD-9pSl-WjILf2VTMt8u2B3qf_0 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15736854737316878703', time=2024-04-17T06:54:40.473102+0000, chain.objs.size()=1 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.ZYEXmnnFsav2Tf7naGckEVM8XaVlcFz_0 2024-04-17T04:57:22.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8884720799690576584', time=2024-04-17T06:54:45.263387+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.aR2uRqNwLZZFqVDGvxs95_rO7K-GrRv_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2056360022795533156', time=2024-04-17T06:55:20.880250+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.aKSDRv5sgijyKzRSG2VgxnOq6G7UPYd_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16943402734711040032', time=2024-04-17T06:55:25.234306+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.gL5zOZvcNj76IWYdhF7jnvHwP1L4nDM_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7195456989011390783', time=2024-04-17T06:55:28.853966+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.h3f20RlMRnW_pOhBqLcqKS7MrsHhA_G_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15361406238803188356', time=2024-04-17T06:55:28.900738+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.TxkwpNafZi4rXKMIdbXMBzgxa-e9Ss1_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17051407009360381038', time=2024-04-17T06:56:09.031381+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.whLldDugeAUSuY0VHYk637ygCdzMoT6_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13147504234135644071', time=2024-04-17T06:56:09.138831+0000, chain.objs.size()=1 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.Nyl5tT3IxTEfDS4LmtZXnYtcA0NUvKG_0 2024-04-17T04:57:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15168544639103586204', time=2024-04-17T06:56:43.889977+0000, chain.objs.size()=1 2024-04-17T04:57:22.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.3ndALnocWaxCKGlJEla6WhGFKE04IUC_0 2024-04-17T04:57:22.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12391307122237405590', time=2024-04-17T06:56:43.891511+0000, chain.objs.size()=1 2024-04-17T04:57:22.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.857+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.Rki2Zvbxz_UwWSQoUeii6Lq0DFB0zSZ_0 2024-04-17T04:57:22.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.861+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8127472308078006586', time=2024-04-17T06:56:43.993806+0000, chain.objs.size()=1 2024-04-17T04:57:22.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.861+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.dzz5jGPDsyNSKoY86rst0pOhUhmdE2u_0 2024-04-17T04:57:22.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.869+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.869+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T04:57:22.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18366126528739179819', time=2024-04-17T06:52:26.783783+0000, chain.objs.size()=1 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.sFNrTzas1xmKINBBE_o66wU_iPkrhDP_0 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10059692401396988354', time=2024-04-17T06:54:40.399729+0000, chain.objs.size()=1 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.hd9zVVW9BA8Rt8uQ0mRrO9jm8wVlhZ5_0 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17119953190141413547', time=2024-04-17T06:56:08.934848+0000, chain.objs.size()=1 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.-IGsFwbUtvE7vk8Yz4dBNu9hkMQxYMn_0 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1610429071192249738', time=2024-04-17T06:56:09.068304+0000, chain.objs.size()=1 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.UleIUzEKGCb_cBislTMI0coCS-8T78o_0 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4218925301128299345', time=2024-04-17T06:56:43.824532+0000, chain.objs.size()=1 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.I4MObjfMvGzkoAdASwpl4U6iIGmZwgn_0 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6804140201624422618', time=2024-04-17T06:56:53.175385+0000, chain.objs.size()=1 2024-04-17T04:57:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:j3mFqnZgBAO8FtNcDx.pWDaqvB48XnG_.C6cvM2gMgFf3aAuNFmPUYTbnHDzOJyc_0 2024-04-17T04:57:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9909331905855297824', time=2024-04-17T06:56:59.431436+0000, chain.objs.size()=1 2024-04-17T04:57:22.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.873+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.19__shadow:klwJ7etnDsFolWvluVN9OwpsJFhYSnG_.to4ZFBJGwgLy22L3Y4l_0i9xYisuUFD_0 2024-04-17T04:57:22.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.877+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T04:57:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_mgfb8vGBLtmT82VV4IL41m0wVaDmY4n', time=2024-04-17T06:52:27.268520+0000, chain.objs.size()=2 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gXHePTiGIPfQEeXmywLmTgP3NtCzkPe_0 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.gXHePTiGIPfQEeXmywLmTgP3NtCzkPe_1 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14993780938410216773', time=2024-04-17T06:52:27.409489+0000, chain.objs.size()=1 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14__shadow_.AjiiJM_XqKo0_HZquO4_vXWE8NXCeKV_0 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14532602795867222162', time=2024-04-17T06:54:01.116945+0000, chain.objs.size()=1 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.sM8G-Kd3375XuFASwR7cBMgA0cMofmE_0 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10917007069673960088', time=2024-04-17T06:54:05.183896+0000, chain.objs.size()=1 2024-04-17T04:57:22.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.885+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.vD8AS1po6kLLQDR2BpcF-nnZE8dGMFD_0 2024-04-17T04:57:22.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.889+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.893+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T04:57:22.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_LY6HrTs7mM1Y4yqUFvFmvHeIMQ6IUpM', time=2024-04-17T06:52:27.137131+0000, chain.objs.size()=1 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.boCaB13pvgUrsPl2KGBx6fLli2TTrBb_0 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1157902591642266296', time=2024-04-17T06:55:20.798229+0000, chain.objs.size()=1 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.SE8_OSVNLEPFqIiY7tfsIQZWgxXSvJP_0 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9458521382247400768', time=2024-04-17T06:55:25.254311+0000, chain.objs.size()=1 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.t5ZyTjOzZuxZ-GR_4eyS5Bo1RGfNRZF_0 2024-04-17T04:57:22.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10457128929415258969', time=2024-04-17T06:55:28.927545+0000, chain.objs.size()=1 2024-04-17T04:57:22.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.o9RzO_J9hluxf07fyNbspRwLq84JRF2_0 2024-04-17T04:57:22.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14926886071660134897', time=2024-04-17T06:56:04.299017+0000, chain.objs.size()=1 2024-04-17T04:57:22.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.TQpK_HEEXQ2Y8gWyltfjwg5mWpxfNxG_0 2024-04-17T04:57:22.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6789337463820982470', time=2024-04-17T06:56:08.925484+0000, chain.objs.size()=1 2024-04-17T04:57:22.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.897+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.5WsJcXqdtLAQZZRfJ5PtImC1C8Fmx8U_0 2024-04-17T04:57:22.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.901+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T04:57:22.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9064418187252474800', time=2024-04-17T06:52:26.801370+0000, chain.objs.size()=1 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.Ij9YJGQP06x-jPBzDGxTSpP8sO_NsPW_0 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4387291275846727288', time=2024-04-17T06:55:28.881187+0000, chain.objs.size()=1 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.84BFh5ezUf1OtXH6R0APQ4KZ_-duTzY_0 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10911436292436007993', time=2024-04-17T06:55:28.957435+0000, chain.objs.size()=1 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.ALa5lcB1uNtsvVjgK2lLKUjWojfe4Xw_0 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18002616288042003003', time=2024-04-17T06:55:28.958972+0000, chain.objs.size()=1 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.5msHthVMnsBGjTCXHbTlIuUhBmWWCmG_0 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7607310847221768930', time=2024-04-17T06:56:45.217723+0000, chain.objs.size()=1 2024-04-17T04:57:22.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.905+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.WxHWuHFJPQ36bYhRWKkaWD91smVu4F2_0 2024-04-17T04:57:22.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.909+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.913+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T04:57:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.913+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.913+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17570049470131095556', time=2024-04-17T06:52:26.814543+0000, chain.objs.size()=1 2024-04-17T04:57:22.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.O5F4XerjAiJyE3muqnfMylvBEyIAszv_0 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1645654568369969872', time=2024-04-17T06:52:26.887624+0000, chain.objs.size()=4 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.1 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.2 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.3 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12__multipart_foo.2~CPjraU1JQDLlJaA7xiJGsCJznPiqoF6.4 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18405166338911737529', time=2024-04-17T06:52:27.190598+0000, chain.objs.size()=3 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.1 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.2 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-FROZEN.2~-WTMKqbYIK5rhelB0OGv26rw4T4IXkf.3 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5720787998017603528', time=2024-04-17T06:52:27.537442+0000, chain.objs.size()=1 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.V5LEjPRr1jIETBN1wSti84YEn40R-bt_0 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17969124964052695248', time=2024-04-17T06:54:40.557519+0000, chain.objs.size()=1 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.uIJ_KGSgEXcagfoqRSkPSLyjL__L-aj_0 2024-04-17T04:57:22.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3224024092780002333', time=2024-04-17T06:55:20.817130+0000, chain.objs.size()=1 2024-04-17T04:57:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.jLmxpKrowaLME6GW_XitAxxaVrwtOOl_0 2024-04-17T04:57:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1465017273547254463', time=2024-04-17T06:55:20.904116+0000, chain.objs.size()=1 2024-04-17T04:57:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.917+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.PVMLbjrbnmTDKl70ciUm2Y0OhN_caEs_0 2024-04-17T04:57:22.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.921+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10207882581440731764', time=2024-04-17T06:56:44.071321+0000, chain.objs.size()=1 2024-04-17T04:57:22.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.921+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.UxjLobnWJxYqR2kySO-LusOZAb9Et9E_0 2024-04-17T04:57:22.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.921+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9783517288952898962', time=2024-04-17T06:56:45.186048+0000, chain.objs.size()=1 2024-04-17T04:57:22.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.921+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.OKNgb6vkdw8qLI1KL4TUov8RhI64deq_0 2024-04-17T04:57:22.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.925+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T04:57:22.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T04:57:22.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13978424803278354288', time=2024-04-17T06:52:27.523597+0000, chain.objs.size()=1 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.VmegyrY_OgQNvKX1V73f6aylQoUaPC9_0 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16754721897483592182', time=2024-04-17T06:54:01.200415+0000, chain.objs.size()=1 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.xZhoc1Yzu4Zi7FHwdtXcLR3N-sXGkxu_0 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14998479808989212726', time=2024-04-17T06:54:09.118432+0000, chain.objs.size()=1 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.f9OR6OaB_V3lY_3spEINDRBGMTjCCOr_0 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6153826142885450862', time=2024-04-17T06:54:40.399619+0000, chain.objs.size()=1 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_._dYSttGY_me3h-4Ovn17BFt0nkU382L_0 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14491995271708865462', time=2024-04-17T06:56:04.129516+0000, chain.objs.size()=1 2024-04-17T04:57:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.929+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.gHn5pq3S4XQUovuOZGoj0skALYEHEsW_0 2024-04-17T04:57:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.933+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T04:57:22.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=13, truncated=0, next_marker='' 2024-04-17T04:57:22.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16703826128154381260', time=2024-04-17T06:52:27.002862+0000, chain.objs.size()=4 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.1 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.2 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.3 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10__multipart_foo.2~7T2m-z4XIDqmVokvdIzFDslcEMqRj1q.4 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15094973298672229375', time=2024-04-17T06:54:05.252866+0000, chain.objs.size()=1 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.38ZABHRi8_oOQTOl_QOuofVT5Pu0vLG_0 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7556981405974537709', time=2024-04-17T06:54:40.389669+0000, chain.objs.size()=1 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.knbpVoJCv8vz1d4nfsexaC5zeP3F0Zs_0 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4071788087801739025', time=2024-04-17T06:54:45.124909+0000, chain.objs.size()=1 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.vox0U7nswa7QswExKZIIxpb9-gBDlng_0 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2917806473354310362', time=2024-04-17T06:54:45.126760+0000, chain.objs.size()=1 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.Y5Zfg7uG8_wElCbdCJhMkKB4Eem5ivq_0 2024-04-17T04:57:22.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.883855448134691317', time=2024-04-17T06:54:45.128914+0000, chain.objs.size()=1 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.9Gm05eHUyrtNpj6F02FQGhddem26hl1_0 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493408415611242194', time=2024-04-17T06:55:20.881105+0000, chain.objs.size()=1 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.PKu4XjkRGnyBeklZpDDiDdZf_IK162w_0 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4154997098879182379', time=2024-04-17T06:55:21.014363+0000, chain.objs.size()=1 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.brYupip7tfumDOyZXPit7BJhpW4YgZ3_0 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17188186117178850593', time=2024-04-17T06:55:25.254931+0000, chain.objs.size()=1 2024-04-17T04:57:22.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.937+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.AS7bfpYRCiLO34em9yg2g8g_YgxKrr9_0 2024-04-17T04:57:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.164705931713649334', time=2024-04-17T06:55:25.256936+0000, chain.objs.size()=1 2024-04-17T04:57:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.e6GAfgdBig621ErN0uSJuP4sSnyi2Hm_0 2024-04-17T04:57:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7950671194247224846', time=2024-04-17T06:55:29.025907+0000, chain.objs.size()=1 2024-04-17T04:57:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.BPhcRDhEoazf7J-neFv4dEgMwl89NT0_0 2024-04-17T04:57:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16815935364952387027', time=2024-04-17T06:56:04.078049+0000, chain.objs.size()=1 2024-04-17T04:57:22.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.GB_aInh1WAg2zs58wmLbbYkvVnxHSZn_0 2024-04-17T04:57:22.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15925792374796013552', time=2024-04-17T06:56:08.882081+0000, chain.objs.size()=1 2024-04-17T04:57:22.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.945+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.x95-gMsIsJf_GXTAFVX5v3XAXPqalT8_0 2024-04-17T04:57:22.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.949+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T04:57:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7625213628631826171', time=2024-04-17T06:52:27.329379+0000, chain.objs.size()=3 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-FROZEN.2~SFqZrD-YUf5hmB8Re8Z6_qCwcQgYVTD.1 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-FROZEN.2~SFqZrD-YUf5hmB8Re8Z6_qCwcQgYVTD.2 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-FROZEN.2~SFqZrD-YUf5hmB8Re8Z6_qCwcQgYVTD.3 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10442656189416446325', time=2024-04-17T06:52:27.516912+0000, chain.objs.size()=1 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8__shadow_.7q-iMuNHOm-srE_a-99unVQKo-A2dHR_0 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13369702719412952550', time=2024-04-17T06:52:27.598543+0000, chain.objs.size()=1 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.8vxzc6SUlNNaYlqsp5dr5F80PjOVyo1_0 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9424396483346911264', time=2024-04-17T06:54:01.119932+0000, chain.objs.size()=1 2024-04-17T04:57:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.2JBpfy2RhoHucZkGSLed-00vYivxcqE_0 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16136467314659305948', time=2024-04-17T06:54:01.221396+0000, chain.objs.size()=1 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.bGesu3O14AO8GuI5rQ1C0jPblvdtyRb_0 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13564057987282717446', time=2024-04-17T06:54:05.243337+0000, chain.objs.size()=1 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.KO_g39Sg3q25MbgeGF2NLxEm8Db5pv2_0 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5155322889077109412', time=2024-04-17T06:54:40.404839+0000, chain.objs.size()=1 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.rQowKQEk2MIHXSHd5985qcsWIqJ9woK_0 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2114414495622758799', time=2024-04-17T06:54:40.530593+0000, chain.objs.size()=1 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.ohYg3GieACwmpi5E4qaYe6yG1FlQ4vd_0 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14069809863980981542', time=2024-04-17T06:54:48.933825+0000, chain.objs.size()=1 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.gceGPymg5wmQ7WY80imD3uP21xgDd2y_0 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16740866978332654885', time=2024-04-17T06:55:20.904257+0000, chain.objs.size()=1 2024-04-17T04:57:22.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.953+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.nTZqPnMmaoGx4Dw0ndiNqWCg_S3NYBM_0 2024-04-17T04:57:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.961+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3708036861046211102', time=2024-04-17T06:56:43.908434+0000, chain.objs.size()=1 2024-04-17T04:57:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.961+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_._qUt__N2iRpeJkdI6RrwQFcwYIXEN8q_0 2024-04-17T04:57:22.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.961+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.965+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T04:57:22.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.965+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1965224351320529598', time=2024-04-17T06:52:26.830068+0000, chain.objs.size()=1 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.d1NUwcs_-cB39LSG7IqJaKYVw0dvaLP_0 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12781097983849889692', time=2024-04-17T06:52:26.838624+0000, chain.objs.size()=1 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.ErXdAMiNIvrHJNR5vHHBbLCfOj_INP8_0 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17561353937998813697', time=2024-04-17T06:52:27.085802+0000, chain.objs.size()=1 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.nYP2Jbo_sybcf3AnLPGWjDl7f9npYEM_0 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6707506847384385726', time=2024-04-17T06:52:27.124123+0000, chain.objs.size()=1 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_0 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_U4T2PiAsjz-TATo8NaDsNGAVNbFsbvi', time=2024-04-17T06:52:27.144582+0000, chain.objs.size()=1 2024-04-17T04:57:22.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.ph0evQ7r_AHKUcWwqxxcq-5wqY3EydL_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10624819203853602309', time=2024-04-17T06:52:27.605555+0000, chain.objs.size()=1 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9__shadow_.wFpa93mvpTcqjhUlrQCGUUefTOD6NCX_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5991094283783352602', time=2024-04-17T06:54:05.317409+0000, chain.objs.size()=1 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.jpPoKyjjDo2CuIyrIe3uhimIBqn2gS0_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13096276194946309565', time=2024-04-17T06:54:40.399346+0000, chain.objs.size()=1 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.7ratxzOA557TRZUf3tXRRk09HzMRtm7_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4683943015566493331', time=2024-04-17T06:55:25.230714+0000, chain.objs.size()=1 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.DdbTAl36C-3QcnyhSJldoLDeJ0aolP9_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15309648076242011276', time=2024-04-17T06:56:08.934736+0000, chain.objs.size()=1 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.YF6Z73kzPYKHjzz4WGe2zDWmU8kjEg8_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6197181161943540198', time=2024-04-17T06:56:43.900289+0000, chain.objs.size()=1 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.BZRCrH44oFYKODpWNfpnJuZtZOdYLAG_0 2024-04-17T04:57:22.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15258613690053543304', time=2024-04-17T06:56:43.902209+0000, chain.objs.size()=1 2024-04-17T04:57:22.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.969+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.N2FF4X7WH62ynENb7xtaTIjqOm7Q-HC_0 2024-04-17T04:57:22.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.977+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.977+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T04:57:22.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_QUGYfh2ApQ5zk3wJTrbBNhTiUhihPSQ', time=2024-04-17T06:52:27.113614+0000, chain.objs.size()=1 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.E86CI1pIPArFwa_rZEjoG9_6gs-dvAo_0 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='_18HCggrqkEE58hUl9SbVBTApWCkp0U9', time=2024-04-17T06:52:27.233196+0000, chain.objs.size()=2 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.TufLWn4UXpTVhO8LntmtEB7mEB-DAWS_0 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__shadow_.TufLWn4UXpTVhO8LntmtEB7mEB-DAWS_1 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1440241217219391637', time=2024-04-17T06:54:01.351588+0000, chain.objs.size()=1 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.Z-wkCuxQ6mvdT245ap1hX5UggJCwA02_0 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9659035881134298331', time=2024-04-17T06:54:45.263291+0000, chain.objs.size()=1 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.gavdmM5uLqEQEKuKItCwnAKNQhYsjhU_0 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8528160826097446309', time=2024-04-17T06:56:09.025507+0000, chain.objs.size()=1 2024-04-17T04:57:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.70aClxxhoNDIcEiib9GMTcVlHZXfYqx_0 2024-04-17T04:57:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4009300284974048138', time=2024-04-17T06:56:09.027412+0000, chain.objs.size()=1 2024-04-17T04:57:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.PHCQvb1D-j6OkVj5iYCKxG_BDmyrzrb_0 2024-04-17T04:57:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17297294082506074549', time=2024-04-17T06:56:09.029233+0000, chain.objs.size()=1 2024-04-17T04:57:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.4ZQCjdBYCq-lQeC4yJqHi-CmxCpWjXw_0 2024-04-17T04:57:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12793540322300285878', time=2024-04-17T06:56:09.138750+0000, chain.objs.size()=1 2024-04-17T04:57:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.981+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.687Y1HWp3SqEYGVRArOMFEvaRA2mOe4_0 2024-04-17T04:57:22.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.985+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:22.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.989+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T04:57:22.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.989+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:22.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.989+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2996146780926949517', time=2024-04-17T06:52:26.956227+0000, chain.objs.size()=4 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.1 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.2 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.3 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11__multipart_foo.2~5kt-8Gdtt9KL-KaC8pb4YnrRbpOFnFH.4 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_1_e7c424f2e243a1fae9dde9d9a5985710', time=2024-04-17T06:52:27.154213+0000, chain.objs.size()=1 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16__shadow_.25K66UPddC_zO6faTVcBJ3KHyhsucS6_0 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17813352488194049708', time=2024-04-17T06:52:27.259649+0000, chain.objs.size()=3 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.1 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.2 2024-04-17T04:57:22.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-STANDARD.2~92I7UlBneDdQyO8UxCAU3HFjr7aCe2C.3 2024-04-17T04:57:22.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15258549198467422455', time=2024-04-17T06:52:27.347077+0000, chain.objs.size()=3 2024-04-17T04:57:22.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-STANDARD.2~EzlEk1NaqjvssmU90zWgfR83tFInWvt.1 2024-04-17T04:57:22.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-STANDARD.2~EzlEk1NaqjvssmU90zWgfR83tFInWvt.2 2024-04-17T04:57:22.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-STANDARD.2~EzlEk1NaqjvssmU90zWgfR83tFInWvt.3 2024-04-17T04:57:22.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9010785875275841542', time=2024-04-17T06:54:05.206702+0000, chain.objs.size()=1 2024-04-17T04:57:22.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.ds2SXAegLWDcPNpSw3_PRe3_b8aCHO1_0 2024-04-17T04:57:22.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8572258483024740258', time=2024-04-17T06:54:40.284628+0000, chain.objs.size()=1 2024-04-17T04:57:22.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.993+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.BasNjgbWXk6iy7YNTBhdC2maxRFD9hi_0 2024-04-17T04:57:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.997+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563112513604096356', time=2024-04-17T06:56:04.072384+0000, chain.objs.size()=1 2024-04-17T04:57:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.997+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.CiLrpsF_9ZDnbzt_apGS4ZCYovTI-m2_0 2024-04-17T04:57:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.997+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16112827868043786915', time=2024-04-17T06:56:45.217768+0000, chain.objs.size()=1 2024-04-17T04:57:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.997+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.h6_srBiN81ZSr1MpvvrQ-hlvQwaaziF_0 2024-04-17T04:57:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.997+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2117661877087972575', time=2024-04-17T06:56:45.243190+0000, chain.objs.size()=1 2024-04-17T04:57:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:22.997+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18__shadow_.JjnlmI1X-5eMSMJmUmvgqX17V0h-y18_0 2024-04-17T04:57:22.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.001+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:23.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T04:57:23.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16640278832355630616', time=2024-04-17T06:52:26.776715+0000, chain.objs.size()=1 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5__shadow_.fonW3ZalRYXQO0vg355LPSbZqHRARzi_0 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7231390364615983272', time=2024-04-17T06:54:01.200761+0000, chain.objs.size()=1 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.LFLdBEsQCyj-r7j1BAS7jSyb_Ja2M9i_0 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2197817733083689638', time=2024-04-17T06:54:05.256293+0000, chain.objs.size()=1 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.ZupU6HOChBZMwcsEimV6vAGMuaE3H1E_0 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16385601071332566010', time=2024-04-17T06:54:05.476799+0000, chain.objs.size()=1 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.Ir7iYOIxIZUDDHE3_2TJqIUhBKEUG7O_0 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11818032967204177980', time=2024-04-17T06:55:20.926995+0000, chain.objs.size()=1 2024-04-17T04:57:23.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.aWMT0TRt3YDKcpPnAXwg348Rz5nurAN_0 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2254869381181826094', time=2024-04-17T06:55:28.963772+0000, chain.objs.size()=1 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.Sxl4apXaJaEUdkccppZ-Ar0OirSWBx5_0 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5205079817441022680', time=2024-04-17T06:55:28.965388+0000, chain.objs.size()=1 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.48wfK_bGVGnNFM2W2hhPYcIZVP3GozV_0 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6407534187193893376', time=2024-04-17T06:56:04.078539+0000, chain.objs.size()=1 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.23mR8sNHy81PsS2Qewxb0oVaFR7Na0b_0 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13245440560653117760', time=2024-04-17T06:56:04.082229+0000, chain.objs.size()=1 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.27__shadow_.wpSOjd8uUPyKnnAQaZnikcQWqOdeZsh_0 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5446153324156042767', time=2024-04-17T06:56:44.013015+0000, chain.objs.size()=1 2024-04-17T04:57:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.005+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.pJ7VHjWZudD5ipstdx74-SyCeNSOWYG_0 2024-04-17T04:57:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.013+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:23.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T04:57:23.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13842275789272848975', time=2024-04-17T06:52:27.337056+0000, chain.objs.size()=3 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-LUKEWARM.2~gCQP2AqwKNkXq_36mLsiUUTd8RS1JYT.1 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-LUKEWARM.2~gCQP2AqwKNkXq_36mLsiUUTd8RS1JYT.2 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15__multipart_foo-LUKEWARM.2~gCQP2AqwKNkXq_36mLsiUUTd8RS1JYT.3 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2674154370502702442', time=2024-04-17T06:54:05.250962+0000, chain.objs.size()=1 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.JR0i-y7PhO4dj2_qv2g2K1KmZMSN1bV_0 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6639380784264490866', time=2024-04-17T06:54:45.013165+0000, chain.objs.size()=1 2024-04-17T04:57:23.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.A8LNzLEWmBh1z8-7MPfuQlts1GNj8RR_0 2024-04-17T04:57:23.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12460122994715344457', time=2024-04-17T06:55:20.797637+0000, chain.objs.size()=1 2024-04-17T04:57:23.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.8R5E7oEq_vzFBj27y6gnPt5dg9_NKZK_0 2024-04-17T04:57:23.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5399226698309329815', time=2024-04-17T06:55:29.101167+0000, chain.objs.size()=1 2024-04-17T04:57:23.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.017+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.fOLep6oH6Tk83Fal8uuRcWWIwHr13K5_0 2024-04-17T04:57:23.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.021+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:23.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T04:57:23.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2_d6820f5c88590181fad3abbf067c4e41', time=2024-04-17T06:52:27.244998+0000, chain.objs.size()=3 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.1 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.2 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17__multipart_foo-LUKEWARM.2~F2bXMdZWOVlr9NMHDAnLyjfbzOIY22H.3 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13445879491146557900', time=2024-04-17T06:54:01.109406+0000, chain.objs.size()=1 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.VytjKF5AwMx-291TkZeYvanhFeT4zvN_0 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18316718829046450670', time=2024-04-17T06:54:01.118581+0000, chain.objs.size()=1 2024-04-17T04:57:23.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.a7jqC540PWg17gToPuBzKL9m1ZHGZT8_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.430660952061260912', time=2024-04-17T06:54:01.354698+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.025+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.20__shadow_.-g9tEoDOVm6yfw8j0Q2ItVKrjYBXe_j_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14281443009956451144', time=2024-04-17T06:54:40.454716+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.PYtfzIWHYhUutskTPhgSWB7yhZjdRwC_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16637384847859980232', time=2024-04-17T06:54:44.015235+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.CErHN2VU-GCBIjHyuAF06YC0-yWXEiI_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8874364178105298842', time=2024-04-17T06:54:44.983056+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.23__shadow_.oH1FRLvcW5yY7ElImxE--Op2CgeIJr8_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8915094569354573484', time=2024-04-17T06:55:21.007629+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_.YNnYt-AFxcs8RttibfOvMmffI16pvGc_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5283449747634910517', time=2024-04-17T06:55:25.240403+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.qnKHG0qjrqAcefvpq130BHhjBeBRL5W_0 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10043294126591896954', time=2024-04-17T06:55:25.254314+0000, chain.objs.size()=1 2024-04-17T04:57:23.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.LHmJyu6CiPBvWU2b3CMXPnY7Jx-sJe6_0 2024-04-17T04:57:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7312017650648678772', time=2024-04-17T06:55:25.404719+0000, chain.objs.size()=1 2024-04-17T04:57:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.25__shadow_.hGquSl3iQdFVfSJAKS-0xT7ejKHUk18_0 2024-04-17T04:57:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13820320275072627579', time=2024-04-17T06:56:43.895591+0000, chain.objs.size()=1 2024-04-17T04:57:23.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.029+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.29__shadow_.fFIRKxhAPDND04_DQSR2EQJuDwV5XS2_0 2024-04-17T04:57:23.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.033+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:23.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T04:57:23.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12703566768547990041', time=2024-04-17T06:54:05.335462+0000, chain.objs.size()=1 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21__shadow_.EXFnZK29WK5WPSxL4pbTIDA6aEsWGCU_0 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5275395615905444333', time=2024-04-17T06:54:40.241988+0000, chain.objs.size()=1 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.22__shadow_.NPh66LEXOzI3dP7l20aQ_0B-Ifu7C9N_0 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8046077112390775569', time=2024-04-17T06:55:20.920716+0000, chain.objs.size()=1 2024-04-17T04:57:23.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.24__shadow_._mCdm9cHm-oaQv93DFoIshF174XXZEA_0 2024-04-17T04:57:23.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5761925452272762032', time=2024-04-17T06:55:29.151850+0000, chain.objs.size()=1 2024-04-17T04:57:23.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.26__shadow_.Mt1bwJbaJWuXyCqoYVNssX_hK36IHzb_0 2024-04-17T04:57:23.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17357646814613252952', time=2024-04-17T06:56:08.899213+0000, chain.objs.size()=1 2024-04-17T04:57:23.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.037+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.28__shadow_.K8F0l20pE5GPukqnjrTPlBIUXYZtYjW_0 2024-04-17T04:57:23.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.041+0000 7f7afceaa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T04:57:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.045+0000 7f79b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:57:23.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.049+0000 7f7afceaa980 20 remove_watcher() i=0 2024-04-17T04:57:23.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.049+0000 7f7afceaa980 2 removed watcher, disabling cache 2024-04-17T04:57:23.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.049+0000 7f7afceaa980 20 remove_watcher() i=1 2024-04-17T04:57:23.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.053+0000 7f7afceaa980 20 remove_watcher() i=2 2024-04-17T04:57:23.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.053+0000 7f7afceaa980 20 remove_watcher() i=3 2024-04-17T04:57:23.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.053+0000 7f7afceaa980 20 remove_watcher() i=4 2024-04-17T04:57:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.053+0000 7f7afceaa980 20 remove_watcher() i=5 2024-04-17T04:57:23.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.057+0000 7f7afceaa980 20 remove_watcher() i=6 2024-04-17T04:57:23.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.057+0000 7f7afceaa980 20 remove_watcher() i=7 2024-04-17T04:57:23.070 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T04:57:23.139 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T04:57:23.139 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T04:57:23.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 realm 2024-04-17T04:57:23.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T04:57:23.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:57:23.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.185+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.189+0000 7fef19945980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T04:57:23.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.189+0000 7fef19945980 20 searching for the correct realm 2024-04-17T04:57:23.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.193+0000 7fef17c61640 0 -- 172.21.15.88:0/3639019249 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fef0c0c4210 msgr2=0x7fef0c007cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:57:23.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.409+0000 7fef17460640 0 -- 172.21.15.88:0/3639019249 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x555e3bccdc90 msgr2=0x555e3bcd00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T04:57:23.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T04:57:23.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 zone default found 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 4 Realm: () 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 zonegroup default 2024-04-17T04:57:23.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.417+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 rados->read ofs=0 len=0 2024-04-17T04:57:23.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T04:57:23.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 started sync module instance, tier type = 2024-04-17T04:57:23.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.421+0000 7fef19945980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T04:57:23.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=0 2024-04-17T04:57:23.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=1 2024-04-17T04:57:23.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=2 2024-04-17T04:57:23.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=3 2024-04-17T04:57:23.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=4 2024-04-17T04:57:23.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=5 2024-04-17T04:57:23.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=6 2024-04-17T04:57:23.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 20 add_watcher() i=7 2024-04-17T04:57:23.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.445+0000 7fef19945980 2 all 8 watchers are set, enabling cache 2024-04-17T04:57:23.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.449+0000 7fedcdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:57:23.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.453+0000 7fef19945980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T04:57:23.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.453+0000 7fef19945980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T04:57:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.537+0000 7fedccfe1640 20 reqs_thread_entry: start 2024-04-17T04:57:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.537+0000 7fef19945980 20 init_complete bucket index max shards: 1 2024-04-17T04:57:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.537+0000 7fef19945980 20 Filter name: none 2024-04-17T04:57:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.537+0000 7fed467fc640 20 reqs_thread_entry: start 2024-04-17T04:57:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.537+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T04:57:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.541+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.541+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.545+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.545+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T04:57:23.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.549+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.549+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.549+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.553+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T04:57:23.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.553+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.553+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.553+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.557+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T04:57:23.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.561+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.561+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.561+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.565+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T04:57:23.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.565+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.569+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.569+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.569+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T04:57:23.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.573+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.573+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.573+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.577+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T04:57:23.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.577+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.581+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.581+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.581+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T04:57:23.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.585+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.589+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.589+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.589+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T04:57:23.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.593+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.593+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.597+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.597+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T04:57:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.601+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.601+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.605+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.605+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T04:57:23.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.609+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.609+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.609+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.613+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T04:57:23.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.617+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.617+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.617+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.621+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T04:57:23.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.625+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.625+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.625+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.629+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T04:57:23.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.633+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.633+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.637+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.637+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T04:57:23.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.641+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.641+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.641+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.645+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T04:57:23.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.649+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.649+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.649+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.653+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T04:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.657+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.657+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.657+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.661+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T04:57:23.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.665+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.665+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.665+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.669+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T04:57:23.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.673+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.673+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.677+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.677+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T04:57:23.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.681+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.685+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.685+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.689+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T04:57:23.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.693+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.697+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.697+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.701+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T04:57:23.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.705+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.705+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.705+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.709+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T04:57:23.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.713+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.713+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.713+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.717+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T04:57:23.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.721+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.721+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.721+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.725+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T04:57:23.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.729+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.729+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.733+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.733+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T04:57:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.737+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.741+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.741+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.741+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T04:57:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.745+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.749+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.749+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.753+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T04:57:23.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.757+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.757+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.757+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.761+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T04:57:23.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.765+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.765+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.769+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.769+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T04:57:23.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.773+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.773+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.777+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.777+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T04:57:23.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.781+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.785+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.785+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.789+0000 7fef19945980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T04:57:23.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.789+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.793+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T04:57:23.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.793+0000 7fef19945980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T04:57:23.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.797+0000 7fedcdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T04:57:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.805+0000 7fef19945980 20 remove_watcher() i=0 2024-04-17T04:57:23.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.805+0000 7fef19945980 2 removed watcher, disabling cache 2024-04-17T04:57:23.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.805+0000 7fef19945980 20 remove_watcher() i=1 2024-04-17T04:57:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.809+0000 7fef19945980 20 remove_watcher() i=2 2024-04-17T04:57:23.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.813+0000 7fef19945980 20 remove_watcher() i=3 2024-04-17T04:57:23.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.813+0000 7fef19945980 20 remove_watcher() i=4 2024-04-17T04:57:23.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.817+0000 7fef19945980 20 remove_watcher() i=5 2024-04-17T04:57:23.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.817+0000 7fef19945980 20 remove_watcher() i=6 2024-04-17T04:57:23.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T04:57:23.821+0000 7fef19945980 20 remove_watcher() i=7 2024-04-17T04:57:23.838 DEBUG:teuthology.orchestra.run.smithi088:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-04-17T04:57:24.455 INFO:teuthology.orchestra.run.smithi088.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-04-17T04:57:28.754 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :clean UP-TO-DATE 2024-04-17T04:57:50.153 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :compileJava 2024-04-17T04:57:50.153 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :processResources 2024-04-17T04:57:50.153 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :classes 2024-04-17T04:57:51.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:51.053 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :compileTestJava 2024-04-17T04:57:51.053 INFO:teuthology.orchestra.run.smithi088.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-04-17T04:57:51.054 INFO:teuthology.orchestra.run.smithi088.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-04-17T04:57:51.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:51.055 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :processTestResources NO-SOURCE 2024-04-17T04:57:51.055 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :testClasses 2024-04-17T04:57:57.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:57.053 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :test 2024-04-17T04:57:57.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:57:57.053 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-04-17T04:58:05.152 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:05.153 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-04-17T04:58:05.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:05.652 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-04-17T04:58:05.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:05.952 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-04-17T04:58:06.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:06.452 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-04-17T04:58:17.653 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:17.653 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-04-17T04:58:18.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:18.353 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-04-17T04:58:18.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:18.353 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-04-17T04:58:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:18.552 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-04-17T04:58:18.752 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:18.752 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-04-17T04:58:19.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:19.354 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-04-17T04:58:27.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:27.953 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-04-17T04:58:29.152 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:29.153 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-04-17T04:58:29.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:58:29.653 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-04-17T04:58:29.653 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 04:58:29,589 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-04-17T04:59:15.154 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:15.154 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-04-17T04:59:16.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:16.252 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-04-17T04:59:16.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:16.353 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-04-17T04:59:16.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:16.952 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-04-17T04:59:17.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:17.253 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-04-17T04:59:18.152 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:18.152 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-04-17T04:59:26.553 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:26.553 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-04-17T04:59:27.152 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:27.152 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-04-17T04:59:27.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:27.252 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-04-17T04:59:27.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:27.452 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-04-17T04:59:38.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:38.253 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-04-17T04:59:39.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:39.953 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests STANDARD_OUT 2024-04-17T04:59:39.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 04:59:39,892 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-17T04:59:39.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 04:59:39,922 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-17T04:59:40.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T04:59:40.353 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-04-17T05:00:04.453 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:00:04.453 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-04-17T05:00:05.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:00:05.452 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-04-17T05:00:09.752 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:00:09.752 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-04-17T05:00:10.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:00:10.252 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-04-17T05:02:17.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:17.255 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-04-17T05:02:17.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:17.853 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-04-17T05:02:27.654 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:27.654 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-04-17T05:02:28.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:28.353 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-04-17T05:02:45.454 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:45.454 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-04-17T05:02:46.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:46.053 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-04-17T05:02:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:49.554 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-04-17T05:02:56.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:56.153 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-04-17T05:02:56.553 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:56.553 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-04-17T05:02:59.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:02:59.153 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-04-17T05:03:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:00.454 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-04-17T05:03:01.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:01.054 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-04-17T05:03:01.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:01.354 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-04-17T05:03:01.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:01.954 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-04-17T05:03:02.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-04-17T05:03:02.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:02.853 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-04-17T05:03:03.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:03.153 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-04-17T05:03:03.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:03.854 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-04-17T05:03:04.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:04.054 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-04-17T05:03:04.754 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:04.754 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-04-17T05:03:04.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:04.953 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-04-17T05:03:05.553 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:05.554 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-04-17T05:03:45.454 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:45.455 INFO:teuthology.orchestra.run.smithi088.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-04-17T05:03:46.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:46.254 INFO:teuthology.orchestra.run.smithi088.stdout:---------------------------------------------------------------------- 2024-04-17T05:03:46.254 INFO:teuthology.orchestra.run.smithi088.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-04-17T05:03:46.254 INFO:teuthology.orchestra.run.smithi088.stdout:---------------------------------------------------------------------- 2024-04-17T05:03:46.279 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:46.281 INFO:teuthology.orchestra.run.smithi088.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-04-17T05:03:46.281 INFO:teuthology.orchestra.run.smithi088.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-04-17T05:03:46.282 INFO:teuthology.orchestra.run.smithi088.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-04-17T05:03:46.282 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:46.282 INFO:teuthology.orchestra.run.smithi088.stdout:BUILD SUCCESSFUL in 6m 22s 2024-04-17T05:03:46.282 INFO:teuthology.orchestra.run.smithi088.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-04-17T05:03:46.610 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:46.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.684+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 realm 2024-04-17T05:03:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:46.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.688+0000 7f006d173980 20 searching for the correct realm 2024-04-17T05:03:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.696+0000 7f006248d640 0 -- 172.21.15.88:0/1382301733 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x564806b51e80 msgr2=0x564806b54340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:46.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.704+0000 7f006ac8e640 0 -- 172.21.15.88:0/1382301733 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f005405f950 msgr2=0x7f0054026990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:46.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 zone default found 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 4 Realm: () 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 zonegroup default 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.712+0000 7f006d173980 20 rados->read ofs=0 len=0 2024-04-17T05:03:46.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.716+0000 7f006d173980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:46.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.716+0000 7f006d173980 20 started sync module instance, tier type = 2024-04-17T05:03:46.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.716+0000 7f006d173980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=0 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=1 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=2 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=3 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=4 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=5 2024-04-17T05:03:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=6 2024-04-17T05:03:46.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 20 add_watcher() i=7 2024-04-17T05:03:46.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.732+0000 7f006d173980 2 all 8 watchers are set, enabling cache 2024-04-17T05:03:46.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.736+0000 7eff217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:46.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.736+0000 7f006d173980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:03:46.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.736+0000 7f006d173980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:03:46.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.808+0000 7efe9bfff640 20 reqs_thread_entry: start 2024-04-17T05:03:46.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.808+0000 7f006d173980 20 init_complete bucket index max shards: 1 2024-04-17T05:03:46.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.808+0000 7f006d173980 20 Filter name: none 2024-04-17T05:03:46.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.808+0000 7efe99ffb640 20 reqs_thread_entry: start 2024-04-17T05:03:46.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.808+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:03:46.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:46.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4445466972927041134', time=2024-04-17T06:58:28.180467+0000, chain.objs.size()=5 2024-04-17T05:03:46.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.1 2024-04-17T05:03:46.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.2 2024-04-17T05:03:46.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.3 2024-04-17T05:03:46.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.4 2024-04-17T05:03:46.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.812+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.5 2024-04-17T05:03:46.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.820+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:03:46.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14902239705032146000', time=2024-04-17T07:02:17.414484+0000, chain.objs.size()=11 2024-04-17T05:03:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.1 2024-04-17T05:03:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.1_1 2024-04-17T05:03:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.2 2024-04-17T05:03:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.2_1 2024-04-17T05:03:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.3 2024-04-17T05:03:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.3_1 2024-04-17T05:03:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.4 2024-04-17T05:03:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.4_1 2024-04-17T05:03:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.828+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.5 2024-04-17T05:03:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.832+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.5_1 2024-04-17T05:03:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.832+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ujrDrci3Yn8CRJTNgQKX15DmtoU3qDv.6 2024-04-17T05:03:46.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.848+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.852+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:03:46.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:46.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7755335699795620876', time=2024-04-17T07:02:17.075773+0000, chain.objs.size()=11 2024-04-17T05:03:46.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.1 2024-04-17T05:03:46.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.1_1 2024-04-17T05:03:46.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.2 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.2_1 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.3 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.3_1 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.4 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.4_1 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.5 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.5_1 2024-04-17T05:03:46.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.856+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~7FP0V-ckCaW19CS4p_WJORwYNsgyde7.6 2024-04-17T05:03:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.868+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.876+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:03:46.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.880+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.880+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.880+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.880+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:03:46.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.884+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:46.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12192695590074543012', time=2024-04-17T07:00:45.642725+0000, chain.objs.size()=11 2024-04-17T05:03:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.1 2024-04-17T05:03:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.2 2024-04-17T05:03:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.3 2024-04-17T05:03:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.4 2024-04-17T05:03:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.5 2024-04-17T05:03:46.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.6 2024-04-17T05:03:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.7 2024-04-17T05:03:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.8 2024-04-17T05:03:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.9 2024-04-17T05:03:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.10 2024-04-17T05:03:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.888+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~DE8RetOjQZVS0i0_psuM_qUet_wprWK.11 2024-04-17T05:03:46.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.904+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.912+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:03:46.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.912+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.912+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.912+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:03:46.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15440501654780145889', time=2024-04-17T06:59:15.385568+0000, chain.objs.size()=5 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~zGEHwyChe6IFihjUwktVUlhGIwBQbzc.1 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__shadow_key1.2~zGEHwyChe6IFihjUwktVUlhGIwBQbzc.1_1 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~zGEHwyChe6IFihjUwktVUlhGIwBQbzc.2 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__shadow_key1.2~zGEHwyChe6IFihjUwktVUlhGIwBQbzc.2_1 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~zGEHwyChe6IFihjUwktVUlhGIwBQbzc.3 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476293904307498557', time=2024-04-17T07:01:38.107836+0000, chain.objs.size()=11 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.1 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.1_1 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.2 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.2_1 2024-04-17T05:03:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.3 2024-04-17T05:03:46.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.3_1 2024-04-17T05:03:46.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.916+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.4 2024-04-17T05:03:46.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.928+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.4_1 2024-04-17T05:03:46.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.928+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.5 2024-04-17T05:03:46.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.928+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__shadow_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.5_1 2024-04-17T05:03:46.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.928+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~ijqoN6qT28XE-3-uxYyBg-TmBp9Jtvr.6 2024-04-17T05:03:46.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.940+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.940+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:03:46.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.944+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.944+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.944+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.948+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:03:46.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.948+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.948+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.948+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.948+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11107268265369857386', time=2024-04-17T07:03:00.698666+0000, chain.objs.size()=1 2024-04-17T05:03:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.948+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.50__shadow_.dsZzR6CyoqybHA1f61FJW0qaO44Eyil_0 2024-04-17T05:03:46.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.952+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.952+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:03:46.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.956+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.956+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.956+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.960+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:03:46.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.960+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.960+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.960+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.964+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:03:46.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.964+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:03:46.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16545046245416159688', time=2024-04-17T06:59:10.704819+0000, chain.objs.size()=5 2024-04-17T05:03:46.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~1D4M5kV6KIeLR1bW4mJYLu2ZQ7FPsPu.1 2024-04-17T05:03:46.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__shadow_key1.2~1D4M5kV6KIeLR1bW4mJYLu2ZQ7FPsPu.1_1 2024-04-17T05:03:46.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~1D4M5kV6KIeLR1bW4mJYLu2ZQ7FPsPu.2 2024-04-17T05:03:46.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__shadow_key1.2~1D4M5kV6KIeLR1bW4mJYLu2ZQ7FPsPu.2_1 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~1D4M5kV6KIeLR1bW4mJYLu2ZQ7FPsPu.3 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5547480437923980666', time=2024-04-17T06:59:15.034756+0000, chain.objs.size()=5 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~gh3XMxgNfFxpFpTWGqLlij4JJh9sBv5.1 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__shadow_key1.2~gh3XMxgNfFxpFpTWGqLlij4JJh9sBv5.1_1 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~gh3XMxgNfFxpFpTWGqLlij4JJh9sBv5.2 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__shadow_key1.2~gh3XMxgNfFxpFpTWGqLlij4JJh9sBv5.2_1 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~gh3XMxgNfFxpFpTWGqLlij4JJh9sBv5.3 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8734644751321404975', time=2024-04-17T07:00:04.960418+0000, chain.objs.size()=1 2024-04-17T05:03:46.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__shadow_.fK8DkD4wpyV2Ja-zluU0GjgvK3ej1nS_0 2024-04-17T05:03:46.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17432615421919454161', time=2024-04-17T07:02:57.037612+0000, chain.objs.size()=1 2024-04-17T05:03:46.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.968+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.49__shadow_.Dh7ECOHPh9rNgRhCSHzG0FxHt0uoJD0_0 2024-04-17T05:03:46.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.984+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:46.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.988+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:03:46.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.988+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:46.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.988+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:46.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10534579057247553793', time=2024-04-17T07:01:20.661966+0000, chain.objs.size()=10 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.1 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.2 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.3 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.4 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.5 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.6 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.7 2024-04-17T05:03:46.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.8 2024-04-17T05:03:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.9 2024-04-17T05:03:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:46.992+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~scwGZhL5AFSfmKLpY26vfhOHBs328AH.10 2024-04-17T05:03:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.000+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.004+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:03:47.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='2~CsFqoVmj3EnVAzT7-k1VXpa1SwpWCyN', time=2024-04-17T06:58:05.040201+0000, chain.objs.size()=5 2024-04-17T05:03:47.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.30__multipart_key1.2~CsFqoVmj3EnVAzT7-k1VXpa1SwpWCyN.1 2024-04-17T05:03:47.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.30__multipart_key1.2~CsFqoVmj3EnVAzT7-k1VXpa1SwpWCyN.2 2024-04-17T05:03:47.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.30__multipart_key1.2~CsFqoVmj3EnVAzT7-k1VXpa1SwpWCyN.3 2024-04-17T05:03:47.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.30__multipart_key1.2~CsFqoVmj3EnVAzT7-k1VXpa1SwpWCyN.4 2024-04-17T05:03:47.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.30__multipart_key1.2~CsFqoVmj3EnVAzT7-k1VXpa1SwpWCyN.5 2024-04-17T05:03:47.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4155561666010807426', time=2024-04-17T07:00:05.060403+0000, chain.objs.size()=1 2024-04-17T05:03:47.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.008+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__shadow_.ErfBG7Q3oZFLPfLla-iLFf_g6DO-vVn_0 2024-04-17T05:03:47.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.016+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.020+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:03:47.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.020+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.024+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.024+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.024+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:03:47.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9883124523295439650', time=2024-04-17T06:59:05.759939+0000, chain.objs.size()=5 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~3AIQLMO7YPX8FIGEVgyC3RicCcsBDEJ.1 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~3AIQLMO7YPX8FIGEVgyC3RicCcsBDEJ.2 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~3AIQLMO7YPX8FIGEVgyC3RicCcsBDEJ.3 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~3AIQLMO7YPX8FIGEVgyC3RicCcsBDEJ.4 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~3AIQLMO7YPX8FIGEVgyC3RicCcsBDEJ.5 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4210942026176296702', time=2024-04-17T06:59:39.614520+0000, chain.objs.size()=5 2024-04-17T05:03:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.42__multipart_key1.2~zl3AwYBiDHPFLjaG5Vj1k6eb4WAgBgl.1 2024-04-17T05:03:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.42__multipart_key1.2~zl3AwYBiDHPFLjaG5Vj1k6eb4WAgBgl.2 2024-04-17T05:03:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.42__multipart_key1.2~zl3AwYBiDHPFLjaG5Vj1k6eb4WAgBgl.3 2024-04-17T05:03:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.42__multipart_key1.2~zl3AwYBiDHPFLjaG5Vj1k6eb4WAgBgl.4 2024-04-17T05:03:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.42__multipart_key1.2~zl3AwYBiDHPFLjaG5Vj1k6eb4WAgBgl.5 2024-04-17T05:03:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7005331480984115837', time=2024-04-17T07:00:04.762477+0000, chain.objs.size()=1 2024-04-17T05:03:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.028+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__shadow_.Cqsnny6cFqVVqusMcyhnhMJyK66J8pJ_0 2024-04-17T05:03:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.040+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.044+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:03:47.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.044+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.048+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.048+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.048+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:03:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15556788853709732052', time=2024-04-17T06:58:17.889366+0000, chain.objs.size()=5 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.32__multipart_key1.2~fFFX-j6uL_r05nH1lrBrqAx9DU7WdYC.1 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.32__multipart_key1.2~fFFX-j6uL_r05nH1lrBrqAx9DU7WdYC.2 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.32__multipart_key1.2~fFFX-j6uL_r05nH1lrBrqAx9DU7WdYC.3 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.32__multipart_key1.2~fFFX-j6uL_r05nH1lrBrqAx9DU7WdYC.4 2024-04-17T05:03:47.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.32__multipart_key1.2~fFFX-j6uL_r05nH1lrBrqAx9DU7WdYC.5 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14792400268838056225', time=2024-04-17T07:03:45.741863+0000, chain.objs.size()=11 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.1 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.2 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.3 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.4 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.5 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.6 2024-04-17T05:03:47.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.052+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.7 2024-04-17T05:03:47.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.060+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.8 2024-04-17T05:03:47.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.064+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.9 2024-04-17T05:03:47.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.064+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.10 2024-04-17T05:03:47.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.064+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.56__multipart_key1.2~T8eBCayq4YbXdbtqRFnP8mk_RW36_y9.11 2024-04-17T05:03:47.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.068+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.072+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.076+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.076+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.076+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.076+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1259347263880019127', time=2024-04-17T06:59:01.413950+0000, chain.objs.size()=5 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.076+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~yJzHxXfOn_TRcCftJLCvi9CVyLg-A9W.1 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.080+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~yJzHxXfOn_TRcCftJLCvi9CVyLg-A9W.2 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.080+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~yJzHxXfOn_TRcCftJLCvi9CVyLg-A9W.3 2024-04-17T05:03:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.080+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~yJzHxXfOn_TRcCftJLCvi9CVyLg-A9W.4 2024-04-17T05:03:47.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.080+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~yJzHxXfOn_TRcCftJLCvi9CVyLg-A9W.5 2024-04-17T05:03:47.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.080+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.084+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:03:47.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV', time=2024-04-17T07:02:53.175038+0000, chain.objs.size()=24 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.1 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.2 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.3 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.4 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.5 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.6 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.7 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.8 2024-04-17T05:03:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.9 2024-04-17T05:03:47.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.10 2024-04-17T05:03:47.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.11 2024-04-17T05:03:47.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.088+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.12 2024-04-17T05:03:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.13 2024-04-17T05:03:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.14 2024-04-17T05:03:47.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.15 2024-04-17T05:03:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.16 2024-04-17T05:03:47.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.17 2024-04-17T05:03:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.18 2024-04-17T05:03:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.19 2024-04-17T05:03:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.20 2024-04-17T05:03:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.23 2024-04-17T05:03:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.26 2024-04-17T05:03:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.096+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.27 2024-04-17T05:03:47.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.104+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.48__multipart_key1.2~6aa3U5TxWFRslhLHoHR1_4jvyOnZfuV.30 2024-04-17T05:03:47.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.108+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.108+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:03:47.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10157183639118056910', time=2024-04-17T07:02:25.968169+0000, chain.objs.size()=1 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.46__shadow_.Qt7LmsZf50SV9BICK3veAJj9-NAPwbj_0 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6428017549030973828', time=2024-04-17T07:02:45.661562+0000, chain.objs.size()=5 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.47__multipart_key1.2~J5l_FZt4rEAAVkphMyF5aceQO7Qk8q3.1 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.47__multipart_key1.2~J5l_FZt4rEAAVkphMyF5aceQO7Qk8q3.2 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.47__multipart_key1.2~J5l_FZt4rEAAVkphMyF5aceQO7Qk8q3.3 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.47__multipart_key1.2~J5l_FZt4rEAAVkphMyF5aceQO7Qk8q3.4 2024-04-17T05:03:47.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.112+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.47__multipart_key1.2~J5l_FZt4rEAAVkphMyF5aceQO7Qk8q3.5 2024-04-17T05:03:47.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.116+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.120+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:03:47.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.120+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.120+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7258337591146956706', time=2024-04-17T06:59:26.788503+0000, chain.objs.size()=5 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.41__multipart_key1.2~H3SfF19Or62iqqXCQ0BQ7olu-eFrcBI.1 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.41__multipart_key1.2~H3SfF19Or62iqqXCQ0BQ7olu-eFrcBI.2 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.41__multipart_key1.2~H3SfF19Or62iqqXCQ0BQ7olu-eFrcBI.3 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.41__multipart_key1.2~H3SfF19Or62iqqXCQ0BQ7olu-eFrcBI.4 2024-04-17T05:03:47.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.124+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.41__multipart_key1.2~H3SfF19Or62iqqXCQ0BQ7olu-eFrcBI.5 2024-04-17T05:03:47.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.128+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.128+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:03:47.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.132+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.132+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.132+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.132+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:03:47.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:47.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1336330366198781423', time=2024-04-17T06:59:15.836890+0000, chain.objs.size()=4 2024-04-17T05:03:47.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.36__shadow_.hzcBppL7Gz2obOK8Nz2bysrkq7qndTx_0 2024-04-17T05:03:47.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.36__shadow_.hzcBppL7Gz2obOK8Nz2bysrkq7qndTx_1 2024-04-17T05:03:47.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.36__shadow_.hzcBppL7Gz2obOK8Nz2bysrkq7qndTx_2 2024-04-17T05:03:47.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.136+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.36__shadow_.hzcBppL7Gz2obOK8Nz2bysrkq7qndTx_3 2024-04-17T05:03:47.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.140+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.144+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:03:47.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.144+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12861506626418947097', time=2024-04-17T06:58:46.907803+0000, chain.objs.size()=5 2024-04-17T05:03:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~Aez42LQQbb8LzphvWTByC-Fk80HJWHi.1 2024-04-17T05:03:47.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~Aez42LQQbb8LzphvWTByC-Fk80HJWHi.2 2024-04-17T05:03:47.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~Aez42LQQbb8LzphvWTByC-Fk80HJWHi.3 2024-04-17T05:03:47.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~Aez42LQQbb8LzphvWTByC-Fk80HJWHi.4 2024-04-17T05:03:47.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.148+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.37__multipart_key1.2~Aez42LQQbb8LzphvWTByC-Fk80HJWHi.5 2024-04-17T05:03:47.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.152+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:03:47.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.2_82e8357a83bc41aa0d5c679155c67966', time=2024-04-17T06:58:28.738599+0000, chain.objs.size()=5 2024-04-17T05:03:47.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.1 2024-04-17T05:03:47.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.2 2024-04-17T05:03:47.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.3 2024-04-17T05:03:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.4 2024-04-17T05:03:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.34__multipart_key1.2~TrZU_BCvhYQQ3BeXu52sR3XbUvPev8g.5 2024-04-17T05:03:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3272138823252421565', time=2024-04-17T07:03:01.565102+0000, chain.objs.size()=1 2024-04-17T05:03:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.156+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.51__shadow_.2NoeZYZ7QVDbDR7rXc5DBRuLhLNPGBD_0 2024-04-17T05:03:47.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.160+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.164+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:03:47.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.164+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.164+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.164+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:03:47.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13777817057774270205', time=2024-04-17T07:01:03.431296+0000, chain.objs.size()=11 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.1 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.2 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.3 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.4 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.5 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.6 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.7 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.8 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.9 2024-04-17T05:03:47.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.10 2024-04-17T05:03:47.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.45__multipart_key1.2~K4lVMQEXw4r3SKFI7xDDbANWHGiN_8o.11 2024-04-17T05:03:47.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10978860969022883111', time=2024-04-17T07:03:02.462173+0000, chain.objs.size()=1 2024-04-17T05:03:47.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.168+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.52__shadow_.pkfRQok3Y3IhDwkAskN0ldYq72bWclo_0 2024-04-17T05:03:47.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.180+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.180+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:03:47.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.184+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.184+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.184+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.184+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:03:47.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11856340849092290188', time=2024-04-17T07:00:04.657583+0000, chain.objs.size()=5 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file.mpg.2~Ksuo4BkHjgmPth-f4C-ae8YZ2OGbap_.1 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file.mpg.2~Ksuo4BkHjgmPth-f4C-ae8YZ2OGbap_.2 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file.mpg.2~Ksuo4BkHjgmPth-f4C-ae8YZ2OGbap_.3 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file.mpg.2~Ksuo4BkHjgmPth-f4C-ae8YZ2OGbap_.4 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file.mpg.2~Ksuo4BkHjgmPth-f4C-ae8YZ2OGbap_.5 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11058283374156350301', time=2024-04-17T07:03:03.369101+0000, chain.objs.size()=1 2024-04-17T05:03:47.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.188+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.53__shadow_.J5MSp1yJ4HxKjnUxlaKz3N_pwONuZb-_0 2024-04-17T05:03:47.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.192+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.196+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:03:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.196+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.196+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2455483687299777285', time=2024-04-17T07:00:04.872798+0000, chain.objs.size()=4 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file2.mpg.2~hrwUIKmpmqRcOta7HDPX9BUUlB4fbW3.1 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file2.mpg.2~hrwUIKmpmqRcOta7HDPX9BUUlB4fbW3.2 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file2.mpg.2~hrwUIKmpmqRcOta7HDPX9BUUlB4fbW3.3 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43__multipart_file2.mpg.2~hrwUIKmpmqRcOta7HDPX9BUUlB4fbW3.4 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10030584639085130762', time=2024-04-17T07:03:04.303999+0000, chain.objs.size()=1 2024-04-17T05:03:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.54__shadow_.LkeWWJRDYfjw16elhu6H83VEO5nE6wp_0 2024-04-17T05:03:47.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.200+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.204+0000 7f006d173980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:03:47.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.204+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13799828827896341582', time=2024-04-17T07:02:27.903182+0000, chain.objs.size()=5 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.46__multipart_key1.2~eVOfUMyLx4PVHhgbAG2PftSQ4NHcyrp.1 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.46__multipart_key1.2~eVOfUMyLx4PVHhgbAG2PftSQ4NHcyrp.2 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.46__multipart_key1.2~eVOfUMyLx4PVHhgbAG2PftSQ4NHcyrp.3 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.46__multipart_key1.2~eVOfUMyLx4PVHhgbAG2PftSQ4NHcyrp.4 2024-04-17T05:03:47.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.208+0000 7f006d173980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.46__multipart_key1.2~eVOfUMyLx4PVHhgbAG2PftSQ4NHcyrp.5 2024-04-17T05:03:47.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.212+0000 7f006d173980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:47.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.216+0000 7eff217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:47.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.220+0000 7f006d173980 20 remove_watcher() i=0 2024-04-17T05:03:47.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.220+0000 7f006d173980 2 removed watcher, disabling cache 2024-04-17T05:03:47.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.220+0000 7f006d173980 20 remove_watcher() i=1 2024-04-17T05:03:47.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.220+0000 7f006d173980 20 remove_watcher() i=2 2024-04-17T05:03:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.224+0000 7f006d173980 20 remove_watcher() i=3 2024-04-17T05:03:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.224+0000 7f006d173980 20 remove_watcher() i=4 2024-04-17T05:03:47.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.224+0000 7f006d173980 20 remove_watcher() i=5 2024-04-17T05:03:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.228+0000 7f006d173980 20 remove_watcher() i=6 2024-04-17T05:03:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.228+0000 7f006d173980 20 remove_watcher() i=7 2024-04-17T05:03:47.241 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:47.308 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:47.308 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:47.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.348+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 realm 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:47.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:47.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.352+0000 7f4088af8980 20 searching for the correct realm 2024-04-17T05:03:47.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.360+0000 7f4085e12640 0 -- 172.21.15.88:0/850142054 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5638962fbfc0 msgr2=0x5638962fe400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:47.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.368+0000 7f4086613640 0 -- 172.21.15.88:0/850142054 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f407c01e930 msgr2=0x7f407c01c400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:47.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:47.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 zone default found 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 4 Realm: () 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 zonegroup default 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.376+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.380+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.380+0000 7f4088af8980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.380+0000 7f4088af8980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.380+0000 7f4088af8980 20 started sync module instance, tier type = 2024-04-17T05:03:47.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.380+0000 7f4088af8980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:03:47.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=0 2024-04-17T05:03:47.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=1 2024-04-17T05:03:47.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=2 2024-04-17T05:03:47.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=3 2024-04-17T05:03:47.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=4 2024-04-17T05:03:47.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=5 2024-04-17T05:03:47.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=6 2024-04-17T05:03:47.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 20 add_watcher() i=7 2024-04-17T05:03:47.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.396+0000 7f4088af8980 2 all 8 watchers are set, enabling cache 2024-04-17T05:03:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.400+0000 7f3f3cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.400+0000 7f4088af8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:03:47.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.400+0000 7f4088af8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:03:47.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.528+0000 7f3eb77fe640 20 reqs_thread_entry: start 2024-04-17T05:03:47.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.528+0000 7f4088af8980 20 init_complete bucket index max shards: 1 2024-04-17T05:03:47.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.528+0000 7f4088af8980 20 Filter name: none 2024-04-17T05:03:47.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.528+0000 7f3eb57fa640 20 reqs_thread_entry: start 2024-04-17T05:03:47.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.528+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:03:47.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.532+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.532+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.532+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.536+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:03:47.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.540+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.540+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.540+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.544+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:03:47.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.544+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.544+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.544+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.548+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:03:47.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.552+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.552+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.552+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.556+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:03:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.560+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.560+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.560+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.564+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:03:47.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.568+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.568+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.568+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.568+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:03:47.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.572+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.572+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.576+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.576+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:03:47.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.580+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.580+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.580+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.584+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:03:47.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.584+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.584+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.588+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.588+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:03:47.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.592+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.592+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.592+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.596+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:03:47.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.600+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.600+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.600+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.604+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:03:47.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.608+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.608+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.608+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.608+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:03:47.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.612+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.616+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.616+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.616+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:03:47.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.620+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.620+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.620+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.620+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:03:47.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.624+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.624+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.624+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.628+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:03:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.632+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.632+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.636+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.636+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:03:47.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.640+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.640+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.640+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.640+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:03:47.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.644+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.644+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.648+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.648+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:03:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.652+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.652+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.652+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.656+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:03:47.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.660+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.660+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.660+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.664+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:03:47.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.668+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.668+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.668+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.668+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:03:47.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.672+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.676+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.676+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.676+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:03:47.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.680+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.680+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.680+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.684+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:03:47.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.688+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.688+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.688+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.692+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:03:47.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.696+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.700+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.700+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.704+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:03:47.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.708+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.708+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.712+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.712+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:03:47.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.716+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.716+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.720+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.720+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:03:47.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.724+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.728+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.728+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.732+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:03:47.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.736+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.736+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.736+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.740+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:03:47.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.744+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.744+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.744+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.748+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:03:47.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.752+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.752+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.752+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.756+0000 7f4088af8980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:03:47.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.760+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.760+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.764+0000 7f4088af8980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:47.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.768+0000 7f3f3cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:47.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.776+0000 7f4088af8980 20 remove_watcher() i=0 2024-04-17T05:03:47.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.776+0000 7f4088af8980 2 removed watcher, disabling cache 2024-04-17T05:03:47.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.776+0000 7f4088af8980 20 remove_watcher() i=1 2024-04-17T05:03:47.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.780+0000 7f4088af8980 20 remove_watcher() i=2 2024-04-17T05:03:47.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.784+0000 7f4088af8980 20 remove_watcher() i=3 2024-04-17T05:03:47.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.784+0000 7f4088af8980 20 remove_watcher() i=4 2024-04-17T05:03:47.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.788+0000 7f4088af8980 20 remove_watcher() i=5 2024-04-17T05:03:47.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.788+0000 7f4088af8980 20 remove_watcher() i=6 2024-04-17T05:03:47.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.792+0000 7f4088af8980 20 remove_watcher() i=7 2024-04-17T05:03:47.811 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:47.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.928+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 realm 2024-04-17T05:03:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:47.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:47.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.932+0000 7f7dfbfe2980 20 searching for the correct realm 2024-04-17T05:03:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:47.940+0000 7f7dfa2fe640 0 -- 172.21.15.88:0/2910296410 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f7df003b040 msgr2=0x7f7df003c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.156+0000 7f7df9afd640 0 -- 172.21.15.88:0/2910296410 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55ce176e07b0 msgr2=0x55ce176e2bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.160+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 zone default found 2024-04-17T05:03:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 4 Realm: () 2024-04-17T05:03:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 zonegroup default 2024-04-17T05:03:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 started sync module instance, tier type = 2024-04-17T05:03:48.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.164+0000 7f7dfbfe2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.184+0000 7f7dfbfe2980 20 add_watcher() i=0 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=1 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=2 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=3 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=4 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=5 2024-04-17T05:03:48.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=6 2024-04-17T05:03:48.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 20 add_watcher() i=7 2024-04-17T05:03:48.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.188+0000 7f7dfbfe2980 2 all 8 watchers are set, enabling cache 2024-04-17T05:03:48.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.192+0000 7f7caffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:48.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.192+0000 7f7dfbfe2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:03:48.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.192+0000 7f7dfbfe2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:03:48.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.280+0000 7f7caefe5640 20 reqs_thread_entry: start 2024-04-17T05:03:48.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.280+0000 7f7dfbfe2980 20 init_complete bucket index max shards: 1 2024-04-17T05:03:48.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.280+0000 7f7dfbfe2980 20 Filter name: none 2024-04-17T05:03:48.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.284+0000 7f7cacfe1640 20 reqs_thread_entry: start 2024-04-17T05:03:48.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.284+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:03:48.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.284+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.284+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.284+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.288+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:03:48.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.292+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.292+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.292+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.296+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:03:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.300+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.300+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.304+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.304+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:03:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.308+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.308+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.308+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.312+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:03:48.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.312+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.316+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.316+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.316+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:03:48.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.320+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.320+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.324+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.324+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:03:48.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.328+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.328+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.328+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.332+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:03:48.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.332+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.336+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.336+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.336+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:03:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.340+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.340+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.344+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.344+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:03:48.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.348+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.348+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.348+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.352+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:03:48.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.352+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.356+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.356+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.360+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:03:48.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.364+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.364+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.364+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.364+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:03:48.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.368+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.368+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.368+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.368+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:03:48.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.372+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.376+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.376+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.380+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:03:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.380+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.380+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.380+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.384+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:03:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.384+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.388+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.388+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.388+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:03:48.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.392+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.392+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.392+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.396+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:03:48.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.400+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.400+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.400+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.404+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.408+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.408+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.408+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.412+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:03:48.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.416+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.416+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.416+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.420+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:03:48.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.424+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.424+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.424+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.428+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:03:48.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.432+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.432+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.432+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.436+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:03:48.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.440+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.440+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.440+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.444+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:03:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.448+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.448+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.448+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.452+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:03:48.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.456+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.456+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.456+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.456+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:03:48.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.460+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.464+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.464+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.468+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:03:48.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.468+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.468+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.472+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.472+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:03:48.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.476+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.476+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.476+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.480+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:03:48.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.484+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.484+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.484+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.488+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:03:48.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.492+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.492+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.496+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.496+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:03:48.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.500+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.500+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.500+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.504+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:03:48.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.508+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.508+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.512+0000 7f7dfbfe2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.516+0000 7f7caffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:48.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.520+0000 7f7dfbfe2980 20 remove_watcher() i=0 2024-04-17T05:03:48.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.520+0000 7f7dfbfe2980 2 removed watcher, disabling cache 2024-04-17T05:03:48.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.528+0000 7f7dfbfe2980 20 remove_watcher() i=1 2024-04-17T05:03:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.528+0000 7f7dfbfe2980 20 remove_watcher() i=2 2024-04-17T05:03:48.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.532+0000 7f7dfbfe2980 20 remove_watcher() i=3 2024-04-17T05:03:48.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.536+0000 7f7dfbfe2980 20 remove_watcher() i=4 2024-04-17T05:03:48.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.536+0000 7f7dfbfe2980 20 remove_watcher() i=5 2024-04-17T05:03:48.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.536+0000 7f7dfbfe2980 20 remove_watcher() i=6 2024-04-17T05:03:48.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.540+0000 7f7dfbfe2980 20 remove_watcher() i=7 2024-04-17T05:03:48.560 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:48.633 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:48.633 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:48.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 realm 2024-04-17T05:03:48.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:48.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:48.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.680+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.684+0000 7fbeed428980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:48.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.684+0000 7fbeed428980 20 searching for the correct realm 2024-04-17T05:03:48.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.688+0000 7fbeea742640 0 -- 172.21.15.88:0/799831342 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbedc0aa030 msgr2=0x7fbedc1043c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:48.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.696+0000 7fbeea742640 0 -- 172.21.15.88:0/799831342 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fbdb4076460 msgr2=0x7fbdb4078890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 zone default found 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 4 Realm: () 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:48.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:48.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 zonegroup default 2024-04-17T05:03:48.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.704+0000 7fbeed428980 20 rados->read ofs=0 len=0 2024-04-17T05:03:48.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.708+0000 7fbeed428980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:48.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.708+0000 7fbeed428980 20 started sync module instance, tier type = 2024-04-17T05:03:48.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.708+0000 7fbeed428980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:03:48.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.724+0000 7fbeed428980 20 add_watcher() i=0 2024-04-17T05:03:48.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.724+0000 7fbeed428980 20 add_watcher() i=1 2024-04-17T05:03:48.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.724+0000 7fbeed428980 20 add_watcher() i=2 2024-04-17T05:03:48.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.724+0000 7fbeed428980 20 add_watcher() i=3 2024-04-17T05:03:48.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.724+0000 7fbeed428980 20 add_watcher() i=4 2024-04-17T05:03:48.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.724+0000 7fbeed428980 20 add_watcher() i=5 2024-04-17T05:03:48.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.728+0000 7fbeed428980 20 add_watcher() i=6 2024-04-17T05:03:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.728+0000 7fbeed428980 20 add_watcher() i=7 2024-04-17T05:03:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.728+0000 7fbeed428980 2 all 8 watchers are set, enabling cache 2024-04-17T05:03:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.728+0000 7fbda1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.728+0000 7fbeed428980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:03:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.728+0000 7fbeed428980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:03:48.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.804+0000 7fbda0fe1640 20 reqs_thread_entry: start 2024-04-17T05:03:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.804+0000 7fbeed428980 20 init_complete bucket index max shards: 1 2024-04-17T05:03:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.804+0000 7fbeed428980 20 Filter name: none 2024-04-17T05:03:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.804+0000 7fbd99ffb640 20 reqs_thread_entry: start 2024-04-17T05:03:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.804+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:03:48.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.808+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.812+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.812+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.812+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:03:48.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.816+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.816+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.816+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.816+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:03:48.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.820+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.820+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.820+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.824+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:03:48.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.828+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.828+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.828+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.828+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:03:48.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.832+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.836+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.836+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.840+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:03:48.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.840+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.844+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.844+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.844+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:03:48.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.848+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.852+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.852+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.852+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:03:48.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.856+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.860+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.860+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.860+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:03:48.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.864+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.868+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.868+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.872+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:03:48.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.876+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.876+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.876+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.880+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:03:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.884+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.884+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.888+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.888+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:03:48.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.892+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.892+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.892+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.896+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:03:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.900+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.900+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.900+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.904+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:03:48.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.908+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.908+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.908+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.912+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:03:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.916+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.916+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.916+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.920+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:03:48.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.924+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.924+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.924+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.928+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:03:48.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.932+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.932+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.936+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.936+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:03:48.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.940+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.944+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.944+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.944+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:03:48.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.952+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.952+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.952+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.956+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:03:48.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.960+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.960+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.964+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.964+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:03:48.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.968+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.972+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.972+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.976+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:03:48.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.980+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.980+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.984+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.984+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:03:48.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.988+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.988+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:48.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.992+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:48.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.992+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:03:48.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:48.996+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.000+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.000+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.004+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:03:49.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.008+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.008+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.008+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.012+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:03:49.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.016+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.016+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.016+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.020+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:03:49.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.024+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.024+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.028+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.032+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:03:49.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.036+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.036+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.036+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.040+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:03:49.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.044+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.044+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.048+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.048+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:03:49.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.052+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.056+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.056+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.060+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:03:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.064+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.064+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.064+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.068+0000 7fbeed428980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:03:49.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.072+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.072+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:49.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.072+0000 7fbeed428980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:49.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.076+0000 7fbda1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:49.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.084+0000 7fbeed428980 20 remove_watcher() i=0 2024-04-17T05:03:49.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.084+0000 7fbeed428980 2 removed watcher, disabling cache 2024-04-17T05:03:49.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.088+0000 7fbeed428980 20 remove_watcher() i=1 2024-04-17T05:03:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.088+0000 7fbeed428980 20 remove_watcher() i=2 2024-04-17T05:03:49.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.096+0000 7fbeed428980 20 remove_watcher() i=3 2024-04-17T05:03:49.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.096+0000 7fbeed428980 20 remove_watcher() i=4 2024-04-17T05:03:49.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.100+0000 7fbeed428980 20 remove_watcher() i=5 2024-04-17T05:03:49.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.104+0000 7fbeed428980 20 remove_watcher() i=6 2024-04-17T05:03:49.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:49.108+0000 7fbeed428980 20 remove_watcher() i=7 2024-04-17T05:03:49.131 DEBUG:teuthology.orchestra.run.smithi088:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-04-17T05:03:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :clean 2024-04-17T05:03:50.669 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :compileJava 2024-04-17T05:03:50.669 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :processResources 2024-04-17T05:03:50.669 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :classes 2024-04-17T05:03:51.270 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:51.270 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :compileTestJava 2024-04-17T05:03:51.270 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:51.270 INFO:teuthology.orchestra.run.smithi088.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-04-17T05:03:51.270 INFO:teuthology.orchestra.run.smithi088.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-04-17T05:03:51.271 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :processTestResources NO-SOURCE 2024-04-17T05:03:51.271 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :testClasses 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :test 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests STANDARD_OUT 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 05:03:52,673 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.271 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-04-17T05:03:53.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.369 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-04-17T05:03:53.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.469 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-04-17T05:03:53.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:53.969 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-04-17T05:03:54.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:54.069 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-04-17T05:03:54.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:54.469 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-04-17T05:03:54.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:54.769 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-04-17T05:03:54.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:54.969 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-04-17T05:03:55.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:55.269 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-04-17T05:03:55.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:55.870 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-04-17T05:03:55.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:55.870 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-04-17T05:03:56.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:56.069 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-04-17T05:03:56.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:56.469 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-04-17T05:03:57.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:57.269 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-04-17T05:03:57.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:57.469 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-04-17T05:03:57.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:57.869 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-04-17T05:03:57.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:57.969 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-04-17T05:03:58.115 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:58.115 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------- 2024-04-17T05:03:58.115 INFO:teuthology.orchestra.run.smithi088.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-04-17T05:03:58.115 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------- 2024-04-17T05:03:58.115 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:58.115 INFO:teuthology.orchestra.run.smithi088.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-04-17T05:03:58.116 INFO:teuthology.orchestra.run.smithi088.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-04-17T05:03:58.116 INFO:teuthology.orchestra.run.smithi088.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-04-17T05:03:58.116 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:03:58.116 INFO:teuthology.orchestra.run.smithi088.stdout:BUILD SUCCESSFUL in 8s 2024-04-17T05:03:58.116 INFO:teuthology.orchestra.run.smithi088.stdout:5 actionable tasks: 5 executed 2024-04-17T05:03:58.123 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:58.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.192+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 realm 2024-04-17T05:03:58.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:58.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:58.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:58.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.196+0000 7fcdbe484980 20 searching for the correct realm 2024-04-17T05:03:58.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.204+0000 7fcdbc7a0640 0 -- 172.21.15.88:0/3338540914 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fcdb001a040 msgr2=0x7fcdb005b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:58.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.420+0000 7fcdbbf9f640 0 -- 172.21.15.88:0/3338540914 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x563edd525240 msgr2=0x563edd527670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:58.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.428+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 zone default found 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 4 Realm: () 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 zonegroup default 2024-04-17T05:03:58.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 started sync module instance, tier type = 2024-04-17T05:03:58.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.432+0000 7fcdbe484980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=0 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=1 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=2 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=3 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=4 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=5 2024-04-17T05:03:58.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.464+0000 7fcdbe484980 20 add_watcher() i=6 2024-04-17T05:03:58.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.468+0000 7fcdbe484980 20 add_watcher() i=7 2024-04-17T05:03:58.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.468+0000 7fcdbe484980 2 all 8 watchers are set, enabling cache 2024-04-17T05:03:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.472+0000 7fcc71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:58.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.472+0000 7fcdbe484980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:03:58.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.472+0000 7fcdbe484980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:03:58.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.568+0000 7fcc70fe1640 20 reqs_thread_entry: start 2024-04-17T05:03:58.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcdbe484980 20 init_complete bucket index max shards: 1 2024-04-17T05:03:58.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcdbe484980 20 Filter name: none 2024-04-17T05:03:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcbea7fc640 20 reqs_thread_entry: start 2024-04-17T05:03:58.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:03:58.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.572+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.576+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:03:58.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.576+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.580+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.580+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.580+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:03:58.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.584+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.584+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.584+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.588+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:03:58.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.592+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.592+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.592+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.596+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:03:58.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.600+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.600+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.604+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.604+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:03:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.608+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.608+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.608+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.612+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:03:58.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.616+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.616+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.616+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.620+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:03:58.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.620+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.624+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.624+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:03:58.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.624+0000 7fcdbe484980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2666225590522195127', time=2024-04-17T07:03:55.475012+0000, chain.objs.size()=1 2024-04-17T05:03:58.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.624+0000 7fcdbe484980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.61__shadow_.r0cdg-_ArxZ2Hyu33vu5-zRT6lmcA3A_0 2024-04-17T05:03:58.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.624+0000 7fcdbe484980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13299556788234363197', time=2024-04-17T07:03:56.610120+0000, chain.objs.size()=1 2024-04-17T05:03:58.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.624+0000 7fcdbe484980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.62__shadow_.ycJXBN-L7tlYwSW9WKTRdCZwyLK7T58_0 2024-04-17T05:03:58.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.632+0000 7fcdbe484980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:03:58.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.636+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:03:58.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.640+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.640+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.640+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.644+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:03:58.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.648+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.648+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.652+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.652+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:03:58.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.656+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.660+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.660+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.660+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:03:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.664+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.668+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.668+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.672+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:03:58.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.672+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.676+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.676+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.676+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:03:58.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.680+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.684+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.684+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.684+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:03:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.692+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.692+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.692+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.696+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:03:58.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.700+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.704+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.704+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.708+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.708+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.712+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.712+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.712+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:03:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.716+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.720+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.720+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.724+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:03:58.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.728+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.728+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.728+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.732+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:03:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.736+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.736+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.740+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.740+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.744+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.748+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.748+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.748+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:03:58.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.752+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.752+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.756+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.756+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:03:58.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.760+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.760+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.764+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.764+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.768+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.768+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.768+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.772+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.776+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.776+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.780+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.780+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:03:58.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.784+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.784+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.784+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.788+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:03:58.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.792+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.792+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.792+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.796+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:03:58.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.800+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.800+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.800+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.804+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:03:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.804+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.808+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.808+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.812+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:03:58.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.812+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.816+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.816+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.816+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:03:58.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.820+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.820+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.820+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.824+0000 7fcdbe484980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:03:58.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.828+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.828+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.828+0000 7fcdbe484980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:58.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.836+0000 7fcc71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.840+0000 7fcdbe484980 20 remove_watcher() i=0 2024-04-17T05:03:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.840+0000 7fcdbe484980 2 removed watcher, disabling cache 2024-04-17T05:03:58.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.844+0000 7fcdbe484980 20 remove_watcher() i=1 2024-04-17T05:03:58.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.844+0000 7fcdbe484980 20 remove_watcher() i=2 2024-04-17T05:03:58.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.848+0000 7fcdbe484980 20 remove_watcher() i=3 2024-04-17T05:03:58.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.848+0000 7fcdbe484980 20 remove_watcher() i=4 2024-04-17T05:03:58.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.852+0000 7fcdbe484980 20 remove_watcher() i=5 2024-04-17T05:03:58.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.852+0000 7fcdbe484980 20 remove_watcher() i=6 2024-04-17T05:03:58.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.856+0000 7fcdbe484980 20 remove_watcher() i=7 2024-04-17T05:03:58.872 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:58.947 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:58.948 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:58.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 realm 2024-04-17T05:03:58.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:58.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:58.996+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.000+0000 7f52d4b57980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.000+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.000+0000 7f52d4b57980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.000+0000 7f52d4b57980 20 searching for the correct realm 2024-04-17T05:03:59.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.008+0000 7f52d2e73640 0 -- 172.21.15.88:0/1261147346 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f52c8074850 msgr2=0x7f52c80850b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:59.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.224+0000 7f52d2672640 0 -- 172.21.15.88:0/1261147346 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x563e20776c20 msgr2=0x563e20779050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:59.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:59.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:59.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:59.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:59.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:59.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:59.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:59.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 zone default found 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 4 Realm: () 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 zonegroup default 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.232+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 started sync module instance, tier type = 2024-04-17T05:03:59.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.236+0000 7f52d4b57980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:03:59.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=0 2024-04-17T05:03:59.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=1 2024-04-17T05:03:59.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=2 2024-04-17T05:03:59.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=3 2024-04-17T05:03:59.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=4 2024-04-17T05:03:59.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=5 2024-04-17T05:03:59.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.260+0000 7f52d4b57980 20 add_watcher() i=6 2024-04-17T05:03:59.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.264+0000 7f52d4b57980 20 add_watcher() i=7 2024-04-17T05:03:59.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.264+0000 7f52d4b57980 2 all 8 watchers are set, enabling cache 2024-04-17T05:03:59.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.268+0000 7f5188fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:59.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.268+0000 7f52d4b57980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:03:59.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.268+0000 7f52d4b57980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:03:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.364+0000 7f51037fe640 20 reqs_thread_entry: start 2024-04-17T05:03:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.364+0000 7f52d4b57980 20 init_complete bucket index max shards: 1 2024-04-17T05:03:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.364+0000 7f52d4b57980 20 Filter name: none 2024-04-17T05:03:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.364+0000 7f51017fa640 20 reqs_thread_entry: start 2024-04-17T05:03:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.364+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:03:59.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.368+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.368+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.368+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.368+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:03:59.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.372+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.372+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.372+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.376+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:03:59.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.380+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.380+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.380+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.384+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:03:59.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.388+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.388+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.388+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.392+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:03:59.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.396+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.396+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.396+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.400+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:03:59.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.404+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.404+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.404+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.408+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:03:59.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.408+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.412+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.412+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.412+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:03:59.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.416+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.420+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.420+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.420+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:03:59.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.424+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.428+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.428+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.428+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:03:59.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.432+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.432+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.432+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.436+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:03:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.440+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.440+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.440+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.448+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:03:59.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.452+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.452+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.452+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.456+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:03:59.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.456+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.460+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.460+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.460+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:03:59.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.464+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.464+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.464+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.468+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:03:59.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.472+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.472+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.472+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.476+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:03:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.476+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.480+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.480+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.480+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:03:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.484+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.484+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.488+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.488+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:03:59.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.492+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.492+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.492+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.496+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:03:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.496+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.500+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.500+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.500+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:03:59.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.504+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.504+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.508+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.508+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:03:59.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.512+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.512+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.512+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.516+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:03:59.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.516+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.520+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.520+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.520+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:03:59.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.524+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.524+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.524+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.528+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:03:59.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.532+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.532+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.532+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.536+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:03:59.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.540+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.540+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.540+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.540+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:03:59.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.544+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.548+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.548+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.548+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:03:59.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.552+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.552+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.552+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.556+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:03:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.560+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.560+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.560+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.564+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:03:59.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.568+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.568+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.572+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.572+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:03:59.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.576+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.580+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.580+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.584+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:03:59.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.584+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.588+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.588+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.588+0000 7f52d4b57980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:03:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.592+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.596+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:03:59.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.596+0000 7f52d4b57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:03:59.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.600+0000 7f5188fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:03:59.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.604+0000 7f52d4b57980 20 remove_watcher() i=0 2024-04-17T05:03:59.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.604+0000 7f52d4b57980 2 removed watcher, disabling cache 2024-04-17T05:03:59.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.608+0000 7f52d4b57980 20 remove_watcher() i=1 2024-04-17T05:03:59.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.608+0000 7f52d4b57980 20 remove_watcher() i=2 2024-04-17T05:03:59.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.612+0000 7f52d4b57980 20 remove_watcher() i=3 2024-04-17T05:03:59.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.616+0000 7f52d4b57980 20 remove_watcher() i=4 2024-04-17T05:03:59.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.616+0000 7f52d4b57980 20 remove_watcher() i=5 2024-04-17T05:03:59.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.616+0000 7f52d4b57980 20 remove_watcher() i=6 2024-04-17T05:03:59.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.620+0000 7f52d4b57980 20 remove_watcher() i=7 2024-04-17T05:03:59.634 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:03:59.702 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:03:59.702 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:03:59.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.748+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 realm 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:03:59.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.752+0000 7f5dd7510980 20 searching for the correct realm 2024-04-17T05:03:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.760+0000 7f5dd582c640 0 -- 172.21.15.88:0/406079265 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f5dcc0c0040 msgr2=0x7f5dcc0ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:59.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.976+0000 7f5dd502b640 0 -- 172.21.15.88:0/406079265 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5642acebb050 msgr2=0x5642acebd440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:03:59.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:03:59.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 zone default found 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 4 Realm: () 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.984+0000 7f5dd7510980 20 zonegroup default 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 rados->read ofs=0 len=0 2024-04-17T05:03:59.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:03:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 started sync module instance, tier type = 2024-04-17T05:03:59.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:03:59.988+0000 7f5dd7510980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=0 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=1 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=2 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=3 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=4 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=5 2024-04-17T05:04:00.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.008+0000 7f5dd7510980 20 add_watcher() i=6 2024-04-17T05:04:00.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.012+0000 7f5dd7510980 20 add_watcher() i=7 2024-04-17T05:04:00.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.012+0000 7f5dd7510980 2 all 8 watchers are set, enabling cache 2024-04-17T05:04:00.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.016+0000 7f5c8bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:04:00.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.016+0000 7f5dd7510980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:04:00.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.016+0000 7f5dd7510980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:04:00.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.092+0000 7f5c8afe5640 20 reqs_thread_entry: start 2024-04-17T05:04:00.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.092+0000 7f5dd7510980 20 init_complete bucket index max shards: 1 2024-04-17T05:04:00.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.092+0000 7f5dd7510980 20 Filter name: none 2024-04-17T05:04:00.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.092+0000 7f5c88fe1640 20 reqs_thread_entry: start 2024-04-17T05:04:00.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.092+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:04:00.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.096+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.096+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.096+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.100+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:04:00.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.100+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.104+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.104+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.104+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:04:00.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.108+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.108+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.112+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.112+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:04:00.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.116+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.116+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.116+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.116+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:04:00.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.120+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.120+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.120+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.124+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:04:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.124+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.128+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.128+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.128+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:04:00.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.132+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.132+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.132+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.136+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:04:00.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.136+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.136+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.136+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.140+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.144+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.144+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.144+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.148+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:04:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.152+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.152+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.152+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.156+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:04:00.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.156+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.160+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.160+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.164+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:04:00.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.164+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.164+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.164+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.168+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:04:00.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.168+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.168+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.168+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.172+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:04:00.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.176+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.176+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.176+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.180+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:04:00.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.184+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.184+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.184+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.188+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:04:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.188+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.192+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.192+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.192+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:04:00.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.196+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.196+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.196+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.200+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:04:00.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.204+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.204+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.208+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.208+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:04:00.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.212+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.216+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.216+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.216+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:04:00.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.220+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.224+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.224+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.228+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:04:00.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.228+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.228+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.232+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.232+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:04:00.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.236+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.236+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.236+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.240+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:04:00.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.244+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.244+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.248+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.248+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:04:00.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.252+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.256+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.256+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.260+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:04:00.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.260+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.264+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.264+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.268+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:04:00.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.272+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.272+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.276+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.276+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:04:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.280+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.284+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.284+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.284+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:04:00.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.288+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.292+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.292+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.296+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:04:00.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.300+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.300+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.300+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.304+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:04:00.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.308+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.308+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.312+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.312+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:04:00.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.316+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.316+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.320+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.320+0000 7f5dd7510980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:04:00.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.324+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.328+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.328+0000 7f5dd7510980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.332+0000 7f5c8bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:04:00.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.336+0000 7f5dd7510980 20 remove_watcher() i=0 2024-04-17T05:04:00.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.336+0000 7f5dd7510980 2 removed watcher, disabling cache 2024-04-17T05:04:00.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.340+0000 7f5dd7510980 20 remove_watcher() i=1 2024-04-17T05:04:00.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.340+0000 7f5dd7510980 20 remove_watcher() i=2 2024-04-17T05:04:00.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.344+0000 7f5dd7510980 20 remove_watcher() i=3 2024-04-17T05:04:00.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.344+0000 7f5dd7510980 20 remove_watcher() i=4 2024-04-17T05:04:00.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.348+0000 7f5dd7510980 20 remove_watcher() i=5 2024-04-17T05:04:00.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.348+0000 7f5dd7510980 20 remove_watcher() i=6 2024-04-17T05:04:00.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.352+0000 7f5dd7510980 20 remove_watcher() i=7 2024-04-17T05:04:00.366 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:04:00.437 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:04:00.437 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:04:00.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.484+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 realm 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:04:00.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:04:00.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.488+0000 7f0320216980 20 searching for the correct realm 2024-04-17T05:04:00.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.496+0000 7f031d530640 0 -- 172.21.15.88:0/141831733 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x55a0eb6900f0 msgr2=0x55a0eb6925b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:04:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.500+0000 7f031dd31640 0 -- 172.21.15.88:0/141831733 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f03140ab980 msgr2=0x7f0314102900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:04:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:04:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:04:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.508+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 zone default found 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 4 Realm: () 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:04:00.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 zonegroup default 2024-04-17T05:04:00.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados->read ofs=0 len=0 2024-04-17T05:04:00.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:04:00.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 started sync module instance, tier type = 2024-04-17T05:04:00.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.512+0000 7f0320216980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:04:00.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=0 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=1 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=2 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=3 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=4 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=5 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=6 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 20 add_watcher() i=7 2024-04-17T05:04:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.528+0000 7f0320216980 2 all 8 watchers are set, enabling cache 2024-04-17T05:04:00.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.532+0000 7f01d3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:04:00.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.532+0000 7f0320216980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:04:00.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.532+0000 7f0320216980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:04:00.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.604+0000 7f01d2fe5640 20 reqs_thread_entry: start 2024-04-17T05:04:00.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.604+0000 7f0320216980 20 init_complete bucket index max shards: 1 2024-04-17T05:04:00.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.604+0000 7f0320216980 20 Filter name: none 2024-04-17T05:04:00.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.604+0000 7f01d0fe1640 20 reqs_thread_entry: start 2024-04-17T05:04:00.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.604+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:04:00.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.608+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.608+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.608+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.608+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:04:00.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.612+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.616+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.616+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.620+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:04:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.620+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.620+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.620+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.624+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:04:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.624+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.624+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.624+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.628+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:04:00.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.632+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.632+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.632+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.636+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:04:00.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.640+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.640+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.640+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.644+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:04:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.644+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.648+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.648+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.648+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:04:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.652+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.652+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.656+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.656+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:04:00.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.660+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.660+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.660+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.664+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:04:00.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.668+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.668+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.672+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.672+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:04:00.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.676+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.676+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.680+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.680+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:04:00.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.684+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.684+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.684+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.684+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:04:00.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.688+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.692+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.692+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.696+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:04:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.700+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.700+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.700+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.700+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:04:00.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.704+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.704+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.708+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.708+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:04:00.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.712+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.712+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.712+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.716+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:04:00.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.720+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.720+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.720+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.724+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:04:00.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.728+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.728+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.728+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.732+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:04:00.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.736+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.736+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.736+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.740+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:04:00.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.744+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.744+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.744+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.748+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:04:00.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.748+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.752+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.752+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.752+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:04:00.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.756+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.756+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.760+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.760+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:04:00.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.764+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.764+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.764+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.768+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:04:00.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.768+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.772+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.772+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.772+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:04:00.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.780+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.780+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.780+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.784+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:04:00.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.788+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.792+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.792+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.796+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:04:00.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.796+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.800+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.800+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.804+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:04:00.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.804+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.808+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.808+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.812+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:04:00.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.816+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.820+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.820+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.820+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:04:00.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.824+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.828+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.828+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.828+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:04:00.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.832+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.836+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.836+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.840+0000 7f0320216980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:04:00.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.840+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.844+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:04:00.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.844+0000 7f0320216980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:04:00.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.848+0000 7f01d3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:04:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.852+0000 7f0320216980 20 remove_watcher() i=0 2024-04-17T05:04:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.852+0000 7f0320216980 2 removed watcher, disabling cache 2024-04-17T05:04:00.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.856+0000 7f0320216980 20 remove_watcher() i=1 2024-04-17T05:04:00.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.860+0000 7f0320216980 20 remove_watcher() i=2 2024-04-17T05:04:00.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.860+0000 7f0320216980 20 remove_watcher() i=3 2024-04-17T05:04:00.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.864+0000 7f0320216980 20 remove_watcher() i=4 2024-04-17T05:04:00.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.864+0000 7f0320216980 20 remove_watcher() i=5 2024-04-17T05:04:00.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.868+0000 7f0320216980 20 remove_watcher() i=6 2024-04-17T05:04:00.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:04:00.868+0000 7f0320216980 20 remove_watcher() i=7 2024-04-17T05:04:00.883 DEBUG:teuthology.orchestra.run.smithi088:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-04-17T05:04:01.598 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :clean 2024-04-17T05:04:02.396 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :compileJava 2024-04-17T05:04:02.396 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :processResources 2024-04-17T05:04:02.396 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :classes 2024-04-17T05:04:02.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:02.896 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :compileTestJava 2024-04-17T05:04:02.896 INFO:teuthology.orchestra.run.smithi088.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-04-17T05:04:02.896 INFO:teuthology.orchestra.run.smithi088.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-04-17T05:04:02.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:02.897 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :processTestResources NO-SOURCE 2024-04-17T05:04:02.897 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :testClasses 2024-04-17T05:04:04.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:04.896 INFO:teuthology.orchestra.run.smithi088.stdout:> Task :test 2024-04-17T05:04:04.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:04.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests STANDARD_OUT 2024-04-17T05:04:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 05:04:04,195 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-04-17T05:04:04.898 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:04.898 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-04-17T05:04:04.898 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:04.898 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-04-17T05:04:05.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:05.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-04-17T05:04:05.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:05.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-04-17T05:04:06.095 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:06.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-04-17T05:04:06.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:06.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-04-17T05:04:06.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:06.919 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-04-17T05:04:10.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:10.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-04-17T05:04:10.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:10.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-04-17T05:04:10.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:10.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-04-17T05:04:11.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:11.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-04-17T05:04:11.900 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:11.901 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-04-17T05:04:12.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:12.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-04-17T05:04:12.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:12.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-04-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-04-17T05:04:12.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:12.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-04-17T05:04:13.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:13.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-04-17T05:04:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:15.299 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-04-17T05:04:15.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:15.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-04-17T05:04:15.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:15.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-04-17T05:04:16.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:16.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-04-17T05:04:16.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:16.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-04-17T05:04:17.295 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:17.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-04-17T05:04:17.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:17.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-04-17T05:04:17.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:17.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-04-17T05:04:18.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:18.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-04-17T05:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-04-17T05:04:18.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:18.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-04-17T05:04:19.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:19.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-04-17T05:04:19.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:19.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-04-17T05:04:19.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:19.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-04-17T05:04:20.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:20.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-04-17T05:04:20.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:20.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-04-17T05:04:33.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:33.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-04-17T05:04:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:04:34.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-04-17T05:05:10.097 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:10.097 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-04-17T05:05:10.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:10.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-04-17T05:05:11.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:11.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-04-17T05:05:11.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:11.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-04-17T05:05:11.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:11.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-04-17T05:05:12.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:12.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-04-17T05:05:15.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:15.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-04-17T05:05:16.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:16.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-04-17T05:05:16.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:16.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-04-17T05:05:16.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:16.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-04-17T05:05:21.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:21.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-04-17T05:05:22.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:22.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests STANDARD_OUT 2024-04-17T05:05:22.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 05:05:22,355 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-17T05:05:22.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17 05:05:22,361 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-17T05:05:22.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:22.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-04-17T05:05:31.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:31.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-04-17T05:05:32.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:32.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-04-17T05:05:43.998 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:43.998 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-04-17T05:05:44.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:05:44.546 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-04-17T05:06:21.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:21.697 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-04-17T05:06:22.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:22.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-04-17T05:06:26.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:26.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-04-17T05:06:27.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:27.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-04-17T05:06:30.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:30.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-04-17T05:06:31.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:31.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-04-17T05:06:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:35.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-04-17T05:06:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:49.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-04-17T05:06:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:49.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-04-17T05:06:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-04-17T05:06:50.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:50.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-04-17T05:06:54.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:54.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-04-17T05:06:54.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:54.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-04-17T05:06:54.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:54.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-04-17T05:06:54.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:54.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-04-17T05:06:55.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:55.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-04-17T05:06:55.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:55.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-04-17T05:06:56.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:56.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-04-17T05:06:56.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:56.297 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-04-17T05:06:56.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:56.697 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-04-17T05:06:56.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:56.897 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-04-17T05:06:57.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:57.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-04-17T05:06:57.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:57.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-04-17T05:06:58.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:58.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-04-17T05:06:59.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:59.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-04-17T05:06:59.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:06:59.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-04-17T05:07:00.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:00.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-04-17T05:07:01.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:01.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-04-17T05:07:01.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:01.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-04-17T05:07:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:02.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-04-17T05:07:03.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:03.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-04-17T05:07:04.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:04.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-04-17T05:07:04.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:04.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-04-17T05:07:05.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:05.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-04-17T05:07:06.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:06.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-04-17T05:07:06.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:06.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-04-17T05:07:07.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:07.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-04-17T05:07:08.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:08.197 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-04-17T05:07:08.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:08.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-04-17T05:07:09.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:09.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-04-17T05:07:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:09.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-04-17T05:07:10.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:10.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-04-17T05:07:11.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:11.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-04-17T05:07:11.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:11.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-04-17T05:07:12.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:12.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-04-17T05:07:13.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:13.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-04-17T05:07:13.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:13.496 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-04-17T05:07:14.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:14.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-04-17T05:07:14.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:14.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-04-17T05:07:15.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:15.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-04-17T05:07:16.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:16.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-04-17T05:07:16.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:16.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-04-17T05:07:17.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:17.397 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-04-17T05:07:18.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:18.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-04-17T05:07:18.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:18.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-04-17T05:07:19.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:19.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-04-17T05:07:19.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:19.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-04-17T05:07:20.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:20.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-04-17T05:07:21.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:21.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-04-17T05:07:21.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:21.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-04-17T05:07:22.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:22.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-04-17T05:07:22.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:22.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-04-17T05:07:23.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:23.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-04-17T05:07:23.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:23.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-04-17T05:07:24.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:24.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-04-17T05:07:25.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:25.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-04-17T05:07:25.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:25.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-04-17T05:07:25.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:25.697 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-04-17T05:07:25.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:25.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-04-17T05:07:26.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:26.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-04-17T05:07:26.497 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:26.497 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-04-17T05:07:26.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:26.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-04-17T05:07:27.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:27.196 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-04-17T05:07:27.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:27.696 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-04-17T05:07:28.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:28.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-04-17T05:07:28.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:28.797 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-04-17T05:07:28.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:28.797 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-04-17T05:07:28.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:28.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-04-17T05:07:29.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:29.296 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-04-17T05:07:29.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:29.796 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-04-17T05:07:30.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:30.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-04-17T05:07:30.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:30.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-04-17T05:07:30.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:30.896 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-04-17T05:07:31.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:31.396 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-04-17T05:07:31.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:31.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-04-17T05:07:31.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:31.996 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-04-17T05:07:36.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:36.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-04-17T05:07:36.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:36.596 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-04-17T05:07:40.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:40.096 INFO:teuthology.orchestra.run.smithi088.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-04-17T05:07:40.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:40.748 INFO:teuthology.orchestra.run.smithi088.stdout:---------------------------------------------------------------------- 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:---------------------------------------------------------------------- 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:BUILD SUCCESSFUL in 3m 39s 2024-04-17T05:07:40.749 INFO:teuthology.orchestra.run.smithi088.stdout:5 actionable tasks: 5 executed 2024-04-17T05:07:40.756 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:07:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:40.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.827+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 realm 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:40.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.831+0000 7fbbcd21c980 20 searching for the correct realm 2024-04-17T05:07:40.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:40.839+0000 7fbbcb538640 0 -- 172.21.15.88:0/1003657548 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fbbc0008050 msgr2=0x7fbbc005a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:41.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.055+0000 7fbbcad37640 0 -- 172.21.15.88:0/1003657548 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55627c2a2d00 msgr2=0x55627c2a5110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:41.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:41.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.063+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 zone default found 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 4 Realm: () 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 zonegroup default 2024-04-17T05:07:41.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 started sync module instance, tier type = 2024-04-17T05:07:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.067+0000 7fbbcd21c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:41.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=0 2024-04-17T05:07:41.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=1 2024-04-17T05:07:41.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=2 2024-04-17T05:07:41.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=3 2024-04-17T05:07:41.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=4 2024-04-17T05:07:41.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=5 2024-04-17T05:07:41.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.087+0000 7fbbcd21c980 20 add_watcher() i=6 2024-04-17T05:07:41.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.091+0000 7fbbcd21c980 20 add_watcher() i=7 2024-04-17T05:07:41.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.091+0000 7fbbcd21c980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:41.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.095+0000 7fba817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:41.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.095+0000 7fbbcd21c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:41.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.095+0000 7fbbcd21c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:41.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.175+0000 7fb9fbfff640 20 reqs_thread_entry: start 2024-04-17T05:07:41.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.175+0000 7fbbcd21c980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:41.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.175+0000 7fbbcd21c980 20 Filter name: none 2024-04-17T05:07:41.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.175+0000 7fb9f9ffb640 20 reqs_thread_entry: start 2024-04-17T05:07:41.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.175+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:07:41.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-17T05:07:41.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13280689708253326783', time=2024-04-17T07:07:03.547760+0000, chain.objs.size()=1 2024-04-17T05:07:41.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.106__shadow_.q23WIPTxrLSwFpsPkSRMKRiwLaJoxUE_0 2024-04-17T05:07:41.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17958342205362493292', time=2024-04-17T07:07:03.660069+0000, chain.objs.size()=1 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.106__shadow_.zLjuG6Yp_91MQrkuuQEbpBMWMq_mqGx_0 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5218659519376404856', time=2024-04-17T07:07:16.307846+0000, chain.objs.size()=1 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.116__shadow_.lyKG-1RedxP0YL4wn2osbPYLHLCJyyn_0 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15195581622563510032', time=2024-04-17T07:07:20.261277+0000, chain.objs.size()=1 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.119__shadow_.Q9VogKuR27n_CGxpmlnE7v4IozNDZmW_0 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13193584119881192782', time=2024-04-17T07:07:22.346044+0000, chain.objs.size()=1 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.179+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.121__shadow_.lSFSPE1Icc2bgn4Ug6ilI7CssrL85mg_0 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.183+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13540110105914619482', time=2024-04-17T07:07:24.569069+0000, chain.objs.size()=1 2024-04-17T05:07:41.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.183+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.123__shadow_.hnvnmJZJp-a1XyKBpMO-yMNS-WYRkSJ_0 2024-04-17T05:07:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.187+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.191+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:07:41.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.195+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9421042137031634011', time=2024-04-17T07:05:31.746787+0000, chain.objs.size()=4 2024-04-17T05:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file2.mpg.2~XuGcw9i9glPlhjeg7Km9Ng0-BW-SuQt.1 2024-04-17T05:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file2.mpg.2~XuGcw9i9glPlhjeg7Km9Ng0-BW-SuQt.2 2024-04-17T05:07:41.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file2.mpg.2~XuGcw9i9glPlhjeg7Km9Ng0-BW-SuQt.3 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file2.mpg.2~XuGcw9i9glPlhjeg7Km9Ng0-BW-SuQt.4 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7450818477671233306', time=2024-04-17T07:07:15.272478+0000, chain.objs.size()=1 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.115__shadow_.foQRR6G8-yhVUdqz8XWvZRimrD3pwPC_0 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7642851000460880336', time=2024-04-17T07:07:18.916715+0000, chain.objs.size()=1 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.118__shadow_.Rr9H4DQFG8BgiK3N3GA_42WnYUAE88x_0 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8702990697722274029', time=2024-04-17T07:07:21.393881+0000, chain.objs.size()=1 2024-04-17T05:07:41.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.199+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.120__shadow_.nlWVGxhoXJ4QfLRZduQcwm3UMXFcWua_0 2024-04-17T05:07:41.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.207+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.215+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:07:41.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17423024616175753455', time=2024-04-17T07:04:16.934640+0000, chain.objs.size()=1 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.75__shadow_.363M3m4JsfP-zY-l5BrXMLCeFGCr91t_0 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1967660692732109805', time=2024-04-17T07:05:09.924117+0000, chain.objs.size()=5 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~artVSTloj0nMGYFdLJ6v8j9KohkbtM9.1 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__shadow_key1.2~artVSTloj0nMGYFdLJ6v8j9KohkbtM9.1_1 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.219+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~artVSTloj0nMGYFdLJ6v8j9KohkbtM9.2 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.223+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__shadow_key1.2~artVSTloj0nMGYFdLJ6v8j9KohkbtM9.2_1 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.223+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~artVSTloj0nMGYFdLJ6v8j9KohkbtM9.3 2024-04-17T05:07:41.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.223+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3318730217835886246', time=2024-04-17T07:05:31.821483+0000, chain.objs.size()=1 2024-04-17T05:07:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.223+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__shadow_.RWZCL0B2krQYpSApX47p3pJyP69ngMq_0 2024-04-17T05:07:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.223+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16656866042103093309', time=2024-04-17T07:07:04.910457+0000, chain.objs.size()=1 2024-04-17T05:07:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.223+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.107__shadow_.Xll5mMw6IniSBFuoJltMj8kJVEoLSXJ_0 2024-04-17T05:07:41.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.231+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.239+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:07:41.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8320592868414734649', time=2024-04-17T07:06:06.447869+0000, chain.objs.size()=5 2024-04-17T05:07:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~jLSkq7gUHk_xjws_SyfDn_TR-EDJOGm.1 2024-04-17T05:07:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~jLSkq7gUHk_xjws_SyfDn_TR-EDJOGm.2 2024-04-17T05:07:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~jLSkq7gUHk_xjws_SyfDn_TR-EDJOGm.3 2024-04-17T05:07:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~jLSkq7gUHk_xjws_SyfDn_TR-EDJOGm.4 2024-04-17T05:07:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~jLSkq7gUHk_xjws_SyfDn_TR-EDJOGm.5 2024-04-17T05:07:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5000288586434226698', time=2024-04-17T07:07:28.180906+0000, chain.objs.size()=1 2024-04-17T05:07:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.243+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.128__shadow_.djyTP6Zo31-WPk2YdcJxqudd8meNjOi_0 2024-04-17T05:07:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.255+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.259+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:07:41.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8099099998098821063', time=2024-04-17T07:04:54.609057+0000, chain.objs.size()=5 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~T2ITV3W-CxBplge60nS-SzQ90m5df46.1 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~T2ITV3W-CxBplge60nS-SzQ90m5df46.2 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~T2ITV3W-CxBplge60nS-SzQ90m5df46.3 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~T2ITV3W-CxBplge60nS-SzQ90m5df46.4 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~T2ITV3W-CxBplge60nS-SzQ90m5df46.5 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8960833187213117347', time=2024-04-17T07:07:07.870685+0000, chain.objs.size()=1 2024-04-17T05:07:41.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.109__shadow_.bOn7PGEeM9du9t7raUDNVCfa1Y6rxSp_0 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3282033012827635746', time=2024-04-17T07:07:10.206131+0000, chain.objs.size()=1 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.111__shadow_.l3OHqfc2PHRUXd0dPU6Jww7sgiRNuEw_0 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17547065369786367592', time=2024-04-17T07:07:13.841648+0000, chain.objs.size()=1 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.114__shadow_.iJdy0YX2g3724izucQM8ul0S-hXkN46_0 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1718714732825964140', time=2024-04-17T07:07:29.475914+0000, chain.objs.size()=1 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.129__shadow_.gYcXcun3gVR_co0edX6ZOUyvgcneXCl_0 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10162272493631817210', time=2024-04-17T07:07:40.360659+0000, chain.objs.size()=1 2024-04-17T05:07:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.263+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134__shadow_.LIH9BC-0Mw_3CDaMWe_tIKUpVZh23wl_0 2024-04-17T05:07:41.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.275+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:07:41.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9396237201117679241', time=2024-04-17T07:04:05.738203+0000, chain.objs.size()=1 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.67__shadow_.8fDyItGzbnqBz4pCSpSNyvq0dG4YFcd_0 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15807398054864327506', time=2024-04-17T07:05:22.362591+0000, chain.objs.size()=5 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.89__multipart_key1.2~YjqvvdluMIsxuI5L6SrZybPduRaqox9.1 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.89__multipart_key1.2~YjqvvdluMIsxuI5L6SrZybPduRaqox9.2 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.89__multipart_key1.2~YjqvvdluMIsxuI5L6SrZybPduRaqox9.3 2024-04-17T05:07:41.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.89__multipart_key1.2~YjqvvdluMIsxuI5L6SrZybPduRaqox9.4 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.283+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.89__multipart_key1.2~YjqvvdluMIsxuI5L6SrZybPduRaqox9.5 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6772151814417232755', time=2024-04-17T07:07:03.428370+0000, chain.objs.size()=1 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.106__shadow_.gEZqcVRYhFR-3Ksq6Pvg0_5up76U3qX_0 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.386325562538802151', time=2024-04-17T07:07:11.524544+0000, chain.objs.size()=1 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.112__shadow_.8L9Fvt23-RFHnVyYsjeRt6sPzysgLaJ_0 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16791043193275030491', time=2024-04-17T07:07:11.601841+0000, chain.objs.size()=1 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.112__shadow_.DXk4x1hp9tkNIPEwEO1vLLdDmShQxwf_0 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9647605440602438869', time=2024-04-17T07:07:22.496884+0000, chain.objs.size()=1 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.121__shadow_.vpNLGxRZKKraz4GIxTGdeJkMddmpK_K_0 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.549028722444846519', time=2024-04-17T07:07:26.643740+0000, chain.objs.size()=1 2024-04-17T05:07:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.287+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.126__shadow_.GV4N4xuSbKTw1YyK_mLOSyLjyasBZQE_0 2024-04-17T05:07:41.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.295+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.299+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:07:41.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.303+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.303+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3342032350112734404', time=2024-04-17T07:07:02.187507+0000, chain.objs.size()=1 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.105__shadow_.DQjt5TbbmGHEKhD0ChAbcmqHgUdSdet_0 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7000270533238742272', time=2024-04-17T07:07:06.474719+0000, chain.objs.size()=1 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.108__shadow_.Z4NSfX-DqpFTxKPFpDK-mi6EAEpjB5t_0 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17004330356984521053', time=2024-04-17T07:07:06.559367+0000, chain.objs.size()=1 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.108__shadow_.07nzgIYxo_5nBhlwanJI4IOHRtMCXxr_0 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1784370599189637248', time=2024-04-17T07:07:17.805515+0000, chain.objs.size()=1 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.117__shadow_.dV9-EvufPriwi-AkARlmGLVrIvpp4wq_0 2024-04-17T05:07:41.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.418316885379111265', time=2024-04-17T07:07:18.842420+0000, chain.objs.size()=1 2024-04-17T05:07:41.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.118__shadow_.jHDuKa5v5h4VgLDaBMhC_1zmxw7V6eJ_0 2024-04-17T05:07:41.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14270774090265262718', time=2024-04-17T07:07:28.010269+0000, chain.objs.size()=1 2024-04-17T05:07:41.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.307+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.128__shadow_.iffKAIYiPP4rgkiGJGVd6SOxUNLYfNJ_0 2024-04-17T05:07:41.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.311+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.315+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:07:41.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9637391483914738470', time=2024-04-17T07:04:34.161338+0000, chain.objs.size()=5 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.1 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.2 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.3 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.4 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.5 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3615004811378562363', time=2024-04-17T07:07:00.675662+0000, chain.objs.size()=1 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.104__shadow_.cmpUfW6e1ubZwrsFow7RpInrIzbtRz2_0 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.628756637435275654', time=2024-04-17T07:07:07.797449+0000, chain.objs.size()=1 2024-04-17T05:07:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.109__shadow_.DOtvsaH8DiIPvm8ZwPq08-QT2L39_T6_0 2024-04-17T05:07:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1702604581714167197', time=2024-04-17T07:07:16.463629+0000, chain.objs.size()=1 2024-04-17T05:07:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.116__shadow_.keAW43uMx3pEzRI1Di_WVDTSenb-Haz_0 2024-04-17T05:07:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5133560566165157156', time=2024-04-17T07:07:16.539531+0000, chain.objs.size()=1 2024-04-17T05:07:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.319+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.116__shadow_.y7hcfxtCCzduVLDsiKdVle84No8r4GH_0 2024-04-17T05:07:41.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.331+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.335+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:07:41.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.335+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.335+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T05:07:41.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16935479825996771219', time=2024-04-17T07:05:15.910183+0000, chain.objs.size()=5 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.88__multipart_key1.2~p-aOdSAVZnCfAH5VZLTYglklyjvyCgm.1 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.88__multipart_key1.2~p-aOdSAVZnCfAH5VZLTYglklyjvyCgm.2 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.88__multipart_key1.2~p-aOdSAVZnCfAH5VZLTYglklyjvyCgm.3 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.88__multipart_key1.2~p-aOdSAVZnCfAH5VZLTYglklyjvyCgm.4 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.88__multipart_key1.2~p-aOdSAVZnCfAH5VZLTYglklyjvyCgm.5 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12967167250071293894', time=2024-04-17T07:07:03.307240+0000, chain.objs.size()=1 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.106__shadow_.oifUm3lg8RWf3mwca9D-Sv6PUCXZ1TU_0 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16958021236303100051', time=2024-04-17T07:07:10.428880+0000, chain.objs.size()=1 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.111__shadow_.iMKWsoKPmLuraWFB3LPCXRRccpIZ62P_0 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8918684034756964248', time=2024-04-17T07:07:13.688463+0000, chain.objs.size()=1 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.114__shadow_.I_3AW6gVMX_rl1Y1VhcZOSuAJeZTxWU_0 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6721150946415835207', time=2024-04-17T07:07:13.762853+0000, chain.objs.size()=1 2024-04-17T05:07:41.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.339+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.114__shadow_.begnXDYVlgZEI2O_shbHda6mQO-qRfU_0 2024-04-17T05:07:41.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.347+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.351+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:07:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14782116007280542318', time=2024-04-17T07:05:31.599224+0000, chain.objs.size()=5 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file.mpg.2~HsfxREzciz4C6IaQl0uR6-7XuCbMk7A.1 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file.mpg.2~HsfxREzciz4C6IaQl0uR6-7XuCbMk7A.2 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file.mpg.2~HsfxREzciz4C6IaQl0uR6-7XuCbMk7A.3 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file.mpg.2~HsfxREzciz4C6IaQl0uR6-7XuCbMk7A.4 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__multipart_file.mpg.2~HsfxREzciz4C6IaQl0uR6-7XuCbMk7A.5 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17050768559623170979', time=2024-04-17T07:06:21.883434+0000, chain.objs.size()=5 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~W4Cfw0gR5VUGQzJlU-hKYX34U5FT1X8.1 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__shadow_key1.2~W4Cfw0gR5VUGQzJlU-hKYX34U5FT1X8.1_1 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~W4Cfw0gR5VUGQzJlU-hKYX34U5FT1X8.2 2024-04-17T05:07:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__shadow_key1.2~W4Cfw0gR5VUGQzJlU-hKYX34U5FT1X8.2_1 2024-04-17T05:07:41.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~W4Cfw0gR5VUGQzJlU-hKYX34U5FT1X8.3 2024-04-17T05:07:41.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17414363197088150695', time=2024-04-17T07:06:31.035312+0000, chain.objs.size()=5 2024-04-17T05:07:41.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.94__multipart_key1.2~2TdPT0A4JxAi3SYi7TgYmKTwiIS4jkE.1 2024-04-17T05:07:41.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.355+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.94__multipart_key1.2~2TdPT0A4JxAi3SYi7TgYmKTwiIS4jkE.2 2024-04-17T05:07:41.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.94__multipart_key1.2~2TdPT0A4JxAi3SYi7TgYmKTwiIS4jkE.3 2024-04-17T05:07:41.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.94__multipart_key1.2~2TdPT0A4JxAi3SYi7TgYmKTwiIS4jkE.4 2024-04-17T05:07:41.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.94__multipart_key1.2~2TdPT0A4JxAi3SYi7TgYmKTwiIS4jkE.5 2024-04-17T05:07:41.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15554791452800776845', time=2024-04-17T07:07:01.926449+0000, chain.objs.size()=1 2024-04-17T05:07:41.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.105__shadow_.e97aMAIIl3ZpxEzPm3SFuNoixPJ5lDG_0 2024-04-17T05:07:41.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4897111136883355938', time=2024-04-17T07:07:15.217735+0000, chain.objs.size()=1 2024-04-17T05:07:41.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.115__shadow_.ax2WInw0GsNZO8oU4tpEWMTgU0DB8q5_0 2024-04-17T05:07:41.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6125376915882540238', time=2024-04-17T07:07:23.500625+0000, chain.objs.size()=1 2024-04-17T05:07:41.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.363+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.122__shadow_.qE0ONB4NXzYhBkQuLYTXzhY8uqXyFeC_0 2024-04-17T05:07:41.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.375+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:07:41.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17778508919676911524', time=2024-04-17T07:06:25.620435+0000, chain.objs.size()=1 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.93__shadow_.bsSeGqsDfRJnYZnV3yyCgltrEQDIsQe_0 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14004953097568440129', time=2024-04-17T07:07:07.719476+0000, chain.objs.size()=1 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.109__shadow_.URmI4ZYLi5n1dTeLmfZPSv2OqtkVeyg_0 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13204804285758696296', time=2024-04-17T07:07:10.276091+0000, chain.objs.size()=1 2024-04-17T05:07:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.379+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.111__shadow_.WCBBCdmhytoYUw_nUnVDa_e3O73Vzts_0 2024-04-17T05:07:41.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.383+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.387+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:07:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.387+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.391+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.391+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:07:41.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.391+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10382629931065652053', time=2024-04-17T07:04:20.034001+0000, chain.objs.size()=1 2024-04-17T05:07:41.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.391+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.80__shadow_.vGCW_GzULbEDmKqf7pYzuoib93zlBhc_0 2024-04-17T05:07:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.391+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:07:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6689071990524540522', time=2024-04-17T07:04:15.482330+0000, chain.objs.size()=2 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.73__shadow_.GicZa11Q_IEJ1XA2mrdD5YuNJqZGcBV_0 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.73__shadow_.GicZa11Q_IEJ1XA2mrdD5YuNJqZGcBV_1 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14088091144493414098', time=2024-04-17T07:07:12.679426+0000, chain.objs.size()=1 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.113__shadow_.DO1gMq7IFlbFdBLIXZ1MA_-ehWGYj8n_0 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16236105082163287390', time=2024-04-17T07:07:23.423905+0000, chain.objs.size()=1 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.122__shadow_.DNvB0llIkZG3n8XrfyqXJvGCYZ4irbM_0 2024-04-17T05:07:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10705891256557634992', time=2024-04-17T07:07:40.280921+0000, chain.objs.size()=5 2024-04-17T05:07:41.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.395+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134__multipart_key1/data/file.mpg.2~nxmz1VtiTSAfFIpjUp1ZUv-UbNqXfWh.1 2024-04-17T05:07:41.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.399+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134__multipart_key1/data/file.mpg.2~nxmz1VtiTSAfFIpjUp1ZUv-UbNqXfWh.2 2024-04-17T05:07:41.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.399+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134__multipart_key1/data/file.mpg.2~nxmz1VtiTSAfFIpjUp1ZUv-UbNqXfWh.3 2024-04-17T05:07:41.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.399+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134__multipart_key1/data/file.mpg.2~nxmz1VtiTSAfFIpjUp1ZUv-UbNqXfWh.4 2024-04-17T05:07:41.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.399+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134__multipart_key1/data/file.mpg.2~nxmz1VtiTSAfFIpjUp1ZUv-UbNqXfWh.5 2024-04-17T05:07:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.407+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.411+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:07:41.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.411+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.411+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14885994242064289174', time=2024-04-17T07:06:09.393080+0000, chain.objs.size()=5 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~fIkaEar3ANKtLaUd14LjgEDn2BcsEfp.1 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__shadow_key1.2~fIkaEar3ANKtLaUd14LjgEDn2BcsEfp.1_1 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~fIkaEar3ANKtLaUd14LjgEDn2BcsEfp.2 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__shadow_key1.2~fIkaEar3ANKtLaUd14LjgEDn2BcsEfp.2_1 2024-04-17T05:07:41.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~fIkaEar3ANKtLaUd14LjgEDn2BcsEfp.3 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5127222010270197177', time=2024-04-17T07:06:59.516234+0000, chain.objs.size()=1 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.103__shadow_.lzfjUEVgjY1VHTTekiwLdA1LUGHsjc2_0 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7749370027289351617', time=2024-04-17T07:07:05.128809+0000, chain.objs.size()=1 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.107__shadow_.DdGvvwvZsTAjcR5xJapNPhJLRb0Tr-1_0 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9939248936532915260', time=2024-04-17T07:07:09.000698+0000, chain.objs.size()=1 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.110__shadow_.yaElMOIFeu1bUQ2fF4sFSKrTocKFQS0_0 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9785060506401030581', time=2024-04-17T07:07:12.626882+0000, chain.objs.size()=1 2024-04-17T05:07:41.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.415+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.113__shadow_.y_ZYr6U-2UGAIG1igUWm1A7SkYmFLTh_0 2024-04-17T05:07:41.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.423+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:07:41.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4291108503709254303', time=2024-04-17T07:07:06.300962+0000, chain.objs.size()=1 2024-04-17T05:07:41.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.108__shadow_.-LBOqvtyZ3CHTqjd9ZlNqhC8gISKQiw_0 2024-04-17T05:07:41.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1821976553374649170', time=2024-04-17T07:07:17.670243+0000, chain.objs.size()=1 2024-04-17T05:07:41.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.427+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.117__shadow_.nNu7h8KtvJuz-jlV5iCvlx5bvkB7F4f_0 2024-04-17T05:07:41.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.431+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:07:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:07:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10043512349145429455', time=2024-04-17T07:04:12.038728+0000, chain.objs.size()=1 2024-04-17T05:07:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.71__shadow_.ftGKa0UF7En-lya7BDtO9BBphJBDYOA_0 2024-04-17T05:07:41.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12241957957674115386', time=2024-04-17T07:06:52.353669+0000, chain.objs.size()=1 2024-04-17T05:07:41.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.96__shadow_.epAeNOHOBecWnjmIwmrZG1qEQMFCL9F_0 2024-04-17T05:07:41.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.850748896096565284', time=2024-04-17T07:07:00.825049+0000, chain.objs.size()=1 2024-04-17T05:07:41.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.104__shadow_.jYr4X2AA61wbkblO20GggV3zVapGpD4_0 2024-04-17T05:07:41.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3731817898809114590', time=2024-04-17T07:07:21.244429+0000, chain.objs.size()=1 2024-04-17T05:07:41.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.435+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.120__shadow_.zt85iWxgf10yOFXe5KrgMXGDg88t3vP_0 2024-04-17T05:07:41.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.439+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.443+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:07:41.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.443+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9473163040336862292', time=2024-04-17T07:05:06.095896+0000, chain.objs.size()=5 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~M3XSW3QxlizD2ySSruu9NhoK0cT7ogy.1 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__shadow_key1.2~M3XSW3QxlizD2ySSruu9NhoK0cT7ogy.1_1 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~M3XSW3QxlizD2ySSruu9NhoK0cT7ogy.2 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__shadow_key1.2~M3XSW3QxlizD2ySSruu9NhoK0cT7ogy.2_1 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~M3XSW3QxlizD2ySSruu9NhoK0cT7ogy.3 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14713547342169766899', time=2024-04-17T07:07:06.383160+0000, chain.objs.size()=1 2024-04-17T05:07:41.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.447+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.108__shadow_.rT7ldmf8C8Yb6kzLx42uOQieQcJbYGl_0 2024-04-17T05:07:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.451+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.455+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:07:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16766259137507997046', time=2024-04-17T07:06:58.276331+0000, chain.objs.size()=1 2024-04-17T05:07:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.102__shadow_.I0Ve04vVKkC3BXsr_xgvl-_2v5dixpr_0 2024-04-17T05:07:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12098036130203502789', time=2024-04-17T07:07:12.745005+0000, chain.objs.size()=1 2024-04-17T05:07:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.113__shadow_.QInYTYaABiaunYPcC243siT63Ol2FYu_0 2024-04-17T05:07:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.459+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.463+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:07:41.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:07:41.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1480741697070168099', time=2024-04-17T07:05:10.219015+0000, chain.objs.size()=4 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.83__shadow_.FFmM1NxeXY0ujzFLPX1Wn4zoUvJv47__0 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.83__shadow_.FFmM1NxeXY0ujzFLPX1Wn4zoUvJv47__1 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.83__shadow_.FFmM1NxeXY0ujzFLPX1Wn4zoUvJv47__2 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.83__shadow_.FFmM1NxeXY0ujzFLPX1Wn4zoUvJv47__3 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12707978992913986395', time=2024-04-17T07:07:02.012103+0000, chain.objs.size()=1 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.105__shadow_.0RcLdZ9X0nup4SFUFRoM3aROppVm9rI_0 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14259732667292980559', time=2024-04-17T07:07:20.211299+0000, chain.objs.size()=1 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.119__shadow_.Mxsjbejk-hIulSGtAAmzPacwwbGAAV3_0 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8822035439706212415', time=2024-04-17T07:07:30.279191+0000, chain.objs.size()=1 2024-04-17T05:07:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.467+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.130__shadow_.tGqV0iBsCyAUwEw24u6tW592Km9MPc1_0 2024-04-17T05:07:41.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.471+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.475+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:07:41.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9152593297422800874', time=2024-04-17T07:07:02.096127+0000, chain.objs.size()=1 2024-04-17T05:07:41.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.105__shadow_.5IgoOI81f7aO8oUDaM2M1mVZUZuOqmR_0 2024-04-17T05:07:41.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11099596691518622687', time=2024-04-17T07:07:05.047973+0000, chain.objs.size()=1 2024-04-17T05:07:41.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.107__shadow_.0Efb9MPv65ZpFw2b7npY1vsNz168k0G_0 2024-04-17T05:07:41.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.479+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.483+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:07:41.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.487+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.487+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.487+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:07:41.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.487+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7787540119105897986', time=2024-04-17T07:07:22.420276+0000, chain.objs.size()=1 2024-04-17T05:07:41.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.487+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.121__shadow_.njSeEbuKgBqqOChOxTMb0TlwEkJRtYI_0 2024-04-17T05:07:41.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.487+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.491+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:07:41.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.491+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7169149520045403285', time=2024-04-17T07:06:21.593364+0000, chain.objs.size()=5 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~eIHMorhaIW6mvsv1-Ase2x7WTlF_PNn.1 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__shadow_key1.2~eIHMorhaIW6mvsv1-Ase2x7WTlF_PNn.1_1 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~eIHMorhaIW6mvsv1-Ase2x7WTlF_PNn.2 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__shadow_key1.2~eIHMorhaIW6mvsv1-Ase2x7WTlF_PNn.2_1 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~eIHMorhaIW6mvsv1-Ase2x7WTlF_PNn.3 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='2~CUBbDorl1quQUo_IpNaNI7matY9nLVH', time=2024-04-17T07:06:46.203909+0000, chain.objs.size()=1 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.13 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='2~CUBbDorl1quQUo_IpNaNI7matY9nLVH', time=2024-04-17T07:06:46.282856+0000, chain.objs.size()=1 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.19 2024-04-17T05:07:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='2~CUBbDorl1quQUo_IpNaNI7matY9nLVH', time=2024-04-17T07:06:46.512317+0000, chain.objs.size()=48 2024-04-17T05:07:41.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.1 2024-04-17T05:07:41.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.2 2024-04-17T05:07:41.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.3 2024-04-17T05:07:41.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.4 2024-04-17T05:07:41.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.495+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.5 2024-04-17T05:07:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.499+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.6 2024-04-17T05:07:41.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.499+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.7 2024-04-17T05:07:41.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.499+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.8 2024-04-17T05:07:41.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.499+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.9 2024-04-17T05:07:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.10 2024-04-17T05:07:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.11 2024-04-17T05:07:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.12 2024-04-17T05:07:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.mrgrGkemAw14FhheSEovLOGYmqXbjOUp.13 2024-04-17T05:07:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.14 2024-04-17T05:07:41.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.15 2024-04-17T05:07:41.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.503+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.16 2024-04-17T05:07:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.507+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.17 2024-04-17T05:07:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.507+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.18 2024-04-17T05:07:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.507+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.A9UGdIismaZ8SzsoPCGU4gng73EDE0hc.19 2024-04-17T05:07:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.507+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.20 2024-04-17T05:07:41.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.21 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.22 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.23 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.24 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.25 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.26 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.27 2024-04-17T05:07:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.28 2024-04-17T05:07:41.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.511+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.29 2024-04-17T05:07:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.30 2024-04-17T05:07:41.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.31 2024-04-17T05:07:41.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.32 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.33 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.34 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.35 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.36 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.37 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.38 2024-04-17T05:07:41.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.515+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.39 2024-04-17T05:07:41.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.519+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.40 2024-04-17T05:07:41.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.519+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.41 2024-04-17T05:07:41.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.42 2024-04-17T05:07:41.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.43 2024-04-17T05:07:41.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.44 2024-04-17T05:07:41.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.45 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.46 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.47 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.95__multipart_key1.2~CUBbDorl1quQUo_IpNaNI7matY9nLVH.48 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16634446547544998546', time=2024-04-17T07:06:58.124265+0000, chain.objs.size()=1 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.102__shadow_.ZaNya8ksp3uiPamsfFHOibO1qfu56ly_0 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16050174993298758471', time=2024-04-17T07:07:00.745810+0000, chain.objs.size()=1 2024-04-17T05:07:41.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.104__shadow_.1xab_OrjppRaYJYucDh13WImdE7cY9a_0 2024-04-17T05:07:41.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4444640842387724334', time=2024-04-17T07:07:04.980972+0000, chain.objs.size()=1 2024-04-17T05:07:41.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.107__shadow_.6s9POs3s78dN3d15XIC4MzWNndDdInp_0 2024-04-17T05:07:41.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17890611434798516207', time=2024-04-17T07:07:09.145112+0000, chain.objs.size()=1 2024-04-17T05:07:41.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.523+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.110__shadow_.i9QmFlAYeNSRjyyrwWzaiLtHppzALgA_0 2024-04-17T05:07:41.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.527+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13628586010030895907', time=2024-04-17T07:07:11.449802+0000, chain.objs.size()=1 2024-04-17T05:07:41.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.527+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.112__shadow_.s5vj4VydqK-_DmATjHC5q_jDoDNyM9K_0 2024-04-17T05:07:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.527+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2798627332202021424', time=2024-04-17T07:07:15.148536+0000, chain.objs.size()=1 2024-04-17T05:07:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.527+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.115__shadow_.ZbPE8BjDgHtBGm2gKQzKwfU2A6xILwd_0 2024-04-17T05:07:41.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.535+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.535+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:07:41.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-17T05:07:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15332598003412030651', time=2024-04-17T07:05:10.594526+0000, chain.objs.size()=5 2024-04-17T05:07:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~NZE1hdJrIjz6eqZcafcSgAqm1X0ULoh.1 2024-04-17T05:07:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__shadow_key1.2~NZE1hdJrIjz6eqZcafcSgAqm1X0ULoh.1_1 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~NZE1hdJrIjz6eqZcafcSgAqm1X0ULoh.2 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__shadow_key1.2~NZE1hdJrIjz6eqZcafcSgAqm1X0ULoh.2_1 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~NZE1hdJrIjz6eqZcafcSgAqm1X0ULoh.3 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9900406982019610605', time=2024-04-17T07:05:31.673500+0000, chain.objs.size()=1 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__shadow_.OIKQ5G9mi0SomglbeDMZdeIXbJqFqyL_0 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6291661498801778885', time=2024-04-17T07:07:19.011546+0000, chain.objs.size()=1 2024-04-17T05:07:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.539+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.118__shadow_.oQbQAfPcKSbdKGEzZ0z5vAhW0awYc0u_0 2024-04-17T05:07:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.543+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.547+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:07:41.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.547+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.547+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8400441636140656812', time=2024-04-17T07:06:03.465256+0000, chain.objs.size()=5 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~YHNM4ASXPKd1tQuQVCIIu0FuIWVDHvD.1 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~YHNM4ASXPKd1tQuQVCIIu0FuIWVDHvD.2 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~YHNM4ASXPKd1tQuQVCIIu0FuIWVDHvD.3 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~YHNM4ASXPKd1tQuQVCIIu0FuIWVDHvD.4 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~YHNM4ASXPKd1tQuQVCIIu0FuIWVDHvD.5 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14699420654688985193', time=2024-04-17T07:06:59.667415+0000, chain.objs.size()=1 2024-04-17T05:07:41.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.103__shadow_.w-ZJ23Ctd1i1yWQ1JO6DU3l9E6RvHc3_0 2024-04-17T05:07:41.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3751316646720930489', time=2024-04-17T07:07:20.321092+0000, chain.objs.size()=1 2024-04-17T05:07:41.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.119__shadow_.qxqkOMAJODkRz_uJ5i2NEML7MmTz1zh_0 2024-04-17T05:07:41.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5935537139233055107', time=2024-04-17T07:07:24.646973+0000, chain.objs.size()=1 2024-04-17T05:07:41.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.551+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.123__shadow_.Ohjfpi_x_rB1XOI_NFDYGVXqSIxhUKp_0 2024-04-17T05:07:41.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.559+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.559+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:07:41.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15582740524800649704', time=2024-04-17T07:07:08.906487+0000, chain.objs.size()=1 2024-04-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.110__shadow_.ELmlHYO_NiYB-ZaYJYqvmzPrT5n1d0-_0 2024-04-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5979811288485095667', time=2024-04-17T07:07:24.730888+0000, chain.objs.size()=1 2024-04-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.123__shadow_.2yCN3x_063rpO6T7xNfJ0Way_Ia5kLt_0 2024-04-17T05:07:41.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.563+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.567+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.567+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12279656838498729524', time=2024-04-17T07:07:11.677586+0000, chain.objs.size()=1 2024-04-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.112__shadow_.my4wx-OH3MvurABY2pXwiT4Av35Wgrc_0 2024-04-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1626913462913023697', time=2024-04-17T07:07:23.570559+0000, chain.objs.size()=1 2024-04-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.122__shadow_.BRf10e7s0N_Uld-PJovDlom2iaJTRpc_0 2024-04-17T05:07:41.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.571+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:07:41.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T05:07:41.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10044461187860647481', time=2024-04-17T07:05:02.644637+0000, chain.objs.size()=5 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~dE9yRaub9uQMkSGa0mUVElkF9i7L_tH.1 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~dE9yRaub9uQMkSGa0mUVElkF9i7L_tH.2 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~dE9yRaub9uQMkSGa0mUVElkF9i7L_tH.3 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~dE9yRaub9uQMkSGa0mUVElkF9i7L_tH.4 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~dE9yRaub9uQMkSGa0mUVElkF9i7L_tH.5 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2382010444689130302', time=2024-04-17T07:06:26.991111+0000, chain.objs.size()=5 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.93__multipart_key1.2~B3FbTGSWWSK1l4phaNXd3Ee_LcmQtF0.1 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.93__multipart_key1.2~B3FbTGSWWSK1l4phaNXd3Ee_LcmQtF0.2 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.93__multipart_key1.2~B3FbTGSWWSK1l4phaNXd3Ee_LcmQtF0.3 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.93__multipart_key1.2~B3FbTGSWWSK1l4phaNXd3Ee_LcmQtF0.4 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.93__multipart_key1.2~B3FbTGSWWSK1l4phaNXd3Ee_LcmQtF0.5 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.608602291995132491', time=2024-04-17T07:06:58.204729+0000, chain.objs.size()=1 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.102__shadow_.B7b9Z3T7DWZS2kSvi8Nw5OUr9j9iL7R_0 2024-04-17T05:07:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10311753497542934796', time=2024-04-17T07:07:00.900139+0000, chain.objs.size()=1 2024-04-17T05:07:41.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.575+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.104__shadow_.PjdcyIHMnTxi_793a4cXOuYd_qD8_0b_0 2024-04-17T05:07:41.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.579+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5819545464283324439', time=2024-04-17T07:07:15.320492+0000, chain.objs.size()=1 2024-04-17T05:07:41.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.579+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.115__shadow_.mSy3hsSshDDczoVKqUcBanq4amlvPls_0 2024-04-17T05:07:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.587+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:07:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13952049449675547007', time=2024-04-17T07:06:59.593655+0000, chain.objs.size()=1 2024-04-17T05:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.103__shadow_.cguDVxg5TPf52I-wDJD8qeXJQO-KYqM_0 2024-04-17T05:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9727077330588827732', time=2024-04-17T07:07:10.355316+0000, chain.objs.size()=1 2024-04-17T05:07:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.591+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.111__shadow_.8UD5T2uPa2M_9kzHr3-VXp6hL5fX-d5_0 2024-04-17T05:07:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.595+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.595+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:07:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-17T05:07:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3974914228543452498', time=2024-04-17T07:04:58.450816+0000, chain.objs.size()=5 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~hyyEzxTz8uRoCLeZboAPfoyWhncm6NN.1 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~hyyEzxTz8uRoCLeZboAPfoyWhncm6NN.2 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~hyyEzxTz8uRoCLeZboAPfoyWhncm6NN.3 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~hyyEzxTz8uRoCLeZboAPfoyWhncm6NN.4 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.84__multipart_key1.2~hyyEzxTz8uRoCLeZboAPfoyWhncm6NN.5 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9615150266254249955', time=2024-04-17T07:05:31.898760+0000, chain.objs.size()=1 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.90__shadow_.RNdSEp0ztaBhQArnatlcjgwHXXNsggQ_0 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13568034848881843498', time=2024-04-17T07:07:09.076058+0000, chain.objs.size()=1 2024-04-17T05:07:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.599+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.110__shadow_.PP0FbYavEPdLid2HjVL2yx5t1sHaB14_0 2024-04-17T05:07:41.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.603+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:07:41.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-17T05:07:41.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1154468622452693162', time=2024-04-17T07:04:16.201033+0000, chain.objs.size()=1 2024-04-17T05:07:41.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.74__shadow_.ACKPnTpJ6r9h03rsjSqQntPwr-jtwGm_0 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.2_fb52277ba0d482e76e566e5c09f1b661', time=2024-04-17T07:04:33.766381+0000, chain.objs.size()=5 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.1 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.2 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.3 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.4 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.81__multipart_key1.2~ObmdhOtEnNrYK7hPr1b8YrZfaxIdHb0.5 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15927992735972798595', time=2024-04-17T07:05:51.053155+0000, chain.objs.size()=5 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~k3J_1nnMiIh0y0J416_Gyh8v_nAmkmS.1 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~k3J_1nnMiIh0y0J416_Gyh8v_nAmkmS.2 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~k3J_1nnMiIh0y0J416_Gyh8v_nAmkmS.3 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~k3J_1nnMiIh0y0J416_Gyh8v_nAmkmS.4 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.92__multipart_key1.2~k3J_1nnMiIh0y0J416_Gyh8v_nAmkmS.5 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13142453542852417228', time=2024-04-17T07:07:17.594311+0000, chain.objs.size()=1 2024-04-17T05:07:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.607+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.117__shadow_.y1mBrJcq-C1Nz1UHhJ-Jq96J8tztGx9_0 2024-04-17T05:07:41.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.615+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4469163515137104078', time=2024-04-17T07:07:20.131564+0000, chain.objs.size()=1 2024-04-17T05:07:41.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.615+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.119__shadow_.JmUSdYVDuU3t39LUFyGEErSaBuhYoT2_0 2024-04-17T05:07:41.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.615+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.619+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:07:41.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.619+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-17T05:07:41.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='2~Usg6DS2X1nszb6Q-kcOq0fYGP9TzaHK', time=2024-04-17T07:04:10.315076+0000, chain.objs.size()=5 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.69__multipart_key1.2~Usg6DS2X1nszb6Q-kcOq0fYGP9TzaHK.1 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.69__multipart_key1.2~Usg6DS2X1nszb6Q-kcOq0fYGP9TzaHK.2 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.69__multipart_key1.2~Usg6DS2X1nszb6Q-kcOq0fYGP9TzaHK.3 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.69__multipart_key1.2~Usg6DS2X1nszb6Q-kcOq0fYGP9TzaHK.4 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.69__multipart_key1.2~Usg6DS2X1nszb6Q-kcOq0fYGP9TzaHK.5 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15850112588354760533', time=2024-04-17T07:06:58.356008+0000, chain.objs.size()=1 2024-04-17T05:07:41.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.102__shadow_.o1JcRByi561W9VUlbH8xM2aZ9-JXiKS_0 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16887091052991527851', time=2024-04-17T07:06:59.440015+0000, chain.objs.size()=1 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.103__shadow_.f4KqRNwoCbfJWas_3ko5yLHRO2LCvfM_0 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12747811260818757865', time=2024-04-17T07:07:07.673395+0000, chain.objs.size()=1 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.109__shadow_.XHrs2xHbSYxJPTfBtbirvonmbqbW0yM_0 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11301067491684812451', time=2024-04-17T07:07:16.387660+0000, chain.objs.size()=1 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.116__shadow_.kIJFyo8RBYp8JMsOlH_C7dMxH6WvxG3_0 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17601093154708956677', time=2024-04-17T07:07:17.737755+0000, chain.objs.size()=1 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.117__shadow_.nckVEjyJnfI7oZkcieIIm6sCOBxZwDK_0 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.209106598807377936', time=2024-04-17T07:07:19.084722+0000, chain.objs.size()=1 2024-04-17T05:07:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.118__shadow_.HnQnz2ZYHcgpRtAAdNf_6k8YbysWmDS_0 2024-04-17T05:07:41.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1675203665647457559', time=2024-04-17T07:07:36.303997+0000, chain.objs.size()=5 2024-04-17T05:07:41.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.623+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.133__multipart_key1.2~7zzP8ynmda4MKdyFHYcQBuqHf_8-SNi.1 2024-04-17T05:07:41.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.627+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.133__multipart_key1.2~7zzP8ynmda4MKdyFHYcQBuqHf_8-SNi.2 2024-04-17T05:07:41.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.627+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.133__multipart_key1.2~7zzP8ynmda4MKdyFHYcQBuqHf_8-SNi.3 2024-04-17T05:07:41.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.631+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.133__multipart_key1.2~7zzP8ynmda4MKdyFHYcQBuqHf_8-SNi.4 2024-04-17T05:07:41.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.631+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.133__multipart_key1.2~7zzP8ynmda4MKdyFHYcQBuqHf_8-SNi.5 2024-04-17T05:07:41.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.635+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.635+0000 7fbbcd21c980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:07:41.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:41.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-17T05:07:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15731711261129096348', time=2024-04-17T07:07:21.314120+0000, chain.objs.size()=1 2024-04-17T05:07:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.120__shadow_.1GkJ-LsdtfQ_FCbvNt-Eh7y-J8l5hHQ_0 2024-04-17T05:07:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14397098547397759181', time=2024-04-17T07:07:27.376976+0000, chain.objs.size()=1 2024-04-17T05:07:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.127__shadow_.sp0UrxlihXDMuCSCIlRxF9SePTj9qA2_0 2024-04-17T05:07:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.639+0000 7fbbcd21c980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:07:41.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.643+0000 7fba817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:41.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.647+0000 7fbbcd21c980 20 remove_watcher() i=0 2024-04-17T05:07:41.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.647+0000 7fbbcd21c980 2 removed watcher, disabling cache 2024-04-17T05:07:41.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.647+0000 7fbbcd21c980 20 remove_watcher() i=1 2024-04-17T05:07:41.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.651+0000 7fbbcd21c980 20 remove_watcher() i=2 2024-04-17T05:07:41.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.651+0000 7fbbcd21c980 20 remove_watcher() i=3 2024-04-17T05:07:41.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.651+0000 7fbbcd21c980 20 remove_watcher() i=4 2024-04-17T05:07:41.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.655+0000 7fbbcd21c980 20 remove_watcher() i=5 2024-04-17T05:07:41.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.655+0000 7fbbcd21c980 20 remove_watcher() i=6 2024-04-17T05:07:41.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.655+0000 7fbbcd21c980 20 remove_watcher() i=7 2024-04-17T05:07:41.667 DEBUG:teuthology.orchestra.run.smithi088:> radosgw-admin gc process --include-all 2024-04-17T05:07:41.740 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:41.740 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.783+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 realm 2024-04-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:41.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:41.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.787+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.791+0000 7fbcbc8e5980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.791+0000 7fbcbc8e5980 20 searching for the correct realm 2024-04-17T05:07:41.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:41.795+0000 7fbcbac01640 0 -- 172.21.15.88:0/613912342 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fbcb00b97b0 msgr2=0x7fbcb00c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:42.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.011+0000 7fbcba400640 0 -- 172.21.15.88:0/613912342 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55ec48e5d5a0 msgr2=0x55ec48e5f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:42.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:42.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.019+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 zone default found 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 4 Realm: () 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 zonegroup default 2024-04-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:42.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:42.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:42.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:42.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados->read ofs=0 len=0 2024-04-17T05:07:42.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:42.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 started sync module instance, tier type = 2024-04-17T05:07:42.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.023+0000 7fbcbc8e5980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:42.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=0 2024-04-17T05:07:42.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=1 2024-04-17T05:07:42.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=2 2024-04-17T05:07:42.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=3 2024-04-17T05:07:42.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=4 2024-04-17T05:07:42.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=5 2024-04-17T05:07:42.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.043+0000 7fbcbc8e5980 20 add_watcher() i=6 2024-04-17T05:07:42.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.047+0000 7fbcbc8e5980 20 add_watcher() i=7 2024-04-17T05:07:42.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.047+0000 7fbcbc8e5980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:42.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.051+0000 7fbb70fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:42.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.051+0000 7fbcbc8e5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:42.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.051+0000 7fbcbc8e5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:42.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.147+0000 7fbaeb7fe640 20 reqs_thread_entry: start 2024-04-17T05:07:42.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.147+0000 7fbcbc8e5980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:42.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.147+0000 7fbcbc8e5980 20 Filter name: none 2024-04-17T05:07:42.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.147+0000 7fbae97fa640 20 reqs_thread_entry: start 2024-04-17T05:07:42.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.147+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:07:42.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.151+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.151+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.151+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.155+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:07:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.159+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.159+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.159+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.163+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:07:42.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.167+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.167+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.167+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.171+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:07:42.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.175+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.175+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.179+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.179+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:07:42.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.183+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.183+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.183+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.187+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:07:42.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.191+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.191+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.191+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.195+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:07:42.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.195+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.199+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.199+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.199+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:07:42.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.203+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.203+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.207+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.207+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:07:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.211+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.215+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.215+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.219+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:07:42.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.223+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.223+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.223+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.227+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:07:42.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.231+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.231+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.231+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.235+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:07:42.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.235+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.239+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.239+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.243+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:07:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.247+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.247+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.247+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.251+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:07:42.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.255+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.255+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.259+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.259+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:07:42.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.263+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.267+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.267+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.271+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:07:42.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.275+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.279+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.279+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.279+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:07:42.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.283+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.287+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.287+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.291+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:07:42.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.295+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.299+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.299+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.299+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:07:42.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.303+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.307+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.307+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.311+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:07:42.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.311+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.315+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.315+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.319+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:07:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.323+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.323+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.323+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.327+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:07:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.331+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.331+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.335+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.335+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:07:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.339+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.343+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.343+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.347+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:07:42.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.351+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.351+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.351+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.355+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:07:42.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.359+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.359+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.359+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.363+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:07:42.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.367+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.367+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.371+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.371+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:07:42.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.375+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.379+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.379+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.383+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:07:42.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.387+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.387+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.391+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.391+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:07:42.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.395+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.399+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.399+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.403+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:07:42.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.407+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.407+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.407+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.411+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:07:42.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.415+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.415+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.415+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.419+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:07:42.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.423+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.423+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:07:42.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.427+0000 7fbcbc8e5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:07:42.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.431+0000 7fbb70fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.435+0000 7fbcbc8e5980 20 remove_watcher() i=0 2024-04-17T05:07:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.435+0000 7fbcbc8e5980 2 removed watcher, disabling cache 2024-04-17T05:07:42.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.439+0000 7fbcbc8e5980 20 remove_watcher() i=1 2024-04-17T05:07:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.443+0000 7fbcbc8e5980 20 remove_watcher() i=2 2024-04-17T05:07:42.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.443+0000 7fbcbc8e5980 20 remove_watcher() i=3 2024-04-17T05:07:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.447+0000 7fbcbc8e5980 20 remove_watcher() i=4 2024-04-17T05:07:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.447+0000 7fbcbc8e5980 20 remove_watcher() i=5 2024-04-17T05:07:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.451+0000 7fbcbc8e5980 20 remove_watcher() i=6 2024-04-17T05:07:42.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:42.451+0000 7fbcbc8e5980 20 remove_watcher() i=7 2024-04-17T05:07:42.469 INFO:teuthology.run_tasks:Running task s3tests... 2024-04-17T05:07:42.487 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 main': 'RGW11111111111111111', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-squid', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}} 2024-04-17T05:07:42.488 INFO:tasks.s3tests:Downloading s3-tests... 2024-04-17T05:07:42.488 INFO:tasks.s3tests:Using branch 'ceph-squid' for s3tests 2024-04-17T05:07:42.488 DEBUG:teuthology.orchestra.run.smithi088:> git clone -b ceph-squid https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-04-17T05:07:42.516 INFO:teuthology.orchestra.run.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-04-17T05:07:43.234 INFO:tasks.s3tests:Creating rgw users... 2024-04-17T05:07:43.235 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW11111111111111111 --account-name Mr.RGW11111111111111111 --email RGW11111111111111111+test@test.test 2024-04-17T05:07:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:43.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 realm 2024-04-17T05:07:43.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:43.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.359+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.363+0000 7f85e393b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:43.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.363+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.363+0000 7f85e393b980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:43.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.363+0000 7f85e393b980 20 searching for the correct realm 2024-04-17T05:07:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.367+0000 7f85e0c55640 0 -- 172.21.15.88:0/1382743169 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x55f575e231b0 msgr2=0x55f575e25670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T05:07:43.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.375+0000 7f85e1456640 0 -- 172.21.15.88:0/1382743169 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f85d4104100 msgr2=0x7f85d4109070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:43.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.591+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 zone default found 2024-04-17T05:07:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 4 Realm: () 2024-04-17T05:07:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 zonegroup default 2024-04-17T05:07:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 started sync module instance, tier type = 2024-04-17T05:07:43.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.595+0000 7f85e393b980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:43.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=0 2024-04-17T05:07:43.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=1 2024-04-17T05:07:43.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=2 2024-04-17T05:07:43.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=3 2024-04-17T05:07:43.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=4 2024-04-17T05:07:43.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=5 2024-04-17T05:07:43.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=6 2024-04-17T05:07:43.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 20 add_watcher() i=7 2024-04-17T05:07:43.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.619+0000 7f85e393b980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:43.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.623+0000 7f84977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:43.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.623+0000 7f85e393b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:43.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.623+0000 7f85e393b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:43.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.623+0000 7f85e393b980 5 note: GC not initialized 2024-04-17T05:07:43.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.623+0000 7f84967e4640 20 reqs_thread_entry: start 2024-04-17T05:07:43.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.623+0000 7f85e393b980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:43.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 Filter name: none 2024-04-17T05:07:43.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f8407fff640 20 reqs_thread_entry: start 2024-04-17T05:07:43.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 : miss 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x0 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 10 adding default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 failed to read name.$Mr.RGW11111111111111111 with: (2) No such file or directory 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 10 cache get: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test : miss 2024-04-17T05:07:43.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 10 cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x0 2024-04-17T05:07:43.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 10 adding default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-04-17T05:07:43.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.627+0000 7f85e393b980 20 failed to read rgw11111111111111111+test@test.test with: (2) No such file or directory 2024-04-17T05:07:43.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.631+0000 7f85e393b980 10 cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-04-17T05:07:43.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.631+0000 7f85e393b980 10 adding default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-04-17T05:07:43.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.631+0000 7f85e393b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW11111111111111111, ofs0, ns] 2024-04-17T05:07:43.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.631+0000 7f84c4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464414 cookie 94512733111360 notifier 7199 bl.length()=352 2024-04-17T05:07:43.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.631+0000 7f84c4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-04-17T05:07:43.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.631+0000 7f84c4ff9640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-04-17T05:07:43.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.635+0000 7f85e393b980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x17 2024-04-17T05:07:43.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.635+0000 7f85e393b980 10 moving default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-04-17T05:07:43.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.635+0000 7f85e393b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW11111111111111111, ofs0, ns] 2024-04-17T05:07:43.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.635+0000 7f84b3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464393 cookie 94512733127680 notifier 7199 bl.length()=225 2024-04-17T05:07:43.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.635+0000 7f84b3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x17 2024-04-17T05:07:43.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.635+0000 7f84b3fff640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-04-17T05:07:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f85e393b980 10 cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x17 2024-04-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f85e393b980 10 moving default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-04-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f85e393b980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw11111111111111111+test@test.test, ofs0, ns] 2024-04-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f84b3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464344 cookie 94512733123904 notifier 7199 bl.length()=234 2024-04-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f84b3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x17 2024-04-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f84b3fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-04-17T05:07:43.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.643+0000 7f84977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.651+0000 7f85e393b980 20 remove_watcher() i=0 2024-04-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.651+0000 7f85e393b980 2 removed watcher, disabling cache 2024-04-17T05:07:43.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.651+0000 7f85e393b980 20 remove_watcher() i=1 2024-04-17T05:07:43.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.655+0000 7f85e393b980 20 remove_watcher() i=2 2024-04-17T05:07:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.655+0000 7f85e393b980 20 remove_watcher() i=3 2024-04-17T05:07:43.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.659+0000 7f85e393b980 20 remove_watcher() i=4 2024-04-17T05:07:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.659+0000 7f85e393b980 20 remove_watcher() i=5 2024-04-17T05:07:43.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.663+0000 7f85e393b980 20 remove_watcher() i=6 2024-04-17T05:07:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.663+0000 7f85e393b980 20 remove_watcher() i=7 2024-04-17T05:07:43.676 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:43.676 INFO:teuthology.orchestra.run.smithi088.stdout: "id": "RGW11111111111111111", 2024-04-17T05:07:43.676 INFO:teuthology.orchestra.run.smithi088.stdout: "tenant": "", 2024-04-17T05:07:43.676 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "Mr.RGW11111111111111111", 2024-04-17T05:07:43.676 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "RGW11111111111111111+test@test.test", 2024-04-17T05:07:43.676 INFO:teuthology.orchestra.run.smithi088.stdout: "quota": { 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_users": 1000, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_roles": 1000, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_groups": 1000, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max_access_keys": 4 2024-04-17T05:07:43.677 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:43.680 DEBUG:tasks.s3tests:Creating user RGW11111111111111111 on client.0 2024-04-17T05:07:43.680 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name Mr.RGW11111111111111111 --email foo+test@test.test --access-key CDRQYLVEEWWVDHUGLBUX --secret voCvQ0Ago2yVy7JAcLTDuP9p9TdP8st+6tncQ8G4I6NNcwjJySo+lQ== --cluster ceph --account-id RGW11111111111111111 --account-root 2024-04-17T05:07:43.771 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:43.771 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:43.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 realm 2024-04-17T05:07:43.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:43.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.815+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.819+0000 7fb4db3f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:43.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.819+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.819+0000 7fb4db3f2980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:43.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.819+0000 7fb4db3f2980 20 searching for the correct realm 2024-04-17T05:07:43.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.823+0000 7fb4d970e640 0 -- 172.21.15.88:0/1929962989 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x5638faffd020 msgr2=0x5638fafff4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:43.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.831+0000 7fb4d3fff640 0 -- 172.21.15.88:0/1929962989 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fb4cc091350 msgr2=0x7fb4cc00c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:43.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 zone default found 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 4 Realm: () 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:43.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 zonegroup default 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 started sync module instance, tier type = 2024-04-17T05:07:43.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.839+0000 7fb4db3f2980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=0 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=1 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=2 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=3 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=4 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=5 2024-04-17T05:07:43.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.855+0000 7fb4db3f2980 20 add_watcher() i=6 2024-04-17T05:07:43.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 20 add_watcher() i=7 2024-04-17T05:07:43.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb38f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 5 note: GC not initialized 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb38e7e4640 20 reqs_thread_entry: start 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 20 Filter name: none 2024-04-17T05:07:43.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb3877fe640 20 reqs_thread_entry: start 2024-04-17T05:07:43.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-04-17T05:07:43.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.859+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache get: name=default.rgw.meta+users.email+foo+test@test.test : miss 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x0 2024-04-17T05:07:43.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 adding default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache get: name=default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX : miss 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX info.flags=0x0 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 adding default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX to cache LRU end 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache get: name=default.rgw.meta+accounts+account.RGW11111111111111111 : miss 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 20 rados_obj.operate() r=0 bl.length=152 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 adding default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache get: name=default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX : hit (negative entry) 2024-04-17T05:07:43.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache get: name=default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX : hit (negative entry) 2024-04-17T05:07:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-17T05:07:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-17T05:07:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb4db3f2980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-04-17T05:07:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb3affff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464394 cookie 94802729232128 notifier 7674 bl.length()=673 2024-04-17T05:07:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb3affff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-17T05:07:43.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.863+0000 7fb3affff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-17T05:07:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.867+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-04-17T05:07:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.867+0000 7fb4db3f2980 10 moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-04-17T05:07:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.867+0000 7fb4db3f2980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2024-04-17T05:07:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.871+0000 7fb3af7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464395 cookie 94802729285040 notifier 7674 bl.length()=185 2024-04-17T05:07:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.871+0000 7fb3af7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-04-17T05:07:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.871+0000 7fb3af7fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-04-17T05:07:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb4db3f2980 10 cache put: name=default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX info.flags=0x7 2024-04-17T05:07:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb4db3f2980 10 moving default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX to cache LRU end 2024-04-17T05:07:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb4db3f2980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.keys:CDRQYLVEEWWVDHUGLBUX, ofs0, ns] 2024-04-17T05:07:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb3af7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464334 cookie 94802729293728 notifier 7674 bl.length()=186 2024-04-17T05:07:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb3af7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX info.flags=0x7 2024-04-17T05:07:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb3af7fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+CDRQYLVEEWWVDHUGLBUX to cache LRU end 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.875+0000 7fb4db3f2980 20 linked user foo.client.0 to account RGW11111111111111111 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "foo.client.0", 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.RGW11111111111111111", 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "foo+test@test.test", 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:43.875 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "foo.client.0", 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "CDRQYLVEEWWVDHUGLBUX", 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "voCvQ0Ago2yVy7JAcLTDuP9p9TdP8st+6tncQ8G4I6NNcwjJySo+lQ==", 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:43.866066Z" 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:43.876 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:43.877 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "root", 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "RGW11111111111111111", 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:43.866051Z", 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:43.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.879+0000 7fb38f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:43.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.883+0000 7fb4db3f2980 20 remove_watcher() i=0 2024-04-17T05:07:43.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.883+0000 7fb4db3f2980 2 removed watcher, disabling cache 2024-04-17T05:07:43.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.883+0000 7fb4db3f2980 20 remove_watcher() i=1 2024-04-17T05:07:43.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.887+0000 7fb4db3f2980 20 remove_watcher() i=2 2024-04-17T05:07:43.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.887+0000 7fb4db3f2980 20 remove_watcher() i=3 2024-04-17T05:07:43.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.891+0000 7fb4db3f2980 20 remove_watcher() i=4 2024-04-17T05:07:43.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.891+0000 7fb4db3f2980 20 remove_watcher() i=5 2024-04-17T05:07:43.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.891+0000 7fb4db3f2980 20 remove_watcher() i=6 2024-04-17T05:07:43.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.891+0000 7fb4db3f2980 20 remove_watcher() i=7 2024-04-17T05:07:43.905 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 3642863866 --totp-seed F6HWPZT6IELT7LBJPHBA27TNNCOTI33L5FK6TMAORC2ZM2M7OZC6CWLSDFKKHPFS --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-17T05:07:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:43.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.983+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 realm 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:43.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:43.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:43.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.987+0000 7f1a97290980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:43.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.991+0000 7f1a97290980 20 searching for the correct realm 2024-04-17T05:07:43.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:43.995+0000 7f1a955ac640 0 -- 172.21.15.88:0/1435310444 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f1a880cc7a0 msgr2=0x7f1a88103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T05:07:44.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.211+0000 7f1a94dab640 0 -- 172.21.15.88:0/1435310444 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x561ead27eed0 msgr2=0x561ead281320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:44.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:44.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:44.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.219+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 zone default found 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 4 Realm: () 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 zonegroup default 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:44.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 started sync module instance, tier type = 2024-04-17T05:07:44.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.223+0000 7f1a97290980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=0 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=1 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=2 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=3 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=4 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=5 2024-04-17T05:07:44.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.247+0000 7f1a97290980 20 add_watcher() i=6 2024-04-17T05:07:44.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.251+0000 7f1a97290980 20 add_watcher() i=7 2024-04-17T05:07:44.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.251+0000 7f1a97290980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:44.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f194afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:44.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:44.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:44.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 5 note: GC not initialized 2024-04-17T05:07:44.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1949fe3640 20 reqs_thread_entry: start 2024-04-17T05:07:44.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:44.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 20 Filter name: none 2024-04-17T05:07:44.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f18bb7fe640 20 reqs_thread_entry: start 2024-04-17T05:07:44.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-04-17T05:07:44.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.255+0000 7f1a97290980 20 rados->read ofs=0 len=0 2024-04-17T05:07:44.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.259+0000 7f1a97290980 20 rados_obj.operate() r=0 bl.length=488 2024-04-17T05:07:44.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.259+0000 7f1a97290980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-04-17T05:07:44.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.259+0000 7f1a97290980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-17T05:07:44.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:44.259+0000 7f1a97290980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:45.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.759+0000 7f1a97290980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-17T05:07:45.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.759+0000 7f1a97290980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-17T05:07:45.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.759+0000 7f1a97290980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-04-17T05:07:45.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.759+0000 7f196b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576398988 cookie 94689754089040 notifier 7689 bl.length()=687 2024-04-17T05:07:45.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.759+0000 7f196b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-17T05:07:45.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.759+0000 7f196b7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-17T05:07:45.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.767+0000 7f194afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:45.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.771+0000 7f1a97290980 20 remove_watcher() i=0 2024-04-17T05:07:45.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.771+0000 7f1a97290980 2 removed watcher, disabling cache 2024-04-17T05:07:45.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.771+0000 7f1a97290980 20 remove_watcher() i=1 2024-04-17T05:07:45.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.775+0000 7f1a97290980 20 remove_watcher() i=2 2024-04-17T05:07:45.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.779+0000 7f1a97290980 20 remove_watcher() i=3 2024-04-17T05:07:45.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.779+0000 7f1a97290980 20 remove_watcher() i=4 2024-04-17T05:07:45.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.783+0000 7f1a97290980 20 remove_watcher() i=5 2024-04-17T05:07:45.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.787+0000 7f1a97290980 20 remove_watcher() i=6 2024-04-17T05:07:45.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.787+0000 7f1a97290980 20 remove_watcher() i=7 2024-04-17T05:07:45.804 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-04-17T05:07:45.805 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name Mr.bar.client.0 --email bar+test@test.test --access-key QCEPFLXOVMPUJCNRUZIJ --secret +w1IvIkb95B0jMCxP8DIdqtjp/lkqKYZ79EwB/A5fWtcZfmtE74mWg== --cluster ceph 2024-04-17T05:07:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:45.891 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:45.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 realm 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.935+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.939+0000 7f958fbdb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:45.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.939+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.939+0000 7f958fbdb980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:45.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.939+0000 7f958fbdb980 20 searching for the correct realm 2024-04-17T05:07:45.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.943+0000 7f958cef5640 0 -- 172.21.15.88:0/4255694202 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f9580063070 msgr2=0x7f958008baa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:45.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.951+0000 7f958cef5640 0 -- 172.21.15.88:0/4255694202 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f94580770f0 msgr2=0x7f9458079520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:45.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 zone default found 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 4 Realm: () 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 zonegroup default 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 started sync module instance, tier type = 2024-04-17T05:07:45.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.959+0000 7f958fbdb980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:45.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=0 2024-04-17T05:07:45.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=1 2024-04-17T05:07:45.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=2 2024-04-17T05:07:45.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=3 2024-04-17T05:07:45.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=4 2024-04-17T05:07:45.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=5 2024-04-17T05:07:45.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.975+0000 7f958fbdb980 20 add_watcher() i=6 2024-04-17T05:07:45.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f958fbdb980 20 add_watcher() i=7 2024-04-17T05:07:45.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f958fbdb980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f94437e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f958fbdb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f958fbdb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f958fbdb980 5 note: GC not initialized 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.979+0000 7f94427e4640 20 reqs_thread_entry: start 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 Filter name: none 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f93b3fff640 20 reqs_thread_entry: start 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-04-17T05:07:45.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache get: name=default.rgw.meta+users.email+bar+test@test.test : miss 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 adding default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache get: name=default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ : miss 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 rados->read ofs=0 len=0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache put: name=default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ info.flags=0x0 2024-04-17T05:07:45.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 adding default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ to cache LRU end 2024-04-17T05:07:45.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache get: name=default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ : hit (negative entry) 2024-04-17T05:07:45.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.983+0000 7f958fbdb980 10 cache get: name=default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ : hit (negative entry) 2024-04-17T05:07:45.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.987+0000 7f958fbdb980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-17T05:07:45.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.987+0000 7f958fbdb980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-17T05:07:45.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.987+0000 7f958fbdb980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-04-17T05:07:45.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.987+0000 7f9470ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576399007 cookie 94791288425632 notifier 7253 bl.length()=645 2024-04-17T05:07:45.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.987+0000 7f9470ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-17T05:07:45.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.987+0000 7f9470ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-17T05:07:45.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.991+0000 7f958fbdb980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-04-17T05:07:45.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.991+0000 7f958fbdb980 10 moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-04-17T05:07:45.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.991+0000 7f958fbdb980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2024-04-17T05:07:45.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.991+0000 7f9470ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576399017 cookie 94791288437648 notifier 7253 bl.length()=185 2024-04-17T05:07:45.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.991+0000 7f9470ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-04-17T05:07:45.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.991+0000 7f9470ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-04-17T05:07:45.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.995+0000 7f958fbdb980 10 cache put: name=default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ info.flags=0x7 2024-04-17T05:07:45.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.995+0000 7f958fbdb980 10 moving default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ to cache LRU end 2024-04-17T05:07:45.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.995+0000 7f958fbdb980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:QCEPFLXOVMPUJCNRUZIJ, ofs0, ns] 2024-04-17T05:07:45.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.995+0000 7f9470ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576399018 cookie 94791288435152 notifier 7253 bl.length()=186 2024-04-17T05:07:45.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.995+0000 7f9470ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ info.flags=0x7 2024-04-17T05:07:45.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.995+0000 7f9470ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+QCEPFLXOVMPUJCNRUZIJ to cache LRU end 2024-04-17T05:07:45.996 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "bar.client.0", 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.bar.client.0", 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "bar+test@test.test", 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "bar.client.0", 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "QCEPFLXOVMPUJCNRUZIJ", 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "+w1IvIkb95B0jMCxP8DIdqtjp/lkqKYZ79EwB/A5fWtcZfmtE74mWg==", 2024-04-17T05:07:45.997 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:45.986635Z" 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:45.998 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "rgw", 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "", 2024-04-17T05:07:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:46.000 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:45.986620Z", 2024-04-17T05:07:46.000 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:46.000 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:46.000 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:46.000 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:46.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:45.999+0000 7f94437e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.003+0000 7f958fbdb980 20 remove_watcher() i=0 2024-04-17T05:07:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.003+0000 7f958fbdb980 2 removed watcher, disabling cache 2024-04-17T05:07:46.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.007+0000 7f958fbdb980 20 remove_watcher() i=1 2024-04-17T05:07:46.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.007+0000 7f958fbdb980 20 remove_watcher() i=2 2024-04-17T05:07:46.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.011+0000 7f958fbdb980 20 remove_watcher() i=3 2024-04-17T05:07:46.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.015+0000 7f958fbdb980 20 remove_watcher() i=4 2024-04-17T05:07:46.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.015+0000 7f958fbdb980 20 remove_watcher() i=5 2024-04-17T05:07:46.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.015+0000 7f958fbdb980 20 remove_watcher() i=6 2024-04-17T05:07:46.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.015+0000 7f958fbdb980 20 remove_watcher() i=7 2024-04-17T05:07:46.030 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 9418111591 --totp-seed CZUPUEMIMAMOCNHL5MXWMTYO4BA3HTDXVT4HHZY3G2SCFRRBTQAISK7433P7KSZG --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-17T05:07:46.112 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:46.113 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 realm 2024-04-17T05:07:46.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:46.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.155+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.159+0000 7fd69e67a980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.159+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.159+0000 7fd69e67a980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.159+0000 7fd69e67a980 20 searching for the correct realm 2024-04-17T05:07:46.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.167+0000 7fd69b994640 0 -- 172.21.15.88:0/3240658554 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7fd68c065680 msgr2=0x7fd68c05c050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:46.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.171+0000 7fd69c996640 0 -- 172.21.15.88:0/3240658554 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fd690018070 msgr2=0x7fd69005da10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:46.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:46.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:46.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:46.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 zone default found 2024-04-17T05:07:46.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 4 Realm: () 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 zonegroup default 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.179+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.183+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.183+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.183+0000 7fd69e67a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.183+0000 7fd69e67a980 20 started sync module instance, tier type = 2024-04-17T05:07:46.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.183+0000 7fd69e67a980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=0 2024-04-17T05:07:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=1 2024-04-17T05:07:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=2 2024-04-17T05:07:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=3 2024-04-17T05:07:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=4 2024-04-17T05:07:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=5 2024-04-17T05:07:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.195+0000 7fd69e67a980 20 add_watcher() i=6 2024-04-17T05:07:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd69e67a980 20 add_watcher() i=7 2024-04-17T05:07:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd69e67a980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd552fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd69e67a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd69e67a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd69e67a980 5 note: GC not initialized 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.199+0000 7fd551fdb640 20 reqs_thread_entry: start 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 20 Filter name: none 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd4c37fe640 20 reqs_thread_entry: start 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-04-17T05:07:46.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 20 rados_obj.operate() r=0 bl.length=460 2024-04-17T05:07:46.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-04-17T05:07:46.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-17T05:07:46.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.203+0000 7fd69e67a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:46.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.207+0000 7fd69e67a980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-17T05:07:46.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.207+0000 7fd69e67a980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-17T05:07:46.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.207+0000 7fd69e67a980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-04-17T05:07:46.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.207+0000 7fd5737fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576399008 cookie 94015068897792 notifier 7728 bl.length()=659 2024-04-17T05:07:46.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.207+0000 7fd5737fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-17T05:07:46.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.207+0000 7fd5737fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-17T05:07:46.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.211+0000 7fd552fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.215+0000 7fd69e67a980 20 remove_watcher() i=0 2024-04-17T05:07:46.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.215+0000 7fd69e67a980 2 removed watcher, disabling cache 2024-04-17T05:07:46.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.219+0000 7fd69e67a980 20 remove_watcher() i=1 2024-04-17T05:07:46.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.223+0000 7fd69e67a980 20 remove_watcher() i=2 2024-04-17T05:07:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.223+0000 7fd69e67a980 20 remove_watcher() i=3 2024-04-17T05:07:46.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.227+0000 7fd69e67a980 20 remove_watcher() i=4 2024-04-17T05:07:46.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.227+0000 7fd69e67a980 20 remove_watcher() i=5 2024-04-17T05:07:46.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.227+0000 7fd69e67a980 20 remove_watcher() i=6 2024-04-17T05:07:46.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.231+0000 7fd69e67a980 20 remove_watcher() i=7 2024-04-17T05:07:46.243 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW33333333333333333 --account-name Mr.RGW33333333333333333 --email RGW33333333333333333+test@test.test --tenant testx 2024-04-17T05:07:46.322 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:46.322 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:46.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.363+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 realm 2024-04-17T05:07:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.367+0000 7fed09608980 20 searching for the correct realm 2024-04-17T05:07:46.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.375+0000 7fed06922640 0 -- 172.21.15.88:0/2850188759 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x55e9c3505500 msgr2=0x55e9c3507970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T05:07:46.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.379+0000 7fed07123640 0 -- 172.21.15.88:0/2850188759 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7fecf010ec20 msgr2=0x7fecf0120040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.387+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 zone default found 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 4 Realm: () 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 zonegroup default 2024-04-17T05:07:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 started sync module instance, tier type = 2024-04-17T05:07:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.391+0000 7fed09608980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=0 2024-04-17T05:07:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=1 2024-04-17T05:07:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=2 2024-04-17T05:07:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=3 2024-04-17T05:07:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=4 2024-04-17T05:07:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=5 2024-04-17T05:07:46.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=6 2024-04-17T05:07:46.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 20 add_watcher() i=7 2024-04-17T05:07:46.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.407+0000 7fed09608980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:46.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7febbe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:46.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:46.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 5 note: GC not initialized 2024-04-17T05:07:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7febbd7e2640 20 reqs_thread_entry: start 2024-04-17T05:07:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 Filter name: none 2024-04-17T05:07:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7febb67fc640 20 reqs_thread_entry: start 2024-04-17T05:07:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 10 cache get: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 : miss 2024-04-17T05:07:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x0 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 10 adding default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 failed to read name.testx$Mr.RGW33333333333333333 with: (2) No such file or directory 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 10 cache get: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test : miss 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.411+0000 7fed09608980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x0 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 10 adding default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-04-17T05:07:46.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 20 failed to read rgw33333333333333333+test@test.test with: (2) No such file or directory 2024-04-17T05:07:46.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-04-17T05:07:46.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-04-17T05:07:46.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7fed09608980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW33333333333333333, ofs0, ns] 2024-04-17T05:07:46.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7febeaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576399009 cookie 94462492616608 notifier 7289 bl.length()=357 2024-04-17T05:07:46.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7febeaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-04-17T05:07:46.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.415+0000 7febeaffd640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-04-17T05:07:46.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.423+0000 7fed09608980 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-04-17T05:07:46.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.423+0000 7fed09608980 10 moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-04-17T05:07:46.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.423+0000 7fed09608980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.testx$Mr.RGW33333333333333333, ofs0, ns] 2024-04-17T05:07:46.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.423+0000 7febea7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576398989 cookie 94462492623424 notifier 7289 bl.length()=230 2024-04-17T05:07:46.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.423+0000 7febea7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-04-17T05:07:46.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.423+0000 7febea7fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-04-17T05:07:46.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.427+0000 7fed09608980 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-04-17T05:07:46.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.427+0000 7fed09608980 10 moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-04-17T05:07:46.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.427+0000 7fed09608980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw33333333333333333+test@test.test, ofs0, ns] 2024-04-17T05:07:46.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.427+0000 7febea7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576398937 cookie 94462492622368 notifier 7289 bl.length()=234 2024-04-17T05:07:46.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.427+0000 7febea7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-04-17T05:07:46.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.427+0000 7febea7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-04-17T05:07:46.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.431+0000 7febbe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.435+0000 7fed09608980 20 remove_watcher() i=0 2024-04-17T05:07:46.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.435+0000 7fed09608980 2 removed watcher, disabling cache 2024-04-17T05:07:46.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.435+0000 7fed09608980 20 remove_watcher() i=1 2024-04-17T05:07:46.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.439+0000 7fed09608980 20 remove_watcher() i=2 2024-04-17T05:07:46.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.439+0000 7fed09608980 20 remove_watcher() i=3 2024-04-17T05:07:46.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.443+0000 7fed09608980 20 remove_watcher() i=4 2024-04-17T05:07:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.443+0000 7fed09608980 20 remove_watcher() i=5 2024-04-17T05:07:46.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.447+0000 7fed09608980 20 remove_watcher() i=6 2024-04-17T05:07:46.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.447+0000 7fed09608980 20 remove_watcher() i=7 2024-04-17T05:07:46.456 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:46.457 INFO:teuthology.orchestra.run.smithi088.stdout: "id": "RGW33333333333333333", 2024-04-17T05:07:46.457 INFO:teuthology.orchestra.run.smithi088.stdout: "tenant": "testx", 2024-04-17T05:07:46.457 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "Mr.RGW33333333333333333", 2024-04-17T05:07:46.457 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "RGW33333333333333333+test@test.test", 2024-04-17T05:07:46.457 INFO:teuthology.orchestra.run.smithi088.stdout: "quota": { 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "max_users": 1000, 2024-04-17T05:07:46.458 INFO:teuthology.orchestra.run.smithi088.stdout: "max_roles": 1000, 2024-04-17T05:07:46.459 INFO:teuthology.orchestra.run.smithi088.stdout: "max_groups": 1000, 2024-04-17T05:07:46.459 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:46.459 INFO:teuthology.orchestra.run.smithi088.stdout: "max_access_keys": 4 2024-04-17T05:07:46.459 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:46.460 DEBUG:tasks.s3tests:Creating user RGW33333333333333333 on client.0 2024-04-17T05:07:46.460 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name Mr.RGW33333333333333333 --email 'testx$tenanteduser+test@test.test' --access-key XYBUNHAQRJRLHKSTDWXQ --secret wQcemNzfgSTieuJa4GPpzzcw8APd41JvXJttCEQxZziiwchgzAfDKw== --cluster ceph --account-id RGW33333333333333333 --account-root 2024-04-17T05:07:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:46.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.591+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 realm 2024-04-17T05:07:46.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:46.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:46.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.595+0000 7f186e3a7980 20 searching for the correct realm 2024-04-17T05:07:46.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.603+0000 7f186b6c1640 0 -- 172.21.15.88:0/2883099349 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x55b7b04557f0 msgr2=0x55b7b0457cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:46.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.607+0000 7f186bec2640 0 -- 172.21.15.88:0/2883099349 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f186008c220 msgr2=0x7f18600017a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.615+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 zone default found 2024-04-17T05:07:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 4 Realm: () 2024-04-17T05:07:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 zonegroup default 2024-04-17T05:07:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 started sync module instance, tier type = 2024-04-17T05:07:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.619+0000 7f186e3a7980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:46.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=0 2024-04-17T05:07:46.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=1 2024-04-17T05:07:46.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=2 2024-04-17T05:07:46.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=3 2024-04-17T05:07:46.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=4 2024-04-17T05:07:46.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=5 2024-04-17T05:07:46.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=6 2024-04-17T05:07:46.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 20 add_watcher() i=7 2024-04-17T05:07:46.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.635+0000 7f186e3a7980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f1721fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 5 note: GC not initialized 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f1720fe1640 20 reqs_thread_entry: start 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 Filter name: none 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f169a7fc640 20 reqs_thread_entry: start 2024-04-17T05:07:46.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test : miss 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.639+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x0 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 adding default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache get: name=default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ : miss 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ info.flags=0x0 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 adding default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ to cache LRU end 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache get: name=default.rgw.meta+accounts+account.RGW33333333333333333 : miss 2024-04-17T05:07:46.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 20 rados_obj.operate() r=0 bl.length=157 2024-04-17T05:07:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-04-17T05:07:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-04-17T05:07:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache get: name=default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ : hit (negative entry) 2024-04-17T05:07:46.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.643+0000 7f186e3a7980 10 cache get: name=default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ : hit (negative entry) 2024-04-17T05:07:46.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.647+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-17T05:07:46.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.647+0000 7f186e3a7980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-17T05:07:46.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.647+0000 7f186e3a7980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-04-17T05:07:46.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.647+0000 7f174affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576399019 cookie 94247424716560 notifier 7764 bl.length()=732 2024-04-17T05:07:46.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.647+0000 7f174affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-17T05:07:46.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.647+0000 7f174affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-17T05:07:46.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.651+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-04-17T05:07:46.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.651+0000 7f186e3a7980 10 moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-04-17T05:07:46.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.651+0000 7f186e3a7980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2024-04-17T05:07:46.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.651+0000 7f174affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576398927 cookie 94247424732656 notifier 7764 bl.length()=215 2024-04-17T05:07:46.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.651+0000 7f174affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-04-17T05:07:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.651+0000 7f174affd640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-04-17T05:07:46.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.655+0000 7f186e3a7980 10 cache put: name=default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ info.flags=0x7 2024-04-17T05:07:46.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.655+0000 7f186e3a7980 10 moving default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ to cache LRU end 2024-04-17T05:07:46.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.655+0000 7f186e3a7980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:XYBUNHAQRJRLHKSTDWXQ, ofs0, ns] 2024-04-17T05:07:46.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.655+0000 7f174affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1129576398990 cookie 94247424731088 notifier 7764 bl.length()=201 2024-04-17T05:07:46.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.655+0000 7f174affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ info.flags=0x7 2024-04-17T05:07:46.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.655+0000 7f174affd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+XYBUNHAQRJRLHKSTDWXQ to cache LRU end 2024-04-17T05:07:46.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.659+0000 7f186e3a7980 20 linked user testx$tenanteduser.client.0 to account RGW33333333333333333 2024-04-17T05:07:46.657 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "testx$tenanteduser.client.0", 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.RGW33333333333333333", 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "testx$tenanteduser+test@test.test", 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "testx$tenanteduser.client.0", 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "XYBUNHAQRJRLHKSTDWXQ", 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "wQcemNzfgSTieuJa4GPpzzcw8APd41JvXJttCEQxZziiwchgzAfDKw==", 2024-04-17T05:07:46.658 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:46.647227Z" 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:46.659 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:46.660 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "root", 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "RGW33333333333333333", 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:46.647216Z", 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:46.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:46.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.659+0000 7f1721fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:46.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.663+0000 7f186e3a7980 20 remove_watcher() i=0 2024-04-17T05:07:46.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.663+0000 7f186e3a7980 2 removed watcher, disabling cache 2024-04-17T05:07:46.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.667+0000 7f186e3a7980 20 remove_watcher() i=1 2024-04-17T05:07:46.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.671+0000 7f186e3a7980 20 remove_watcher() i=2 2024-04-17T05:07:46.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.671+0000 7f186e3a7980 20 remove_watcher() i=3 2024-04-17T05:07:46.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.675+0000 7f186e3a7980 20 remove_watcher() i=4 2024-04-17T05:07:46.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.675+0000 7f186e3a7980 20 remove_watcher() i=5 2024-04-17T05:07:46.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.675+0000 7f186e3a7980 20 remove_watcher() i=6 2024-04-17T05:07:46.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.675+0000 7f186e3a7980 20 remove_watcher() i=7 2024-04-17T05:07:46.689 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 2104921279 --totp-seed VQ2MAYBBGF5QZGO344TWLQLYMQY3Q4DY2JW62HXZX5YYSCOULHIBPUZGPTGCEYE7 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-17T05:07:46.732 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:46.732 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.783+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.783+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.783+0000 7ff672084980 20 realm 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.783+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:46.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.787+0000 7ff672084980 20 searching for the correct realm 2024-04-17T05:07:46.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:46.807+0000 7ff6703a0640 0 -- 172.21.15.88:0/2331115492 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7ff664013030 msgr2=0x7ff66402c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:47.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.023+0000 7ff66fb9f640 0 -- 172.21.15.88:0/2331115492 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5630a55ff300 msgr2=0x5630a5601730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 zone default found 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 4 Realm: () 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 zonegroup default 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.031+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.035+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.035+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.035+0000 7ff672084980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.035+0000 7ff672084980 20 started sync module instance, tier type = 2024-04-17T05:07:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.035+0000 7ff672084980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=0 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=1 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=2 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=3 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=4 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=5 2024-04-17T05:07:47.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=6 2024-04-17T05:07:47.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 20 add_watcher() i=7 2024-04-17T05:07:47.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.059+0000 7ff672084980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:47.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff525fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:47.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:47.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:47.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 5 note: GC not initialized 2024-04-17T05:07:47.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff524fe1640 20 reqs_thread_entry: start 2024-04-17T05:07:47.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:47.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 20 Filter name: none 2024-04-17T05:07:47.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff49e7fc640 20 reqs_thread_entry: start 2024-04-17T05:07:47.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-04-17T05:07:47.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 20 rados_obj.operate() r=0 bl.length=532 2024-04-17T05:07:47.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-04-17T05:07:47.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-17T05:07:47.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.067+0000 7ff672084980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:47.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.071+0000 7ff672084980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-17T05:07:47.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.071+0000 7ff672084980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-17T05:07:47.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.071+0000 7ff672084980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-04-17T05:07:47.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.071+0000 7ff54effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366316 cookie 94766932953488 notifier 7785 bl.length()=746 2024-04-17T05:07:47.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.071+0000 7ff54effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-17T05:07:47.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.071+0000 7ff54effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-17T05:07:47.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.075+0000 7ff525fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.083+0000 7ff672084980 20 remove_watcher() i=0 2024-04-17T05:07:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.083+0000 7ff672084980 2 removed watcher, disabling cache 2024-04-17T05:07:47.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.083+0000 7ff672084980 20 remove_watcher() i=1 2024-04-17T05:07:47.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.087+0000 7ff672084980 20 remove_watcher() i=2 2024-04-17T05:07:47.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.087+0000 7ff672084980 20 remove_watcher() i=3 2024-04-17T05:07:47.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.091+0000 7ff672084980 20 remove_watcher() i=4 2024-04-17T05:07:47.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.091+0000 7ff672084980 20 remove_watcher() i=5 2024-04-17T05:07:47.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.095+0000 7ff672084980 20 remove_watcher() i=6 2024-04-17T05:07:47.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.095+0000 7ff672084980 20 remove_watcher() i=7 2024-04-17T05:07:47.113 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-04-17T05:07:47.114 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name Mr.foobar.client.0 --email foobar+test@test.test --access-key UUWPESPMUGVLFTUQPKPD --secret hPxLynn5POk8xRcI9fCIxF66U2jQnzhDhfBbLsSA274zzRinogxKvg== --cluster ceph 2024-04-17T05:07:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:47.200 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:47.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 realm 2024-04-17T05:07:47.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:47.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.243+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.247+0000 7feb2eea0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:47.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.247+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.247+0000 7feb2eea0980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:47.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.247+0000 7feb2eea0980 20 searching for the correct realm 2024-04-17T05:07:47.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.255+0000 7feb2d1bc640 0 -- 172.21.15.88:0/3121339875 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7feb20049940 msgr2=0x7feb20029400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.471+0000 7feb2c9bb640 0 -- 172.21.15.88:0/3121339875 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x56171fbfb500 msgr2=0x56171fbfd930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:47.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:47.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:47.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:47.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 zone default found 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 4 Realm: () 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 zonegroup default 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.479+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.483+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.483+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.483+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.483+0000 7feb2eea0980 20 started sync module instance, tier type = 2024-04-17T05:07:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.483+0000 7feb2eea0980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:47.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=0 2024-04-17T05:07:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=1 2024-04-17T05:07:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=2 2024-04-17T05:07:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=3 2024-04-17T05:07:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=4 2024-04-17T05:07:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=5 2024-04-17T05:07:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.503+0000 7feb2eea0980 20 add_watcher() i=6 2024-04-17T05:07:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.507+0000 7feb2eea0980 20 add_watcher() i=7 2024-04-17T05:07:47.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.507+0000 7feb2eea0980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7fe9e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 5 note: GC not initialized 2024-04-17T05:07:47.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7fe9e1fe3640 20 reqs_thread_entry: start 2024-04-17T05:07:47.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 Filter name: none 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7fe9537fe640 20 reqs_thread_entry: start 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-04-17T05:07:47.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:47.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 10 cache get: name=default.rgw.meta+users.email+foobar+test@test.test : miss 2024-04-17T05:07:47.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.511+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x0 2024-04-17T05:07:47.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 adding default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-04-17T05:07:47.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 cache get: name=default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD : miss 2024-04-17T05:07:47.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 cache put: name=default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD info.flags=0x0 2024-04-17T05:07:47.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 adding default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD to cache LRU end 2024-04-17T05:07:47.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 cache get: name=default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD : hit (negative entry) 2024-04-17T05:07:47.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 cache get: name=default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD : hit (negative entry) 2024-04-17T05:07:47.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-17T05:07:47.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:47.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.515+0000 7feb2eea0980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-04-17T05:07:47.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.519+0000 7fea03fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366224 cookie 94657316945744 notifier 7340 bl.length()=660 2024-04-17T05:07:47.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.519+0000 7fea03fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-17T05:07:47.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.519+0000 7fea03fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:47.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.523+0000 7feb2eea0980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-04-17T05:07:47.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.523+0000 7feb2eea0980 10 moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-04-17T05:07:47.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.523+0000 7feb2eea0980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2024-04-17T05:07:47.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.523+0000 7fea03fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366306 cookie 94657316929600 notifier 7340 bl.length()=191 2024-04-17T05:07:47.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.523+0000 7fea03fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-04-17T05:07:47.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.523+0000 7fea03fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-04-17T05:07:47.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.531+0000 7feb2eea0980 10 cache put: name=default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD info.flags=0x7 2024-04-17T05:07:47.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.531+0000 7feb2eea0980 10 moving default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD to cache LRU end 2024-04-17T05:07:47.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.531+0000 7feb2eea0980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:UUWPESPMUGVLFTUQPKPD, ofs0, ns] 2024-04-17T05:07:47.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.531+0000 7fea037fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366287 cookie 94657316934368 notifier 7340 bl.length()=189 2024-04-17T05:07:47.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.531+0000 7fea037fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD info.flags=0x7 2024-04-17T05:07:47.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.531+0000 7fea037fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+UUWPESPMUGVLFTUQPKPD to cache LRU end 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "foobar.client.0", 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.foobar.client.0", 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "foobar+test@test.test", 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "foobar.client.0", 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "UUWPESPMUGVLFTUQPKPD", 2024-04-17T05:07:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "hPxLynn5POk8xRcI9fCIxF66U2jQnzhDhfBbLsSA274zzRinogxKvg==", 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:47.518618Z" 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "rgw", 2024-04-17T05:07:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "", 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:47.518591Z", 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:47.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.535+0000 7fe9e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:47.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.539+0000 7feb2eea0980 20 remove_watcher() i=0 2024-04-17T05:07:47.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.539+0000 7feb2eea0980 2 removed watcher, disabling cache 2024-04-17T05:07:47.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.543+0000 7feb2eea0980 20 remove_watcher() i=1 2024-04-17T05:07:47.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.543+0000 7feb2eea0980 20 remove_watcher() i=2 2024-04-17T05:07:47.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.547+0000 7feb2eea0980 20 remove_watcher() i=3 2024-04-17T05:07:47.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.547+0000 7feb2eea0980 20 remove_watcher() i=4 2024-04-17T05:07:47.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.547+0000 7feb2eea0980 20 remove_watcher() i=5 2024-04-17T05:07:47.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.551+0000 7feb2eea0980 20 remove_watcher() i=6 2024-04-17T05:07:47.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.551+0000 7feb2eea0980 20 remove_watcher() i=7 2024-04-17T05:07:47.567 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 9647937757 --totp-seed 5OVNORA75X66XN3A24NSWPUWGZOFLR3FCKOCS7YRVVUH3JPNGXF3XOGW6EXCY423 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-17T05:07:47.609 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:47.609 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:47.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 20 realm 2024-04-17T05:07:47.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.651+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.655+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.655+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.655+0000 7f8c448e7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:47.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.655+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.655+0000 7f8c448e7980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:47.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.655+0000 7f8c448e7980 20 searching for the correct realm 2024-04-17T05:07:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.663+0000 7f8c42c03640 0 -- 172.21.15.88:0/1288046957 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f8c3800a070 msgr2=0x7f8c3802e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.879+0000 7f8c42402640 0 -- 172.21.15.88:0/1288046957 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55e3f55d51e0 msgr2=0x55e3f55d7610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:47.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:47.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 zone default found 2024-04-17T05:07:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 4 Realm: () 2024-04-17T05:07:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 zonegroup default 2024-04-17T05:07:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.887+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.891+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.891+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.891+0000 7f8c448e7980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.891+0000 7f8c448e7980 20 started sync module instance, tier type = 2024-04-17T05:07:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.891+0000 7f8c448e7980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:47.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=0 2024-04-17T05:07:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=1 2024-04-17T05:07:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=2 2024-04-17T05:07:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=3 2024-04-17T05:07:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=4 2024-04-17T05:07:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=5 2024-04-17T05:07:47.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=6 2024-04-17T05:07:47.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 20 add_watcher() i=7 2024-04-17T05:07:47.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.915+0000 7f8c448e7980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.919+0000 7f8af8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.919+0000 7f8c448e7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.919+0000 7f8c448e7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:47.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 5 note: GC not initialized 2024-04-17T05:07:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8a737fe640 20 reqs_thread_entry: start 2024-04-17T05:07:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 20 Filter name: none 2024-04-17T05:07:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8a717fa640 20 reqs_thread_entry: start 2024-04-17T05:07:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-04-17T05:07:47.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 20 rados->read ofs=0 len=0 2024-04-17T05:07:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 20 rados_obj.operate() r=0 bl.length=472 2024-04-17T05:07:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-04-17T05:07:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:47.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.923+0000 7f8c448e7980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.927+0000 7f8c448e7980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-17T05:07:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.927+0000 7f8c448e7980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:47.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.927+0000 7f8c448e7980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-04-17T05:07:47.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.931+0000 7f8b25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366225 cookie 94437562523216 notifier 7824 bl.length()=674 2024-04-17T05:07:47.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.931+0000 7f8b25ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-17T05:07:47.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.931+0000 7f8b25ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:47.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.931+0000 7f8af8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.939+0000 7f8c448e7980 20 remove_watcher() i=0 2024-04-17T05:07:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.939+0000 7f8c448e7980 2 removed watcher, disabling cache 2024-04-17T05:07:47.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.939+0000 7f8c448e7980 20 remove_watcher() i=1 2024-04-17T05:07:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.943+0000 7f8c448e7980 20 remove_watcher() i=2 2024-04-17T05:07:47.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.943+0000 7f8c448e7980 20 remove_watcher() i=3 2024-04-17T05:07:47.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.947+0000 7f8c448e7980 20 remove_watcher() i=4 2024-04-17T05:07:47.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.947+0000 7f8c448e7980 20 remove_watcher() i=5 2024-04-17T05:07:47.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.951+0000 7f8c448e7980 20 remove_watcher() i=6 2024-04-17T05:07:47.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:47.951+0000 7f8c448e7980 20 remove_watcher() i=7 2024-04-17T05:07:47.966 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*;roles=*;user-policy=*' --cluster ceph 2024-04-17T05:07:48.047 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:48.048 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:48.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 realm 2024-04-17T05:07:48.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:48.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.091+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.095+0000 7f1995381980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:48.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.095+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.095+0000 7f1995381980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:48.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.095+0000 7f1995381980 20 searching for the correct realm 2024-04-17T05:07:48.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.103+0000 7f199369d640 0 -- 172.21.15.88:0/2850058925 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f197c02a410 msgr2=0x7f197c02a890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:48.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.111+0000 7f199369d640 0 -- 172.21.15.88:0/2850058925 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f185c077100 msgr2=0x7f185c079530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.115+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:48.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 zone default found 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 4 Realm: () 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 zonegroup default 2024-04-17T05:07:48.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 started sync module instance, tier type = 2024-04-17T05:07:48.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.119+0000 7f1995381980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=0 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=1 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=2 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=3 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=4 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=5 2024-04-17T05:07:48.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=6 2024-04-17T05:07:48.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 20 add_watcher() i=7 2024-04-17T05:07:48.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.135+0000 7f1995381980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:48.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f18497e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:48.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 5 note: GC not initialized 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f17c3fff640 20 reqs_thread_entry: start 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 20 Filter name: none 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f17c1ffb640 20 reqs_thread_entry: start 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-04-17T05:07:48.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 20 rados_obj.operate() r=0 bl.length=486 2024-04-17T05:07:48.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-04-17T05:07:48.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:48.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.139+0000 7f1995381980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:48.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.143+0000 7f1995381980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-17T05:07:48.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.143+0000 7f1995381980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:48.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.143+0000 7f1995381980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-04-17T05:07:48.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.143+0000 7f18767fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366226 cookie 94326927256416 notifier 7839 bl.length()=727 2024-04-17T05:07:48.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.143+0000 7f18767fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-17T05:07:48.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.143+0000 7f18767fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "foobar.client.0", 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.foobar.client.0", 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "foobar+test@test.test", 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "foobar.client.0", 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "UUWPESPMUGVLFTUQPKPD", 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "hPxLynn5POk8xRcI9fCIxF66U2jQnzhDhfBbLsSA274zzRinogxKvg==", 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:47.518618Z" 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [ 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "oidc-provider", 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "perm": "*" 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "roles", 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: "perm": "*" 2024-04-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "user-policy", 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "perm": "*" 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "rgw", 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [ 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "9647937757" 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "", 2024-04-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:47.518591Z", 2024-04-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.147+0000 7f18497e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:48.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.151+0000 7f1995381980 20 remove_watcher() i=0 2024-04-17T05:07:48.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.151+0000 7f1995381980 2 removed watcher, disabling cache 2024-04-17T05:07:48.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.151+0000 7f1995381980 20 remove_watcher() i=1 2024-04-17T05:07:48.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.155+0000 7f1995381980 20 remove_watcher() i=2 2024-04-17T05:07:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.155+0000 7f1995381980 20 remove_watcher() i=3 2024-04-17T05:07:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.159+0000 7f1995381980 20 remove_watcher() i=4 2024-04-17T05:07:48.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.159+0000 7f1995381980 20 remove_watcher() i=5 2024-04-17T05:07:48.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.159+0000 7f1995381980 20 remove_watcher() i=6 2024-04-17T05:07:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.163+0000 7f1995381980 20 remove_watcher() i=7 2024-04-17T05:07:48.174 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW88888888888888888 --account-name Mr.RGW88888888888888888 --email RGW88888888888888888+test@test.test 2024-04-17T05:07:48.256 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:48.257 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:48.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.299+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 realm 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:48.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.303+0000 7f60fab0a980 20 searching for the correct realm 2024-04-17T05:07:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.311+0000 7f60f7e24640 0 -- 172.21.15.88:0/4182328611 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x558e6204b1a0 msgr2=0x558e6204d660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:48.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.315+0000 7f60f7e24640 0 -- 172.21.15.88:0/4182328611 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f60e80e5410 msgr2=0x7f60e80e5830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T05:07:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:48.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:48.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.323+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:48.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 zone default found 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 4 Realm: () 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:48.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 zonegroup default 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 started sync module instance, tier type = 2024-04-17T05:07:48.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.327+0000 7f60fab0a980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=0 2024-04-17T05:07:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=1 2024-04-17T05:07:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=2 2024-04-17T05:07:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=3 2024-04-17T05:07:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=4 2024-04-17T05:07:48.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=5 2024-04-17T05:07:48.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=6 2024-04-17T05:07:48.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 20 add_watcher() i=7 2024-04-17T05:07:48.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.343+0000 7f60fab0a980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f5faefdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 5 note: GC not initialized 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f5fadfdb640 20 reqs_thread_entry: start 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 Filter name: none 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f5f1f7fe640 20 reqs_thread_entry: start 2024-04-17T05:07:48.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 : miss 2024-04-17T05:07:48.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x0 2024-04-17T05:07:48.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 10 adding default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-04-17T05:07:48.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 failed to read name.$Mr.RGW88888888888888888 with: (2) No such file or directory 2024-04-17T05:07:48.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 10 cache get: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test : miss 2024-04-17T05:07:48.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x0 2024-04-17T05:07:48.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 10 adding default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-04-17T05:07:48.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.347+0000 7f60fab0a980 20 failed to read rgw88888888888888888+test@test.test with: (2) No such file or directory 2024-04-17T05:07:48.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.351+0000 7f60fab0a980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-04-17T05:07:48.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.351+0000 7f60fab0a980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-04-17T05:07:48.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.351+0000 7f60fab0a980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2024-04-17T05:07:48.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.351+0000 7f5fcf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366307 cookie 94070018243008 notifier 7860 bl.length()=352 2024-04-17T05:07:48.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.351+0000 7f5fcf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-04-17T05:07:48.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.351+0000 7f5fcf7fe640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-04-17T05:07:48.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.355+0000 7f60fab0a980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-04-17T05:07:48.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.355+0000 7f60fab0a980 10 moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-04-17T05:07:48.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.355+0000 7f60fab0a980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2024-04-17T05:07:48.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.359+0000 7f5fcf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366288 cookie 94070018253120 notifier 7860 bl.length()=225 2024-04-17T05:07:48.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.359+0000 7f5fcf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-04-17T05:07:48.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.359+0000 7f5fcf7fe640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-04-17T05:07:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.363+0000 7f60fab0a980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-04-17T05:07:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.363+0000 7f60fab0a980 10 moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-04-17T05:07:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.363+0000 7f60fab0a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2024-04-17T05:07:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.363+0000 7f5fcf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366234 cookie 94070018249568 notifier 7860 bl.length()=234 2024-04-17T05:07:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.363+0000 7f5fcf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-04-17T05:07:48.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.363+0000 7f5fcf7fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-04-17T05:07:48.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.367+0000 7f5faefdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.371+0000 7f60fab0a980 20 remove_watcher() i=0 2024-04-17T05:07:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.371+0000 7f60fab0a980 2 removed watcher, disabling cache 2024-04-17T05:07:48.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.371+0000 7f60fab0a980 20 remove_watcher() i=1 2024-04-17T05:07:48.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.583+0000 7f60fab0a980 20 remove_watcher() i=2 2024-04-17T05:07:48.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.583+0000 7f60fab0a980 20 remove_watcher() i=3 2024-04-17T05:07:48.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.587+0000 7f60fab0a980 20 remove_watcher() i=4 2024-04-17T05:07:48.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.591+0000 7f60fab0a980 20 remove_watcher() i=5 2024-04-17T05:07:48.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.591+0000 7f60fab0a980 20 remove_watcher() i=6 2024-04-17T05:07:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.595+0000 7f60fab0a980 20 remove_watcher() i=7 2024-04-17T05:07:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: "id": "RGW88888888888888888", 2024-04-17T05:07:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: "tenant": "", 2024-04-17T05:07:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "Mr.RGW88888888888888888", 2024-04-17T05:07:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "RGW88888888888888888+test@test.test", 2024-04-17T05:07:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: "quota": { 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_users": 1000, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_roles": 1000, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_groups": 1000, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max_access_keys": 4 2024-04-17T05:07:48.616 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:48.619 DEBUG:tasks.s3tests:Creating user RGW88888888888888888 on client.0 2024-04-17T05:07:48.620 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root1.client.0 --display-name Mr.RGW88888888888888888 --email root1+test@test.test --access-key JIHYYJKRORZDQJCHPBVZ --secret rWNcYdBzibDkW/Yt0zz9wa7UxzbWcd4wngbAlWv0xZJC71zVYGsYcg== --cluster ceph --account-id RGW88888888888888888 --account-root 2024-04-17T05:07:48.713 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:48.713 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:48.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 realm 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.755+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.759+0000 7f584c80c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:48.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.759+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.759+0000 7f584c80c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:48.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.759+0000 7f584c80c980 20 searching for the correct realm 2024-04-17T05:07:48.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.767+0000 7f584ab28640 0 -- 172.21.15.88:0/3568594969 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7f5840008050 msgr2=0x7f5840018410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-17T05:07:48.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.983+0000 7f584a327640 0 -- 172.21.15.88:0/3568594969 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55624ca7b7f0 msgr2=0x55624ca7dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:48.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:48.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:48.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:48.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.991+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:48.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 zone default found 2024-04-17T05:07:48.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 4 Realm: () 2024-04-17T05:07:48.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:48.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:48.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:48.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 zonegroup default 2024-04-17T05:07:48.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:48.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:48.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 started sync module instance, tier type = 2024-04-17T05:07:48.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:48.995+0000 7f584c80c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:49.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=0 2024-04-17T05:07:49.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=1 2024-04-17T05:07:49.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=2 2024-04-17T05:07:49.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=3 2024-04-17T05:07:49.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=4 2024-04-17T05:07:49.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=5 2024-04-17T05:07:49.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=6 2024-04-17T05:07:49.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 20 add_watcher() i=7 2024-04-17T05:07:49.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.023+0000 7f584c80c980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.027+0000 7f57017e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.027+0000 7f584c80c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.027+0000 7f584c80c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.027+0000 7f584c80c980 5 note: GC not initialized 2024-04-17T05:07:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.027+0000 7f56fb7fe640 20 reqs_thread_entry: start 2024-04-17T05:07:49.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:49.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 Filter name: none 2024-04-17T05:07:49.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f56f97fa640 20 reqs_thread_entry: start 2024-04-17T05:07:49.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-04-17T05:07:49.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2024-04-17T05:07:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-17T05:07:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache get: name=default.rgw.meta+users.email+root1+test@test.test : miss 2024-04-17T05:07:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x0 2024-04-17T05:07:49.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 adding default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache get: name=default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ : miss 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache put: name=default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ info.flags=0x0 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 adding default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ to cache LRU end 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2024-04-17T05:07:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.031+0000 7f584c80c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 20 rados_obj.operate() r=0 bl.length=152 2024-04-17T05:07:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-04-17T05:07:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-04-17T05:07:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 cache get: name=default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ : hit (negative entry) 2024-04-17T05:07:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 cache get: name=default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ : hit (negative entry) 2024-04-17T05:07:49.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-17T05:07:49.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-17T05:07:49.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.035+0000 7f584c80c980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-04-17T05:07:49.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.039+0000 7f572dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366308 cookie 93880681223184 notifier 7878 bl.length()=681 2024-04-17T05:07:49.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.039+0000 7f572dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-17T05:07:49.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.039+0000 7f572dffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-17T05:07:49.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.043+0000 7f584c80c980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-04-17T05:07:49.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.043+0000 7f584c80c980 10 moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-04-17T05:07:49.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.043+0000 7f584c80c980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2024-04-17T05:07:49.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.043+0000 7f572d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366317 cookie 93880681260032 notifier 7878 bl.length()=189 2024-04-17T05:07:49.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.043+0000 7f572d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-04-17T05:07:49.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.043+0000 7f572d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-04-17T05:07:49.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.047+0000 7f584c80c980 10 cache put: name=default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ info.flags=0x7 2024-04-17T05:07:49.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.047+0000 7f584c80c980 10 moving default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ to cache LRU end 2024-04-17T05:07:49.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.047+0000 7f584c80c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:JIHYYJKRORZDQJCHPBVZ, ofs0, ns] 2024-04-17T05:07:49.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.047+0000 7f572d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366318 cookie 93880681265344 notifier 7878 bl.length()=188 2024-04-17T05:07:49.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.047+0000 7f572d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ info.flags=0x7 2024-04-17T05:07:49.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.047+0000 7f572d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+JIHYYJKRORZDQJCHPBVZ to cache LRU end 2024-04-17T05:07:49.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.055+0000 7f584c80c980 20 linked user root1.client.0 to account RGW88888888888888888 2024-04-17T05:07:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "root1.client.0", 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.RGW88888888888888888", 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "root1+test@test.test", 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "root1.client.0", 2024-04-17T05:07:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "JIHYYJKRORZDQJCHPBVZ", 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "rWNcYdBzibDkW/Yt0zz9wa7UxzbWcd4wngbAlWv0xZJC71zVYGsYcg==", 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:49.037073Z" 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "root", 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T05:07:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "RGW88888888888888888", 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:49.037047Z", 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:49.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.055+0000 7f57017e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.059+0000 7f584c80c980 20 remove_watcher() i=0 2024-04-17T05:07:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.059+0000 7f584c80c980 2 removed watcher, disabling cache 2024-04-17T05:07:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.063+0000 7f584c80c980 20 remove_watcher() i=1 2024-04-17T05:07:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.067+0000 7f584c80c980 20 remove_watcher() i=2 2024-04-17T05:07:49.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.067+0000 7f584c80c980 20 remove_watcher() i=3 2024-04-17T05:07:49.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.071+0000 7f584c80c980 20 remove_watcher() i=4 2024-04-17T05:07:49.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.071+0000 7f584c80c980 20 remove_watcher() i=5 2024-04-17T05:07:49.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.075+0000 7f584c80c980 20 remove_watcher() i=6 2024-04-17T05:07:49.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.075+0000 7f584c80c980 20 remove_watcher() i=7 2024-04-17T05:07:49.089 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root1.client.0 --totp-serial 5179722921 --totp-seed FGSZFCIFJS4SPAHUCYBMUMGLGMBLSVT3DLILFOYQMSXGR7I7KQRKSJ6CPA7SFSQH --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-17T05:07:49.134 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:49.134 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.175+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 realm 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:49.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:49.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.179+0000 7faef4040980 20 searching for the correct realm 2024-04-17T05:07:49.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.187+0000 7faef235c640 0 -- 172.21.15.88:0/188853057 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7faee8034060 msgr2=0x7faee80838e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.403+0000 7faef1b5b640 0 -- 172.21.15.88:0/188853057 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x55e5d9239530 msgr2=0x55e5d923b980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:49.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.411+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 zone default found 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 4 Realm: () 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 zonegroup default 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 started sync module instance, tier type = 2024-04-17T05:07:49.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.415+0000 7faef4040980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:49.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=0 2024-04-17T05:07:49.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=1 2024-04-17T05:07:49.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=2 2024-04-17T05:07:49.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=3 2024-04-17T05:07:49.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=4 2024-04-17T05:07:49.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=5 2024-04-17T05:07:49.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.439+0000 7faef4040980 20 add_watcher() i=6 2024-04-17T05:07:49.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.443+0000 7faef4040980 20 add_watcher() i=7 2024-04-17T05:07:49.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.443+0000 7faef4040980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:49.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7fada37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:49.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:49.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:49.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 5 note: GC not initialized 2024-04-17T05:07:49.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7fada27fc640 20 reqs_thread_entry: start 2024-04-17T05:07:49.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:49.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 20 Filter name: none 2024-04-17T05:07:49.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7fad13fff640 20 reqs_thread_entry: start 2024-04-17T05:07:49.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-04-17T05:07:49.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.447+0000 7faef4040980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 20 rados_obj.operate() r=0 bl.length=494 2024-04-17T05:07:49.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2024-04-17T05:07:49.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-17T05:07:49.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:49.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-17T05:07:49.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-17T05:07:49.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.451+0000 7faef4040980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-04-17T05:07:49.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.455+0000 7fadd57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366309 cookie 94445678893248 notifier 7424 bl.length()=695 2024-04-17T05:07:49.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.455+0000 7fadd57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-17T05:07:49.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.455+0000 7fadd57fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-17T05:07:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.459+0000 7fada37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:49.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.463+0000 7faef4040980 20 remove_watcher() i=0 2024-04-17T05:07:49.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.463+0000 7faef4040980 2 removed watcher, disabling cache 2024-04-17T05:07:49.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.467+0000 7faef4040980 20 remove_watcher() i=1 2024-04-17T05:07:49.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.467+0000 7faef4040980 20 remove_watcher() i=2 2024-04-17T05:07:49.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.471+0000 7faef4040980 20 remove_watcher() i=3 2024-04-17T05:07:49.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.471+0000 7faef4040980 20 remove_watcher() i=4 2024-04-17T05:07:49.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.475+0000 7faef4040980 20 remove_watcher() i=5 2024-04-17T05:07:49.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.475+0000 7faef4040980 20 remove_watcher() i=6 2024-04-17T05:07:49.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.475+0000 7faef4040980 20 remove_watcher() i=7 2024-04-17T05:07:49.490 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW99999999999999999 --account-name Mr.RGW99999999999999999 --email RGW99999999999999999+test@test.test 2024-04-17T05:07:49.528 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:49.528 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:49.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.779+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 realm 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:49.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.783+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.787+0000 7f1baa8e6980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:49.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.787+0000 7f1baa8e6980 20 searching for the correct realm 2024-04-17T05:07:49.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.791+0000 7f1ba8c02640 0 -- 172.21.15.88:0/1110215821 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x56513cd249f0 msgr2=0x56513cd26e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:49.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.799+0000 7f1ba7c00640 0 -- 172.21.15.88:0/1110215821 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f1b98117070 msgr2=0x7f1b98002050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:49.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:49.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 zone default found 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 4 Realm: () 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 zonegroup default 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 started sync module instance, tier type = 2024-04-17T05:07:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.807+0000 7f1baa8e6980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:49.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=0 2024-04-17T05:07:49.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=1 2024-04-17T05:07:49.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=2 2024-04-17T05:07:49.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=3 2024-04-17T05:07:49.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=4 2024-04-17T05:07:49.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=5 2024-04-17T05:07:49.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=6 2024-04-17T05:07:49.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 20 add_watcher() i=7 2024-04-17T05:07:49.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.827+0000 7f1baa8e6980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:49.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1a5efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:49.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:49.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:49.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 5 note: GC not initialized 2024-04-17T05:07:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1a5dfdb640 20 reqs_thread_entry: start 2024-04-17T05:07:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 Filter name: none 2024-04-17T05:07:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f19cf7fe640 20 reqs_thread_entry: start 2024-04-17T05:07:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 : miss 2024-04-17T05:07:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x0 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 10 adding default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 failed to read name.$Mr.RGW99999999999999999 with: (2) No such file or directory 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 10 cache get: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test : miss 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 rados->read ofs=0 len=0 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x0 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 10 adding default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-04-17T05:07:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.831+0000 7f1baa8e6980 20 failed to read rgw99999999999999999+test@test.test with: (2) No such file or directory 2024-04-17T05:07:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.835+0000 7f1baa8e6980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-04-17T05:07:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.835+0000 7f1baa8e6980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-04-17T05:07:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.835+0000 7f1baa8e6980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2024-04-17T05:07:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.835+0000 7f1a7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366310 cookie 94906912806320 notifier 7911 bl.length()=352 2024-04-17T05:07:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.835+0000 7f1a7effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-04-17T05:07:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.835+0000 7f1a7effd640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-04-17T05:07:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.839+0000 7f1baa8e6980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-04-17T05:07:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.839+0000 7f1baa8e6980 10 moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-04-17T05:07:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.839+0000 7f1baa8e6980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2024-04-17T05:07:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.839+0000 7f1a7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366289 cookie 94906912808656 notifier 7911 bl.length()=225 2024-04-17T05:07:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.839+0000 7f1a7effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-04-17T05:07:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.839+0000 7f1a7effd640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-04-17T05:07:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.847+0000 7f1baa8e6980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-04-17T05:07:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.847+0000 7f1baa8e6980 10 moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-04-17T05:07:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.847+0000 7f1baa8e6980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2024-04-17T05:07:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.847+0000 7f1a7f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366235 cookie 94906912758208 notifier 7911 bl.length()=234 2024-04-17T05:07:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.847+0000 7f1a7f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-04-17T05:07:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.847+0000 7f1a7f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-04-17T05:07:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.851+0000 7f1a5efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:49.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.855+0000 7f1baa8e6980 20 remove_watcher() i=0 2024-04-17T05:07:49.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.855+0000 7f1baa8e6980 2 removed watcher, disabling cache 2024-04-17T05:07:49.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.859+0000 7f1baa8e6980 20 remove_watcher() i=1 2024-04-17T05:07:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.859+0000 7f1baa8e6980 20 remove_watcher() i=2 2024-04-17T05:07:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.863+0000 7f1baa8e6980 20 remove_watcher() i=3 2024-04-17T05:07:49.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.863+0000 7f1baa8e6980 20 remove_watcher() i=4 2024-04-17T05:07:49.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.863+0000 7f1baa8e6980 20 remove_watcher() i=5 2024-04-17T05:07:49.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.867+0000 7f1baa8e6980 20 remove_watcher() i=6 2024-04-17T05:07:49.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:49.867+0000 7f1baa8e6980 20 remove_watcher() i=7 2024-04-17T05:07:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:49.880 INFO:teuthology.orchestra.run.smithi088.stdout: "id": "RGW99999999999999999", 2024-04-17T05:07:49.880 INFO:teuthology.orchestra.run.smithi088.stdout: "tenant": "", 2024-04-17T05:07:49.880 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "Mr.RGW99999999999999999", 2024-04-17T05:07:49.880 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "RGW99999999999999999+test@test.test", 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "quota": { 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_users": 1000, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_roles": 1000, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_groups": 1000, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max_access_keys": 4 2024-04-17T05:07:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:49.883 DEBUG:tasks.s3tests:Creating user RGW99999999999999999 on client.0 2024-04-17T05:07:49.884 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root2.client.0 --display-name Mr.RGW99999999999999999 --email root2+test@test.test --access-key DLFBOYCFGVEEFBWBJXPB --secret 1aJgT62IhvsinHCIVt1CoprEWChLiqVXu9pVQ+V53uyAobp/PxodPg== --cluster ceph --account-id RGW99999999999999999 --account-root 2024-04-17T05:07:49.975 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:49.975 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:50.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 realm 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.019+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.023+0000 7f4bbcf3c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:50.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.023+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.023+0000 7f4bbcf3c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:50.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.023+0000 7f4bbcf3c980 20 searching for the correct realm 2024-04-17T05:07:50.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.031+0000 7f4bbb258640 0 -- 172.21.15.88:0/3226204967 >> [v2:172.21.15.88:6816/3314042889,v1:172.21.15.88:6817/3314042889] conn(0x564167ef7330 msgr2=0x564167ef97f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:50.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.035+0000 7f4bba256640 0 -- 172.21.15.88:0/3226204967 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f4bac027040 msgr2=0x7f4bac044930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:50.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.043+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:50.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 zone default found 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 4 Realm: () 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 zonegroup default 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 started sync module instance, tier type = 2024-04-17T05:07:50.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.047+0000 7f4bbcf3c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=0 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=1 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=2 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=3 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=4 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=5 2024-04-17T05:07:50.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=6 2024-04-17T05:07:50.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 20 add_watcher() i=7 2024-04-17T05:07:50.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.063+0000 7f4bbcf3c980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:50.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4a717e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:50.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:50.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:50.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 5 note: GC not initialized 2024-04-17T05:07:50.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f49ebfff640 20 reqs_thread_entry: start 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 Filter name: none 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f49e9ffb640 20 reqs_thread_entry: start 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 cache get: name=default.rgw.meta+users.email+root2+test@test.test : miss 2024-04-17T05:07:50.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x0 2024-04-17T05:07:50.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 adding default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-04-17T05:07:50.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 cache get: name=default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB : miss 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB info.flags=0x0 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.067+0000 7f4bbcf3c980 10 adding default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB to cache LRU end 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 20 rados_obj.operate() r=0 bl.length=152 2024-04-17T05:07:50.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-04-17T05:07:50.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-04-17T05:07:50.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:50.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 cache get: name=default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB : hit (negative entry) 2024-04-17T05:07:50.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 cache get: name=default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB : hit (negative entry) 2024-04-17T05:07:50.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-17T05:07:50.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-17T05:07:50.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.071+0000 7f4bbcf3c980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-04-17T05:07:50.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.075+0000 7f4a9e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366311 cookie 94838916613536 notifier 7932 bl.length()=681 2024-04-17T05:07:50.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.075+0000 7f4a9e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-17T05:07:50.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.075+0000 7f4a9e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-17T05:07:50.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-04-17T05:07:50.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4bbcf3c980 10 moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-04-17T05:07:50.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4bbcf3c980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2024-04-17T05:07:50.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4a9dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366319 cookie 94838916671488 notifier 7932 bl.length()=189 2024-04-17T05:07:50.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4a9dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-04-17T05:07:50.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4a9dffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-04-17T05:07:50.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4bbcf3c980 10 cache put: name=default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB info.flags=0x7 2024-04-17T05:07:50.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4bbcf3c980 10 moving default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB to cache LRU end 2024-04-17T05:07:50.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.079+0000 7f4bbcf3c980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:DLFBOYCFGVEEFBWBJXPB, ofs0, ns] 2024-04-17T05:07:50.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.083+0000 7f4a9dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366290 cookie 94838916615744 notifier 7932 bl.length()=188 2024-04-17T05:07:50.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.083+0000 7f4a9dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB info.flags=0x7 2024-04-17T05:07:50.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.083+0000 7f4a9dffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+DLFBOYCFGVEEFBWBJXPB to cache LRU end 2024-04-17T05:07:50.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.087+0000 7f4bbcf3c980 20 linked user root2.client.0 to account RGW99999999999999999 2024-04-17T05:07:50.085 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:07:50.085 INFO:teuthology.orchestra.run.smithi088.stdout: "user_id": "root2.client.0", 2024-04-17T05:07:50.085 INFO:teuthology.orchestra.run.smithi088.stdout: "display_name": "Mr.RGW99999999999999999", 2024-04-17T05:07:50.086 INFO:teuthology.orchestra.run.smithi088.stdout: "email": "root2+test@test.test", 2024-04-17T05:07:50.086 INFO:teuthology.orchestra.run.smithi088.stdout: "suspended": 0, 2024-04-17T05:07:50.086 INFO:teuthology.orchestra.run.smithi088.stdout: "max_buckets": 1000, 2024-04-17T05:07:50.086 INFO:teuthology.orchestra.run.smithi088.stdout: "subusers": [], 2024-04-17T05:07:50.086 INFO:teuthology.orchestra.run.smithi088.stdout: "keys": [ 2024-04-17T05:07:50.086 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:07:50.087 INFO:teuthology.orchestra.run.smithi088.stdout: "user": "root2.client.0", 2024-04-17T05:07:50.087 INFO:teuthology.orchestra.run.smithi088.stdout: "access_key": "DLFBOYCFGVEEFBWBJXPB", 2024-04-17T05:07:50.087 INFO:teuthology.orchestra.run.smithi088.stdout: "secret_key": "1aJgT62IhvsinHCIVt1CoprEWChLiqVXu9pVQ+V53uyAobp/PxodPg==", 2024-04-17T05:07:50.087 INFO:teuthology.orchestra.run.smithi088.stdout: "active": true, 2024-04-17T05:07:50.087 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:50.073949Z" 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: "swift_keys": [], 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": [], 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: "op_mask": "read, write, delete", 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: "default_placement": "", 2024-04-17T05:07:50.088 INFO:teuthology.orchestra.run.smithi088.stdout: "default_storage_class": "", 2024-04-17T05:07:50.089 INFO:teuthology.orchestra.run.smithi088.stdout: "placement_tags": [], 2024-04-17T05:07:50.089 INFO:teuthology.orchestra.run.smithi088.stdout: "bucket_quota": { 2024-04-17T05:07:50.089 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:50.089 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:50.089 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: "user_quota": { 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": false, 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: "check_on_raw": false, 2024-04-17T05:07:50.090 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size": -1, 2024-04-17T05:07:50.091 INFO:teuthology.orchestra.run.smithi088.stdout: "max_size_kb": 0, 2024-04-17T05:07:50.091 INFO:teuthology.orchestra.run.smithi088.stdout: "max_objects": -1 2024-04-17T05:07:50.091 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:07:50.091 INFO:teuthology.orchestra.run.smithi088.stdout: "temp_url_keys": [], 2024-04-17T05:07:50.091 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "root", 2024-04-17T05:07:50.091 INFO:teuthology.orchestra.run.smithi088.stdout: "mfa_ids": [], 2024-04-17T05:07:50.092 INFO:teuthology.orchestra.run.smithi088.stdout: "account_id": "RGW99999999999999999", 2024-04-17T05:07:50.092 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/", 2024-04-17T05:07:50.092 INFO:teuthology.orchestra.run.smithi088.stdout: "create_date": "2024-04-17T05:07:50.073936Z", 2024-04-17T05:07:50.092 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-04-17T05:07:50.092 INFO:teuthology.orchestra.run.smithi088.stdout: "group_ids": [] 2024-04-17T05:07:50.092 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:07:50.093 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:07:50.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.087+0000 7f4a717e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:50.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.091+0000 7f4bbcf3c980 20 remove_watcher() i=0 2024-04-17T05:07:50.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.091+0000 7f4bbcf3c980 2 removed watcher, disabling cache 2024-04-17T05:07:50.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.095+0000 7f4bbcf3c980 20 remove_watcher() i=1 2024-04-17T05:07:50.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.095+0000 7f4bbcf3c980 20 remove_watcher() i=2 2024-04-17T05:07:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.099+0000 7f4bbcf3c980 20 remove_watcher() i=3 2024-04-17T05:07:50.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.099+0000 7f4bbcf3c980 20 remove_watcher() i=4 2024-04-17T05:07:50.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.099+0000 7f4bbcf3c980 20 remove_watcher() i=5 2024-04-17T05:07:50.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.099+0000 7f4bbcf3c980 20 remove_watcher() i=6 2024-04-17T05:07:50.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.103+0000 7f4bbcf3c980 20 remove_watcher() i=7 2024-04-17T05:07:50.113 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root2.client.0 --totp-serial 8945774460 --totp-seed EPNYCVJN2NZ76VTVRYZLOAXUOONSRR327JB5VVK2KQWMEMAVRUDMNZ6JDU7FILGP --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-17T05:07:50.151 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:07:50.151 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:07:50.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 realm 2024-04-17T05:07:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:07:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.191+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.195+0000 7fae07d08980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.195+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.195+0000 7fae07d08980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:07:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.195+0000 7fae07d08980 20 searching for the correct realm 2024-04-17T05:07:50.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.203+0000 7fae06024640 0 -- 172.21.15.88:0/4243554724 >> [v2:172.21.15.154:6811/1439702329,v1:172.21.15.154:6815/1439702329] conn(0x7fadfc0f2920 msgr2=0x7fadfc0d8410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:50.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.419+0000 7fae05823640 0 -- 172.21.15.88:0/4243554724 >> [v2:172.21.15.154:6809/3628934341,v1:172.21.15.154:6813/3628934341] conn(0x5566d8001990 msgr2=0x5566d8003de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:07:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:07:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.427+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 zone default found 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 4 Realm: () 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 zonegroup default 2024-04-17T05:07:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:07:50.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 started sync module instance, tier type = 2024-04-17T05:07:50.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.431+0000 7fae07d08980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:07:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=0 2024-04-17T05:07:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=1 2024-04-17T05:07:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=2 2024-04-17T05:07:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=3 2024-04-17T05:07:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=4 2024-04-17T05:07:50.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=5 2024-04-17T05:07:50.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.455+0000 7fae07d08980 20 add_watcher() i=6 2024-04-17T05:07:50.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.459+0000 7fae07d08980 20 add_watcher() i=7 2024-04-17T05:07:50.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.459+0000 7fae07d08980 2 all 8 watchers are set, enabling cache 2024-04-17T05:07:50.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7facb77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:50.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:07:50.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:07:50.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 5 note: GC not initialized 2024-04-17T05:07:50.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7facb67fc640 20 reqs_thread_entry: start 2024-04-17T05:07:50.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 20 init_complete bucket index max shards: 1 2024-04-17T05:07:50.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 20 Filter name: none 2024-04-17T05:07:50.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fac27fff640 20 reqs_thread_entry: start 2024-04-17T05:07:50.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-04-17T05:07:50.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.463+0000 7fae07d08980 20 rados->read ofs=0 len=0 2024-04-17T05:07:50.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.467+0000 7fae07d08980 20 rados_obj.operate() r=0 bl.length=494 2024-04-17T05:07:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.467+0000 7fae07d08980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2024-04-17T05:07:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.467+0000 7fae07d08980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-17T05:07:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.467+0000 7fae07d08980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:07:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.471+0000 7fae07d08980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-17T05:07:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.471+0000 7fae07d08980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-17T05:07:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.471+0000 7fae07d08980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-04-17T05:07:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.471+0000 7face97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366312 cookie 93900198945904 notifier 7472 bl.length()=695 2024-04-17T05:07:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.471+0000 7face97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-17T05:07:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.471+0000 7face97fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-17T05:07:50.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.475+0000 7facb77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:07:50.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.483+0000 7fae07d08980 20 remove_watcher() i=0 2024-04-17T05:07:50.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.483+0000 7fae07d08980 2 removed watcher, disabling cache 2024-04-17T05:07:50.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.487+0000 7fae07d08980 20 remove_watcher() i=1 2024-04-17T05:07:50.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.487+0000 7fae07d08980 20 remove_watcher() i=2 2024-04-17T05:07:50.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.491+0000 7fae07d08980 20 remove_watcher() i=3 2024-04-17T05:07:50.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.495+0000 7fae07d08980 20 remove_watcher() i=4 2024-04-17T05:07:50.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.495+0000 7fae07d08980 20 remove_watcher() i=5 2024-04-17T05:07:50.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.499+0000 7fae07d08980 20 remove_watcher() i=6 2024-04-17T05:07:50.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:07:50.499+0000 7fae07d08980 20 remove_watcher() i=7 2024-04-17T05:07:50.515 INFO:tasks.s3tests:Configuring s3-tests... 2024-04-17T05:07:50.516 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T05:07:50.516 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-04-17T05:07:50.561 INFO:tasks.s3tests:Configuring boto... 2024-04-17T05:07:50.561 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-17T05:07:50.562 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-04-17T05:07:50.609 DEBUG:teuthology.orchestra.run.smithi088:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-04-17T05:07:50.770 INFO:teuthology.orchestra.run.smithi088.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-04-17T05:07:50.964 INFO:teuthology.orchestra.run.smithi088.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-04-17T05:07:51.237 INFO:teuthology.orchestra.run.smithi088.stdout:py installdeps: -rrequirements.txt 2024-04-17T05:09:34.058 INFO:teuthology.orchestra.run.smithi088.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-04-17T05:09:36.062 INFO:teuthology.orchestra.run.smithi088.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.2.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.0,filelock==3.13.4,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.7,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.2.1,munch==4.0.0,packaging==24.0,platformdirs==4.2.0,pluggy==1.4.0,pyparsing==3.1.2,pyproject-api==1.6.1,pytest==8.1.1,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.31.0,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=0e7f6050065d0d046e61607bf80100f643d7c2cd91a7a95a26ee483418d95cf7,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.14.2,urllib3==1.26.18,virtualenv==20.25.2,zope.event==5.0,zope.interface==6.3 2024-04-17T05:09:36.063 INFO:teuthology.orchestra.run.smithi088.stdout:py run-test-pre: PYTHONHASHSEED='2098500876' 2024-04-17T05:09:36.063 INFO:teuthology.orchestra.run.smithi088.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-04-17T05:09:36.252 INFO:teuthology.orchestra.run.smithi088.stdout:============================= test session starts ============================== 2024-04-17T05:09:36.252 INFO:teuthology.orchestra.run.smithi088.stdout:platform linux -- Python 3.10.6, pytest-8.1.1, pluggy-1.4.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-04-17T05:09:36.252 INFO:teuthology.orchestra.run.smithi088.stdout:cachedir: .tox/py/.pytest_cache 2024-04-17T05:09:36.253 INFO:teuthology.orchestra.run.smithi088.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-04-17T05:09:36.253 INFO:teuthology.orchestra.run.smithi088.stdout:configfile: pytest.ini 2024-04-17T05:09:37.939 INFO:teuthology.orchestra.run.smithi088.stdout:collecting ... collected 859 items / 127 deselected / 732 selected 2024-04-17T05:09:37.940 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:09:38.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-04-17T05:09:39.117 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-04-17T05:09:39.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-04-17T05:09:40.398 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-04-17T05:09:40.900 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-04-17T05:09:41.377 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-04-17T05:09:42.138 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 0%] 2024-04-17T05:09:42.592 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-04-17T05:09:42.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-04-17T05:09:42.932 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-04-17T05:09:43.429 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-04-17T05:09:43.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-04-17T05:09:44.429 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-04-17T05:09:44.926 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 1%] 2024-04-17T05:09:45.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-04-17T05:09:45.324 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-04-17T05:09:45.450 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-04-17T05:09:45.570 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-04-17T05:09:45.687 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-04-17T05:09:45.810 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 2%] 2024-04-17T05:09:45.933 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 2%] 2024-04-17T05:09:46.054 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.297 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.418 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.540 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 3%] 2024-04-17T05:09:46.908 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 3%] 2024-04-17T05:09:47.029 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.148 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.267 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.386 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.508 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.628 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 4%] 2024-04-17T05:09:47.868 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-04-17T05:09:47.993 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-04-17T05:09:48.120 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-04-17T05:09:48.240 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 5%] 2024-04-17T05:09:48.360 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 5%] 2024-04-17T05:09:48.479 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 5%] 2024-04-17T05:09:48.603 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 5%] 2024-04-17T05:09:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-04-17T05:09:48.845 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-04-17T05:09:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-04-17T05:09:49.107 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 6%] 2024-04-17T05:09:49.227 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 6%] 2024-04-17T05:09:49.347 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 6%] 2024-04-17T05:09:49.469 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 6%] 2024-04-17T05:09:49.590 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 6%] 2024-04-17T05:09:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-04-17T05:09:49.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-04-17T05:09:50.930 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 7%] 2024-04-17T05:09:56.823 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 7%] 2024-04-17T05:10:03.486 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 7%] 2024-04-17T05:10:12.964 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 7%] 2024-04-17T05:10:26.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 7%] 2024-04-17T05:10:30.187 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-04-17T05:10:34.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 8%] 2024-04-17T05:10:36.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 8%] 2024-04-17T05:10:36.610 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 8%] 2024-04-17T05:10:37.755 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 8%] 2024-04-17T05:10:38.898 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 8%] 2024-04-17T05:10:40.710 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 8%] 2024-04-17T05:10:41.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-04-17T05:10:42.517 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 9%] 2024-04-17T05:10:42.521 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 9%] 2024-04-17T05:10:42.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 9%] 2024-04-17T05:10:42.526 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 9%] 2024-04-17T05:10:42.528 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 9%] 2024-04-17T05:10:42.531 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 9%] 2024-04-17T05:10:42.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 9%] 2024-04-17T05:10:42.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-04-17T05:10:42.538 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-04-17T05:10:42.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-04-17T05:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-04-17T05:10:42.546 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-04-17T05:10:42.550 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 10%] 2024-04-17T05:10:42.552 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-04-17T05:10:42.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-04-17T05:10:42.558 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-04-17T05:10:42.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-04-17T05:10:42.563 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-04-17T05:10:42.566 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-04-17T05:10:42.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-04-17T05:10:42.572 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 11%] 2024-04-17T05:10:42.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 12%] 2024-04-17T05:10:42.577 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 12%] 2024-04-17T05:10:42.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 12%] 2024-04-17T05:10:42.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 12%] 2024-04-17T05:10:42.583 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 12%] 2024-04-17T05:10:42.585 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 12%] 2024-04-17T05:10:42.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 12%] 2024-04-17T05:10:42.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 12%] 2024-04-17T05:10:42.702 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 13%] 2024-04-17T05:10:44.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 13%] 2024-04-17T05:10:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 13%] 2024-04-17T05:10:54.295 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 13%] 2024-04-17T05:10:55.543 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 13%] 2024-04-17T05:10:56.831 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 13%] 2024-04-17T05:10:58.079 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 13%] 2024-04-17T05:10:59.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 14%] 2024-04-17T05:11:00.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 14%] 2024-04-17T05:11:01.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 14%] 2024-04-17T05:11:02.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 14%] 2024-04-17T05:11:03.827 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 14%] 2024-04-17T05:11:04.839 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 14%] 2024-04-17T05:11:05.803 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 14%] 2024-04-17T05:11:06.719 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 15%] 2024-04-17T05:11:07.271 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 15%] 2024-04-17T05:11:08.315 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 15%] 2024-04-17T05:11:09.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 15%] 2024-04-17T05:11:10.707 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 15%] 2024-04-17T05:11:11.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 15%] 2024-04-17T05:11:12.675 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 15%] 2024-04-17T05:11:13.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 15%] 2024-04-17T05:11:14.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 16%] 2024-04-17T05:11:15.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 16%] 2024-04-17T05:11:16.520 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 16%] 2024-04-17T05:11:17.411 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 16%] 2024-04-17T05:11:17.995 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 16%] 2024-04-17T05:11:18.571 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 16%] 2024-04-17T05:11:19.163 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 16%] 2024-04-17T05:11:19.803 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 17%] 2024-04-17T05:11:20.380 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-04-17T05:11:21.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 17%] 2024-04-17T05:11:22.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 17%] 2024-04-17T05:11:22.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 17%] 2024-04-17T05:11:23.795 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 17%] 2024-04-17T05:11:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 17%] 2024-04-17T05:11:25.307 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 18%] 2024-04-17T05:11:26.283 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 18%] 2024-04-17T05:11:27.247 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 18%] 2024-04-17T05:11:28.475 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 18%] 2024-04-17T05:11:29.291 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 18%] 2024-04-17T05:11:30.207 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 18%] 2024-04-17T05:11:31.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 18%] 2024-04-17T05:11:32.707 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 18%] 2024-04-17T05:11:34.135 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:35.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:36.887 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:38.603 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:41.235 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:43.463 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:45.247 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 19%] 2024-04-17T05:11:46.999 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 20%] 2024-04-17T05:11:48.239 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 20%] 2024-04-17T05:11:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 20%] 2024-04-17T05:11:50.835 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 20%] 2024-04-17T05:11:53.467 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_create PASSED [ 20%] 2024-04-17T05:11:55.087 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_case_insensitive_name PASSED [ 20%] 2024-04-17T05:11:57.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_delete PASSED [ 20%] 2024-04-17T05:12:01.047 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list PASSED [ 21%] 2024-04-17T05:12:06.207 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list_path_prefix PASSED [ 21%] 2024-04-17T05:12:09.079 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_name PASSED [ 21%] 2024-04-17T05:12:11.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_path PASSED [ 21%] 2024-04-17T05:12:13.844 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_create PASSED [ 21%] 2024-04-17T05:12:15.271 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_create PASSED [ 21%] 2024-04-17T05:12:17.715 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_update PASSED [ 21%] 2024-04-17T05:12:19.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_update PASSED [ 21%] 2024-04-17T05:12:21.747 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_delete PASSED [ 22%] 2024-04-17T05:12:23.304 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_delete PASSED [ 22%] 2024-04-17T05:12:26.295 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_list PASSED [ 22%] 2024-04-17T05:12:28.735 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_bucket_policy_allow PASSED [ 22%] 2024-04-17T05:12:32.019 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy PASSED [ 22%] 2024-04-17T05:12:35.292 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_managed PASSED [ 22%] 2024-04-17T05:12:37.592 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_allow PASSED [ 22%] 2024-04-17T05:12:39.683 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_create PASSED [ 23%] 2024-04-17T05:12:41.335 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_case_insensitive_name PASSED [ 23%] 2024-04-17T05:12:45.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_list PASSED [ 23%] 2024-04-17T05:12:48.223 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_update PASSED [ 23%] 2024-04-17T05:12:51.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy PASSED [ 23%] 2024-04-17T05:12:54.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy PASSED [ 23%] 2024-04-17T05:12:57.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy_allow PASSED [ 23%] 2024-04-17T05:13:00.455 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy_allow PASSED [ 24%] 2024-04-17T05:13:03.067 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_create PASSED [ 24%] 2024-04-17T05:13:04.807 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_case_insensitive_name PASSED [ 24%] 2024-04-17T05:13:06.895 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_delete PASSED [ 24%] 2024-04-17T05:13:10.531 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list PASSED [ 24%] 2024-04-17T05:13:15.183 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list_path_prefix PASSED [ 24%] 2024-04-17T05:13:17.195 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_update PASSED [ 24%] 2024-04-17T05:13:20.176 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy PASSED [ 25%] 2024-04-17T05:13:23.147 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_managed PASSED [ 25%] 2024-04-17T05:13:26.067 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow PASSED [ 25%] 2024-04-17T05:13:29.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_same_account_role_policy_allow PASSED [ 25%] 2024-04-17T05:13:32.480 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_role_policy_allow PASSED [ 25%] 2024-04-17T05:13:35.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_create_bucket PASSED [ 25%] 2024-04-17T05:13:38.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_get_role PASSED [ 25%] 2024-04-17T05:13:40.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_account_oidc_provider PASSED [ 25%] 2024-04-17T05:13:43.623 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_user_arn PASSED [ 26%] 2024-04-17T05:13:46.539 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_user_arn PASSED [ 26%] 2024-04-17T05:13:49.467 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_arn PASSED [ 26%] 2024-04-17T05:13:52.387 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_arn PASSED [ 26%] 2024-04-17T05:13:55.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_id PASSED [ 26%] 2024-04-17T05:13:58.127 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_id PASSED [ 26%] 2024-04-17T05:14:00.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_user_id PASSED [ 26%] 2024-04-17T05:14:03.919 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_user_id PASSED [ 27%] 2024-04-17T05:14:06.831 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_canonical_id PASSED [ 27%] 2024-04-17T05:14:09.723 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_canonical_id PASSED [ 27%] 2024-04-17T05:14:12.595 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_account_email PASSED [ 27%] 2024-04-17T05:14:15.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_account_email PASSED [ 27%] 2024-04-17T05:14:17.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_arn PASSED [ 27%] 2024-04-17T05:14:19.191 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_id PASSED [ 27%] 2024-04-17T05:14:20.951 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_canonical_id PASSED [ 28%] 2024-04-17T05:14:22.691 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_account_email PASSED [ 28%] 2024-04-17T05:14:23.954 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 28%] 2024-04-17T05:14:25.538 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 28%] 2024-04-17T05:14:27.123 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 28%] 2024-04-17T05:14:28.759 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 28%] 2024-04-17T05:14:30.455 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 28%] 2024-04-17T05:14:32.078 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 28%] 2024-04-17T05:14:33.707 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 29%] 2024-04-17T05:14:35.358 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 29%] 2024-04-17T05:14:36.979 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 29%] 2024-04-17T05:14:39.290 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 29%] 2024-04-17T05:14:41.610 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 29%] 2024-04-17T05:14:42.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-04-17T05:14:44.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-04-17T05:14:45.650 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 30%] 2024-04-17T05:14:47.238 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 30%] 2024-04-17T05:14:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 30%] 2024-04-17T05:14:51.918 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 30%] 2024-04-17T05:14:53.558 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 30%] 2024-04-17T05:14:55.182 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 30%] 2024-04-17T05:14:56.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 30%] 2024-04-17T05:14:58.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 31%] 2024-04-17T05:15:00.023 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 31%] 2024-04-17T05:15:01.590 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 31%] 2024-04-17T05:15:03.279 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 31%] 2024-04-17T05:15:04.919 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 31%] 2024-04-17T05:15:06.534 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 31%] 2024-04-17T05:15:08.166 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 31%] 2024-04-17T05:15:09.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 31%] 2024-04-17T05:15:11.379 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 32%] 2024-04-17T05:15:12.935 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 32%] 2024-04-17T05:15:14.411 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 32%] 2024-04-17T05:15:15.863 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 32%] 2024-04-17T05:15:17.479 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 32%] 2024-04-17T05:15:19.103 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 32%] 2024-04-17T05:17:33.060 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 32%] 2024-04-17T05:17:34.583 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 33%] 2024-04-17T05:17:36.071 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 33%] 2024-04-17T05:17:37.571 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 33%] 2024-04-17T05:17:39.071 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 33%] 2024-04-17T05:17:40.563 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 33%] 2024-04-17T05:17:42.047 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 33%] 2024-04-17T05:17:43.571 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 33%] 2024-04-17T05:17:45.079 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 34%] 2024-04-17T05:17:46.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 34%] 2024-04-17T05:17:48.023 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 34%] 2024-04-17T05:17:49.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 34%] 2024-04-17T05:17:50.987 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 34%] 2024-04-17T05:17:52.655 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 34%] 2024-04-17T05:17:54.283 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 34%] 2024-04-17T05:17:55.879 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 34%] 2024-04-17T05:17:57.507 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 35%] 2024-04-17T05:17:59.111 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-04-17T05:18:00.723 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-04-17T05:18:02.355 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-04-17T05:18:03.995 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-04-17T05:18:05.611 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-04-17T05:18:07.207 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-04-17T05:18:08.919 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 36%] 2024-04-17T05:18:10.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 36%] 2024-04-17T05:18:12.259 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 36%] 2024-04-17T05:18:13.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 36%] 2024-04-17T05:18:15.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 36%] 2024-04-17T05:18:17.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 36%] 2024-04-17T05:18:17.839 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 36%] 2024-04-17T05:18:19.047 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 37%] 2024-04-17T05:18:23.939 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 37%] 2024-04-17T05:18:28.923 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 37%] 2024-04-17T05:18:30.531 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 37%] 2024-04-17T05:18:32.139 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 37%] 2024-04-17T05:18:33.943 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 37%] 2024-04-17T05:18:35.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 37%] 2024-04-17T05:18:37.251 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 37%] 2024-04-17T05:18:39.064 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 38%] 2024-04-17T05:18:40.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 38%] 2024-04-17T05:18:42.359 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 38%] 2024-04-17T05:18:43.955 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 38%] 2024-04-17T05:18:45.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 38%] 2024-04-17T05:18:47.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 38%] 2024-04-17T05:18:48.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 38%] 2024-04-17T05:18:50.979 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 39%] 2024-04-17T05:18:53.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 39%] 2024-04-17T05:18:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 39%] 2024-04-17T05:18:55.803 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 39%] 2024-04-17T05:18:56.779 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 39%] 2024-04-17T05:18:57.751 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 39%] 2024-04-17T05:18:58.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 39%] 2024-04-17T05:18:59.043 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 40%] 2024-04-17T05:18:59.655 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 40%] 2024-04-17T05:19:00.895 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 40%] 2024-04-17T05:19:04.806 INFO:tasks.rgw.client.0.smithi088.stdout:2024-04-17T05:19:04.794+0000 34179640 -1 req 16483070181208950048 0.111997291s s3:delete_bucket ERROR: could not remove bucket test-29fkkr1kn7gqxd76i3nfcoel-156 2024-04-17T05:19:05.295 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 40%] 2024-04-17T05:19:05.656 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 40%] 2024-04-17T05:19:05.656 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket ERROR [ 40%] 2024-04-17T05:19:06.519 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 40%] 2024-04-17T05:19:06.519 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding ERROR [ 40%] 2024-04-17T05:19:07.277 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 40%] 2024-04-17T05:19:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete ERROR [ 40%] 2024-04-17T05:19:07.938 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 40%] 2024-04-17T05:19:07.938 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist ERROR [ 40%] 2024-04-17T05:19:08.636 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 41%] 2024-04-17T05:19:08.636 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error ERROR [ 41%] 2024-04-17T05:19:13.322 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 41%] 2024-04-17T05:19:13.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete ERROR [ 41%] 2024-04-17T05:19:14.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 41%] 2024-04-17T05:19:14.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete ERROR [ 41%] 2024-04-17T05:19:16.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 41%] 2024-04-17T05:19:16.270 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete ERROR [ 41%] 2024-04-17T05:21:29.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 41%] 2024-04-17T05:21:29.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit ERROR [ 41%] 2024-04-17T05:23:42.026 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 41%] 2024-04-17T05:23:42.026 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit ERROR [ 41%] 2024-04-17T05:23:42.894 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 41%] 2024-04-17T05:23:42.894 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes ERROR [ 41%] 2024-04-17T05:23:43.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 42%] 2024-04-17T05:23:43.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag ERROR [ 42%] 2024-04-17T05:23:44.657 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 42%] 2024-04-17T05:23:44.658 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control ERROR [ 42%] 2024-04-17T05:23:45.595 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 42%] 2024-04-17T05:23:45.595 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires ERROR [ 42%] 2024-04-17T05:23:46.773 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 42%] 2024-04-17T05:23:46.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete ERROR [ 42%] 2024-04-17T05:23:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 42%] 2024-04-17T05:23:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good ERROR [ 42%] 2024-04-17T05:23:48.586 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 42%] 2024-04-17T05:23:48.586 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty ERROR [ 42%] 2024-04-17T05:23:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 42%] 2024-04-17T05:23:49.764 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty ERROR [ 42%] 2024-04-17T05:23:50.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 43%] 2024-04-17T05:23:50.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put ERROR [ 43%] 2024-04-17T05:23:51.631 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 43%] 2024-04-17T05:23:51.631 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file ERROR [ 43%] 2024-04-17T05:23:52.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 43%] 2024-04-17T05:23:52.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request ERROR [ 43%] 2024-04-17T05:23:53.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 43%] 2024-04-17T05:23:53.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request ERROR [ 43%] 2024-04-17T05:23:54.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 43%] 2024-04-17T05:23:54.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type ERROR [ 43%] 2024-04-17T05:23:55.154 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 43%] 2024-04-17T05:23:55.154 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key ERROR [ 43%] 2024-04-17T05:23:55.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 43%] 2024-04-17T05:23:55.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code ERROR [ 43%] 2024-04-17T05:23:56.794 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 43%] 2024-04-17T05:23:56.794 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code ERROR [ 43%] 2024-04-17T05:23:58.042 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 44%] 2024-04-17T05:23:58.042 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk ERROR [ 44%] 2024-04-17T05:23:58.987 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 44%] 2024-04-17T05:23:58.987 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename ERROR [ 44%] 2024-04-17T05:23:59.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 44%] 2024-04-17T05:23:59.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header ERROR [ 44%] 2024-04-17T05:24:00.558 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 44%] 2024-04-17T05:24:00.558 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields ERROR [ 44%] 2024-04-17T05:24:01.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 44%] 2024-04-17T05:24:01.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values ERROR [ 44%] 2024-04-17T05:24:02.512 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 44%] 2024-04-17T05:24:02.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action ERROR [ 44%] 2024-04-17T05:24:03.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 44%] 2024-04-17T05:24:03.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature ERROR [ 44%] 2024-04-17T05:24:03.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 45%] 2024-04-17T05:24:03.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key ERROR [ 45%] 2024-04-17T05:24:04.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 45%] 2024-04-17T05:24:04.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format ERROR [ 45%] 2024-04-17T05:24:05.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 45%] 2024-04-17T05:24:05.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified ERROR [ 45%] 2024-04-17T05:24:05.876 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 45%] 2024-04-17T05:24:05.876 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature ERROR [ 45%] 2024-04-17T05:24:06.553 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 45%] 2024-04-17T05:24:06.553 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition ERROR [ 45%] 2024-04-17T05:24:07.492 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 45%] 2024-04-17T05:24:07.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header ERROR [ 45%] 2024-04-17T05:24:08.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 45%] 2024-04-17T05:24:08.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field ERROR [ 45%] 2024-04-17T05:24:08.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 46%] 2024-04-17T05:24:08.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive ERROR [ 46%] 2024-04-17T05:24:09.511 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 46%] 2024-04-17T05:24:09.511 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive ERROR [ 46%] 2024-04-17T05:24:10.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 46%] 2024-04-17T05:24:10.157 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy ERROR [ 46%] 2024-04-17T05:24:11.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 46%] 2024-04-17T05:24:11.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket ERROR [ 46%] 2024-04-17T05:24:11.869 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 46%] 2024-04-17T05:24:11.869 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value ERROR [ 46%] 2024-04-17T05:24:12.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 46%] 2024-04-17T05:24:12.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition ERROR [ 46%] 2024-04-17T05:24:13.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 46%] 2024-04-17T05:24:13.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list ERROR [ 46%] 2024-04-17T05:24:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 46%] 2024-04-17T05:24:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded ERROR [ 46%] 2024-04-17T05:24:14.658 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 47%] 2024-04-17T05:24:14.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument ERROR [ 47%] 2024-04-17T05:24:15.318 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 47%] 2024-04-17T05:24:15.318 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument ERROR [ 47%] 2024-04-17T05:24:16.026 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 47%] 2024-04-17T05:24:16.026 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum ERROR [ 47%] 2024-04-17T05:24:17.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 47%] 2024-04-17T05:24:17.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug ERROR [ 47%] 2024-04-17T05:24:18.476 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 47%] 2024-04-17T05:24:18.476 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions ERROR [ 47%] 2024-04-17T05:24:19.486 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 47%] 2024-04-17T05:24:19.486 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good ERROR [ 47%] 2024-04-17T05:24:20.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 47%] 2024-04-17T05:24:20.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed ERROR [ 47%] 2024-04-17T05:24:21.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 48%] 2024-04-17T05:24:21.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good ERROR [ 48%] 2024-04-17T05:24:22.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 48%] 2024-04-17T05:24:22.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed ERROR [ 48%] 2024-04-17T05:24:23.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 48%] 2024-04-17T05:24:23.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good ERROR [ 48%] 2024-04-17T05:24:25.221 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 48%] 2024-04-17T05:24:25.222 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed ERROR [ 48%] 2024-04-17T05:24:26.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 48%] 2024-04-17T05:24:26.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good ERROR [ 48%] 2024-04-17T05:24:27.123 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 48%] 2024-04-17T05:24:27.123 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed ERROR [ 48%] 2024-04-17T05:24:28.285 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 48%] 2024-04-17T05:24:28.286 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good ERROR [ 48%] 2024-04-17T05:24:29.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 49%] 2024-04-17T05:24:29.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed ERROR [ 49%] 2024-04-17T05:24:30.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 49%] 2024-04-17T05:24:30.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good ERROR [ 49%] 2024-04-17T05:24:31.578 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 49%] 2024-04-17T05:24:31.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed ERROR [ 49%] 2024-04-17T05:24:32.683 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 49%] 2024-04-17T05:24:32.683 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good ERROR [ 49%] 2024-04-17T05:24:33.949 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 49%] 2024-04-17T05:24:33.949 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed ERROR [ 49%] 2024-04-17T05:24:34.858 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 49%] 2024-04-17T05:24:34.858 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good ERROR [ 49%] 2024-04-17T05:24:36.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 49%] 2024-04-17T05:24:36.040 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed ERROR [ 49%] 2024-04-17T05:24:36.872 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 50%] 2024-04-17T05:24:36.872 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get ERROR [ 50%] 2024-04-17T05:24:37.599 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 50%] 2024-04-17T05:24:37.599 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone ERROR [ 50%] 2024-04-17T05:24:38.311 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 50%] 2024-04-17T05:24:38.311 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone ERROR [ 50%] 2024-04-17T05:24:39.148 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 50%] 2024-04-17T05:24:39.148 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone ERROR [ 50%] 2024-04-17T05:24:39.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 50%] 2024-04-17T05:24:39.830 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head ERROR [ 50%] 2024-04-17T05:24:40.145 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 50%] 2024-04-17T05:24:40.146 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist ERROR [ 50%] 2024-04-17T05:24:41.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 50%] 2024-04-17T05:24:41.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended ERROR [ 50%] 2024-04-17T05:24:42.307 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 50%] 2024-04-17T05:24:42.310 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl ERROR [ 50%] 2024-04-17T05:24:43.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 51%] 2024-04-17T05:24:43.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl ERROR [ 51%] 2024-04-17T05:24:47.016 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 51%] 2024-04-17T05:24:47.016 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime ERROR [ 51%] 2024-04-17T05:24:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 51%] 2024-04-17T05:24:47.879 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated ERROR [ 51%] 2024-04-17T05:24:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 51%] 2024-04-17T05:24:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers ERROR [ 51%] 2024-04-17T05:24:49.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 51%] 2024-04-17T05:24:49.740 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl ERROR [ 51%] 2024-04-17T05:24:50.557 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 51%] 2024-04-17T05:24:50.558 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl ERROR [ 51%] 2024-04-17T05:24:51.335 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 51%] 2024-04-17T05:24:51.335 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone ERROR [ 51%] 2024-04-17T05:24:52.202 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 52%] 2024-04-17T05:24:52.204 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone ERROR [ 52%] 2024-04-17T05:24:53.233 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 52%] 2024-04-17T05:24:53.233 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired ERROR [ 52%] 2024-04-17T05:24:53.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 52%] 2024-04-17T05:24:53.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant ERROR [ 52%] 2024-04-17T05:24:54.749 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 52%] 2024-04-17T05:24:54.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero ERROR [ 52%] 2024-04-17T05:24:55.537 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 52%] 2024-04-17T05:24:55.538 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range ERROR [ 52%] 2024-04-17T05:24:56.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 52%] 2024-04-17T05:24:56.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range ERROR [ 52%] 2024-04-17T05:24:57.183 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 52%] 2024-04-17T05:24:57.184 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put ERROR [ 52%] 2024-04-17T05:24:58.042 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 53%] 2024-04-17T05:24:58.042 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access ERROR [ 53%] 2024-04-17T05:24:58.826 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 53%] 2024-04-17T05:24:58.826 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated ERROR [ 53%] 2024-04-17T05:24:59.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl PASSED [ 53%] 2024-04-17T05:24:59.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl ERROR [ 53%] 2024-04-17T05:25:00.327 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant PASSED [ 53%] 2024-04-17T05:25:00.328 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant ERROR [ 53%] 2024-04-17T05:25:01.157 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 53%] 2024-04-17T05:25:01.157 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired ERROR [ 53%] 2024-04-17T05:25:01.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 53%] 2024-04-17T05:25:01.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha ERROR [ 53%] 2024-04-17T05:25:01.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 53%] 2024-04-17T05:25:01.671 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one ERROR [ 53%] 2024-04-17T05:25:01.987 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 53%] 2024-04-17T05:25:01.987 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two ERROR [ 53%] 2024-04-17T05:25:02.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 54%] 2024-04-17T05:25:02.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 ERROR [ 54%] 2024-04-17T05:25:03.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 54%] 2024-04-17T05:25:03.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 ERROR [ 54%] 2024-04-17T05:25:03.801 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 54%] 2024-04-17T05:25:03.801 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 ERROR [ 54%] 2024-04-17T05:25:04.480 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 54%] 2024-04-17T05:25:04.480 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 ERROR [ 54%] 2024-04-17T05:25:05.186 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 54%] 2024-04-17T05:25:05.186 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name ERROR [ 54%] 2024-04-17T05:25:05.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 54%] 2024-04-17T05:25:05.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip ERROR [ 54%] 2024-04-17T05:25:05.788 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 54%] 2024-04-17T05:25:05.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore ERROR [ 54%] 2024-04-17T05:25:06.461 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 55%] 2024-04-17T05:25:06.461 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long ERROR [ 55%] 2024-04-17T05:25:06.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 55%] 2024-04-17T05:25:06.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end ERROR [ 55%] 2024-04-17T05:25:07.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 55%] 2024-04-17T05:25:07.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot ERROR [ 55%] 2024-04-17T05:25:07.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 55%] 2024-04-17T05:25:07.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash ERROR [ 55%] 2024-04-17T05:25:07.577 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 55%] 2024-04-17T05:25:07.577 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot ERROR [ 55%] 2024-04-17T05:25:08.535 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 55%] 2024-04-17T05:25:08.535 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists ERROR [ 55%] 2024-04-17T05:25:09.253 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 55%] 2024-04-17T05:25:09.253 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location ERROR [ 55%] 2024-04-17T05:25:09.947 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 56%] 2024-04-17T05:25:09.948 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner ERROR [ 56%] 2024-04-17T05:25:10.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 56%] 2024-04-17T05:25:10.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl ERROR [ 56%] 2024-04-17T05:25:11.406 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 56%] 2024-04-17T05:25:11.406 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl ERROR [ 56%] 2024-04-17T05:25:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 56%] 2024-04-17T05:25:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default ERROR [ 56%] 2024-04-17T05:25:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 56%] 2024-04-17T05:25:12.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create ERROR [ 56%] 2024-04-17T05:25:13.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 56%] 2024-04-17T05:25:13.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned ERROR [ 56%] 2024-04-17T05:25:14.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 56%] 2024-04-17T05:25:14.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite ERROR [ 56%] 2024-04-17T05:25:15.036 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 56%] 2024-04-17T05:25:15.037 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread ERROR [ 56%] 2024-04-17T05:25:15.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 57%] 2024-04-17T05:25:15.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default ERROR [ 57%] 2024-04-17T05:25:16.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 57%] 2024-04-17T05:25:16.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create ERROR [ 57%] 2024-04-17T05:25:18.074 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 57%] 2024-04-17T05:25:18.074 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned ERROR [ 57%] 2024-04-17T05:25:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 57%] 2024-04-17T05:25:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite ERROR [ 57%] 2024-04-17T05:25:19.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 57%] 2024-04-17T05:25:19.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread ERROR [ 57%] 2024-04-17T05:25:21.032 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 57%] 2024-04-17T05:25:21.032 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread ERROR [ 57%] 2024-04-17T05:25:22.173 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 57%] 2024-04-17T05:25:22.173 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol ERROR [ 57%] 2024-04-17T05:25:23.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 58%] 2024-04-17T05:25:23.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner ERROR [ 58%] 2024-04-17T05:25:24.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 58%] 2024-04-17T05:25:24.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes ERROR [ 58%] 2024-04-17T05:25:25.241 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 58%] 2024-04-17T05:25:25.241 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private ERROR [ 58%] 2024-04-17T05:25:26.418 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 58%] 2024-04-17T05:25:26.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl ERROR [ 58%] 2024-04-17T05:25:27.542 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 58%] 2024-04-17T05:25:27.542 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write ERROR [ 58%] 2024-04-17T05:25:28.691 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 58%] 2024-04-17T05:25:28.691 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp ERROR [ 58%] 2024-04-17T05:25:29.895 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 58%] 2024-04-17T05:25:29.895 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read ERROR [ 58%] 2024-04-17T05:25:31.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 59%] 2024-04-17T05:25:31.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp ERROR [ 59%] 2024-04-17T05:25:32.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 59%] 2024-04-17T05:25:32.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol ERROR [ 59%] 2024-04-17T05:25:33.572 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 59%] 2024-04-17T05:25:33.572 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read ERROR [ 59%] 2024-04-17T05:25:34.773 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 59%] 2024-04-17T05:25:34.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp ERROR [ 59%] 2024-04-17T05:25:35.993 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 59%] 2024-04-17T05:25:35.993 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write ERROR [ 59%] 2024-04-17T05:25:37.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 59%] 2024-04-17T05:25:37.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp ERROR [ 59%] 2024-04-17T05:25:37.932 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 59%] 2024-04-17T05:25:37.932 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user ERROR [ 59%] 2024-04-17T05:25:38.800 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 59%] 2024-04-17T05:25:38.800 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants ERROR [ 59%] 2024-04-17T05:25:39.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 60%] 2024-04-17T05:25:39.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants ERROR [ 60%] 2024-04-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 60%] 2024-04-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email ERROR [ 60%] 2024-04-17T05:25:41.488 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 60%] 2024-04-17T05:25:41.488 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist ERROR [ 60%] 2024-04-17T05:25:42.639 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 60%] 2024-04-17T05:25:42.639 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all ERROR [ 60%] 2024-04-17T05:25:44.279 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 60%] 2024-04-17T05:25:44.279 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private ERROR [ 60%] 2024-04-17T05:25:46.035 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 60%] 2024-04-17T05:25:46.035 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private ERROR [ 60%] 2024-04-17T05:25:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 60%] 2024-04-17T05:25:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread ERROR [ 60%] 2024-04-17T05:25:49.302 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 61%] 2024-04-17T05:25:49.305 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread ERROR [ 61%] 2024-04-17T05:25:50.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 61%] 2024-04-17T05:25:50.953 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite ERROR [ 61%] 2024-04-17T05:25:52.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 61%] 2024-04-17T05:25:52.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite ERROR [ 61%] 2024-04-17T05:25:54.333 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 61%] 2024-04-17T05:25:54.333 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private ERROR [ 61%] 2024-04-17T05:25:56.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 61%] 2024-04-17T05:25:56.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread ERROR [ 61%] 2024-04-17T05:25:57.758 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 61%] 2024-04-17T05:25:57.758 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite ERROR [ 61%] 2024-04-17T05:25:59.550 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 61%] 2024-04-17T05:25:59.550 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private ERROR [ 61%] 2024-04-17T05:26:01.415 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 62%] 2024-04-17T05:26:01.415 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread ERROR [ 62%] 2024-04-17T05:26:03.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 62%] 2024-04-17T05:26:03.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite ERROR [ 62%] 2024-04-17T05:26:05.628 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 62%] 2024-04-17T05:26:05.628 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list ERROR [ 62%] 2024-04-17T05:26:07.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 62%] 2024-04-17T05:26:07.766 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime ERROR [ 62%] 2024-04-17T05:26:08.030 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 62%] 2024-04-17T05:26:08.030 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous ERROR [ 62%] 2024-04-17T05:26:08.370 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 62%] 2024-04-17T05:26:08.370 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth ERROR [ 62%] 2024-04-17T05:26:08.605 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 62%] 2024-04-17T05:26:08.605 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth ERROR [ 62%] 2024-04-17T05:26:09.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 62%] 2024-04-17T05:26:09.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha ERROR [ 62%] 2024-04-17T05:26:10.178 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 63%] 2024-04-17T05:26:10.179 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit ERROR [ 63%] 2024-04-17T05:26:10.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 63%] 2024-04-17T05:26:10.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period ERROR [ 63%] 2024-04-17T05:26:11.423 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 63%] 2024-04-17T05:26:11.423 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen ERROR [ 63%] 2024-04-17T05:26:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 63%] 2024-04-17T05:26:12.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding ERROR [ 63%] 2024-04-17T05:26:17.483 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 63%] 2024-04-17T05:26:17.483 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names ERROR [ 63%] 2024-04-17T05:26:19.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 63%] 2024-04-17T05:26:19.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix ERROR [ 63%] 2024-04-17T05:26:20.365 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 63%] 2024-04-17T05:26:20.366 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size ERROR [ 63%] 2024-04-17T05:26:24.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 64%] 2024-04-17T05:26:24.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m ERROR [ 64%] 2024-04-17T05:26:26.038 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 64%] 2024-04-17T05:26:26.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket ERROR [ 64%] 2024-04-17T05:26:27.215 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 64%] 2024-04-17T05:26:27.216 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype ERROR [ 64%] 2024-04-17T05:26:28.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 64%] 2024-04-17T05:26:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself ERROR [ 64%] 2024-04-17T05:26:29.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 64%] 2024-04-17T05:26:29.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata ERROR [ 64%] 2024-04-17T05:26:30.943 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 64%] 2024-04-17T05:26:30.943 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket ERROR [ 64%] 2024-04-17T05:26:31.914 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 64%] 2024-04-17T05:26:31.914 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket ERROR [ 64%] 2024-04-17T05:26:33.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 65%] 2024-04-17T05:26:33.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket ERROR [ 65%] 2024-04-17T05:26:34.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 65%] 2024-04-17T05:26:34.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl ERROR [ 65%] 2024-04-17T05:26:36.998 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 65%] 2024-04-17T05:26:36.999 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata ERROR [ 65%] 2024-04-17T05:26:39.109 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 65%] 2024-04-17T05:26:39.109 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata ERROR [ 65%] 2024-04-17T05:26:39.807 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 65%] 2024-04-17T05:26:39.807 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found ERROR [ 65%] 2024-04-17T05:26:40.578 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 65%] 2024-04-17T05:26:40.578 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found ERROR [ 65%] 2024-04-17T05:26:44.548 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 65%] 2024-04-17T05:26:44.548 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket ERROR [ 65%] 2024-04-17T05:26:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 65%] 2024-04-17T05:26:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding ERROR [ 65%] 2024-04-17T05:26:57.238 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 66%] 2024-04-17T05:26:57.239 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload ERROR [ 66%] 2024-04-17T05:26:58.067 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 66%] 2024-04-17T05:26:58.067 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty ERROR [ 66%] 2024-04-17T05:26:59.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 66%] 2024-04-17T05:26:59.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small ERROR [ 66%] 2024-04-17T05:27:02.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 66%] 2024-04-17T05:27:02.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small ERROR [ 66%] 2024-04-17T05:27:03.993 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 66%] 2024-04-17T05:27:03.993 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range ERROR [ 66%] 2024-04-17T05:27:06.499 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 66%] 2024-04-17T05:27:06.499 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range ERROR [ 66%] 2024-04-17T05:27:16.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 66%] 2024-04-17T05:27:16.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names ERROR [ 66%] 2024-04-17T05:27:29.393 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 67%] 2024-04-17T05:27:29.393 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload ERROR [ 67%] 2024-04-17T05:27:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 67%] 2024-04-17T05:27:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned ERROR [ 67%] 2024-04-17T05:29:10.992 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 67%] 2024-04-17T05:29:10.993 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part ERROR [ 67%] 2024-04-17T05:29:22.869 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 67%] 2024-04-17T05:29:22.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes ERROR [ 67%] 2024-04-17T05:29:37.571 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 67%] 2024-04-17T05:29:37.571 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes ERROR [ 67%] 2024-04-17T05:29:39.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 67%] 2024-04-17T05:29:39.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small ERROR [ 67%] 2024-04-17T05:29:45.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 67%] 2024-04-17T05:29:45.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents ERROR [ 67%] 2024-04-17T05:29:49.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 68%] 2024-04-17T05:29:49.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object ERROR [ 68%] 2024-04-17T05:29:52.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 68%] 2024-04-17T05:29:52.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload ERROR [ 68%] 2024-04-17T05:29:53.345 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 68%] 2024-04-17T05:29:53.345 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found ERROR [ 68%] 2024-04-17T05:29:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 68%] 2024-04-17T05:29:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload ERROR [ 68%] 2024-04-17T05:29:59.095 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 68%] 2024-04-17T05:29:59.095 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner ERROR [ 68%] 2024-04-17T05:30:00.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 68%] 2024-04-17T05:30:00.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part ERROR [ 68%] 2024-04-17T05:30:01.101 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 68%] 2024-04-17T05:30:01.101 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag ERROR [ 68%] 2024-04-17T05:30:21.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 68%] 2024-04-17T05:30:21.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part ERROR [ 68%] 2024-04-17T05:30:22.422 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 69%] 2024-04-17T05:30:22.422 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part ERROR [ 69%] 2024-04-17T05:30:23.139 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 69%] 2024-04-17T05:30:23.139 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue ERROR [ 69%] 2024-04-17T05:30:24.237 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 69%] 2024-04-17T05:30:24.237 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors ERROR [ 69%] 2024-04-17T05:30:29.260 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response PASSED [ 69%] 2024-04-17T05:30:29.260 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response ERROR [ 69%] 2024-04-17T05:30:33.115 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 69%] 2024-04-17T05:30:33.116 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard ERROR [ 69%] 2024-04-17T05:30:36.935 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option PASSED [ 69%] 2024-04-17T05:30:36.935 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option ERROR [ 69%] 2024-04-17T05:30:37.917 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 69%] 2024-04-17T05:30:37.917 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object ERROR [ 69%] 2024-04-17T05:30:38.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 70%] 2024-04-17T05:30:38.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant ERROR [ 70%] 2024-04-17T05:30:39.628 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object PASSED [ 70%] 2024-04-17T05:30:39.628 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object ERROR [ 70%] 2024-04-17T05:30:40.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl PASSED [ 70%] 2024-04-17T05:30:40.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl ERROR [ 70%] 2024-04-17T05:30:41.235 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant PASSED [ 70%] 2024-04-17T05:30:41.235 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant ERROR [ 70%] 2024-04-17T05:30:41.856 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl PASSED [ 70%] 2024-04-17T05:30:41.856 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl ERROR [ 70%] 2024-04-17T05:30:42.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging PASSED [ 70%] 2024-04-17T05:30:42.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging ERROR [ 70%] 2024-04-17T05:30:44.614 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb PASSED [ 70%] 2024-04-17T05:30:44.614 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb ERROR [ 70%] 2024-04-17T05:30:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb PASSED [ 71%] 2024-04-17T05:30:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb ERROR [ 71%] 2024-04-17T05:30:50.981 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb PASSED [ 71%] 2024-04-17T05:30:50.982 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb ERROR [ 71%] 2024-04-17T05:30:53.196 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb PASSED [ 71%] 2024-04-17T05:30:53.196 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb ERROR [ 71%] 2024-04-17T05:30:56.383 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb PASSED [ 71%] 2024-04-17T05:30:56.383 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb ERROR [ 71%] 2024-04-17T05:31:01.111 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb PASSED [ 71%] 2024-04-17T05:31:01.111 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb ERROR [ 71%] 2024-04-17T05:31:03.265 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 71%] 2024-04-17T05:31:03.265 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb ERROR [ 71%] 2024-04-17T05:31:07.578 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 71%] 2024-04-17T05:31:07.578 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb ERROR [ 71%] 2024-04-17T05:31:15.543 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 71%] 2024-04-17T05:31:15.544 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb ERROR [ 71%] 2024-04-17T05:31:17.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 72%] 2024-04-17T05:31:17.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb ERROR [ 72%] 2024-04-17T05:31:18.813 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 72%] 2024-04-17T05:31:18.813 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write ERROR [ 72%] 2024-04-17T05:31:20.726 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 72%] 2024-04-17T05:31:20.726 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last ERROR [ 72%] 2024-04-17T05:31:21.732 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code PASSED [ 72%] 2024-04-17T05:31:21.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code ERROR [ 72%] 2024-04-17T05:31:28.359 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 72%] 2024-04-17T05:31:28.359 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code ERROR [ 72%] 2024-04-17T05:31:29.272 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 72%] 2024-04-17T05:31:29.272 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code ERROR [ 72%] 2024-04-17T05:31:30.177 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 72%] 2024-04-17T05:31:30.177 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code ERROR [ 72%] 2024-04-17T05:31:31.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 73%] 2024-04-17T05:31:31.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range ERROR [ 73%] 2024-04-17T05:31:32.104 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 73%] 2024-04-17T05:31:32.105 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object ERROR [ 73%] 2024-04-17T05:31:33.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 73%] 2024-04-17T05:31:33.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend ERROR [ 73%] 2024-04-17T05:31:56.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 73%] 2024-04-17T05:31:56.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove ERROR [ 73%] 2024-04-17T05:31:59.007 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 73%] 2024-04-17T05:31:59.008 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head ERROR [ 73%] 2024-04-17T05:32:00.333 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 73%] 2024-04-17T05:32:00.333 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal ERROR [ 73%] 2024-04-17T05:32:02.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 73%] 2024-04-17T05:32:02.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite ERROR [ 73%] 2024-04-17T05:32:04.056 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 74%] 2024-04-17T05:32:04.056 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended ERROR [ 74%] 2024-04-17T05:32:13.464 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 74%] 2024-04-17T05:32:13.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions ERROR [ 74%] 2024-04-17T05:32:15.517 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy PASSED [ 74%] 2024-04-17T05:32:15.518 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy ERROR [ 74%] 2024-04-17T05:32:26.464 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 74%] 2024-04-17T05:32:26.464 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all ERROR [ 74%] 2024-04-17T05:33:08.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 74%] 2024-04-17T05:33:08.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names ERROR [ 74%] 2024-04-17T05:33:25.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 74%] 2024-04-17T05:33:25.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart ERROR [ 74%] 2024-04-17T05:33:29.892 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 74%] 2024-04-17T05:33:29.892 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker ERROR [ 74%] 2024-04-17T05:33:34.390 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 75%] 2024-04-17T05:33:34.391 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version ERROR [ 75%] 2024-04-17T05:33:36.049 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 75%] 2024-04-17T05:33:36.049 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete ERROR [ 75%] 2024-04-17T05:33:37.899 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 75%] 2024-04-17T05:33:37.899 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker ERROR [ 75%] 2024-04-17T05:33:39.104 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 75%] 2024-04-17T05:33:39.104 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create ERROR [ 75%] 2024-04-17T05:33:41.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl PASSED [ 75%] 2024-04-17T05:33:41.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl ERROR [ 75%] 2024-04-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 75%] 2024-04-17T05:33:43.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified ERROR [ 75%] 2024-04-17T05:33:47.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove PASSED [ 75%] 2024-04-17T05:33:47.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove ERROR [ 75%] 2024-04-17T05:33:48.046 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set PASSED [ 75%] 2024-04-17T05:33:48.046 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set ERROR [ 75%] 2024-04-17T05:33:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get PASSED [ 76%] 2024-04-17T05:33:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get ERROR [ 76%] 2024-04-17T05:33:49.690 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id PASSED [ 76%] 2024-04-17T05:33:49.690 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id ERROR [ 76%] 2024-04-17T05:33:50.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long PASSED [ 76%] 2024-04-17T05:33:50.320 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long ERROR [ 76%] 2024-04-17T05:33:51.011 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id PASSED [ 76%] 2024-04-17T05:33:51.011 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id ERROR [ 76%] 2024-04-17T05:33:51.830 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status PASSED [ 76%] 2024-04-17T05:33:51.830 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status ERROR [ 76%] 2024-04-17T05:33:52.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date PASSED [ 76%] 2024-04-17T05:33:52.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date ERROR [ 76%] 2024-04-17T05:33:53.142 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date PASSED [ 76%] 2024-04-17T05:33:53.142 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date ERROR [ 76%] 2024-04-17T05:33:54.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent PASSED [ 77%] 2024-04-17T05:33:54.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent ERROR [ 77%] 2024-04-17T05:33:55.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker PASSED [ 77%] 2024-04-17T05:33:55.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker ERROR [ 77%] 2024-04-17T05:33:55.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter PASSED [ 77%] 2024-04-17T05:33:55.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter ERROR [ 77%] 2024-04-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter PASSED [ 77%] 2024-04-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter ERROR [ 77%] 2024-04-17T05:33:57.248 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart PASSED [ 77%] 2024-04-17T05:33:57.248 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart ERROR [ 77%] 2024-04-17T05:33:57.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date PASSED [ 77%] 2024-04-17T05:33:57.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date ERROR [ 77%] 2024-04-17T05:33:58.821 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b PASSED [ 77%] 2024-04-17T05:33:58.821 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b ERROR [ 77%] 2024-04-17T05:33:59.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb PASSED [ 78%] 2024-04-17T05:33:59.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb ERROR [ 78%] 2024-04-17T05:34:01.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB PASSED [ 78%] 2024-04-17T05:34:01.257 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB ERROR [ 78%] 2024-04-17T05:34:02.221 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b PASSED [ 78%] 2024-04-17T05:34:02.221 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b ERROR [ 78%] 2024-04-17T05:34:03.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head PASSED [ 78%] 2024-04-17T05:34:03.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head ERROR [ 78%] 2024-04-17T05:34:04.190 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present PASSED [ 78%] 2024-04-17T05:34:04.190 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present ERROR [ 78%] 2024-04-17T05:34:05.151 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key PASSED [ 78%] 2024-04-17T05:34:05.151 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key ERROR [ 78%] 2024-04-17T05:34:05.889 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 PASSED [ 78%] 2024-04-17T05:34:05.889 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 ERROR [ 78%] 2024-04-17T05:34:06.649 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 PASSED [ 78%] 2024-04-17T05:34:06.649 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 ERROR [ 78%] 2024-04-17T05:34:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key PASSED [ 79%] 2024-04-17T05:34:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key ERROR [ 79%] 2024-04-17T05:34:08.164 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c PASSED [ 79%] 2024-04-17T05:34:08.165 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c ERROR [ 79%] 2024-04-17T05:34:47.798 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload PASSED [ 79%] 2024-04-17T05:34:47.798 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload ERROR [ 79%] 2024-04-17T05:35:25.925 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload PASSED [ 79%] 2024-04-17T05:35:25.926 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload ERROR [ 79%] 2024-04-17T05:35:35.418 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download PASSED [ 79%] 2024-04-17T05:35:35.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download ERROR [ 79%] 2024-04-17T05:35:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request PASSED [ 79%] 2024-04-17T05:35:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request ERROR [ 79%] 2024-04-17T05:35:37.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head PASSED [ 79%] 2024-04-17T05:35:37.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head ERROR [ 79%] 2024-04-17T05:35:38.415 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present PASSED [ 80%] 2024-04-17T05:35:38.415 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present ERROR [ 80%] 2024-04-17T05:35:39.091 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key PASSED [ 80%] 2024-04-17T05:35:39.091 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key ERROR [ 80%] 2024-04-17T05:35:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared PASSED [ 80%] 2024-04-17T05:35:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared ERROR [ 80%] 2024-04-17T05:36:08.447 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload PASSED [ 80%] 2024-04-17T05:36:08.447 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload ERROR [ 80%] 2024-04-17T05:36:16.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 PASSED [ 80%] 2024-04-17T05:36:16.910 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 ERROR [ 80%] 2024-04-17T05:36:25.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 PASSED [ 80%] 2024-04-17T05:36:25.641 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 ERROR [ 80%] 2024-04-17T05:36:26.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request PASSED [ 80%] 2024-04-17T05:36:26.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request ERROR [ 80%] 2024-04-17T05:36:27.544 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b PASSED [ 81%] 2024-04-17T05:36:27.544 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b ERROR [ 81%] 2024-04-17T05:36:28.422 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb PASSED [ 81%] 2024-04-17T05:36:28.422 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb ERROR [ 81%] 2024-04-17T05:36:29.832 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB PASSED [ 81%] 2024-04-17T05:36:29.832 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB ERROR [ 81%] 2024-04-17T05:36:30.873 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b PASSED [ 81%] 2024-04-17T05:36:30.873 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b ERROR [ 81%] 2024-04-17T05:36:31.816 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare PASSED [ 81%] 2024-04-17T05:36:31.817 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare ERROR [ 81%] 2024-04-17T05:36:32.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy PASSED [ 81%] 2024-04-17T05:36:32.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy ERROR [ 81%] 2024-04-17T05:36:33.824 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy PASSED [ 81%] 2024-04-17T05:36:33.825 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy ERROR [ 81%] 2024-04-17T05:36:35.067 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl PASSED [ 81%] 2024-04-17T05:36:35.067 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl ERROR [ 81%] 2024-04-17T05:36:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl PASSED [ 82%] 2024-04-17T05:36:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl ERROR [ 82%] 2024-04-17T05:36:38.139 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket PASSED [ 82%] 2024-04-17T05:36:38.140 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket ERROR [ 82%] 2024-04-17T05:36:40.066 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket PASSED [ 82%] 2024-04-17T05:36:40.066 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket ERROR [ 82%] 2024-04-17T05:36:42.573 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging PASSED [ 82%] 2024-04-17T05:36:42.573 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging ERROR [ 82%] 2024-04-17T05:36:44.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging PASSED [ 82%] 2024-04-17T05:36:44.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging ERROR [ 82%] 2024-04-17T05:36:46.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags PASSED [ 82%] 2024-04-17T05:36:46.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags ERROR [ 82%] 2024-04-17T05:36:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags PASSED [ 82%] 2024-04-17T05:36:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags ERROR [ 82%] 2024-04-17T05:36:50.874 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags PASSED [ 83%] 2024-04-17T05:36:50.875 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags ERROR [ 83%] 2024-04-17T05:36:52.899 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags PASSED [ 83%] 2024-04-17T05:36:52.899 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags ERROR [ 83%] 2024-04-17T05:36:55.025 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags PASSED [ 83%] 2024-04-17T05:36:55.025 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags ERROR [ 83%] 2024-04-17T05:36:57.252 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags PASSED [ 83%] 2024-04-17T05:36:57.253 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags ERROR [ 83%] 2024-04-17T05:36:59.649 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags PASSED [ 83%] 2024-04-17T05:36:59.649 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags ERROR [ 83%] 2024-04-17T05:37:00.644 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request PASSED [ 83%] 2024-04-17T05:37:00.644 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request ERROR [ 83%] 2024-04-17T05:37:01.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request PASSED [ 83%] 2024-04-17T05:37:01.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request ERROR [ 83%] 2024-04-17T05:37:02.751 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags PASSED [ 84%] 2024-04-17T05:37:02.751 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags ERROR [ 84%] 2024-04-17T05:37:05.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public PASSED [ 84%] 2024-04-17T05:37:05.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public ERROR [ 84%] 2024-04-17T05:37:07.354 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public PASSED [ 84%] 2024-04-17T05:37:07.354 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public ERROR [ 84%] 2024-04-17T05:37:09.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public PASSED [ 84%] 2024-04-17T05:37:09.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public ERROR [ 84%] 2024-04-17T05:37:12.280 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id PASSED [ 84%] 2024-04-17T05:37:12.280 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id ERROR [ 84%] 2024-04-17T05:37:34.404 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id PASSED [ 84%] 2024-04-17T05:37:34.404 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id ERROR [ 84%] 2024-04-17T05:37:36.232 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag PASSED [ 84%] 2024-04-17T05:37:36.232 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag ERROR [ 84%] 2024-04-17T05:37:38.184 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag PASSED [ 84%] 2024-04-17T05:37:38.184 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag ERROR [ 84%] 2024-04-17T05:37:40.013 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag PASSED [ 85%] 2024-04-17T05:37:40.013 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag ERROR [ 85%] 2024-04-17T05:37:42.620 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source PASSED [ 85%] 2024-04-17T05:37:42.620 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source ERROR [ 85%] 2024-04-17T05:37:44.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta PASSED [ 85%] 2024-04-17T05:37:44.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta ERROR [ 85%] 2024-04-17T05:37:45.775 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl PASSED [ 85%] 2024-04-17T05:37:45.775 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl ERROR [ 85%] 2024-04-17T05:37:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant PASSED [ 85%] 2024-04-17T05:37:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant ERROR [ 85%] 2024-04-17T05:37:48.448 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 PASSED [ 85%] 2024-04-17T05:37:48.449 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 ERROR [ 85%] 2024-04-17T05:37:49.101 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms PASSED [ 85%] 2024-04-17T05:37:49.102 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms ERROR [ 85%] 2024-04-17T05:37:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms PASSED [ 86%] 2024-04-17T05:37:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms ERROR [ 86%] 2024-04-17T05:37:50.371 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms PASSED [ 86%] 2024-04-17T05:37:50.371 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms ERROR [ 86%] 2024-04-17T05:37:51.469 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc PASSED [ 86%] 2024-04-17T05:37:51.469 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc ERROR [ 86%] 2024-04-17T05:37:52.262 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 PASSED [ 86%] 2024-04-17T05:37:52.263 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 ERROR [ 86%] 2024-04-17T05:37:54.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag PASSED [ 86%] 2024-04-17T05:37:54.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag ERROR [ 86%] 2024-04-17T05:37:55.076 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock PASSED [ 86%] 2024-04-17T05:37:55.076 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock ERROR [ 86%] 2024-04-17T05:37:55.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket PASSED [ 86%] 2024-04-17T05:37:55.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket ERROR [ 86%] 2024-04-17T05:37:56.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years PASSED [ 87%] 2024-04-17T05:37:56.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years ERROR [ 87%] 2024-04-17T05:37:57.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days PASSED [ 87%] 2024-04-17T05:37:57.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days ERROR [ 87%] 2024-04-17T05:37:58.056 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years PASSED [ 87%] 2024-04-17T05:37:58.056 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years ERROR [ 87%] 2024-04-17T05:37:58.892 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode PASSED [ 87%] 2024-04-17T05:37:58.892 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode ERROR [ 87%] 2024-04-17T05:37:59.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status PASSED [ 87%] 2024-04-17T05:37:59.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status ERROR [ 87%] 2024-04-17T05:38:00.286 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning PASSED [ 87%] 2024-04-17T05:38:00.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning ERROR [ 87%] 2024-04-17T05:38:01.132 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock PASSED [ 87%] 2024-04-17T05:38:01.132 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock ERROR [ 87%] 2024-04-17T05:38:01.923 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket PASSED [ 87%] 2024-04-17T05:38:01.923 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket ERROR [ 87%] 2024-04-17T05:38:03.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention PASSED [ 88%] 2024-04-17T05:38:03.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention ERROR [ 88%] 2024-04-17T05:38:04.024 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket PASSED [ 88%] 2024-04-17T05:38:04.024 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket ERROR [ 88%] 2024-04-17T05:38:05.183 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode PASSED [ 88%] 2024-04-17T05:38:05.183 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode ERROR [ 88%] 2024-04-17T05:38:06.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention PASSED [ 88%] 2024-04-17T05:38:06.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention ERROR [ 88%] 2024-04-17T05:38:07.559 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 PASSED [ 88%] 2024-04-17T05:38:07.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 ERROR [ 88%] 2024-04-17T05:38:08.492 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket PASSED [ 88%] 2024-04-17T05:38:08.492 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket ERROR [ 88%] 2024-04-17T05:38:09.911 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid PASSED [ 88%] 2024-04-17T05:38:09.911 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid ERROR [ 88%] 2024-04-17T05:38:11.161 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention PASSED [ 89%] 2024-04-17T05:38:11.161 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention ERROR [ 89%] 2024-04-17T05:38:12.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period PASSED [ 89%] 2024-04-17T05:38:12.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period ERROR [ 89%] 2024-04-17T05:38:13.591 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period PASSED [ 89%] 2024-04-17T05:38:13.591 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period ERROR [ 89%] 2024-04-17T05:38:14.845 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass PASSED [ 89%] 2024-04-17T05:38:14.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass ERROR [ 89%] 2024-04-17T05:38:15.973 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention PASSED [ 89%] 2024-04-17T05:38:15.974 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention ERROR [ 89%] 2024-04-17T05:38:17.448 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention PASSED [ 89%] 2024-04-17T05:38:17.448 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention ERROR [ 89%] 2024-04-17T05:38:18.905 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker PASSED [ 89%] 2024-04-17T05:38:18.905 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker ERROR [ 89%] 2024-04-17T05:38:20.277 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention PASSED [ 90%] 2024-04-17T05:38:20.277 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention ERROR [ 90%] 2024-04-17T05:38:21.473 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold PASSED [ 90%] 2024-04-17T05:38:21.473 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold ERROR [ 90%] 2024-04-17T05:38:22.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket PASSED [ 90%] 2024-04-17T05:38:22.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket ERROR [ 90%] 2024-04-17T05:38:23.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status PASSED [ 90%] 2024-04-17T05:38:23.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status ERROR [ 90%] 2024-04-17T05:38:24.906 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold PASSED [ 90%] 2024-04-17T05:38:24.907 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold ERROR [ 90%] 2024-04-17T05:38:25.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket PASSED [ 90%] 2024-04-17T05:38:25.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket ERROR [ 90%] 2024-04-17T05:38:27.182 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on PASSED [ 90%] 2024-04-17T05:38:27.183 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on ERROR [ 90%] 2024-04-17T05:38:28.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on PASSED [ 90%] 2024-04-17T05:38:28.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on ERROR [ 90%] 2024-04-17T05:38:29.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off PASSED [ 91%] 2024-04-17T05:38:29.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off ERROR [ 91%] 2024-04-17T05:38:30.997 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata PASSED [ 91%] 2024-04-17T05:38:30.998 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata ERROR [ 91%] 2024-04-17T05:38:32.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj PASSED [ 91%] 2024-04-17T05:38:32.167 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj ERROR [ 91%] 2024-04-17T05:38:42.777 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass PASSED [ 91%] 2024-04-17T05:38:42.777 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass ERROR [ 91%] 2024-04-17T05:38:43.850 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass PASSED [ 91%] 2024-04-17T05:38:43.850 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass ERROR [ 91%] 2024-04-17T05:38:53.483 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance PASSED [ 91%] 2024-04-17T05:38:53.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance ERROR [ 91%] 2024-04-17T05:38:54.595 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good PASSED [ 91%] 2024-04-17T05:38:54.595 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good ERROR [ 91%] 2024-04-17T05:38:55.751 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed PASSED [ 92%] 2024-04-17T05:38:55.751 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed ERROR [ 92%] 2024-04-17T05:38:56.427 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status PASSED [ 92%] 2024-04-17T05:38:56.427 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status ERROR [ 92%] 2024-04-17T05:38:57.245 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status PASSED [ 92%] 2024-04-17T05:38:57.245 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status ERROR [ 92%] 2024-04-17T05:38:58.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status PASSED [ 92%] 2024-04-17T05:38:58.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status ERROR [ 92%] 2024-04-17T05:38:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-04-17T05:38:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-04-17T05:38:59.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-04-17T05:38:59.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-04-17T05:39:00.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status PASSED [ 92%] 2024-04-17T05:39:00.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status ERROR [ 92%] 2024-04-17T05:39:01.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block PASSED [ 93%] 2024-04-17T05:39:01.493 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block ERROR [ 93%] 2024-04-17T05:39:02.633 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy PASSED [ 93%] 2024-04-17T05:39:02.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy ERROR [ 93%] 2024-04-17T05:39:03.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block PASSED [ 93%] 2024-04-17T05:39:03.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block ERROR [ 93%] 2024-04-17T05:39:04.423 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls PASSED [ 93%] 2024-04-17T05:39:04.423 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls ERROR [ 93%] 2024-04-17T05:39:05.361 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls PASSED [ 93%] 2024-04-17T05:39:05.361 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls ERROR [ 93%] 2024-04-17T05:39:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy PASSED [ 93%] 2024-04-17T05:39:06.252 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy ERROR [ 93%] 2024-04-17T05:39:07.729 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls PASSED [ 93%] 2024-04-17T05:39:07.729 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls ERROR [ 93%] 2024-04-17T05:39:17.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy PASSED [ 93%] 2024-04-17T05:39:17.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy ERROR [ 93%] 2024-04-17T05:39:18.355 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms PASSED [ 94%] 2024-04-17T05:39:18.355 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms ERROR [ 94%] 2024-04-17T05:39:19.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms PASSED [ 94%] 2024-04-17T05:39:19.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms ERROR [ 94%] 2024-04-17T05:39:20.155 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms PASSED [ 94%] 2024-04-17T05:39:20.155 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms ERROR [ 94%] 2024-04-17T05:39:21.128 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request PASSED [ 94%] 2024-04-17T05:39:21.128 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request ERROR [ 94%] 2024-04-17T05:39:23.636 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent PASSED [ 94%] 2024-04-17T05:39:23.636 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent ERROR [ 94%] 2024-04-17T05:39:36.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_where_clause PASSED [ 94%] 2024-04-17T05:39:48.192 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_projection PASSED [ 94%] 2024-04-17T05:39:49.292 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_count_operation PASSED [ 95%] 2024-04-17T05:39:50.912 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_count_json_operation PASSED [ 95%] 2024-04-17T05:40:29.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_json_column_sum_min_max PASSED [ 95%] 2024-04-17T05:40:47.432 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_json_nullif_expressions PASSED [ 95%] 2024-04-17T05:40:52.112 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_column_sum_min_max PASSED [ 95%] 2024-04-17T05:41:00.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_nullif_expressions PASSED [ 95%] 2024-04-17T05:41:03.468 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_nulliftrue_expressions PASSED [ 95%] 2024-04-17T05:41:09.916 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_is_not_null_expressions PASSED [ 96%] 2024-04-17T05:41:11.004 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_lowerupper_expressions PASSED [ 96%] 2024-04-17T05:41:15.948 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_in_expressions PASSED [ 96%] 2024-04-17T05:41:21.932 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_in_expressions PASSED [ 96%] 2024-04-17T05:41:40.276 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_like_expressions PASSED [ 96%] 2024-04-17T05:41:44.496 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalselike_expressions PASSED [ 96%] 2024-04-17T05:41:51.288 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_complex_expressions PASSED [ 96%] 2024-04-17T05:41:53.696 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_alias PASSED [ 96%] 2024-04-17T05:41:55.292 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_alias_cyclic_refernce PASSED [ 97%] 2024-04-17T05:42:09.777 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_datetime PASSED [ 97%] 2024-04-17T05:42:13.916 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_datetime PASSED [ 97%] 2024-04-17T05:42:15.572 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_parser PASSED [ 97%] 2024-04-17T05:42:17.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_definition PASSED [ 97%] 2024-04-17T05:42:25.145 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_schema_definition PASSED [ 97%] 2024-04-17T05:42:29.920 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_when_then_else_expressions PASSED [ 97%] 2024-04-17T05:42:36.216 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_coalesce_expressions PASSED [ 98%] 2024-04-17T05:42:38.661 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_cast_expressions PASSED [ 98%] 2024-04-17T05:42:39.216 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_version PASSED [ 98%] 2024-04-17T05:42:43.880 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_trim_expressions PASSED [ 98%] 2024-04-17T05:42:48.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalse_trim_expressions PASSED [ 98%] 2024-04-17T05:42:51.624 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_escape_expressions PASSED [ 98%] 2024-04-17T05:42:56.856 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_case_value_expressions PASSED [ 98%] 2024-04-17T05:42:58.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_bool_cast_expressions PASSED [ 99%] 2024-04-17T05:45:39.988 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_progress_expressions PASSED [ 99%] 2024-04-17T05:45:40.556 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3select.py::test_output_serial_expressions PASSED [ 99%] 2024-04-17T05:45:42.443 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic PASSED [ 99%] 2024-04-17T05:45:43.897 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic PASSED [ 99%] 2024-04-17T05:45:45.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic_publish PASSED [ 99%] 2024-04-17T05:45:47.231 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic_publish PASSED [ 99%] 2024-04-17T05:45:47.285 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_utils.py::test_generate PASSED [100%] 2024-04-17T05:45:47.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.285 INFO:teuthology.orchestra.run.smithi088.stdout:==================================== ERRORS ==================================== 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_object_write_to_nonexist_bucket ___________ 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.286 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.287 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.288 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.290 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.291 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.292 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_write_with_chunked_transfer_encoding _____ 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.294 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.296 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.297 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.298 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_bucket_create_delete ________________ 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.300 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.303 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.303 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.304 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.305 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.306 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.307 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.308 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_object_read_not_exist ________________ 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.309 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.311 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.313 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.314 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_requestid_matches_header_on_error ______ 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.315 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.316 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.317 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.318 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.319 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.320 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.321 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_versioning_concurrent_multi_object_delete ______ 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.322 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.323 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.324 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.325 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.327 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_multi_object_delete _________________ 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.328 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.329 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.330 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.331 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.332 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.333 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.334 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_multi_objectv2_delete ________________ 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.335 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.336 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.337 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.338 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.339 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.340 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.341 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_multi_object_delete_key_limit ____________ 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.342 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.343 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.344 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.346 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.347 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.348 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_multi_objectv2_delete_key_limit ___________ 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.349 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.350 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.352 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.353 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.354 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_object_head_zero_bytes _______________ 2024-04-17T05:45:47.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.358 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.358 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.358 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.358 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.360 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.361 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.362 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_write_check_etag _______________ 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.363 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.364 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.367 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.368 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.369 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.370 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_write_cache_control _____________ 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.371 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.372 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.374 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.375 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.376 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_object_write_expires ________________ 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.377 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.378 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.379 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.381 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.383 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_write_read_update_read_delete ________ 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.384 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.385 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.386 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.387 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.388 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.389 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.389 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.389 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.389 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.390 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.390 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.390 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.390 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.391 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.391 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.391 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.392 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.392 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.392 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.392 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.392 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.393 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.393 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.393 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.393 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.393 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_set_get_metadata_none_to_good ________ 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.396 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.397 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.398 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.399 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_object_set_get_metadata_none_to_empty ________ 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.401 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.402 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.406 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_set_get_metadata_overwrite_to_empty _____ 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.407 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.409 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.411 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.412 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_object_metadata_replaced_on_put ___________ 2024-04-17T05:45:47.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.414 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.415 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.417 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.418 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.419 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_object_write_file __________________ 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.421 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.426 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.427 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.428 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_post_object_anonymous_request ____________ 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.429 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.430 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.431 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.434 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_post_object_authenticated_request __________ 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.435 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.436 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.437 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.438 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.439 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.440 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_post_object_authenticated_no_content_type ______ 2024-04-17T05:45:47.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.442 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.443 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.444 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.444 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.444 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.446 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.447 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.448 INFO:teuthology.orchestra.run.smithi088.stdout:__ ERROR at teardown of test_post_object_authenticated_request_bad_access_key __ 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.450 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.451 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.452 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.453 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.454 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_post_object_set_success_code ____________ 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.455 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.458 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.459 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.460 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_post_object_set_invalid_success_code ________ 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.461 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.462 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.463 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.464 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.465 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.466 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_post_object_upload_larger_than_chunk ________ 2024-04-17T05:45:47.467 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.468 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.469 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.471 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.472 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.473 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_post_object_set_key_from_filename __________ 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.474 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.475 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.476 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.477 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.478 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.479 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_post_object_ignored_header _____________ 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.480 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.481 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.482 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.483 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.484 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.485 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.486 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_post_object_case_insensitive_condition_fields ____ 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.487 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.488 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.489 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.490 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.491 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.492 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_post_object_escaped_field_values __________ 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.494 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.495 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.496 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.497 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.498 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_post_object_success_redirect_action _________ 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.499 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.500 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.501 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.502 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.503 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.504 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.505 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_post_object_invalid_signature ____________ 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.506 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.507 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.508 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.509 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.510 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.511 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_post_object_invalid_access_key ___________ 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.512 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.513 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.514 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.515 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.516 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.517 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.518 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_post_object_invalid_date_format ___________ 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.519 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.520 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.523 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.524 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_post_object_no_key_specified ____________ 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.525 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.526 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.527 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.528 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.529 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.530 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.531 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_post_object_missing_signature ____________ 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.533 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.534 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.534 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.534 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.534 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.558 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.559 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.560 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_post_object_missing_policy_condition ________ 2024-04-17T05:45:47.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.563 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.564 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.565 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.566 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.567 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_post_object_user_specified_header __________ 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.568 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.569 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.570 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.572 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_post_object_request_missing_policy_specified_field _ 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.574 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.575 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.577 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.578 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.579 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_post_object_condition_is_case_sensitive _______ 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.580 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.582 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.583 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.584 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.585 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.586 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_post_object_expires_is_case_sensitive ________ 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.589 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.591 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.592 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_post_object_expired_policy _____________ 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.593 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.594 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.597 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.598 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_post_object_wrong_bucket ______________ 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.599 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.600 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.601 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.602 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.603 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.604 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_post_object_invalid_request_field_value _______ 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.605 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.607 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.609 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.610 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.611 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_post_object_missing_expires_condition ________ 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.612 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.613 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.614 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.615 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.616 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.617 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_post_object_missing_conditions_list _________ 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.618 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.619 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.620 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.621 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.622 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.623 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_post_object_upload_size_limit_exceeded _______ 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.624 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.626 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.627 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.629 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.630 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.631 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.632 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_post_object_missing_content_length_argument _____ 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.633 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.634 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.635 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.637 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.638 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.639 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_post_object_invalid_content_length_argument _____ 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.640 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.641 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.642 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.643 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.644 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.645 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_post_object_upload_size_below_minimum ________ 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.647 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.648 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.649 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.650 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.651 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_post_object_upload_size_rgw_chunk_size_bug _____ 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.652 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.653 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.654 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.655 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.656 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.657 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.658 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_post_object_empty_conditions ____________ 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.659 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.661 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.662 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.663 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.664 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_get_object_ifmatch_good _______________ 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.665 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.667 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.669 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.670 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.671 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_get_object_ifmatch_failed ______________ 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.672 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.673 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.674 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.675 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.676 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.677 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_get_object_ifnonematch_good _____________ 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.678 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.679 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.680 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.681 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.682 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.683 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.684 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_get_object_ifnonematch_failed ____________ 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.686 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.688 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_get_object_ifmodifiedsince_good ___________ 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.691 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.692 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.693 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.694 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.695 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.696 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_get_object_ifmodifiedsince_failed __________ 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.697 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.699 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.701 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.702 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_get_object_ifunmodifiedsince_good __________ 2024-04-17T05:45:47.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.704 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.705 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.707 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.708 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.709 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_get_object_ifunmodifiedsince_failed _________ 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.711 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.712 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.713 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.714 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.715 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_put_object_ifmatch_good _______________ 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.716 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.718 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.720 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.721 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_put_object_ifmatch_failed ______________ 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.722 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.723 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.724 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.725 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.726 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.727 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.728 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_put_object_ifmatch_overwrite_existed_good ______ 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.729 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.730 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.731 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.733 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_put_object_ifmatch_nonexisted_failed ________ 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.735 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.736 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.737 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.739 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.740 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_put_object_ifnonmatch_good _____________ 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.741 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.742 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.743 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.744 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.745 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.746 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.747 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_put_object_ifnonmatch_failed ____________ 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.748 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.750 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.751 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.752 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.753 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.754 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_put_object_ifnonmatch_nonexisted_good ________ 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.755 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.756 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.758 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_put_object_ifnonmatch_overwrite_existed_failed ___ 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.761 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.763 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.765 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.766 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout:___________________ ERROR at teardown of test_object_raw_get ___________________ 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.767 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.768 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.769 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.770 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.772 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_gone _____________ 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.774 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.775 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.777 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.778 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.779 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_delete_key_bucket_gone ____________ 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.780 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.781 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.782 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.783 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.784 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.785 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_raw_get_object_gone _____________ 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.787 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.788 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.789 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.790 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.791 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout:____________________ ERROR at teardown of test_bucket_head _____________________ 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.792 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.793 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.794 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.796 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.797 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.798 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_bucket_head_notexist ________________ 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.799 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.800 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.801 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.802 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.804 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_bucket_head_extended ________________ 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.806 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.807 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.808 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.809 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.810 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_acl ______________ 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.811 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.812 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.813 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.814 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.815 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.816 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_raw_get_object_acl ______________ 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.817 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.818 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.819 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.820 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.821 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.822 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.823 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_object_put_acl_mtime ________________ 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.824 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.825 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.826 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.827 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.829 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_raw_authenticated ______________ 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.830 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.831 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.831 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.831 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.831 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.831 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.834 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.835 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.836 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.837 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.838 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_object_raw_response_headers _____________ 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.840 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.841 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.842 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_acl _________ 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.845 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.846 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.847 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.848 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.849 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.851 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_acl _________ 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.852 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.853 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.854 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.855 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.857 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_gone ________ 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.858 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.859 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.860 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.861 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.862 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.863 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_gone ________ 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.864 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.865 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.866 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.868 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.869 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.870 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired ______ 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.871 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.872 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.874 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.875 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.876 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired_tenant ___ 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.878 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.879 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.880 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.881 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.882 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_range_zero _____ 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.883 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.884 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.886 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.887 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.888 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.889 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_max_range _____ 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.892 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.893 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.894 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_out_positive_range ___ 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.899 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.900 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.901 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_object_anon_put ___________________ 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.903 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.904 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.905 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.906 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_object_anon_put_write_access ____________ 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.908 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.909 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.911 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.912 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.913 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_put_authenticated ______________ 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.916 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.918 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.919 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_presigned_put_object_with_acl ________ 2024-04-17T05:45:47.920 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.921 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.922 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.924 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.925 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_presigned_put_object_with_acl_tenant _____ 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.928 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.929 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.930 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.931 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.932 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_raw_put_authenticated_expired ________ 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.933 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.934 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.936 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.937 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.938 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_bucket_create_naming_bad_starts_nonalpha ______ 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.939 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.941 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.943 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.944 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.945 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_one _________ 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.946 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.947 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.948 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.949 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.950 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.951 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_two _________ 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.954 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.955 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.956 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.957 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_60 __________ 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.959 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.960 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.961 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.962 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.963 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_61 __________ 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.964 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.965 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.966 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.967 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.968 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.969 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.970 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_62 __________ 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.972 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.974 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.975 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.976 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_63 __________ 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.977 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.978 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.979 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.981 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.982 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_bucket_list_long_name ________________ 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.983 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.984 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.985 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.987 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.988 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_bucket_create_naming_bad_ip _____________ 2024-04-17T05:45:47.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.990 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.991 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.993 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:47.994 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.995 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_underscore _________ 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:47.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:47.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:47.999 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.000 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.001 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucket_create_naming_dns_long ____________ 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.003 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.005 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.006 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.007 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_dash_at_end ________ 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.008 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.010 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.012 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.013 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.014 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_create_naming_dns_dot_dot __________ 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.015 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.016 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.018 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.019 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.020 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dot_dash __________ 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.022 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.024 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.025 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.026 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dash_dot __________ 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.027 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.028 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.029 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.031 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.032 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_bucket_create_exists ________________ 2024-04-17T05:45:48.033 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.034 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.035 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.035 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.035 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.035 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.035 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.038 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.038 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.039 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.040 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.041 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.042 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_bucket_get_location _________________ 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.043 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.044 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.045 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.046 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.047 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.048 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucket_create_exists_nonowner ____________ 2024-04-17T05:45:48.049 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.050 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.051 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.052 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.054 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.055 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucket_recreate_overwrite_acl ____________ 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.056 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.058 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.059 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.060 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.061 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_bucket_recreate_new_acl _______________ 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.062 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.063 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.064 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.065 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.066 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.067 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_bucket_acl_default _________________ 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.068 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.069 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.070 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.072 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.073 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.074 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_acl_canned_during_create ___________ 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.075 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.076 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.077 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.078 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.079 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.080 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_bucket_acl_canned __________________ 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.081 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.083 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.084 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.085 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.086 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_acl_canned_publicreadwrite __________ 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.088 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.089 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.090 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.091 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.092 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_bucket_acl_canned_authenticatedread _________ 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.093 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.094 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.095 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.096 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.097 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.098 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_object_acl_default _________________ 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.099 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.101 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.103 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.104 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.105 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_object_acl_canned_during_create ___________ 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.106 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.107 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.108 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.109 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.110 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.111 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_object_acl_canned __________________ 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.112 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.113 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.114 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.115 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.116 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.117 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_object_acl_canned_publicreadwrite __________ 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.119 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.120 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.121 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.122 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.123 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_acl_canned_authenticatedread _________ 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.124 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.125 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.127 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.129 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_object_acl_canned_bucketownerread __________ 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.130 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.131 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.134 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.135 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.136 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_acl_canned_bucketownerfullcontrol ______ 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.137 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.138 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.139 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.140 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.141 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.142 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_acl_full_control_verify_owner ________ 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.144 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.145 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.146 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.147 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.148 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_acl_full_control_verify_attributes ______ 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.149 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.151 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.152 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.154 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_bucket_acl_canned_private_to_private ________ 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.155 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.157 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.159 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.160 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.161 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:_____________________ ERROR at teardown of test_object_acl _____________________ 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.163 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.164 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.166 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.167 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_object_acl_write __________________ 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.169 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.170 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.171 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.172 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.173 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_object_acl_writeacp _________________ 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.174 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.175 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.177 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.178 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.179 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_object_acl_read ___________________ 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.181 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.184 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.185 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_object_acl_readacp _________________ 2024-04-17T05:45:48.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.187 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.188 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.190 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.191 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.192 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_bucket_acl_grant_userid_fullcontrol _________ 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.194 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.195 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.196 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.197 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.198 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_bucket_acl_grant_userid_read ____________ 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.200 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.202 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.203 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.204 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_readacp ___________ 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.205 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.207 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.208 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.209 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.210 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucket_acl_grant_userid_write ____________ 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.212 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.213 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.214 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.215 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.216 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.217 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_writeacp __________ 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.218 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.219 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.220 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.221 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.222 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.223 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucket_acl_grant_nonexist_user ___________ 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.224 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.225 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.227 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.228 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.229 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_header_acl_grants ______________ 2024-04-17T05:45:48.230 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.231 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.232 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.234 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.235 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.236 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_bucket_header_acl_grants ______________ 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.237 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.238 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.239 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.240 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.241 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.242 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_bucket_acl_grant_email _______________ 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.243 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.244 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.246 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.247 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.248 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_acl_grant_email_not_exist __________ 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.249 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.250 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.252 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.253 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.254 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.255 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_bucket_acl_revoke_all ________________ 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.256 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.257 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.258 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.259 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.260 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.261 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_access_bucket_private_object_private ________ 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.262 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.263 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.265 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.266 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.267 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_access_bucket_private_objectv2_private _______ 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.270 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.271 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.272 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.273 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_access_bucket_private_object_publicread _______ 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.274 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.275 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.277 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.278 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.279 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_access_bucket_private_objectv2_publicread ______ 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.280 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.281 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.282 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.284 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.285 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.286 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_access_bucket_private_object_publicreadwrite ____ 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.288 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.290 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.291 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.292 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_access_bucket_private_objectv2_publicreadwrite ___ 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.293 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.294 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.296 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.297 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.298 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_access_bucket_publicread_object_private _______ 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.299 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.300 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.301 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.302 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.303 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.304 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_access_bucket_publicread_object_publicread _____ 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.305 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.306 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.307 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.308 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.309 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.310 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout:__ ERROR at teardown of test_access_bucket_publicread_object_publicreadwrite ___ 2024-04-17T05:45:48.311 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.312 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.313 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.315 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.316 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.317 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_access_bucket_publicreadwrite_object_private ____ 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.318 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.319 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.320 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.323 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout:__ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicread ___ 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.324 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.326 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.327 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.328 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.329 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicreadwrite _ 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.331 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.332 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.334 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.335 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.336 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_buckets_create_then_list ______________ 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.337 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.338 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.339 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.340 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.341 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.342 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_buckets_list_ctime _________________ 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.343 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.344 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.345 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.346 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.347 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.348 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_list_buckets_anonymous _______________ 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.349 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.350 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.351 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.352 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.353 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.354 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_list_buckets_invalid_auth ______________ 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.355 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.356 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.357 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.358 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.359 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.360 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_list_buckets_bad_auth ________________ 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.362 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.363 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.364 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.365 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.366 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.367 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_alpha _______ 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.369 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.370 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.371 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.372 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.373 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:Done with cleanup of buckets in tests. 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:Done with cleanup of buckets in tests. 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_digit _______ 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.374 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.375 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.376 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.377 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.378 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.379 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:Done with cleanup of buckets in tests. 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-17T05:45:48.380 INFO:teuthology.orchestra.run.smithi088.stdout:Done with cleanup of buckets in tests. 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_period ______ 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.381 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.383 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.384 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.385 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.386 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_hyphen ______ 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.387 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.389 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.390 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.391 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.392 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucket_recreate_not_overriding ___________ 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.393 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.394 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.395 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.396 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.397 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.398 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_create_special_key_names ___________ 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.399 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.400 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.401 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.403 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.404 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_bucket_list_special_prefix _____________ 2024-04-17T05:45:48.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.406 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.407 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.408 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.409 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.410 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.411 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_object_copy_zero_size ________________ 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.412 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.414 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.415 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.416 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_object_copy_16m ___________________ 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.418 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.419 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.420 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.421 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.422 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.423 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_copy_same_bucket _______________ 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.424 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.425 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.426 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.427 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.428 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.430 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_copy_verify_contenttype ___________ 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.431 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.432 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.433 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.434 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.435 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.436 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_object_copy_to_itself ________________ 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.437 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.438 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.439 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.440 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.442 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_copy_to_itself_with_metadata _________ 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.443 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.444 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.445 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.446 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.447 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.447 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.447 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.447 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.447 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.450 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.451 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_copy_diff_bucket _______________ 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.452 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.453 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.455 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.457 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.458 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_object_copy_not_owned_bucket ____________ 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.459 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.460 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.461 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.462 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.463 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.464 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_object_copy_not_owned_object_bucket _________ 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.466 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.467 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.468 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.469 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.470 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_object_copy_canned_acl _______________ 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.471 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.472 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.474 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.475 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.476 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_copy_retaining_metadata ___________ 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.477 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.479 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.481 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.482 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_copy_replacing_metadata ___________ 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.483 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.484 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.485 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.486 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.487 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.488 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_object_copy_bucket_not_found ____________ 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.490 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.491 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.494 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.495 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_copy_key_not_found ______________ 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.496 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.497 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.498 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.499 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.500 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.501 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_object_copy_versioned_bucket ____________ 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.502 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.504 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.506 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.507 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.508 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_object_copy_versioned_url_encoding _________ 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.509 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.510 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.511 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.512 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.513 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.514 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_copy_versioning_multipart_upload _______ 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.516 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.517 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.518 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.519 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.520 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_multipart_upload_empty _______________ 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.521 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.522 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.523 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.524 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.525 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.526 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_multipart_upload_small _______________ 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.527 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.528 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.529 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.530 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.531 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.532 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.533 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_multipart_copy_small ________________ 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.534 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.535 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.536 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.537 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.538 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.539 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_multipart_copy_invalid_range ____________ 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.540 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.541 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.542 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.543 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.544 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.545 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_multipart_copy_without_range ____________ 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.546 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.547 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.548 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.549 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.550 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.551 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_multipart_copy_special_names ____________ 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.553 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.553 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.553 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.554 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.555 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.556 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.557 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.558 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.559 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_multipart_upload __________________ 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.560 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.561 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.562 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.565 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_multipart_copy_versioned ______________ 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.566 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.567 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.568 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.570 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.572 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_multipart_upload_resend_part ____________ 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.573 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.575 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.577 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.578 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_multipart_upload_multiple_sizes ___________ 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.579 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.580 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.581 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.583 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.584 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_multipart_copy_multiple_sizes ____________ 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.586 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.586 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.586 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.586 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.586 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.587 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.588 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.589 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.590 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.591 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_multipart_upload_size_too_small ___________ 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.592 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.593 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.594 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.596 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.597 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.598 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.599 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_multipart_upload_contents ______________ 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.600 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.601 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.602 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.604 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.606 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_multipart_upload_overwrite_existing_object _____ 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.608 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.610 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.611 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.613 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_abort_multipart_upload _______________ 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.615 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.616 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.617 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.618 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.619 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_abort_multipart_upload_not_found __________ 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.620 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.621 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.623 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.625 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_list_multipart_upload ________________ 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.627 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.628 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.630 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.631 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.632 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_list_multipart_upload_owner _____________ 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.633 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.634 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.635 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.636 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.638 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_multipart_upload_missing_part ____________ 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.639 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.640 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.641 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.642 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.643 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_multipart_upload_incorrect_etag ___________ 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.645 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.646 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.648 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.650 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.651 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.651 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.654 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_multipart_get_part _________________ 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.655 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.656 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.657 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.658 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.659 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.660 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_non_multipart_get_part _______________ 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.661 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.662 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.663 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.664 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.665 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.666 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout:____________________ ERROR at teardown of test_100_continue ____________________ 2024-04-17T05:45:48.667 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.668 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.670 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.671 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.672 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.673 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:______________________ ERROR at teardown of test_set_cors ______________________ 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.674 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.675 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.676 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.677 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.679 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_cors_origin_response ________________ 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.681 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.682 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.683 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.684 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.685 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.686 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_cors_origin_wildcard ________________ 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.687 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.688 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.689 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.690 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.691 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.692 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_cors_header_option _________________ 2024-04-17T05:45:48.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.694 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.695 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.696 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.697 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.698 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.699 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_cors_presigned_get_object ______________ 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.700 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.701 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.702 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.704 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.705 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.706 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_cors_presigned_get_object_tenant __________ 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.707 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.708 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.710 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.711 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_cors_presigned_put_object ______________ 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.713 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.714 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.715 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.716 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.717 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.718 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.719 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_cors_presigned_put_object_with_acl _________ 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.720 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.721 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.722 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.723 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.724 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.725 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_cors_presigned_put_object_tenant __________ 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.726 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.727 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.728 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.729 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.730 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.731 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_cors_presigned_put_object_tenant_with_acl ______ 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.732 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.733 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.734 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.735 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.736 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.737 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.738 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_set_bucket_tagging _________________ 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.739 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.740 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.741 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.742 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.743 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.744 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_atomic_read_1mb ___________________ 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.745 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.746 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.747 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.748 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.749 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.750 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_atomic_read_4mb ___________________ 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.751 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.752 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.753 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.754 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.755 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.756 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.757 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_atomic_read_8mb ___________________ 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.758 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.759 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.760 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.763 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_atomic_write_1mb __________________ 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.764 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.765 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.766 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.768 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.769 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.770 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_atomic_write_4mb __________________ 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.771 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.772 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.773 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.774 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.775 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.776 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_atomic_write_8mb __________________ 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.778 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.779 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.781 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.782 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.783 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_atomic_dual_write_1mb ________________ 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.784 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.785 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.787 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.788 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_atomic_dual_write_4mb ________________ 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.790 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.791 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.792 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.793 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.794 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.795 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.796 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_atomic_dual_write_8mb ________________ 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.797 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.798 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.799 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.800 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.801 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.802 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_atomic_conditional_write_1mb ____________ 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.803 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.804 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.806 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.808 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_atomic_multipart_upload_write ____________ 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.809 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.810 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.811 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.813 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.814 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.815 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_multipart_resend_first_finishes_last ________ 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.816 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.817 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.819 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.820 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.821 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_ranged_request_response_code ____________ 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.822 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.823 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.824 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.825 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.826 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.827 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_ranged_big_request_response_code __________ 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.828 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.829 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.832 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.833 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.834 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:__ ERROR at teardown of test_ranged_request_skip_leading_bytes_response_code ___ 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.835 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.836 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.837 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.838 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.839 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.840 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_ranged_request_return_trailing_bytes_response_code _ 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.841 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.842 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.844 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.845 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.847 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_ranged_request_invalid_range ____________ 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.849 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.850 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.851 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.852 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.853 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_ranged_request_empty_object _____________ 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.855 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.857 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.858 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.859 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.860 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_versioning_bucket_create_suspend __________ 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.861 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.863 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.864 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.865 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.866 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.867 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.868 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.869 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_versioning_obj_create_read_remove __________ 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.870 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.871 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.872 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.873 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.874 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.875 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_versioning_obj_create_read_remove_head _______ 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.877 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.878 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.879 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.880 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.881 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_versioning_obj_plain_null_version_removal ______ 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.882 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.883 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.884 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.885 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.886 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.887 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.888 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite _____ 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.889 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.890 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.891 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.892 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.893 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.894 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite_suspended _ 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.896 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.898 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.899 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.900 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_versioning_obj_suspend_versions ___________ 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.901 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.902 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.903 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.904 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.905 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.906 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_versioning_obj_suspended_copy ____________ 2024-04-17T05:45:48.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.908 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.909 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.910 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.911 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.912 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_versioning_obj_create_versions_remove_all ______ 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.914 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.915 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.916 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.917 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.918 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.919 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_versioning_obj_create_versions_remove_special_names _ 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.920 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.921 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.922 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.923 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.924 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.925 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_versioning_obj_create_overwrite_multipart ______ 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.926 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.927 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.928 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.929 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.930 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.931 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.932 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_versioning_obj_list_marker _____________ 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.933 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.934 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.935 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.936 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.938 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_versioning_copy_obj_version _____________ 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.939 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.940 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.941 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.942 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.943 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_versioning_multi_object_delete ___________ 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.945 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.946 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.947 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.948 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.949 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.950 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_versioning_multi_object_delete_with_marker _____ 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.951 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.952 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.953 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.954 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.955 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.956 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.957 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_versioning_multi_object_delete_with_marker_create __ 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.958 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.959 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.960 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.961 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.962 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.963 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_versioned_object_acl ________________ 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.964 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.965 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.966 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.967 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.968 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.969 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_versioned_object_acl_no_version_specified ______ 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.970 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.971 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.972 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.973 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.974 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.975 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_versioned_concurrent_object_create_and_remove ____ 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.976 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.977 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.978 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.979 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.980 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.981 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:------------------------------ Captured log call ------------------------------- 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:WARNING urllib3.connectionpool:connectionpool.py:323 Connection pool is full, discarding connection: smithi088.front.sepia.ceph.com. Connection pool size: 10 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:___________________ ERROR at teardown of test_lifecycle_set ____________________ 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.982 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.983 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.984 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.985 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.986 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.987 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.988 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:___________________ ERROR at teardown of test_lifecycle_get ____________________ 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.989 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.990 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.991 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.992 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:48.993 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.994 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_lifecycle_get_no_id _________________ 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:48.995 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:48.996 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:48.997 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:48.998 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:48.999 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.000 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_lifecycle_id_too_long ________________ 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.001 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.002 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.003 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.004 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.005 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.006 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.007 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_lifecycle_same_id __________________ 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.008 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.009 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.010 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.011 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.012 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.013 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_lifecycle_invalid_status ______________ 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.014 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.015 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.016 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.018 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.019 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_lifecycle_set_date _________________ 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.020 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.021 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.022 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.023 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.024 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.025 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_lifecycle_set_invalid_date _____________ 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.027 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.028 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.030 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.031 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_lifecycle_set_noncurrent ______________ 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.032 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.033 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.034 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.035 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.036 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_lifecycle_set_deletemarker _____________ 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.038 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.039 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.040 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.041 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.042 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.043 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_lifecycle_set_filter ________________ 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.044 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.045 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.046 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.047 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.048 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.049 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.050 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_lifecycle_set_empty_filter _____________ 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.051 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.052 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.053 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.054 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.055 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.056 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_lifecycle_set_multipart _______________ 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.057 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.058 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.059 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.060 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.061 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.062 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.066 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.066 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.066 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.066 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.066 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_lifecycle_transition_set_invalid_date ________ 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.067 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.068 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.069 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.070 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.071 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.072 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.073 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1b ________________ 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.074 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.075 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.076 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.077 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.078 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.079 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1kb _______________ 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.080 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.081 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.082 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.083 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.084 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.085 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1MB _______________ 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.088 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.089 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.090 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.091 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_encrypted_transfer_13b _______________ 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.092 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.093 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.094 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.095 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.096 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.097 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.098 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_encryption_sse_c_method_head ____________ 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.099 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.100 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.101 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.102 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.103 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_encryption_sse_c_present ______________ 2024-04-17T05:45:49.104 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.105 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.106 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.107 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.108 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.109 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.110 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_encryption_sse_c_other_key _____________ 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.111 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.112 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.114 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.115 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.116 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_encryption_sse_c_invalid_md5 ____________ 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.118 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.119 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.120 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.121 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.122 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_md5 _______________ 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.123 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.124 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.125 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.126 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.127 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.128 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_key _______________ 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.129 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.130 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.131 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.132 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.133 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.135 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_encryption_key_no_sse_c _______________ 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.136 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.137 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.138 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.139 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.140 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.141 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_encryption_sse_c_multipart_upload __________ 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.142 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.143 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.144 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.145 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.146 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.147 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_encryption_sse_c_unaligned_multipart_upload _____ 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.148 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.149 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.150 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.151 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.152 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.153 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.154 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_encryption_sse_c_multipart_bad_download _______ 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.155 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.156 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.157 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.158 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.159 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.160 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_encryption_sse_c_post_object_authenticated_request _ 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.161 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.162 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.163 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.164 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.165 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.166 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_method_head _________________ 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.168 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.169 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.170 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.171 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.172 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_sse_kms_present ___________________ 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.173 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.174 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.175 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.176 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.177 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.178 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout:___________________ ERROR at teardown of test_sse_kms_no_key ___________________ 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.180 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.181 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.182 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.183 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.184 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.185 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_not_declared ________________ 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.186 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.187 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.188 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.189 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.190 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.191 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_sse_kms_multipart_upload ______________ 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.192 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.193 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.194 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.195 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.196 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.197 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_1 _________ 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.199 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.200 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.202 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.203 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_2 _________ 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.204 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.205 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.206 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.207 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.208 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.209 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_sse_kms_post_object_authenticated_request ______ 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.211 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.212 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.213 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.214 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.215 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1b _________________ 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.217 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.218 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.219 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.220 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.221 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.222 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1kb ________________ 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.223 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.224 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.225 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.226 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.227 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.228 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1MB ________________ 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.230 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.231 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.232 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.233 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.234 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_transfer_13b ________________ 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.235 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.236 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.237 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.238 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.239 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.240 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_sse_kms_read_declare ________________ 2024-04-17T05:45:49.241 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.242 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.243 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.244 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.245 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.246 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.247 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:___________________ ERROR at teardown of test_bucket_policy ____________________ 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.248 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.249 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.250 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.251 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.252 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_bucketv2_policy ___________________ 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.254 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.255 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.256 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.257 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.258 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.259 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.260 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_bucket_policy_acl __________________ 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.261 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.262 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.263 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.264 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.265 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.266 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_bucketv2_policy_acl _________________ 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.267 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.268 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.269 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.270 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.271 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.272 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_bucket_policy_another_bucket ____________ 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.273 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.274 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.275 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.276 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.277 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.278 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_bucketv2_policy_another_bucket ___________ 2024-04-17T05:45:49.279 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.280 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.281 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.282 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.283 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.284 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.285 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_get_obj_tagging ___________________ 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.286 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.287 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.288 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.289 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.290 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.291 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_get_obj_head_tagging ________________ 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.293 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.294 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.295 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.296 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.297 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.298 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:____________________ ERROR at teardown of test_put_max_tags ____________________ 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.299 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.300 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.301 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.302 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.303 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.304 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_put_excess_tags ___________________ 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.305 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.306 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.307 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.308 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.309 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.310 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.311 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_put_max_kvsize_tags _________________ 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.312 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.313 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.314 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.315 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.316 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.317 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_put_excess_key_tags _________________ 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.318 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.319 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.320 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.321 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_put_excess_val_tags _________________ 2024-04-17T05:45:49.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.325 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.326 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.327 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.328 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.329 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.330 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_put_modify_tags ___________________ 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.331 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.332 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.333 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.334 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.335 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.336 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_put_delete_tags ___________________ 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.337 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.338 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.339 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.340 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.341 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.342 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.343 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_post_object_tags_anonymous_request _________ 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.344 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.345 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.346 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.349 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_post_object_tags_authenticated_request _______ 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.351 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.352 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.353 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.354 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.355 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_put_obj_with_tags __________________ 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.356 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.357 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.358 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.359 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.360 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.361 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.362 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_get_tags_acl_public _________________ 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.363 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.364 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.365 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.366 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.367 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_put_tags_acl_public _________________ 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.369 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.370 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.371 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.372 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.374 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:_______________ ERROR at teardown of test_delete_tags_obj_public _______________ 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.375 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.376 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.377 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.378 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.379 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.380 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_versioning_bucket_atomic_upload_return_version_id __ 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.381 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.382 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.383 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.384 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.385 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.386 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_versioning_bucket_multipart_upload_return_version_id _ 2024-04-17T05:45:49.387 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.388 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.389 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.390 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.391 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.392 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.393 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_policy_get_obj_existing_tag _________ 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.394 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.395 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.396 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.397 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.398 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.399 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_bucket_policy_get_obj_tagging_existing_tag _____ 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.400 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.401 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.402 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.403 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.404 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.405 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.406 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_bucket_policy_put_obj_tagging_existing_tag _____ 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.407 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.408 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.409 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.410 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.411 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.412 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:_________ ERROR at teardown of test_bucket_policy_put_obj_copy_source __________ 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.413 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.414 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.415 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.416 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.417 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.418 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_bucket_policy_put_obj_copy_source_meta _______ 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.419 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.420 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.421 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.422 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.423 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.424 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.425 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_bucket_policy_put_obj_acl ______________ 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.426 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.427 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.428 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.429 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.430 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.431 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_grant _____________ 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.432 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.433 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.434 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.435 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.436 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.437 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.438 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_s3 ______________ 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.439 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.440 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.441 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.442 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.443 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.444 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_kms _____________ 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.445 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.446 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.447 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.448 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.449 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.450 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_put_obj_enc_conflict_s3_kms _____________ 2024-04-17T05:45:49.451 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.452 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.453 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.454 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.455 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.457 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_put_obj_enc_conflict_bad_enc_kms __________ 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.458 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.459 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.460 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.461 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.462 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.463 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_bucket_policy_put_obj_kms_noenc ___________ 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.464 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.465 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.466 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.467 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.468 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.469 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_kms_s3 ____________ 2024-04-17T05:45:49.470 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.471 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.472 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.473 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.474 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.475 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.476 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_bucket_policy_get_obj_acl_existing_tag _______ 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.477 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.478 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.479 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.480 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.481 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_lock_put_obj_lock ______________ 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.483 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.484 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.485 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.486 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.487 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.488 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_bucket _______ 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.490 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.491 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.492 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.494 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.495 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_lock_put_obj_lock_with_days_and_years ____ 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.496 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.497 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.498 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.499 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.500 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.501 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_days ________ 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.503 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.504 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.505 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.506 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.507 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.508 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_years _______ 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.509 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.510 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.511 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.512 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.513 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.514 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.515 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_mode ________ 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.516 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.517 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.518 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.519 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.520 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.521 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_status _______ 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.522 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.523 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.524 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.525 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.526 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.527 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_lock_suspend_versioning ___________ 2024-04-17T05:45:49.528 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.529 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.530 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.531 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.532 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.533 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.534 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_object_lock_get_obj_lock ______________ 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.535 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.536 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.537 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.538 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.539 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.540 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_lock_get_obj_lock_invalid_bucket _______ 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.541 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.542 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.543 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.544 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.545 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.546 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.547 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_lock_put_obj_retention ____________ 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.548 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.549 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.550 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.551 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.552 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.553 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_bucket ____ 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.554 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.555 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.556 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.557 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.558 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.559 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.560 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_mode _____ 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.561 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.562 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.563 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.564 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.565 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.566 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_object_lock_get_obj_retention ____________ 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.567 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.568 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.569 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.570 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.571 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.572 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.573 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:_______ ERROR at teardown of test_object_lock_get_obj_retention_iso8601 ________ 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.574 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.575 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.576 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.577 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.578 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.579 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_lock_get_obj_retention_invalid_bucket ____ 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.580 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.581 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.582 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.583 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.584 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.585 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_lock_put_obj_retention_versionid _______ 2024-04-17T05:45:49.586 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.587 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.588 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.589 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.590 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.591 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.592 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_override_default_retention _ 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.593 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.594 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.596 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.597 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.598 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_object_lock_put_obj_retention_increase_period ____ 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.600 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.601 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.602 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.603 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.604 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.605 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period ____ 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.606 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.607 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.608 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.609 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.610 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period_bypass _ 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.612 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.614 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.615 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.616 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.617 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_object_lock_delete_object_with_retention ______ 2024-04-17T05:45:49.618 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.619 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.620 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.621 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.622 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.623 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_retention _ 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.625 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.626 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.627 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.628 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.629 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.630 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_delete_object_with_retention_and_marker _ 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.631 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.632 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.633 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.634 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.635 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.636 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_object_lock_multi_delete_object_with_retention ___ 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.638 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.639 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.640 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.641 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.642 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.643 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_lock_put_legal_hold _____________ 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.644 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.645 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.646 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.648 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.649 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_bucket ______ 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.650 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.651 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.652 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.653 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.654 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.655 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_status ______ 2024-04-17T05:45:49.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.657 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.658 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.659 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.660 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.661 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.662 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_lock_get_legal_hold _____________ 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.663 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.664 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.665 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.666 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.667 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.668 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_lock_get_legal_hold_invalid_bucket ______ 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.670 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.671 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.672 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.673 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.674 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.675 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.675 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.677 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.677 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_on ____ 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.679 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.680 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.681 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.682 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_legal_hold_on _ 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_off ____ 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.691 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_object_lock_get_obj_metadata ____________ 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.698 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.703 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_object_lock_uploading_obj ______________ 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_with_bypass _ 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.711 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.712 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.713 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.714 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.715 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.716 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:nuke_bucket test-29fkkr1kn7gqxd76i3nfcoel-550 waiting 9.226571 seconds for object locks to expire 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_without_bypass _ 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.717 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.718 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.719 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.720 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.722 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_object_lock_changing_mode_from_compliance ______ 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.723 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.724 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.725 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.726 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.727 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.728 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.729 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout:nuke_bucket test-29fkkr1kn7gqxd76i3nfcoel-552 waiting 8.305727 seconds for object locks to expire 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_copy_object_ifmatch_good ______________ 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.731 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.732 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.733 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.735 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout:___________ ERROR at teardown of test_copy_object_ifnonematch_failed ___________ 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.736 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.737 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.738 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.739 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.740 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.741 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.742 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:______________ ERROR at teardown of test_get_bucket_policy_status ______________ 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.743 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.744 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.745 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.747 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.748 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_get_public_acl_bucket_policy_status _________ 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.749 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.750 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.751 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.752 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.753 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.754 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_get_authpublic_acl_bucket_policy_status _______ 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.756 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.757 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.759 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.760 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.761 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout:_____ ERROR at teardown of test_get_publicpolicy_acl_bucket_policy_status ______ 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.762 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.763 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.764 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.765 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.766 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.768 INFO:teuthology.orchestra.run.smithi088.stdout:____ ERROR at teardown of test_get_nonpublicpolicy_acl_bucket_policy_status ____ 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.769 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.772 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.773 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.774 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:___ ERROR at teardown of test_get_nonpublicpolicy_deny_bucket_policy_status ____ 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.776 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.777 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.778 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.779 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.780 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_get_undefined_public_block _____________ 2024-04-17T05:45:49.781 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.782 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.784 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.785 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.786 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.787 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout:________ ERROR at teardown of test_get_public_block_deny_bucket_policy _________ 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.788 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.789 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.790 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.791 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.792 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.793 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.794 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout:__________________ ERROR at teardown of test_put_public_block __________________ 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.795 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.796 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.797 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.798 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.799 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.800 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_block_public_put_bucket_acls ____________ 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.801 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.802 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.803 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.804 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.805 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.806 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:__________ ERROR at teardown of test_block_public_object_canned_acls ___________ 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.808 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.809 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.810 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.812 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.813 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout:________________ ERROR at teardown of test_block_public_policy _________________ 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.814 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.815 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.816 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.817 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.818 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.819 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.820 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_ignore_public_acls _________________ 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.821 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.822 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.823 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.824 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.825 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.826 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout:______ ERROR at teardown of test_multipart_upload_on_a_bucket_with_policy ______ 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.828 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.829 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.830 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.831 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.832 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.833 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_put_bucket_encryption_kms ______________ 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.834 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.836 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.837 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.838 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.839 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout:_____________ ERROR at teardown of test_get_bucket_encryption_kms ______________ 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.840 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.841 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.842 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.843 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.844 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.845 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.846 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:____________ ERROR at teardown of test_delete_bucket_encryption_kms ____________ 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.847 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.848 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.849 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.850 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.851 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.852 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout:_ ERROR at teardown of test_sse_kms_default_post_object_authenticated_request __ 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.853 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.854 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.855 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.856 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.857 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.858 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.859 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:_________________ ERROR at teardown of test_get_object_torrent _________________ 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:configfile = 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.fixture(autouse=True) 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: def setup_teardown(configfile): 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: yield 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:> teardown() 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-17T05:45:49.860 INFO:teuthology.orchestra.run.smithi088.stdout: raise err 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout: nuke_bucket(client, bucket_name) 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout: listing = client.list_object_versions(**kwargs) 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:self = 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:operation_name = 'ListObjectVersions' 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout:api_params = {'Bucket': 'test-29fkkr1kn7gqxd76i3nfcoel-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-17T05:45:49.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: service_name = self._service_model.service_name 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: history_recorder.record( 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: 'API_CALL', 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: 'service': service_name, 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: 'operation': operation_name, 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: 'params': api_params, 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: if operation_model.deprecated: 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: logger.debug( 2024-04-17T05:45:49.862 INFO:teuthology.orchestra.run.smithi088.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: request_context = { 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_region': self.meta.region_name, 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: 'client_config': self.meta.config, 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: 'auth_type': operation_model.auth_type, 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: request_dict = self._convert_to_request_dict( 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: api_params, operation_model, context=request_context 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.863 INFO:teuthology.orchestra.run.smithi088.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: params=request_dict, 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: request_signer=self._request_signer, 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: if event_response is not None: 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = event_response 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: else: 2024-04-17T05:45:49.864 INFO:teuthology.orchestra.run.smithi088.stdout: apply_request_checksum(request_dict) 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: http, parsed_response = self._make_request( 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: operation_model, request_dict, request_context 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: self.meta.events.emit( 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: service_id=service_id, operation_name=operation_name 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: ), 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: http_response=http, 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: parsed=parsed_response, 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: model=operation_model, 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: context=request_context, 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: ) 2024-04-17T05:45:49.865 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: if http.status_code >= 300: 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout:> raise error_class(parsed_response, operation_name) 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout:=============================== warnings summary =============================== 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: import imp 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py:120 2024-04-17T05:45:49.866 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:120: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: if boto_type is 'S3Connection': 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_headers.py:124 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:124: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: elif boto_type is 'HTTPRequest': 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3.py:359 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3.py:359: DeprecationWarning: invalid escape sequence '\W' 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: extra_sc = re.split('\W+', config['main']['storage_classes']) 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py:176 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:176: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-04-17T05:45:49.867 INFO:teuthology.orchestra.run.smithi088.stdout: if k is 'BucketName': 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py:253 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:253: PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.mark.timeout(10) 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests/functional/test_s3_website.py:1077 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: s3tests/functional/test_s3_website.py:1077: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_routing_generator will be ignored 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: @pytest.mark.s3website_routing_rules 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2213 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2213: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2228 2024-04-17T05:45:49.868 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2228: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2234 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2234: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: response = client.get_object(Bucket=bucket_name, Key='\$foo.txt') 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2291 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2291: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2306 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2306: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-04-17T05:45:49.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2324 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2324: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2339 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2339: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id[::-1]),\ 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2357 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2357: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.870 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2372 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2372: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2422 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2422: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout:s3tests_boto3/functional/test_s3.py:2454 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2454: DeprecationWarning: invalid escape sequence '\$' 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout:=========================== short test summary info ============================ 2024-04-17T05:45:49.871 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_delete - botoco... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_read_not_exist - botoc... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete - botocor... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete - botoc... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes - boto... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_check_etag - bot... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_cache_control - ... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_expires - botoco... 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good 2024-04-17T05:45:49.872 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_file - botocore.... 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header - ... 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields 2024-04-17T05:45:49.873 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy - ... 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket - bo... 2024-04-17T05:45:49.874 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good - bot... 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed - b... 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good 2024-04-17T05:45:49.875 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good - bot... 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed - b... 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good - ... 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get - botocore.err... 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone - ... 2024-04-17T05:45:49.876 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone - ... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head - botocore.errorf... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist - botoco... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_extended - botoco... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl - b... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl - b... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime - botoco... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated - bo... 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone 2024-04-17T05:45:49.877 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put - botocore.er... 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_authenticated - bo... 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one 2024-04-17T05:45:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name - botoc... 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists - botoco... 2024-04-17T05:45:49.879 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_get_location - botocor... 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl - bot... 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_default - botocore... 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned - botocore.... 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_default - botocore... 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned - botocore.... 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread 2024-04-17T05:45:49.880 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl - botocore.errorfa... 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_write - botocore.e... 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp - botocor... 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_read - botocore.er... 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_readacp - botocore... 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp 2024-04-17T05:45:49.881 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants - bo... 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants - bo... 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email - boto... 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all - botoc... 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite 2024-04-17T05:45:49.882 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list - bo... 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime - botocore... 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous - boto... 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth - b... 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth - botoc... 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period 2024-04-17T05:45:49.883 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix - ... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size - botoc... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_16m - botocore.er... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket - bot... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself - botoc... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket - bot... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl - boto... 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata 2024-04-17T05:45:49.884 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found - b... 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty - boto... 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_small - boto... 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_small - botoco... 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload - botocore.e... 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned - bo... 2024-04-17T05:45:49.885 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents - b... 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload - boto... 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload - botoc... 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_get_part - botocore... 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part - boto... 2024-04-17T05:45:49.886 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_100_continue - botocore.error... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_cors - botocore.errorfact... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_response - botoco... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard - botoco... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_header_option - botocore... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object - b... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object - b... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging - botocore... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb - botocore.er... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb - botocore.er... 2024-04-17T05:45:49.887 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb - botocore.er... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb - botocore.e... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb - botocore.e... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb - botocore.e... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb - botoc... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb - botoc... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb - botoc... 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code 2024-04-17T05:45:49.888 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker - ... 2024-04-17T05:45:49.889 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl - botoco... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set - botocore.erro... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get - botocore.erro... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id - botocor... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long - botoc... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id - botocore.... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status - bo... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date - botocore... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date - ... 2024-04-17T05:45:49.890 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent - bo... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker - ... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter - botoco... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter - ... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart - bot... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b - botoc... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb - boto... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB - boto... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b - boto... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present - bo... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key - ... 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 2024-04-17T05:45:49.891 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 - bot... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key - bot... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c - bot... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head - botocor... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_present - botocore.er... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key - botocore.err... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared - botoco... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload - bo... 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request 2024-04-17T05:45:49.892 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b - botocor... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb - botoco... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB - botoco... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b - botoco... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare - botoco... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy - botocore.erro... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy - botocore.er... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl - botocore.... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl - botocor... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_tagging - botocore.er... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging - botoco... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_tags - botocore.error... 2024-04-17T05:45:49.893 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_tags - botocore.er... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags - botocor... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags - botocor... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags - botocor... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_modify_tags - botocore.er... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_delete_tags - botocore.er... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags - botocore.... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public - botocor... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public - botocor... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public - boto... 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag 2024-04-17T05:45:49.894 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl - b... 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 - b... 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms - ... 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag 2024-04-17T05:45:49.895 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock - bo... 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock - bo... 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention 2024-04-17T05:45:49.896 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold - ... 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket 2024-04-17T05:45:49.897 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold - ... 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj - b... 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good - bo... 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status - bo... 2024-04-17T05:45:49.898 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block - ... 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_public_block - botocore.e... 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_policy - botocor... 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ignore_public_acls - botocore... 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms - b... 2024-04-17T05:45:49.899 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms - b... 2024-04-17T05:45:49.900 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms 2024-04-17T05:45:49.900 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request 2024-04-17T05:45:49.900 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_torrent - botocore... 2024-04-17T05:45:49.900 INFO:teuthology.orchestra.run.smithi088.stdout:= 666 passed, 65 skipped, 127 deselected, 1 xfailed, 18 warnings, 397 errors in 2171.18s (0:36:11) = 2024-04-17T05:45:49.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:45:49.902 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR: InvocationError for command /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' (exited with code 1) 2024-04-17T05:45:49.902 INFO:teuthology.orchestra.run.smithi088.stdout:___________________________________ summary ____________________________________ 2024-04-17T05:45:49.902 INFO:teuthology.orchestra.run.smithi088.stdout:ERROR: py: commands failed 2024-04-17T05:45:49.902 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi088 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-04-17T05:45:49.904 INFO:tasks.s3tests:Cleaning up boto... 2024-04-17T05:45:49.904 DEBUG:teuthology.orchestra.run.smithi088:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-04-17T05:45:49.911 INFO:tasks.s3tests:Removing s3-tests... 2024-04-17T05:45:49.911 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-04-17T05:45:50.101 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi088 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-04-17T05:45:54.284 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ca2c46e2ffa94cc5b9f7c43ac858b932 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph_41c500dd60c93c8d83053b72cffc3dd1a952a336/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi088 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-04-17T05:45:54.288 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-04-17T05:45:54.300 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-04-17T05:45:54.337 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-04-17T05:45:54.337 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-04-17T05:45:54.337 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-04-17T05:45:54.393 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-04-17T05:45:54.393 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-04-17T05:45:54.472 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:45:54.473 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:45:54.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.512+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 realm 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:45:54.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:45:54.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.516+0000 7f378d20c980 20 searching for the correct realm 2024-04-17T05:45:54.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.524+0000 7f378a526640 0 -- 172.21.15.88:0/676960925 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f377c0562a0 msgr2=0x7f377c0579e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:45:54.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.532+0000 7f378a526640 0 -- 172.21.15.88:0/676960925 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f3654077100 msgr2=0x7f3654079530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:45:54.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:45:54.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:45:54.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:45:54.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:45:54.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:45:54.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:45:54.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:45:54.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:45:54.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 zone default found 2024-04-17T05:45:54.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 4 Realm: () 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 zonegroup default 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:45:54.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 started sync module instance, tier type = 2024-04-17T05:45:54.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.540+0000 7f378d20c980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:45:54.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=0 2024-04-17T05:45:54.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=1 2024-04-17T05:45:54.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=2 2024-04-17T05:45:54.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=3 2024-04-17T05:45:54.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=4 2024-04-17T05:45:54.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=5 2024-04-17T05:45:54.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=6 2024-04-17T05:45:54.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 add_watcher() i=7 2024-04-17T05:45:54.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 2 all 8 watchers are set, enabling cache 2024-04-17T05:45:54.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f36417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:45:54.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:45:54.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.560+0000 7f378d20c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:45:54.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f35bbfff640 20 reqs_thread_entry: start 2024-04-17T05:45:54.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 20 init_complete bucket index max shards: 1 2024-04-17T05:45:54.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 20 Filter name: none 2024-04-17T05:45:54.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f35b9ffb640 20 reqs_thread_entry: start 2024-04-17T05:45:54.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-04-17T05:45:54.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 20 rados->read ofs=0 len=0 2024-04-17T05:45:54.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 20 rados_obj.operate() r=0 bl.length=482 2024-04-17T05:45:54.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x13 2024-04-17T05:45:54.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-17T05:45:54.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-17T05:45:54.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.644+0000 7f378d20c980 10 removing key index: ONWMPQGBYPSBLBMIEXSY 2024-04-17T05:45:54.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.648+0000 7f378d20c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:ONWMPQGBYPSBLBMIEXSY, ofs0, ns] 2024-04-17T05:45:54.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.648+0000 7f366e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366581 cookie 94290567585904 notifier 7490 bl.length()=170 2024-04-17T05:45:54.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.648+0000 7f378d20c980 10 removing email index: tester.client.0_test@test.test 2024-04-17T05:45:54.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.652+0000 7f378d20c980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-04-17T05:45:54.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.652+0000 7f366dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366435 cookie 94290567571168 notifier 7490 bl.length()=181 2024-04-17T05:45:54.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.652+0000 7f378d20c980 10 removing user buckets index 2024-04-17T05:45:54.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.656+0000 7f378d20c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0.buckets, ofs0, ns] 2024-04-17T05:45:54.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.656+0000 7f366dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871367513 cookie 94290567593504 notifier 7490 bl.length()=172 2024-04-17T05:45:54.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.660+0000 7f378d20c980 10 removing user index: tester.client.0 2024-04-17T05:45:54.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.660+0000 7f378d20c980 10 removing default.rgw.meta+users.uid+tester.client.0 from cache 2024-04-17T05:45:54.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.660+0000 7f378d20c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-04-17T05:45:54.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.664+0000 7f366e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366582 cookie 94290567585904 notifier 7490 bl.length()=164 2024-04-17T05:45:54.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.664+0000 7f36417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:45:54.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.672+0000 7f378d20c980 20 remove_watcher() i=0 2024-04-17T05:45:54.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.672+0000 7f378d20c980 2 removed watcher, disabling cache 2024-04-17T05:45:54.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.672+0000 7f378d20c980 20 remove_watcher() i=1 2024-04-17T05:45:54.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.676+0000 7f378d20c980 20 remove_watcher() i=2 2024-04-17T05:45:54.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.680+0000 7f378d20c980 20 remove_watcher() i=3 2024-04-17T05:45:54.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.680+0000 7f378d20c980 20 remove_watcher() i=4 2024-04-17T05:45:54.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.684+0000 7f378d20c980 20 remove_watcher() i=5 2024-04-17T05:45:54.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.684+0000 7f378d20c980 20 remove_watcher() i=6 2024-04-17T05:45:54.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:45:54.688+0000 7f378d20c980 20 remove_watcher() i=7 2024-04-17T05:45:54.705 DEBUG:teuthology.run_tasks:Unwinding manager ragweed 2024-04-17T05:45:54.761 DEBUG:teuthology.run_tasks:Unwinding manager tox 2024-04-17T05:45:54.773 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/tox-venv 2024-04-17T05:45:54.817 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-17T05:45:54.828 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-04-17T05:45:54.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:45:54.871 INFO:tasks.rgw.client.0.smithi088.stdout:2024-04-17T05:45:54.864+0000 e52e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 37531) UID: 0 2024-04-17T05:45:54.871 INFO:tasks.rgw.client.0.smithi088.stdout:2024-04-17T05:45:54.868+0000 8e96e80 -1 shutting down 2024-04-17T05:45:59.183 INFO:tasks.rgw.client.0.smithi088.stderr:==00:01:09:20.670 37533== could not unlink /tmp/vgdb-pipe-from-vgdb-to-37533-by-root-on-??? 2024-04-17T05:45:59.184 INFO:tasks.rgw.client.0.smithi088.stderr:==00:01:09:20.670 37533== could not unlink /tmp/vgdb-pipe-to-vgdb-from-37533-by-root-on-??? 2024-04-17T05:45:59.184 INFO:tasks.rgw.client.0.smithi088.stderr:==00:01:09:20.670 37533== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-37533-by-root-on-??? 2024-04-17T05:46:00.930 INFO:tasks.rgw.client.0:Stopped 2024-04-17T05:46:00.930 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-04-17T05:46:00.937 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/vault-root-token 2024-04-17T05:46:00.994 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-17T05:46:01.049 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-04-17T05:46:01.049 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-04-17T05:46:01.049 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-04-17T05:46:01.128 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setuser ceph since I am not root 2024-04-17T05:46:01.128 INFO:teuthology.orchestra.run.smithi088.stderr:ignoring --setgroup ceph since I am not root 2024-04-17T05:46:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.168+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 realm 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:46:01.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 rados_obj.operate() r=0 bl.length=1139 2024-04-17T05:46:01.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.172+0000 7f039ff7f980 20 searching for the correct realm 2024-04-17T05:46:01.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.180+0000 7f039d299640 0 -- 172.21.15.88:0/3940810918 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f039805d040 msgr2=0x7f039805f7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:46:01.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.184+0000 7f039d299640 0 -- 172.21.15.88:0/3940810918 >> [v2:172.21.15.154:6808/2413638230,v1:172.21.15.154:6810/2413638230] conn(0x7f0268077100 msgr2=0x7f0268079530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:46:01.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 RGWRados::pool_iterate: got zonegroup_info.e86b0663-24e2-4326-bb34-b167c7a25311 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 RGWRados::pool_iterate: got zone_info.52c3b34b-e732-40a8-b5aa-dfbef5c9e391 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 RGWRados::pool_iterate: got default.zone. 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 RGWRados::pool_iterate: got zone_names.default 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 rados_obj.operate() r=0 bl.length=46 2024-04-17T05:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.192+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados_obj.operate() r=0 bl.length=470 2024-04-17T05:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 zone default found 2024-04-17T05:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 4 Realm: () 2024-04-17T05:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 4 ZoneGroup: default (e86b0663-24e2-4326-bb34-b167c7a25311) 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 4 Zone: default (52c3b34b-e732-40a8-b5aa-dfbef5c9e391) 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 zonegroup default 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados->read ofs=0 len=0 2024-04-17T05:46:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 rados_obj.operate() r=-2 bl.length=0 2024-04-17T05:46:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 started sync module instance, tier type = 2024-04-17T05:46:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.196+0000 7f039ff7f980 20 started zone id=52c3b34b-e732-40a8-b5aa-dfbef5c9e391 (name=default) with tier type = 2024-04-17T05:46:01.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=0 2024-04-17T05:46:01.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=1 2024-04-17T05:46:01.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=2 2024-04-17T05:46:01.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=3 2024-04-17T05:46:01.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=4 2024-04-17T05:46:01.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=5 2024-04-17T05:46:01.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.212+0000 7f039ff7f980 20 add_watcher() i=6 2024-04-17T05:46:01.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.216+0000 7f039ff7f980 20 add_watcher() i=7 2024-04-17T05:46:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.216+0000 7f039ff7f980 2 all 8 watchers are set, enabling cache 2024-04-17T05:46:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.216+0000 7f024f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:46:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.216+0000 7f039ff7f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-17T05:46:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.216+0000 7f039ff7f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-17T05:46:01.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.304+0000 7f024e7fc640 20 reqs_thread_entry: start 2024-04-17T05:46:01.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.304+0000 7f039ff7f980 20 init_complete bucket index max shards: 1 2024-04-17T05:46:01.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.304+0000 7f039ff7f980 20 Filter name: none 2024-04-17T05:46:01.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.304+0000 7f01bffff640 20 reqs_thread_entry: start 2024-04-17T05:46:01.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-17T05:46:01.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45465' 2024-04-17T05:46:01.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9390464709162160050', time=2024-04-17T07:10:35.313178+0000, chain.objs.size()=1 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.156__shadow_.vJQSxem0MksTN5Us65YO5VD-7qHGJLO_0 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5222453188449370937', time=2024-04-17T07:11:02.087951+0000, chain.objs.size()=1 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.172__shadow_.jOskjF6b8s06jbrW2ESedxgTO9__ax9_0 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2774048128650700102', time=2024-04-17T07:11:36.001775+0000, chain.objs.size()=1 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.189__shadow_.c6TRLbzkd5HtbmNUd_XU9nwQ3XXT6n2_0 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12798840959013736938', time=2024-04-17T07:14:51.248239+0000, chain.objs.size()=1 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.234__shadow_.9-AAx1lPos2MQSLmba1cL_6XNtSlzXs_0 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9101650085513565470', time=2024-04-17T07:15:04.278837+0000, chain.objs.size()=1 2024-04-17T05:46:01.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.242__shadow_.ysKQpies4v6wL_bFExzBhh9dCiL4Avk_0 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16707146243312437693', time=2024-04-17T07:17:02.094646+0000, chain.objs.size()=1 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.IC_jBwOl8VeGWrq1qc2NJ9X5kLcWvQ7_0 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16961681940738018184', time=2024-04-17T07:17:03.958857+0000, chain.objs.size()=1 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wI5HUNaWn2dkQNS534xgcqpB8MEr6Ah_0 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5195593292171836007', time=2024-04-17T07:17:04.210748+0000, chain.objs.size()=1 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-Vq6FDbH3u-hBA1gy6RS_2LBJFzCJxB_0 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6884063137573490042', time=2024-04-17T07:17:06.036511+0000, chain.objs.size()=1 2024-04-17T05:46:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GZvJ14O1cEuNxPNlyEbpUprvjI3wceS_0 2024-04-17T05:46:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7132712005824432716', time=2024-04-17T07:17:08.492642+0000, chain.objs.size()=1 2024-04-17T05:46:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TGx0W7w5o6H99Cqj1mGtlaYpg66UDv4_0 2024-04-17T05:46:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2686562460615874184', time=2024-04-17T07:17:08.494223+0000, chain.objs.size()=1 2024-04-17T05:46:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4YxrbcwiXp70dvvj5aPTnC5OqHmIYLh_0 2024-04-17T05:46:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2941424782122870516', time=2024-04-17T07:17:08.673353+0000, chain.objs.size()=1 2024-04-17T05:46:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Pl3wQe2kx0LehoOfhEIY3J1IW2hTmq5_0 2024-04-17T05:46:01.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3591277332276973501', time=2024-04-17T07:17:10.665691+0000, chain.objs.size()=1 2024-04-17T05:46:01.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bW8CLKbx25Rk2aIDA-l3OhjdeyWlx_n_0 2024-04-17T05:46:01.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.273241137195020048', time=2024-04-17T07:17:10.882879+0000, chain.objs.size()=1 2024-04-17T05:46:01.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sYym6V8F67NcW5pGqCXVEISlwzksI0I_0 2024-04-17T05:46:01.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6187487851708200090', time=2024-04-17T07:17:11.434833+0000, chain.objs.size()=1 2024-04-17T05:46:01.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aaR_M39Z--cOCrgGAgper6XpX6u3y_O_0 2024-04-17T05:46:01.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3045634234491502180', time=2024-04-17T07:17:11.658263+0000, chain.objs.size()=1 2024-04-17T05:46:01.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.asKlsTiXQRZHkYPXrg_t_dJzCv8cGKj_0 2024-04-17T05:46:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10165084273957627207', time=2024-04-17T07:17:11.883974+0000, chain.objs.size()=1 2024-04-17T05:46:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QXdQEKmSwfF1_IUAsz9OnD4pRzH8XI6_0 2024-04-17T05:46:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10353925497256407362', time=2024-04-17T07:17:12.117682+0000, chain.objs.size()=1 2024-04-17T05:46:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0cvtJ0N6plHvXU1IonO7KFVoWHx9ach_0 2024-04-17T05:46:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14439632720915146550', time=2024-04-17T07:17:12.365833+0000, chain.objs.size()=1 2024-04-17T05:46:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lAS8-pMmMd0Jel9QowKJNI4SMQeLBKK_0 2024-04-17T05:46:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7079404840448903635', time=2024-04-17T07:17:12.684377+0000, chain.objs.size()=1 2024-04-17T05:46:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4cVMV1i4nDFzqBvfihaNkLJN2XbLB7F_0 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17795236616756525830', time=2024-04-17T07:17:14.112813+0000, chain.objs.size()=1 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MO2O4d2Mupg7AdIeH69fux3bpgAbz3P_0 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6313904256220092201', time=2024-04-17T07:17:14.887814+0000, chain.objs.size()=1 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GQhSJ-WM2L05M_I4Ii9NqpI3Q2HjxCr_0 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6521132782128669522', time=2024-04-17T07:17:15.683548+0000, chain.objs.size()=1 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.p4jRC1gRxfVjqOqnVAd0wb1PL7eA3bu_0 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7033352732621714479', time=2024-04-17T07:17:16.261141+0000, chain.objs.size()=1 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.h7-Iq1ytE4d3A6IwLf09PyX3G1TT5JO_0 2024-04-17T05:46:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13726654092143662505', time=2024-04-17T07:17:16.362914+0000, chain.objs.size()=1 2024-04-17T05:46:01.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ry1ficeu4SQAMUYJjlE7o0FFyusYp05_0 2024-04-17T05:46:01.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15949677882013454131', time=2024-04-17T07:17:16.594629+0000, chain.objs.size()=1 2024-04-17T05:46:01.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wbuaO3Q_zrW2aXilvjtU01iqT7lw3iG_0 2024-04-17T05:46:01.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5064120475435510261', time=2024-04-17T07:17:18.865817+0000, chain.objs.size()=1 2024-04-17T05:46:01.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VD7J1VSnKLNd7ae_A5cE43Y_IIfm7p4_0 2024-04-17T05:46:01.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13490899520993470297', time=2024-04-17T07:17:19.666590+0000, chain.objs.size()=1 2024-04-17T05:46:01.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.E8i8_q7H4cIH3WF1YgsZ0FKemCc-QX-_0 2024-04-17T05:46:01.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2360815394759373873', time=2024-04-17T07:17:21.990997+0000, chain.objs.size()=1 2024-04-17T05:46:01.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c3pp33tRg5enhQxDcXDQKgCr1pytume_0 2024-04-17T05:46:01.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18440124077142959005', time=2024-04-17T07:17:22.441904+0000, chain.objs.size()=1 2024-04-17T05:46:01.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xdpWBD8o_XIkdfxkMiNMwivQHDupSZL_0 2024-04-17T05:46:01.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13401684874573811001', time=2024-04-17T07:17:22.706838+0000, chain.objs.size()=1 2024-04-17T05:46:01.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.WmBiwfLpWsQWFg6nwgZ6Oe6TI6fWejg_0 2024-04-17T05:46:01.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4934124421745814855', time=2024-04-17T07:17:22.771181+0000, chain.objs.size()=1 2024-04-17T05:46:01.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VzGLk0YD8g6H8-_RBo4F5C_HOSoRBjU_0 2024-04-17T05:46:01.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7631783850433171237', time=2024-04-17T07:17:22.885639+0000, chain.objs.size()=1 2024-04-17T05:46:01.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ix29lNzp3wsY8bOJiRW1sKlHqmeH0js_0 2024-04-17T05:46:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6636624488907901836', time=2024-04-17T07:17:23.535809+0000, chain.objs.size()=1 2024-04-17T05:46:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dgydPwkG0XIbOu2P6qlLYRReleocYaP_0 2024-04-17T05:46:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11458796789283836504', time=2024-04-17T07:17:24.055680+0000, chain.objs.size()=1 2024-04-17T05:46:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.R1KdCF_HYICd8t39aIlknzoJeyLSLNQ_0 2024-04-17T05:46:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13338321501451374348', time=2024-04-17T07:17:24.189495+0000, chain.objs.size()=1 2024-04-17T05:46:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jm694pPOUH4UBKLC_3P66uIlOH6Xuu__0 2024-04-17T05:46:01.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.729989523178336819', time=2024-04-17T07:17:24.743781+0000, chain.objs.size()=1 2024-04-17T05:46:01.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5wKeNdgxaxYLuaurgymC-d9ZiVlZ-EM_0 2024-04-17T05:46:01.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9670427277960438606', time=2024-04-17T07:17:26.030991+0000, chain.objs.size()=1 2024-04-17T05:46:01.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TO4TzvcdXKLXtjm5fnrV-bvZWis2Vf5_0 2024-04-17T05:46:01.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4274857717087774490', time=2024-04-17T07:17:26.159860+0000, chain.objs.size()=1 2024-04-17T05:46:01.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.n1w2EksPEUVZMCtay-aad854ywQD0xc_0 2024-04-17T05:46:01.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7882414472904768560', time=2024-04-17T07:17:27.121313+0000, chain.objs.size()=1 2024-04-17T05:46:01.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lUjYZp9QqW5rEhMS4R66QVR6JAPcwC2_0 2024-04-17T05:46:01.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10018568890862271179', time=2024-04-17T07:17:29.232253+0000, chain.objs.size()=1 2024-04-17T05:46:01.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._l6nnbe67qW9i5AhEcJf9vJm79QCj3-_0 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12647904948383111967', time=2024-04-17T07:17:29.929666+0000, chain.objs.size()=1 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wuemC3gnQC-mn4wS4_wM2hokDQuk5ue_0 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12562831389364737798', time=2024-04-17T07:17:31.199902+0000, chain.objs.size()=1 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.m0uo3zaQXV8SwoaLR0q96LNmT2nWR5S_0 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6136285655539333792', time=2024-04-17T07:17:31.962070+0000, chain.objs.size()=1 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.S__X78_z6PpWFQHtVNfBotGKRWVC0jH_0 2024-04-17T05:46:01.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1070202051154597751', time=2024-04-17T07:18:27.984431+0000, chain.objs.size()=1 2024-04-17T05:46:01.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.gJmd9dKUNcAXZd5kNef9rE9k2rf2YEZ_0 2024-04-17T05:46:01.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12597615493594332412', time=2024-04-17T07:18:27.986324+0000, chain.objs.size()=1 2024-04-17T05:46:01.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.Zaj5FO-JstT5brS8h7I6AAffmRWo1iX_0 2024-04-17T05:46:01.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13423538129097444636', time=2024-04-17T07:18:36.601111+0000, chain.objs.size()=1 2024-04-17T05:46:01.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.288__shadow_.n-eafI-KRkqFoiLTNh3Ghunk5jMCBM9_0 2024-04-17T05:46:01.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10816022351230764862', time=2024-04-17T07:18:43.319763+0000, chain.objs.size()=1 2024-04-17T05:46:01.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.292__shadow_.qOqva-sbbI5ZcB67YfKGKOWeaR9I-qK_0 2024-04-17T05:46:01.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18001772440623948953', time=2024-04-17T07:21:00.305308+0000, chain.objs.size()=1 2024-04-17T05:46:01.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GFNDQMuJr4sLrq03ZlnzsSZXl8hr-fu_0 2024-04-17T05:46:01.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.951282512742070171', time=2024-04-17T07:21:02.783441+0000, chain.objs.size()=1 2024-04-17T05:46:01.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aj6PGUNr7PS5FHvlhEHIlwHe-uzLLE4_0 2024-04-17T05:46:01.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14464451667122039886', time=2024-04-17T07:21:02.785490+0000, chain.objs.size()=1 2024-04-17T05:46:01.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HMwGDZFkxvpEFc6zKhrq-Wmsyv4LmPV_0 2024-04-17T05:46:01.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.617494936751510667', time=2024-04-17T07:21:04.004352+0000, chain.objs.size()=1 2024-04-17T05:46:01.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t7FlkVwvMXl2sF6Y0Msa4hguTUE2mTk_0 2024-04-17T05:46:01.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15307158912761771822', time=2024-04-17T07:21:04.582786+0000, chain.objs.size()=1 2024-04-17T05:46:01.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.29Bp44B_5jfAGw6x5fSKkCrQd_7qmA4_0 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4330153263870356178', time=2024-04-17T07:21:05.560814+0000, chain.objs.size()=1 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nPQkIBOKzTcAsewtAK1yKwfOvOeY2Zf_0 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1223856593346345747', time=2024-04-17T07:21:05.562976+0000, chain.objs.size()=1 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.n60BwjPDgCmzKivMMQg7gog1bNyaiEQ_0 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2539618908880165404', time=2024-04-17T07:21:05.887556+0000, chain.objs.size()=1 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.n0ag7bK2ak1YtdKpqbqOfc1Ml8hP0NP_0 2024-04-17T05:46:01.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2428644200866547667', time=2024-04-17T07:21:08.051481+0000, chain.objs.size()=1 2024-04-17T05:46:01.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5l7twovgeN4csLxIFpDILdugye2xvd0_0 2024-04-17T05:46:01.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17721668368725821321', time=2024-04-17T07:21:10.417572+0000, chain.objs.size()=1 2024-04-17T05:46:01.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DBZp4ro9h1JfJh4B75u_9h-lqnvjUce_0 2024-04-17T05:46:01.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4024766654105027029', time=2024-04-17T07:21:10.667879+0000, chain.objs.size()=1 2024-04-17T05:46:01.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.x2_E9G61tVTNeJ8Wx284J5WCxF3S_Um_0 2024-04-17T05:46:01.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15045794971355401509', time=2024-04-17T07:21:10.785616+0000, chain.objs.size()=1 2024-04-17T05:46:01.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.OWtoenGuMpUvDv6Ho4O8IxAHpcc9RbT_0 2024-04-17T05:46:01.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11701094684792199953', time=2024-04-17T07:21:11.822280+0000, chain.objs.size()=1 2024-04-17T05:46:01.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0d1Qlvs-CU6kNxCKFMh_gAECSmeh9YY_0 2024-04-17T05:46:01.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13044569789293550238', time=2024-04-17T07:21:12.983333+0000, chain.objs.size()=1 2024-04-17T05:46:01.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XlQJusVjBaKmpT2ZscpQFQNjKVQm0Wg_0 2024-04-17T05:46:01.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7211783981833945754', time=2024-04-17T07:21:13.556446+0000, chain.objs.size()=1 2024-04-17T05:46:01.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.emhuxWXtVz6WlGhZj6emtjKjOoPNWd0_0 2024-04-17T05:46:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3026894600072366310', time=2024-04-17T07:21:13.993801+0000, chain.objs.size()=1 2024-04-17T05:46:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tXO3FT_3KRLOvmYrRfGObVall3g77D8_0 2024-04-17T05:46:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3884306919489936860', time=2024-04-17T07:21:14.631185+0000, chain.objs.size()=1 2024-04-17T05:46:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LhjYi19hyaBP37ZvBUHSIY7xIWX-XIH_0 2024-04-17T05:46:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16248036064248266157', time=2024-04-17T07:21:15.872314+0000, chain.objs.size()=1 2024-04-17T05:46:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HnaSPDDvJg1ExsJh7BWjTefeBKWgIqg_0 2024-04-17T05:46:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3948299484475130689', time=2024-04-17T07:21:16.857895+0000, chain.objs.size()=1 2024-04-17T05:46:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7XfMRMv0zR7sIQ32yjaUzP-vHnDLrYh_0 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2053227796747110599', time=2024-04-17T07:21:17.264039+0000, chain.objs.size()=1 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.L2BazL0qOGRqMBUPTf0hL9iOJLAELuf_0 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13936431608747616214', time=2024-04-17T07:21:18.531620+0000, chain.objs.size()=1 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CwWrcpQahF7qTv3oSqdJe4Wu_9cFznb_0 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5982616699324545260', time=2024-04-17T07:21:18.724537+0000, chain.objs.size()=1 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dBBCIc6o0SF8dXJPAFWXCkQc32MvFbA_0 2024-04-17T05:46:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.975233976140940545', time=2024-04-17T07:21:20.081973+0000, chain.objs.size()=1 2024-04-17T05:46:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EuDbpoobDO7neIvk6CFnbeb856u3H8s_0 2024-04-17T05:46:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15190205018021517291', time=2024-04-17T07:21:20.675280+0000, chain.objs.size()=1 2024-04-17T05:46:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IsqbAKl7H8FWDi_lCuL_0RZDCSxXAS0_0 2024-04-17T05:46:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7659579657473197133', time=2024-04-17T07:21:22.174826+0000, chain.objs.size()=1 2024-04-17T05:46:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.M2sriPR1yAArHUVW2y5VTdFnTK3kHbT_0 2024-04-17T05:46:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11912628109436634047', time=2024-04-17T07:21:22.353225+0000, chain.objs.size()=1 2024-04-17T05:46:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZxsP5jjEn_Yi3M-wRlAOj-rajp7Apak_0 2024-04-17T05:46:01.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17985454944048380342', time=2024-04-17T07:21:23.293265+0000, chain.objs.size()=1 2024-04-17T05:46:01.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.S8thQ_Os7vxdfM1U0mjzVBiQzD-HNPK_0 2024-04-17T05:46:01.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1528865350406952740', time=2024-04-17T07:21:23.769471+0000, chain.objs.size()=1 2024-04-17T05:46:01.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mQE21LLR_UdkzMcIjx06Jf_oh2lie_U_0 2024-04-17T05:46:01.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3889509724672751052', time=2024-04-17T07:21:25.083776+0000, chain.objs.size()=1 2024-04-17T05:46:01.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.p70_drGjPe1PQ1y-ggmn9lzbUSU2YMs_0 2024-04-17T05:46:01.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12519936324101206067', time=2024-04-17T07:21:25.684348+0000, chain.objs.size()=1 2024-04-17T05:46:01.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.G3YjHTiHiREbqOfgu6IzSDnFzthOHlI_0 2024-04-17T05:46:01.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17443397121891932788', time=2024-04-17T07:21:28.223468+0000, chain.objs.size()=1 2024-04-17T05:46:01.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1R8D4UjKarCfuvb65v-x1w2QWi5IV6L_0 2024-04-17T05:46:01.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10487224778160613615', time=2024-04-17T07:21:28.497525+0000, chain.objs.size()=1 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.96J2FX9hulsWOJnvEhZojV9TaYUesb6_0 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1468853327836678211', time=2024-04-17T07:21:28.947451+0000, chain.objs.size()=1 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2c53zT6E0LZRBVEwAJJMAyw7mPfPFhY_0 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6123485786181002189', time=2024-04-17T07:23:14.353358+0000, chain.objs.size()=1 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.zKJQzL-ZUOCzOggouEB18hJm6cp_9FG_0 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4470975975454547378', time=2024-04-17T07:23:15.180999+0000, chain.objs.size()=1 2024-04-17T05:46:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ILKflRNJm8ghieQJhDgLcomv1DBcFHo_0 2024-04-17T05:46:01.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15011973095506349376', time=2024-04-17T07:23:15.458001+0000, chain.objs.size()=1 2024-04-17T05:46:01.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7YIPEt-ZDYWACf7Im8-sKzDZZoQBpPb_0 2024-04-17T05:46:01.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.890880383829471862', time=2024-04-17T07:23:18.164516+0000, chain.objs.size()=1 2024-04-17T05:46:01.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gFHkE7inx0ljnMPB7jQBnVmhSG0OvtR_0 2024-04-17T05:46:01.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10992672500074684802', time=2024-04-17T07:23:19.018686+0000, chain.objs.size()=1 2024-04-17T05:46:01.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xOKax3HB4OpayZ62eDx-R8refwLwHND_0 2024-04-17T05:46:01.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12572763735728000883', time=2024-04-17T07:23:19.159488+0000, chain.objs.size()=1 2024-04-17T05:46:01.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wRdh-G4aYy2m3WyVldrVWXKhkiOnB8W_0 2024-04-17T05:46:01.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2931748649743670761', time=2024-04-17T07:23:19.723608+0000, chain.objs.size()=1 2024-04-17T05:46:01.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.zmMS1DNJ_sX6T1Lf10J_BhHHHsDPFpT_0 2024-04-17T05:46:01.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14432512655245282205', time=2024-04-17T07:23:20.836671+0000, chain.objs.size()=1 2024-04-17T05:46:01.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.T_d1rJvrmcfRsPGrTEaY6InUxjA-V6k_0 2024-04-17T05:46:01.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3494407168866966587', time=2024-04-17T07:23:21.246185+0000, chain.objs.size()=1 2024-04-17T05:46:01.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CJRY_mvknNnuQ70tK8JXWZ1SiVBaqCG_0 2024-04-17T05:46:01.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17036862562173942271', time=2024-04-17T07:23:22.141811+0000, chain.objs.size()=1 2024-04-17T05:46:01.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PJLEWQGVOTz8gzrLaJBtSuBDY5Qb7Oc_0 2024-04-17T05:46:01.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13691856475841234421', time=2024-04-17T07:23:23.012829+0000, chain.objs.size()=1 2024-04-17T05:46:01.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XkAxjuJKpLq6YxGUXR1WTPXOhWjWIly_0 2024-04-17T05:46:01.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15450585771442940649', time=2024-04-17T07:23:27.194845+0000, chain.objs.size()=1 2024-04-17T05:46:01.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DqO5JabQJL24qINDzUAbdx-aIDfgpHz_0 2024-04-17T05:46:01.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12120448834365181972', time=2024-04-17T07:23:27.385194+0000, chain.objs.size()=1 2024-04-17T05:46:01.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.V3nmtoNoDi1ojNGHlAIvp9zviVe8_IJ_0 2024-04-17T05:46:01.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9776529015385835227', time=2024-04-17T07:23:30.292578+0000, chain.objs.size()=1 2024-04-17T05:46:01.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.C7Btbbi4TDo6a19ms0HcwZ8KUqpO3dQ_0 2024-04-17T05:46:01.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13739167562251563410', time=2024-04-17T07:23:30.517882+0000, chain.objs.size()=1 2024-04-17T05:46:01.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Ak_ItXqrs53XvoBEtx7BeFq1xA6YZZT_0 2024-04-17T05:46:01.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13717209576278337418', time=2024-04-17T07:23:30.940456+0000, chain.objs.size()=1 2024-04-17T05:46:01.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.C3wBRSsQvm02CYgbanoRa3nGQ2OFrlo_0 2024-04-17T05:46:01.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5553846221977184232', time=2024-04-17T07:23:31.286180+0000, chain.objs.size()=1 2024-04-17T05:46:01.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GRlwoSKbceXCBeBw022hklGIEAWfYo2_0 2024-04-17T05:46:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16516574720916045565', time=2024-04-17T07:23:32.246225+0000, chain.objs.size()=1 2024-04-17T05:46:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2X06rKYlTRK_IBET_ofH0EZpL-tqRYj_0 2024-04-17T05:46:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2397084506289821088', time=2024-04-17T07:23:32.818941+0000, chain.objs.size()=1 2024-04-17T05:46:01.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wtQemr8ITX6STfDmHHVrHu95taTd6Tr_0 2024-04-17T05:46:01.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45465 2024-04-17T05:46:01.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=22, truncated=0, next_marker='' 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7272105165011969134', time=2024-04-17T07:23:34.660353+0000, chain.objs.size()=1 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eu16Sxs8rVQgDu-yS5iJ-MQL_gpGWOP_0 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2289092040745454323', time=2024-04-17T07:23:36.582382+0000, chain.objs.size()=1 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hOAMs1XHS_cnulx3wDLFgPBEc1GRHXV_0 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17961095391138426378', time=2024-04-17T07:23:36.878707+0000, chain.objs.size()=1 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.I5RWVzng6yYin7kKSmfaoATCgJRv1oq_0 2024-04-17T05:46:01.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7448646143522860366', time=2024-04-17T07:23:38.053903+0000, chain.objs.size()=1 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JFB2kuHf5wds14gy0_ocZ-J1yPpdARF_0 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6739113400420996765', time=2024-04-17T07:23:39.527445+0000, chain.objs.size()=1 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Y4h_jJL_BpUjArT4zpmMH7JxhtcfPB7_0 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12398014915065270289', time=2024-04-17T07:23:41.640031+0000, chain.objs.size()=1 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VhHjMQ_IlVfE7vv8DcWzZK4bX5sv1MG_0 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8991689878188999881', time=2024-04-17T07:25:52.404526+0000, chain.objs.size()=1 2024-04-17T05:46:01.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.450__shadow_.EuPKiIi5A0rwbNxsO-fQIKXjdd9G_5O_0 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.486__shadow_.WL_H6Uy_YNGNX8uK5DUJ2c6hMVkqF1L_1_4f41c87e77a2a969bc9678245f114e6f', time=2024-04-17T07:26:34.424955+0000, chain.objs.size()=1 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.486__shadow_.WL_H6Uy_YNGNX8uK5DUJ2c6hMVkqF1L_0 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.488__shadow_.iG18N--mlQ4Drns8sXiYqtuo0gsJaIT_1_5db1b7719a88f6da0ab5010828c516d4', time=2024-04-17T07:26:36.735578+0000, chain.objs.size()=1 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.488__shadow_.iG18N--mlQ4Drns8sXiYqtuo0gsJaIT_0 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15024746909241667924', time=2024-04-17T07:26:59.060627+0000, chain.objs.size()=1 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.501__multipart_mymultipart.2~XZhZ7QMbtm6I6AcHX4_Xg6PYQG76-GG.1 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14426262780381528861', time=2024-04-17T07:30:22.164361+0000, chain.objs.size()=1 2024-04-17T05:46:01.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.526__shadow_.6uUk4BTZ_Swpw3jEq6yJKH1BlPJ2ODn_0 2024-04-17T05:46:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10171797583714155663', time=2024-04-17T07:30:50.057971+0000, chain.objs.size()=1 2024-04-17T05:46:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.541__shadow_.1B3A5ssgPT5y0MQKD4WhtZQuni2X9TZ_0 2024-04-17T05:46:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4932968351384365976', time=2024-04-17T07:31:00.190287+0000, chain.objs.size()=1 2024-04-17T05:46:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.544__shadow_.s96_Ye3E2wOHtvgwEPeq-diDgcDhtCb_0 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1394218081320830710', time=2024-04-17T07:32:17.428704+0000, chain.objs.size()=1 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:CxxDs2IAl0afvquEKgpVThTsNHVJgGb_.4uBwq2JqUf_3SIx5YrE16Y51AYzRFy__0 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13438693726995334926', time=2024-04-17T07:32:36.513542+0000, chain.objs.size()=1 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:V0oWh7uyVYr5hd6o9xpK3Kzk916vVMJ_.yprkWLV9ip7hqGOhlpqocDyviRivjXx_0 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13560703031077311541', time=2024-04-17T07:32:42.598971+0000, chain.objs.size()=1 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:.iFd4AA72ZistlH7-vxVak6shQYddLp_.UXxK9zdSV0ESxnz3mnGltEZwGrHPDT-_0 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11126812911808855254', time=2024-04-17T07:32:52.788778+0000, chain.objs.size()=1 2024-04-17T05:46:01.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:g7bnEYL3rGOPraU.ukJOj2JpbwGtTJG_.L5a3qGVI1LshDhYlz6TBG7OheRAWzz4_0 2024-04-17T05:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.238793909214020703', time=2024-04-17T07:32:56.938283+0000, chain.objs.size()=1 2024-04-17T05:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:CQ5BNl9QZo1AO7-LeVsZKWABJ35GkHs_.L2zDnopT-VKH7y5tlOAQRqL9OYGD15A_0 2024-04-17T05:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1427781503342847034', time=2024-04-17T07:33:29.554860+0000, chain.objs.size()=1 2024-04-17T05:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:W5xgK3WJXpjvErEeoqmQ1jdJQ2pyOln_.KrMedMCCvCk_Ha3VY5ZFVLnmBxfqlBk_0 2024-04-17T05:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14351441577506438378', time=2024-04-17T07:34:01.966714+0000, chain.objs.size()=1 2024-04-17T05:46:01.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.594__shadow_.X1ax-frhF6sBKbbHKdATRVE4X__KtVY_0 2024-04-17T05:46:01.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13119317773140678880', time=2024-04-17T07:35:36.039590+0000, chain.objs.size()=1 2024-04-17T05:46:01.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.605__shadow_.Xmqy-dnxgh8diuJpAR8Wn2FRx6nroYi_0 2024-04-17T05:46:01.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15269291700114132473', time=2024-04-17T07:38:53.202751+0000, chain.objs.size()=1 2024-04-17T05:46:01.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.702__shadow:1gJe.3xx-ShqPqOFU.7M0z05jUM1sx4_.jXj8cyNIO0QsHETdirK5pzjK_DdJ21s_0 2024-04-17T05:46:01.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:01.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-17T05:46:01.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44506' 2024-04-17T05:46:01.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_O892fNQ1lnzYVDy7KNm4IL--ntrPOvL', time=2024-04-17T07:10:12.606691+0000, chain.objs.size()=2 2024-04-17T05:46:01.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.QS4QlzC2iHB6h71q7kF8EdG_RmXJyk7_0 2024-04-17T05:46:01.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.QS4QlzC2iHB6h71q7kF8EdG_RmXJyk7_1 2024-04-17T05:46:01.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.120029929291823890', time=2024-04-17T07:11:46.175496+0000, chain.objs.size()=1 2024-04-17T05:46:01.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.194__shadow_.tDXlRZjmPD8UD6z4lq-2BkOeMzda1sb_0 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16822496086211728021', time=2024-04-17T07:14:36.294391+0000, chain.objs.size()=1 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.226__shadow_.qOCuPnNKhOqJh4UlOVTmM4KIMBaPZpP_0 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5425940794338609216', time=2024-04-17T07:14:38.638658+0000, chain.objs.size()=1 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.227__shadow_.iDsq3QXzqeLqjf_suI-8xDHB_vtDKz0_0 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4919072189487228814', time=2024-04-17T07:14:38.668838+0000, chain.objs.size()=1 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.227__shadow_.p9xG_xhpljCPnQTmh1fx--dkZynJL3Z_0 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8399982240696233793', time=2024-04-17T07:15:05.902657+0000, chain.objs.size()=1 2024-04-17T05:46:01.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.243__shadow_.58g7i3TSBbXwQnl56PzA1sRRWL3uXzT_0 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2011520670751733461', time=2024-04-17T07:15:09.142049+0000, chain.objs.size()=1 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.245__shadow_.3fGPSCLGksArvJXzoIkQA9NStrUe6Nv_0 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9904489328841004823', time=2024-04-17T07:15:16.843453+0000, chain.objs.size()=1 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.250__shadow_.seJkFocyLJivY6WGzsJ400ShpWy_eoP_0 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14222029061967369821', time=2024-04-17T07:17:01.499360+0000, chain.objs.size()=1 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SL61b-AdZrUig6Pj4PUYkxA3rN2kExI_0 2024-04-17T05:46:01.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7033666652090450372', time=2024-04-17T07:17:02.094191+0000, chain.objs.size()=1 2024-04-17T05:46:01.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._a9aWFjIIQX2M0guZzLN-4qhbd18ZOl_0 2024-04-17T05:46:01.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10696968383584880388', time=2024-04-17T07:17:02.572048+0000, chain.objs.size()=1 2024-04-17T05:46:01.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.B1WcSlUOMagbavCFeZ_f85DSKxoRI3S_0 2024-04-17T05:46:01.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12048473612908099158', time=2024-04-17T07:17:04.282434+0000, chain.objs.size()=1 2024-04-17T05:46:01.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qbh6UEL--OGrQ07A5sx_UIzoY_KuUTz_0 2024-04-17T05:46:01.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11564884686068830346', time=2024-04-17T07:17:05.493822+0000, chain.objs.size()=1 2024-04-17T05:46:01.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wi4lyf0T_NUW06UPqXjZAwIrpbnmXOS_0 2024-04-17T05:46:01.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17195650281141316840', time=2024-04-17T07:17:06.426161+0000, chain.objs.size()=1 2024-04-17T05:46:01.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Qhb6knnpMinZfFz1-fLPcLhxA7KVIo5_0 2024-04-17T05:46:01.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3499913727251372170', time=2024-04-17T07:17:07.719645+0000, chain.objs.size()=1 2024-04-17T05:46:01.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5UnIM3f7czFukFFvOuYDVV6yCp7nR9L_0 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15426168390640771495', time=2024-04-17T07:17:07.960996+0000, chain.objs.size()=1 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RVd7bXouswHHtCIV7cPjuyJL-htVLtY_0 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13377207458248977779', time=2024-04-17T07:17:10.440406+0000, chain.objs.size()=1 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZAXjj-eA66DsFeTXkZMwEmyPJmg54tC_0 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16504583185760776751', time=2024-04-17T07:17:11.121153+0000, chain.objs.size()=1 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1yZOXdvh2_l2iKGPQg5rxI6YIdTeLey_0 2024-04-17T05:46:01.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1048161676326475608', time=2024-04-17T07:17:12.285995+0000, chain.objs.size()=1 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KyiECuts8kW2hN5m5fYQaJ01JeQVvpi_0 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8382640169506988011', time=2024-04-17T07:17:16.032268+0000, chain.objs.size()=1 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tpBNoLEuqpd044JQEbH_0PPDWU9M399_0 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8722934905948090236', time=2024-04-17T07:17:16.298562+0000, chain.objs.size()=1 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Au9dFO2SuOfCZRdpI_zw_pj5N5djxLn_0 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10407874539779869490', time=2024-04-17T07:17:16.362825+0000, chain.objs.size()=1 2024-04-17T05:46:01.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vIZxMNb3O2b1o6G__kZ8DpUnIiygmnf_0 2024-04-17T05:46:01.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17184224492095421770', time=2024-04-17T07:17:16.428014+0000, chain.objs.size()=1 2024-04-17T05:46:01.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Xwco3Kr3tJHkx5z1ZHX6AlBPOMXmRF__0 2024-04-17T05:46:01.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12748698674578501926', time=2024-04-17T07:17:16.629065+0000, chain.objs.size()=1 2024-04-17T05:46:01.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.r7RLA0MYzeLbaS_w1bdOWv9eWMSVndn_0 2024-04-17T05:46:01.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8321087955316703334', time=2024-04-17T07:17:16.955243+0000, chain.objs.size()=1 2024-04-17T05:46:01.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mVfQ1VmJm0v1T9eZYIUBLdiyhiOClSK_0 2024-04-17T05:46:01.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2893295968342497316', time=2024-04-17T07:17:20.236318+0000, chain.objs.size()=1 2024-04-17T05:46:01.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GFzlYLMyc-3U__-tE_6EGlzoqkxekyw_0 2024-04-17T05:46:01.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16258722861880725808', time=2024-04-17T07:17:23.633961+0000, chain.objs.size()=1 2024-04-17T05:46:01.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jloSXkY1C9FrTvxhoGssJp3HDJ-mU_E_0 2024-04-17T05:46:01.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10908768950843183129', time=2024-04-17T07:17:24.685999+0000, chain.objs.size()=1 2024-04-17T05:46:01.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iu1apAju28ogRpr-Q9AF-WzELOFOtah_0 2024-04-17T05:46:01.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3034959014033984652', time=2024-04-17T07:17:24.693264+0000, chain.objs.size()=1 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zPnoLwqefEMru_rp2gEKNs1XT2b_tdm_0 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18171144302407112061', time=2024-04-17T07:17:26.477755+0000, chain.objs.size()=1 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XOcJ3DBcYk3UMehhNCqK5iamteqcsK3_0 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8506017040532510404', time=2024-04-17T07:17:26.690248+0000, chain.objs.size()=1 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ajKksu172w9RHdWgyxc7fNXtzF2AgU3_0 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4851589635615628087', time=2024-04-17T07:17:28.159058+0000, chain.objs.size()=1 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.D0qOnBxCKyq9me4VFnvq5sRlFxmfXFo_0 2024-04-17T05:46:01.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4868624539077562208', time=2024-04-17T07:17:28.678585+0000, chain.objs.size()=1 2024-04-17T05:46:01.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JvhzjxLNLVNc1wAFVl0IUX3El8kV0Q8_0 2024-04-17T05:46:01.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17991243168592218211', time=2024-04-17T07:17:29.168910+0000, chain.objs.size()=1 2024-04-17T05:46:01.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tme95W2f5c2c3PDiuNMLfcwKLNXs9m0_0 2024-04-17T05:46:01.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4237997362175091868', time=2024-04-17T07:17:30.322784+0000, chain.objs.size()=1 2024-04-17T05:46:01.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9S6mx2t6ErtcievKjPDyA0rU0qOKX5O_0 2024-04-17T05:46:01.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8410566639064576239', time=2024-04-17T07:17:30.645220+0000, chain.objs.size()=1 2024-04-17T05:46:01.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6eV_GyoKDdW7v-2l8_jB890KX97usgV_0 2024-04-17T05:46:01.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15017257499263876209', time=2024-04-17T07:17:31.201676+0000, chain.objs.size()=1 2024-04-17T05:46:01.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FqczCJolrbBfbRzGhxmk8ck1YbdP3nT_0 2024-04-17T05:46:01.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1848893596786755689', time=2024-04-17T07:17:31.739218+0000, chain.objs.size()=1 2024-04-17T05:46:01.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2y5S56p_TIQgUTPkg6ig-JEb2PgN-3D_0 2024-04-17T05:46:01.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9912444540257087662', time=2024-04-17T07:17:32.223791+0000, chain.objs.size()=1 2024-04-17T05:46:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.z2F0wXjTjBAxvKdW2lOQdAcphaeVJ36_0 2024-04-17T05:46:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12948740141393083114', time=2024-04-17T07:17:32.364650+0000, chain.objs.size()=1 2024-04-17T05:46:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NimpYNuK1Gw9gQ9GPFzDfvQ_d2LGzwN_0 2024-04-17T05:46:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6722689606868005560', time=2024-04-17T07:18:01.681590+0000, chain.objs.size()=1 2024-04-17T05:46:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.271__shadow_.w_13jWB6vZkyAajCDgsJvmAhRuTrHcE_0 2024-04-17T05:46:01.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6573858133796452319', time=2024-04-17T07:18:01.708810+0000, chain.objs.size()=1 2024-04-17T05:46:01.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.271__shadow_.7lMTJWADFhJE6SNe6_ox6XdbGj-_OCg_0 2024-04-17T05:46:01.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7452867568916092223', time=2024-04-17T07:18:13.221763+0000, chain.objs.size()=1 2024-04-17T05:46:01.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.278__shadow_.Z0nHJarH3UKbf8VGAk1kRPU7lfZY5HK_0 2024-04-17T05:46:01.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9624240255564773372', time=2024-04-17T07:18:28.025951+0000, chain.objs.size()=1 2024-04-17T05:46:01.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.rstz6PAwY54RxroCCuztd8O3yOv6wfC_0 2024-04-17T05:46:01.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15755419341859093261', time=2024-04-17T07:18:33.303798+0000, chain.objs.size()=1 2024-04-17T05:46:01.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.286__shadow_.jSVL9MF-_VWupyhA1gUyuut3eiB8RY9_0 2024-04-17T05:46:01.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8935933866506752774', time=2024-04-17T07:20:59.917670+0000, chain.objs.size()=1 2024-04-17T05:46:01.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fgPITVtgGpx7hOj2CD53Vq6fcaMYa31_0 2024-04-17T05:46:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6461773653383651930', time=2024-04-17T07:21:01.867855+0000, chain.objs.size()=1 2024-04-17T05:46:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.__ALOnFlHig8uNxRu0JIqd5A5AJvarK_0 2024-04-17T05:46:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16292893027943280905', time=2024-04-17T07:21:01.870113+0000, chain.objs.size()=1 2024-04-17T05:46:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.R8vA3zfTEUnGt3tepA_YnOaIb1KNdoG_0 2024-04-17T05:46:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16667695289883118615', time=2024-04-17T07:21:02.028927+0000, chain.objs.size()=1 2024-04-17T05:46:01.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.of1acl6P7aovYaHC7AxBKbDiu7n4J88_0 2024-04-17T05:46:01.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9339597864516550585', time=2024-04-17T07:21:02.495344+0000, chain.objs.size()=1 2024-04-17T05:46:01.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.s3335tya_aVA4KVaVLGOVU4oegd3FxN_0 2024-04-17T05:46:01.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12971687841605149534', time=2024-04-17T07:21:03.876483+0000, chain.objs.size()=1 2024-04-17T05:46:01.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.qdGfLZhBQgYPDOmv1qqeN1GtHsDhCl7_0 2024-04-17T05:46:01.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1563248095391866660', time=2024-04-17T07:21:04.006210+0000, chain.objs.size()=1 2024-04-17T05:46:01.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CYVG64EQkNYR1Q2neBcMZLTPY34feLr_0 2024-04-17T05:46:01.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10172621689160545284', time=2024-04-17T07:21:05.619009+0000, chain.objs.size()=1 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.olOgzORasohDC-NLItHXGkc6TF4j6dv_0 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15153698715066036226', time=2024-04-17T07:21:06.531031+0000, chain.objs.size()=1 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nr0RqorqWnv6SPgrHmc1Qe7HfOSnj9v_0 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12233763948502643047', time=2024-04-17T07:21:06.789955+0000, chain.objs.size()=1 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._QDrCLNNdK1eq0QPyD34s_FNDJ_kHY0_0 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14025163509950410216', time=2024-04-17T07:21:08.051424+0000, chain.objs.size()=1 2024-04-17T05:46:01.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UMunQAKzDSNrg_PGNkPmo1S6yTpQm-n_0 2024-04-17T05:46:01.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7689947510773446018', time=2024-04-17T07:21:08.871949+0000, chain.objs.size()=1 2024-04-17T05:46:01.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GgwOppjhpnZ6lIn2SEakbB68Mo4zz0U_0 2024-04-17T05:46:01.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1861003534379500658', time=2024-04-17T07:21:09.434631+0000, chain.objs.size()=1 2024-04-17T05:46:01.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GjJaQvTtdQRZjm9pkdQ0BaxFO2B7h5z_0 2024-04-17T05:46:01.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2880791174136426531', time=2024-04-17T07:21:09.954363+0000, chain.objs.size()=1 2024-04-17T05:46:01.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eH86E2dI5bCHZ0yQmE8WBmjTHiIWV_d_0 2024-04-17T05:46:01.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7260119812510521313', time=2024-04-17T07:21:09.974485+0000, chain.objs.size()=1 2024-04-17T05:46:01.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._eF_X5Kgj07AKlEgdzPjK3cYl8oWmE__0 2024-04-17T05:46:01.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15885088451489822135', time=2024-04-17T07:21:10.254723+0000, chain.objs.size()=1 2024-04-17T05:46:01.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jBnI8WQNeuJcOsJbNJUE-Nm4RNgP_U3_0 2024-04-17T05:46:01.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6038948213210724090', time=2024-04-17T07:21:10.783977+0000, chain.objs.size()=1 2024-04-17T05:46:01.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vSF-IH3mCn02J9VPOrUFxCp-cma3K3R_0 2024-04-17T05:46:01.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7953326542050019705', time=2024-04-17T07:21:10.785623+0000, chain.objs.size()=1 2024-04-17T05:46:01.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ofF0GKMq7kd1EEwLdvEHu9w71iM-FmH_0 2024-04-17T05:46:01.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12693185345479963796', time=2024-04-17T07:21:11.943385+0000, chain.objs.size()=1 2024-04-17T05:46:01.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.L6WVnTngNo-2SvpSV7cRBd2c0vHg_MQ_0 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17396501471591753908', time=2024-04-17T07:21:12.614306+0000, chain.objs.size()=1 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tESZTWueoVVeGhakzW3c_8SB4EzcAdf_0 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9785575867854759233', time=2024-04-17T07:21:12.980516+0000, chain.objs.size()=1 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gKnGubX0Gnofdd7iFxdOjxlMwweyN3X_0 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7833781494738011844', time=2024-04-17T07:21:13.233572+0000, chain.objs.size()=1 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RNy830emtSPd6bY4s_QNGxooLMDYlxY_0 2024-04-17T05:46:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13721203665126114242', time=2024-04-17T07:21:13.387644+0000, chain.objs.size()=1 2024-04-17T05:46:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._KAA0FcinPWatx7fHU9xug2WRtMZJx0_0 2024-04-17T05:46:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12644407535587639048', time=2024-04-17T07:21:14.631365+0000, chain.objs.size()=1 2024-04-17T05:46:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zDBKenjoZ5jPQbK2eodTBcmhegTIII-_0 2024-04-17T05:46:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3268510990606920697', time=2024-04-17T07:21:15.872404+0000, chain.objs.size()=1 2024-04-17T05:46:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uk21yUTGvveDGAeAQ97ZG6sxSJJ0eWx_0 2024-04-17T05:46:01.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2757022781565781831', time=2024-04-17T07:21:17.321888+0000, chain.objs.size()=1 2024-04-17T05:46:01.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.F6Rr9Sf9FvBSLzel60ld9VBXl4hNmM7_0 2024-04-17T05:46:01.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16935355309625488567', time=2024-04-17T07:21:18.598807+0000, chain.objs.size()=1 2024-04-17T05:46:01.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UboDfPQmkhML74o9Oh36CzL6gcRdOOR_0 2024-04-17T05:46:01.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5453361628472020429', time=2024-04-17T07:21:19.487451+0000, chain.objs.size()=1 2024-04-17T05:46:01.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xv-Urf72aEu0x8mhZEvY6S3Hw1DRT8Z_0 2024-04-17T05:46:01.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17755559310336959193', time=2024-04-17T07:21:21.548357+0000, chain.objs.size()=1 2024-04-17T05:46:01.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iJMTRHgkVTIhGwavtNAsxafLORldJM__0 2024-04-17T05:46:01.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3986892281690860572', time=2024-04-17T07:21:23.294226+0000, chain.objs.size()=1 2024-04-17T05:46:01.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.feXAF0InzM_Uo6iqITmNLUhi5cZ2luT_0 2024-04-17T05:46:01.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11509659319601052698', time=2024-04-17T07:21:23.756749+0000, chain.objs.size()=1 2024-04-17T05:46:01.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.G5DntwElSfnNOdgXdRogXf8fjo0T3kk_0 2024-04-17T05:46:01.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3952603822713369229', time=2024-04-17T07:21:24.801654+0000, chain.objs.size()=1 2024-04-17T05:46:01.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7VgI3bobHnfLw6BTCL7Br3mFwoejg84_0 2024-04-17T05:46:01.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3423969373049293651', time=2024-04-17T07:21:25.359317+0000, chain.objs.size()=1 2024-04-17T05:46:01.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JecB1PbCeGA_oTdmzxYGtLLIsb0y3ME_0 2024-04-17T05:46:01.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18174549368388105074', time=2024-04-17T07:21:25.787571+0000, chain.objs.size()=1 2024-04-17T05:46:01.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rBugHkr32lBWc4gYUgSfwBpGvKwSlHK_0 2024-04-17T05:46:01.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2372896295998493829', time=2024-04-17T07:21:27.458885+0000, chain.objs.size()=1 2024-04-17T05:46:01.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CT87gtoIVx-NNJYQ9dYZW7RBjQui70m_0 2024-04-17T05:46:01.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2847238654974210088', time=2024-04-17T07:21:27.460374+0000, chain.objs.size()=1 2024-04-17T05:46:01.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.AnQ0b2AMZbl2y_ypASkf9uvs99ShCwe_0 2024-04-17T05:46:01.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18040707107026827330', time=2024-04-17T07:21:27.968684+0000, chain.objs.size()=1 2024-04-17T05:46:01.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-ItnXbblRPRdNtUA7XOS_c41EEJovk7_0 2024-04-17T05:46:01.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13075033255756468042', time=2024-04-17T07:21:28.658342+0000, chain.objs.size()=1 2024-04-17T05:46:01.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.d7vidp-G8DgVyI3wmUQqBMLYrxmyK6O_0 2024-04-17T05:46:01.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6010356802350320789', time=2024-04-17T07:23:12.750573+0000, chain.objs.size()=1 2024-04-17T05:46:01.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gf94tkclSmJYIhciZWEGDg0tgBfhgqd_0 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14676494367203935636', time=2024-04-17T07:23:13.941095+0000, chain.objs.size()=1 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uVLxGmUEelmAOng94_-bzzQvZpbyrep_0 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3285602893025895150', time=2024-04-17T07:23:15.307549+0000, chain.objs.size()=1 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.i_w11FVnn-JXpB2CBI2gM5QQkA9tcy0_0 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12098716443314560436', time=2024-04-17T07:23:15.733743+0000, chain.objs.size()=1 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CGAvsxPcoidhoJgIAx7C04gboStTm1N_0 2024-04-17T05:46:01.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15898809770342577396', time=2024-04-17T07:23:15.784173+0000, chain.objs.size()=1 2024-04-17T05:46:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.T0I3u-DicE5cULvWNoLyHD-RPt2tkKR_0 2024-04-17T05:46:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2452834034668200588', time=2024-04-17T07:23:18.757188+0000, chain.objs.size()=1 2024-04-17T05:46:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aMND2jPG5wBdp5lmnmLONTXaTKXsU6S_0 2024-04-17T05:46:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8391173612600078336', time=2024-04-17T07:23:21.139322+0000, chain.objs.size()=1 2024-04-17T05:46:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.01uTDELXHlD3oFSa_ccuT6tjjcTIN-i_0 2024-04-17T05:46:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.184178703459985745', time=2024-04-17T07:23:21.687574+0000, chain.objs.size()=1 2024-04-17T05:46:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rw2MRZf5E62KhCbTc32O8eeKQDHvdAh_0 2024-04-17T05:46:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16790560658513226397', time=2024-04-17T07:23:23.818081+0000, chain.objs.size()=1 2024-04-17T05:46:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NkHoJ0gpUQ8X_MsLgX_s7ZRDa5Bg1hZ_0 2024-04-17T05:46:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6638812937899674088', time=2024-04-17T07:23:29.247362+0000, chain.objs.size()=1 2024-04-17T05:46:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hMT8hQxRhc6zNLY42B9D49-7t2WocOG_0 2024-04-17T05:46:01.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13365939909921479273', time=2024-04-17T07:23:30.291507+0000, chain.objs.size()=1 2024-04-17T05:46:01.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Qm67oc-pi6s_Z4t2KTs00pgMYO0E6Dm_0 2024-04-17T05:46:01.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13243456401231938945', time=2024-04-17T07:23:30.293206+0000, chain.objs.size()=1 2024-04-17T05:46:01.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YnL_ttrtHNKSEyj_hV7Hf11D1xGT-EY_0 2024-04-17T05:46:01.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15886158011943005556', time=2024-04-17T07:23:31.332580+0000, chain.objs.size()=1 2024-04-17T05:46:01.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7W8snBmspwqCR7kTp_7xCbGsnYRx6gz_0 2024-04-17T05:46:01.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15833110311867208841', time=2024-04-17T07:23:32.164649+0000, chain.objs.size()=1 2024-04-17T05:46:01.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.u3hlbpxlkAxjk6SOrrkTqnauj6v1xOX_0 2024-04-17T05:46:01.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3157677089903005264', time=2024-04-17T07:23:33.284449+0000, chain.objs.size()=1 2024-04-17T05:46:01.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.K5P64H73J9vMiwp6yvBAFhWczqmC-un_0 2024-04-17T05:46:01.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14175694974445682241', time=2024-04-17T07:23:33.739469+0000, chain.objs.size()=1 2024-04-17T05:46:01.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5GwOcV36fMAJHo7id32WvuA0EoOTtOW_0 2024-04-17T05:46:01.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14684443700911278614', time=2024-04-17T07:23:34.661335+0000, chain.objs.size()=1 2024-04-17T05:46:01.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Mwve0azV62hxONWpa_Vzm4RARKhmcRo_0 2024-04-17T05:46:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/44506 2024-04-17T05:46:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=26, truncated=0, next_marker='' 2024-04-17T05:46:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3007813856681242308', time=2024-04-17T07:23:35.022940+0000, chain.objs.size()=1 2024-04-17T05:46:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XtBuXXyDBrlix0oW9VS4dpZ_AsYdyyY_0 2024-04-17T05:46:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.209564735816535497', time=2024-04-17T07:23:36.326948+0000, chain.objs.size()=1 2024-04-17T05:46:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oxtyOp6Gt9Hcb9J_7DUwMff28_QCkuI_0 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.419791453680245027', time=2024-04-17T07:23:36.582418+0000, chain.objs.size()=1 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZJGDCcg4aZDtN03AcMSH1DFiIOZmuLD_0 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6888354715364036856', time=2024-04-17T07:23:37.685261+0000, chain.objs.size()=1 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._P4cmoBXsuuIAVbqGKqsoD8XXDk2CEq_0 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5013647995625606069', time=2024-04-17T07:23:37.774805+0000, chain.objs.size()=1 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.w076IL8DxzuDgtVs30OPa42Qvc_asOi_0 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1558457820751089668', time=2024-04-17T07:23:39.860621+0000, chain.objs.size()=1 2024-04-17T05:46:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.74aR80h2KjyEp0Wd7aovw4SYM9AHDye_0 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8633041131837623349', time=2024-04-17T07:23:39.962005+0000, chain.objs.size()=1 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jaQwfDfdwGUrlJTBUobruaZu9iN0jZU_0 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2014599125536313351', time=2024-04-17T07:23:39.963526+0000, chain.objs.size()=1 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Tja7NMDTVqd9Ir5hjd1Ej2oRGlXaHpC_0 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11217676920181553406', time=2024-04-17T07:23:40.262238+0000, chain.objs.size()=1 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6o0e7y7kiK3P67czGr9A1rWY22Y96U5_0 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17461787248464825649', time=2024-04-17T07:23:40.662347+0000, chain.objs.size()=1 2024-04-17T05:46:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.POZI9nB8c-wz0wGTUFakhMVc6NzPyVW_0 2024-04-17T05:46:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7772913029659509614', time=2024-04-17T07:25:32.172308+0000, chain.objs.size()=1 2024-04-17T05:46:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.434__shadow_.Q-Hl_SrDITugTM0PqOj2HpQqHfFO6As_0 2024-04-17T05:46:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12767678152585764713', time=2024-04-17T07:25:49.037211+0000, chain.objs.size()=1 2024-04-17T05:46:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.448__shadow_.qc7yr0iT8KibbN4uGhCkTq6cn1-ReIa_0 2024-04-17T05:46:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17625643736374223609', time=2024-04-17T07:26:17.207632+0000, chain.objs.size()=1 2024-04-17T05:46:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.kkzZ800JZrvd_s0_cLlYi3UMlL3QnhK_0 2024-04-17T05:46:01.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1429042150533855654', time=2024-04-17T07:26:18.748976+0000, chain.objs.size()=1 2024-04-17T05:46:01.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.474__shadow_.flAPTOhalTGWNmzriZ1vzNM5hMV9yre_0 2024-04-17T05:46:01.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13792536949305234519', time=2024-04-17T07:26:18.774821+0000, chain.objs.size()=1 2024-04-17T05:46:01.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.474__shadow_.5kwnBCYP6yTPPl837UB2U3zw1F4hski_0 2024-04-17T05:46:01.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_1_b0630391e6b949add819d592dd33fb01', time=2024-04-17T07:26:43.913912+0000, chain.objs.size()=1 2024-04-17T05:46:01.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_0 2024-04-17T05:46:01.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2_3f655602819d9a077c8d58eb45fe7a18', time=2024-04-17T07:26:56.970647+0000, chain.objs.size()=6 2024-04-17T05:46:01.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.1 2024-04-17T05:46:01.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2 2024-04-17T05:46:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.3 2024-04-17T05:46:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.4 2024-04-17T05:46:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.5 2024-04-17T05:46:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.6 2024-04-17T05:46:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7395718681997443927', time=2024-04-17T07:27:49.041145+0000, chain.objs.size()=1 2024-04-17T05:46:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.510__shadow:iH8BaBEbZojdtAqOk4kZwtcwSln03Bp_.JYTXDihmAd1D0w5IIkhGfjjWv2i9o7j_0 2024-04-17T05:46:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.524+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc', time=2024-04-17T07:28:26.261732+0000, chain.objs.size()=1 2024-04-17T05:46:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.524+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc.3 2024-04-17T05:46:01.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.524+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13315814818374870979', time=2024-04-17T07:31:16.720792+0000, chain.objs.size()=1 2024-04-17T05:46:01.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.524+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.548__shadow_.Ujs6sqCy_NwBwJlSBLXSP3ro7PaLeTv_0 2024-04-17T05:46:01.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3609970550621318229', time=2024-04-17T07:35:38.158789+0000, chain.objs.size()=1 2024-04-17T05:46:01.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.607__shadow_.eCaDQzJctpGf0gOOF0tcM9WvlMnkkKG_0 2024-04-17T05:46:01.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6888851700656389187', time=2024-04-17T07:36:44.308958+0000, chain.objs.size()=1 2024-04-17T05:46:01.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.628__shadow_.WfQBp3y2sijpgaZE9_wVaUWYpBYAaE4_0 2024-04-17T05:46:01.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5924746632361218727', time=2024-04-17T07:36:52.639070+0000, chain.objs.size()=1 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.632__shadow_.NMAr3w-XDe8DG8a4u-IEaBVCKGl1nnn_0 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.nkv5fDUv7YDEmrvqdHF7khPn4g4y8KP_1_d60c89fab3c0a188190fbc6351240850', time=2024-04-17T07:37:42.288332+0000, chain.objs.size()=1 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.nkv5fDUv7YDEmrvqdHF7khPn4g4y8KP_0 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8664766465271807077', time=2024-04-17T07:37:53.946513+0000, chain.objs.size()=1 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.664__shadow_.wODzcHt54RpvRsRpEZHdQWukJVercGf_0 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13331437141136247183', time=2024-04-17T07:38:05.754854+0000, chain.objs.size()=1 2024-04-17T05:46:01.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.678__shadow:cRyMkovgJ666JqHTV3CupXj03aYvDMp_.gZqCPNbNH8eQmtQoICvPectRJCl5usy_0 2024-04-17T05:46:01.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.536+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:01.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.536+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-17T05:46:01.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44026' 2024-04-17T05:46:01.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4619531973610491139', time=2024-04-17T07:14:31.428805+0000, chain.objs.size()=1 2024-04-17T05:46:01.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.223__shadow_.nlBRdvlr4f1z9-rGonrZ0h7KOZlecBK_0 2024-04-17T05:46:01.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13903026341543961528', time=2024-04-17T07:14:48.913737+0000, chain.objs.size()=1 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.233__shadow_.aeUOTAtz38MR_4noBhfKY2cC75o7SJE_0 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6177751172701228453', time=2024-04-17T07:14:52.889427+0000, chain.objs.size()=1 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.235__shadow_.y8eyeTlroGxC9oX40I1DTbTEkxuG5_s_0 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4524084518513161683', time=2024-04-17T07:14:56.130835+0000, chain.objs.size()=1 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.237__shadow_.At31OG-SaWo7244_jGI_K_p-ZdixlC7_0 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18110636220117163638', time=2024-04-17T07:17:01.311015+0000, chain.objs.size()=1 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Md3Q9ohgiczvkUWUqoRufj6_MLtk7_K_0 2024-04-17T05:46:01.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10927234529844492145', time=2024-04-17T07:17:01.312256+0000, chain.objs.size()=1 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TxcaQe-WUJbVEmhr_bZ9R5G0FDtn0TD_0 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18380350665495534565', time=2024-04-17T07:17:01.313137+0000, chain.objs.size()=1 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yXkwdqkIs9Wfs7GXX9tIeOeOVicCycj_0 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8569853733492324134', time=2024-04-17T07:17:01.757390+0000, chain.objs.size()=1 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ePcYoy0nvusiABFiocJWKi2RG3pLExJ_0 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4023024253842362401', time=2024-04-17T07:17:02.159267+0000, chain.objs.size()=1 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SgPxlijG1DciZTIkTiWmAzRRV_rMinn_0 2024-04-17T05:46:01.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8214597320591737972', time=2024-04-17T07:17:02.406012+0000, chain.objs.size()=1 2024-04-17T05:46:01.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NKQGT67N4AezTOKS7vOcNCG4aJ1XnKs_0 2024-04-17T05:46:01.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10677058528923810822', time=2024-04-17T07:17:02.948692+0000, chain.objs.size()=1 2024-04-17T05:46:01.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OVJjWAgzD8ZDY5Q8nF2k3aBs1O5zccm_0 2024-04-17T05:46:01.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9500320633864727959', time=2024-04-17T07:17:05.571356+0000, chain.objs.size()=1 2024-04-17T05:46:01.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.o8KKkwrURBuoyZkYQY2sOgOjpDp6JJ5_0 2024-04-17T05:46:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9803523697381835861', time=2024-04-17T07:17:06.364096+0000, chain.objs.size()=1 2024-04-17T05:46:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6JD27JfjCrQDKRqyoidMqC7RoL7pOOm_0 2024-04-17T05:46:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7314620968810481541', time=2024-04-17T07:17:06.450639+0000, chain.objs.size()=1 2024-04-17T05:46:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fAUaBPcAvkUXvDCov775vAUR9TTZlng_0 2024-04-17T05:46:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.634875003666271378', time=2024-04-17T07:17:06.452283+0000, chain.objs.size()=1 2024-04-17T05:46:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3A9fh5zBbG273x4WhUYM8np6HnMNldX_0 2024-04-17T05:46:01.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13957052062718615280', time=2024-04-17T07:17:06.707272+0000, chain.objs.size()=1 2024-04-17T05:46:01.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.A2R-lDukINMuBQpzr5SWXeSM_rllP30_0 2024-04-17T05:46:01.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5781172191006418381', time=2024-04-17T07:17:06.797218+0000, chain.objs.size()=1 2024-04-17T05:46:01.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lUXC78W5RP4r5j_jsdUbwESucy5MM-B_0 2024-04-17T05:46:01.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16456057799514232594', time=2024-04-17T07:17:08.654858+0000, chain.objs.size()=1 2024-04-17T05:46:01.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2K4pKHP-n9RnMjj1ou7RV7-ZX0YVe9P_0 2024-04-17T05:46:01.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14469754152833397785', time=2024-04-17T07:17:08.721319+0000, chain.objs.size()=1 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c_X3A7ACHVJTgRLiOucrRuExE5qnECx_0 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13048005203750928241', time=2024-04-17T07:17:10.685008+0000, chain.objs.size()=1 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.f2wHHOmfLQLHDERPy6W8fw1XyPoC4KW_0 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10430927065931548199', time=2024-04-17T07:17:12.081657+0000, chain.objs.size()=1 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RTzNqT4q_6IyvX2P3lIK9EMJwFMZmke_0 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1049512872874084973', time=2024-04-17T07:17:14.686125+0000, chain.objs.size()=1 2024-04-17T05:46:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bGSVSJH7j8GVGPs_q-9MlRytgwPWbOJ_0 2024-04-17T05:46:01.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5901096506225946925', time=2024-04-17T07:17:14.751692+0000, chain.objs.size()=1 2024-04-17T05:46:01.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jiPL79j_HCzLKkIFK-mkVZbzTycUqd6_0 2024-04-17T05:46:01.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13888206754451967142', time=2024-04-17T07:17:15.571324+0000, chain.objs.size()=1 2024-04-17T05:46:01.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tfg6192VeXHzVVmhQh2W0O9u2GCIrOP_0 2024-04-17T05:46:01.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4943089763687527987', time=2024-04-17T07:17:15.908104+0000, chain.objs.size()=1 2024-04-17T05:46:01.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1Loaxjj6prLFHtlSdaYBMxI0iLTWhAe_0 2024-04-17T05:46:01.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15691837804165619532', time=2024-04-17T07:17:16.778197+0000, chain.objs.size()=1 2024-04-17T05:46:01.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.odT9HkmkEquLl3AvYyOK5bUabdSBiY7_0 2024-04-17T05:46:01.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9293138118153356163', time=2024-04-17T07:17:18.413700+0000, chain.objs.size()=1 2024-04-17T05:46:01.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DP3YgWDULzcIbsWpm04UT9hNtPXxrCK_0 2024-04-17T05:46:01.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5211706559306790253', time=2024-04-17T07:17:22.063473+0000, chain.objs.size()=1 2024-04-17T05:46:01.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.S9lWAAMj7rJsolMIh4SXkSLBJABvYWt_0 2024-04-17T05:46:01.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2560296108403134542', time=2024-04-17T07:17:22.353549+0000, chain.objs.size()=1 2024-04-17T05:46:01.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QoAFv9HeBKMhe_T6HSy3Rxz_CxtZlVm_0 2024-04-17T05:46:01.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2113027772975155455', time=2024-04-17T07:17:22.354928+0000, chain.objs.size()=1 2024-04-17T05:46:01.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2X6Ap1B4szLj710w04RJrONkYHZc6c6_0 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17512908481719113394', time=2024-04-17T07:17:22.770564+0000, chain.objs.size()=1 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.X40Ky_Gq70D6w8OTxFBxteP838LJYD2_0 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.327926587688889442', time=2024-04-17T07:17:22.844950+0000, chain.objs.size()=1 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3WxP04Kc91ejwOGz80nGTHkStNGqovl_0 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9350118479973938478', time=2024-04-17T07:17:23.116156+0000, chain.objs.size()=1 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CjkWIZAkzctNYaWADt0nJ6PKo7wL0QD_0 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17581230276620296486', time=2024-04-17T07:17:24.158884+0000, chain.objs.size()=1 2024-04-17T05:46:01.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VBjd5ARtcx16fmL1TDrhc0660MbMAkF_0 2024-04-17T05:46:01.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4595728000542732804', time=2024-04-17T07:17:24.700730+0000, chain.objs.size()=1 2024-04-17T05:46:01.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8vAhYhF9ogtG0tp-K_LBmet8CiAzaUC_0 2024-04-17T05:46:01.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15819404938205317105', time=2024-04-17T07:17:24.879626+0000, chain.objs.size()=1 2024-04-17T05:46:01.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._LRbyMaI7S78omSy_vXgtKAvnMobVoa_0 2024-04-17T05:46:01.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13557270904443110275', time=2024-04-17T07:17:25.118424+0000, chain.objs.size()=1 2024-04-17T05:46:01.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EQKoZ1JkEZ127NJYe1A5qNyJnP5arCX_0 2024-04-17T05:46:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17524759548869734672', time=2024-04-17T07:17:27.090153+0000, chain.objs.size()=1 2024-04-17T05:46:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.n5kT_gLU18B95PAxyh8yvtf_vmtIwuH_0 2024-04-17T05:46:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4193510696278205622', time=2024-04-17T07:17:27.721836+0000, chain.objs.size()=1 2024-04-17T05:46:01.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GbwnLNDq3Uv05qOmv0pyUR20LwOA_cZ_0 2024-04-17T05:46:01.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2235212023181552521', time=2024-04-17T07:17:27.968019+0000, chain.objs.size()=1 2024-04-17T05:46:01.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PwyQB4B89DNDtYPj2v4Lir7jVNT2FG__0 2024-04-17T05:46:01.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6949395255358837070', time=2024-04-17T07:17:29.879454+0000, chain.objs.size()=1 2024-04-17T05:46:01.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.AljA2mYfvvPiJm9WJWmPMlLQq0j3AhE_0 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.681886984066006074', time=2024-04-17T07:17:30.422909+0000, chain.objs.size()=1 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1CrsI9BO_LKw5YOX_HzVuUS2dgYNeAG_0 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15131167408749338759', time=2024-04-17T07:17:30.932448+0000, chain.objs.size()=1 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.AMN5NTly03o_gpsS82H6W7EPePAp3Ap_0 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15737209298231704427', time=2024-04-17T07:17:32.274582+0000, chain.objs.size()=1 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7iphpxipX4PO-uXVmcJGGuMXsN8xVok_0 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12633272700640652295', time=2024-04-17T07:17:36.902084+0000, chain.objs.size()=1 2024-04-17T05:46:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.255__shadow_.qwq0tMSYn6wnOGTUaoXwVhnuAe9FhPe_0 2024-04-17T05:46:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4369885922114115696', time=2024-04-17T07:17:38.414142+0000, chain.objs.size()=1 2024-04-17T05:46:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.256__shadow_.fABk50uyOfNqkqfNC8X7_am71xf680A_0 2024-04-17T05:46:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17357924903036930540', time=2024-04-17T07:17:44.447239+0000, chain.objs.size()=1 2024-04-17T05:46:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.260__shadow_.I9W0WjQy05gfajJE9VBHNfou58ngdHy_0 2024-04-17T05:46:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11635083310466034960', time=2024-04-17T07:18:06.562720+0000, chain.objs.size()=1 2024-04-17T05:46:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.274__shadow_.agZX81kCgSsJM0deHUDnerJmhhwdZYt_0 2024-04-17T05:46:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.843904026167268336', time=2024-04-17T07:21:00.113627+0000, chain.objs.size()=1 2024-04-17T05:46:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KRzU2sRCzXXl-sksReZFLYekQCs0RvY_0 2024-04-17T05:46:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1119777419092695328', time=2024-04-17T07:21:00.256992+0000, chain.objs.size()=1 2024-04-17T05:46:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7bW4lgqdZGikoTc7MQkOOqZyEq-IJIu_0 2024-04-17T05:46:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10315826880697438192', time=2024-04-17T07:21:01.231907+0000, chain.objs.size()=1 2024-04-17T05:46:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.L4-hfaXUB19TkS0wMKHt9oBcyTSAYKx_0 2024-04-17T05:46:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10739860156730217670', time=2024-04-17T07:21:01.572538+0000, chain.objs.size()=1 2024-04-17T05:46:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oqrCg6ug1PYHdjRUxJQBBWsxUS1UuE8_0 2024-04-17T05:46:01.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7281332916476569307', time=2024-04-17T07:21:05.715558+0000, chain.objs.size()=1 2024-04-17T05:46:01.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.x4Hr15k_DDpdK0hiBWCTekSrtPZUYFb_0 2024-04-17T05:46:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12461066475416915442', time=2024-04-17T07:21:06.526015+0000, chain.objs.size()=1 2024-04-17T05:46:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-e_JEovaSN9lfXgsklC1f1IqlunH4Yn_0 2024-04-17T05:46:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14245197072103409058', time=2024-04-17T07:21:07.957804+0000, chain.objs.size()=1 2024-04-17T05:46:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rs_codRLXj01l4_P8Bu0OC97GWyz4UD_0 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6579298523396883260', time=2024-04-17T07:21:07.959826+0000, chain.objs.size()=1 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9fFb_or01gutBnrlYN2lgmCVoebkO6i_0 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16423660871820847205', time=2024-04-17T07:21:08.356963+0000, chain.objs.size()=1 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sCUuuAsxF21dca9ftTY5oPJ11Q_V7DB_0 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3349305425189891805', time=2024-04-17T07:21:09.233804+0000, chain.objs.size()=1 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._a43F7-GkbcspXrhtGckCaXbeqRWjnN_0 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11481833718139725074', time=2024-04-17T07:21:11.967363+0000, chain.objs.size()=1 2024-04-17T05:46:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zCVup_nmoqXuTnC2WPrGHhn4lgm4JtF_0 2024-04-17T05:46:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.199940660995816915', time=2024-04-17T07:21:12.238112+0000, chain.objs.size()=1 2024-04-17T05:46:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xsd5OHeKTSiO2sCMyglKUBF7RmVhKwa_0 2024-04-17T05:46:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11793461692224529134', time=2024-04-17T07:21:13.071773+0000, chain.objs.size()=1 2024-04-17T05:46:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uKr3bF_JAwuGttWT8yjK8VIGUY0vgqm_0 2024-04-17T05:46:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6584775003889944125', time=2024-04-17T07:21:14.612343+0000, chain.objs.size()=1 2024-04-17T05:46:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.T0ZXcbEGUXZaUzPZdALZOCipWYFg0e8_0 2024-04-17T05:46:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6998080220216576816', time=2024-04-17T07:21:14.640093+0000, chain.objs.size()=1 2024-04-17T05:46:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2FfqZmXX7iPKOAHTtBpw_UTHjHvacML_0 2024-04-17T05:46:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.821067976651270645', time=2024-04-17T07:21:16.310011+0000, chain.objs.size()=1 2024-04-17T05:46:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.j1V-i2gm0n1F2WCQTsycJjom4FFi1Lz_0 2024-04-17T05:46:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15669935303161614507', time=2024-04-17T07:21:16.408463+0000, chain.objs.size()=1 2024-04-17T05:46:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.h4_Kac7igjOfqyvCqhmycBs9ApSw5iI_0 2024-04-17T05:46:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.253821118797328388', time=2024-04-17T07:21:16.560030+0000, chain.objs.size()=1 2024-04-17T05:46:01.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JEgUYuFoJNwe1fg4_yBZKpEurHL8zvM_0 2024-04-17T05:46:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13226198756875519211', time=2024-04-17T07:21:17.512730+0000, chain.objs.size()=1 2024-04-17T05:46:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sWGIEIXvCixYTH3lK56Ex811HYQeZGp_0 2024-04-17T05:46:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10999073659902850637', time=2024-04-17T07:21:17.632394+0000, chain.objs.size()=1 2024-04-17T05:46:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mUlCwuu0QA_8o9ibN2ouCM7SBCeF1XQ_0 2024-04-17T05:46:01.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11134225280207895594', time=2024-04-17T07:21:17.677231+0000, chain.objs.size()=1 2024-04-17T05:46:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Lij434tMJIi8rYpWUeS_9ZcqHgCc-gl_0 2024-04-17T05:46:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7540776131330158325', time=2024-04-17T07:21:17.844286+0000, chain.objs.size()=1 2024-04-17T05:46:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Yn92injBaLiSUJ154oo8ykk-MqWV5ZB_0 2024-04-17T05:46:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11272805621211440165', time=2024-04-17T07:21:18.277320+0000, chain.objs.size()=1 2024-04-17T05:46:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oIeZrvaD73Mlz7-EX7lIUcRR0tMgm8V_0 2024-04-17T05:46:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14792876736911097400', time=2024-04-17T07:21:18.778700+0000, chain.objs.size()=1 2024-04-17T05:46:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9Q6rOmK1160jnhKgR-XHHUM1FYVZFhv_0 2024-04-17T05:46:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9966472312294674029', time=2024-04-17T07:21:19.710611+0000, chain.objs.size()=1 2024-04-17T05:46:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zFKrMCES4yVrbZHhUYL7FQ7YtlquNex_0 2024-04-17T05:46:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9745935876403063924', time=2024-04-17T07:21:20.436715+0000, chain.objs.size()=1 2024-04-17T05:46:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QsXrseNX6eeK_u6LnwQeZ1RDphNeeEj_0 2024-04-17T05:46:01.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6944006420801764494', time=2024-04-17T07:21:20.598453+0000, chain.objs.size()=1 2024-04-17T05:46:01.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kjPjPLM147rKysQqg65vsVT_9EXvFk-_0 2024-04-17T05:46:01.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17890100963203302230', time=2024-04-17T07:21:20.813459+0000, chain.objs.size()=1 2024-04-17T05:46:01.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GHYdAmM4KtHS8lCGnoOOvLkhGQvOWyA_0 2024-04-17T05:46:01.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10734817960958799701', time=2024-04-17T07:21:21.432585+0000, chain.objs.size()=1 2024-04-17T05:46:01.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.REBzfI_0F0yFqyR9gBSv8UK8zyZ2lCz_0 2024-04-17T05:46:01.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4113383215205974048', time=2024-04-17T07:21:21.687788+0000, chain.objs.size()=1 2024-04-17T05:46:01.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DQIZN4NPXJ5obUpPu3VssNG6zl-QRuU_0 2024-04-17T05:46:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17367723288321935485', time=2024-04-17T07:21:23.237582+0000, chain.objs.size()=1 2024-04-17T05:46:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.do9YuOcjYRgdVBEYk6X1tWCFB6TZu94_0 2024-04-17T05:46:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2620251613056503823', time=2024-04-17T07:21:23.375848+0000, chain.objs.size()=1 2024-04-17T05:46:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SSE_YUhOJ6DV7a9mMwqnbXq03nU1UxG_0 2024-04-17T05:46:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2436575364572541006', time=2024-04-17T07:21:24.762840+0000, chain.objs.size()=1 2024-04-17T05:46:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2ajUtf0dCP_vxIKTDMl_vb0jsGufUdu_0 2024-04-17T05:46:01.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1320112324664208954', time=2024-04-17T07:21:25.108318+0000, chain.objs.size()=1 2024-04-17T05:46:01.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aXjEx7Lp8o5kBhZEDz0vDU1ZF5E6HuC_0 2024-04-17T05:46:01.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12093658802719867685', time=2024-04-17T07:21:26.195034+0000, chain.objs.size()=1 2024-04-17T05:46:01.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aPzBkHjeAwjM91PacZHYsrcuFAKOEfh_0 2024-04-17T05:46:01.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8134937874358268642', time=2024-04-17T07:21:27.520040+0000, chain.objs.size()=1 2024-04-17T05:46:01.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.h5edSufezAz1DUCFEqvI-ihTXPBwcii_0 2024-04-17T05:46:01.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16719174945258029772', time=2024-04-17T07:21:28.864636+0000, chain.objs.size()=1 2024-04-17T05:46:01.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WkqG6_s3IZHd8ROoRDX72C3S2yp7Wb1_0 2024-04-17T05:46:01.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5143960204332025747', time=2024-04-17T07:23:13.997999+0000, chain.objs.size()=1 2024-04-17T05:46:01.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Cf2S_jqZR5GAspllHXaYIKSJxHqo7kQ_0 2024-04-17T05:46:01.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7920790673224808863', time=2024-04-17T07:23:14.962138+0000, chain.objs.size()=1 2024-04-17T05:46:01.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1s_HR4fTRtIanVXc39oHlZsi0gzlK1G_0 2024-04-17T05:46:01.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1181603980760868467', time=2024-04-17T07:23:14.964246+0000, chain.objs.size()=1 2024-04-17T05:46:01.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VdvCdt2DIeF-XoXmGmKFkWk0DeqLPn1_0 2024-04-17T05:46:01.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9625111679036161201', time=2024-04-17T07:23:15.706918+0000, chain.objs.size()=1 2024-04-17T05:46:01.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xE9V0ONidsg5I-_zAKhlK5Fl0k7HN-w_0 2024-04-17T05:46:01.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8940837020213840805', time=2024-04-17T07:23:17.102594+0000, chain.objs.size()=1 2024-04-17T05:46:01.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gKudIrmZAj8vzF4EZNplDKAVkvpIlgX_0 2024-04-17T05:46:01.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5896172244736906357', time=2024-04-17T07:23:17.104353+0000, chain.objs.size()=1 2024-04-17T05:46:01.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._67BTibWhYMqWZeK8lGTof6p4g6pGbI_0 2024-04-17T05:46:01.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10003978504346125030', time=2024-04-17T07:23:17.852525+0000, chain.objs.size()=1 2024-04-17T05:46:01.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9g1HpgmdMfT7rrgMFW55CmbRn7wroAO_0 2024-04-17T05:46:01.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3528934107769780386', time=2024-04-17T07:23:18.864892+0000, chain.objs.size()=1 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.456fRTjX9FMpX_zrGr7ig4qBXUDkMhx_0 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10439167678010351091', time=2024-04-17T07:23:18.927902+0000, chain.objs.size()=1 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-SVCoHAfXblbWJQids1gqM46_Uhntey_0 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17278106206759851736', time=2024-04-17T07:23:19.158306+0000, chain.objs.size()=1 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.q0rfuyJTidN336vNc0VkoUDxfOc5-5S_0 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1623505029651320380', time=2024-04-17T07:23:19.379162+0000, chain.objs.size()=1 2024-04-17T05:46:01.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bKSvEBHhH03ZWdhh3d6yASSNKXtB3Ci_0 2024-04-17T05:46:01.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5400883767641477682', time=2024-04-17T07:23:21.010097+0000, chain.objs.size()=1 2024-04-17T05:46:01.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7xamuQgLLeF7Te062AGTzW8VK8s7pGw_0 2024-04-17T05:46:01.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5425468709313992614', time=2024-04-17T07:23:21.504541+0000, chain.objs.size()=1 2024-04-17T05:46:01.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nDM93d72D4oXwb_qsxcIhF1NwraeZi-_0 2024-04-17T05:46:01.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5995655997803382739', time=2024-04-17T07:23:21.890825+0000, chain.objs.size()=1 2024-04-17T05:46:01.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5K70tWhCIXn7nh5wGJGnN0TNR82d0ax_0 2024-04-17T05:46:01.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17194919256954562051', time=2024-04-17T07:23:22.136875+0000, chain.objs.size()=1 2024-04-17T05:46:01.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.arb18gJkSdIwG-xKYrHZJFxmQODgOms_0 2024-04-17T05:46:01.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/44026 2024-04-17T05:46:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=44, truncated=0, next_marker='' 2024-04-17T05:46:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10526688146290034680', time=2024-04-17T07:23:22.606719+0000, chain.objs.size()=1 2024-04-17T05:46:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5xYTLRU02sK1EsXHq_8PwDUBzWMZ2cN_0 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16411889214941976961', time=2024-04-17T07:23:25.698457+0000, chain.objs.size()=1 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DhbslyDCMV3k18zOCKqygJktUJEHCZg_0 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8537348771493632868', time=2024-04-17T07:23:26.701818+0000, chain.objs.size()=1 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mFNNrgRdBtpvKCUzOlTR-ofrcVtKLiz_0 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7741874260710202216', time=2024-04-17T07:23:26.704506+0000, chain.objs.size()=1 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.I7XN0lIx0tuhEYtU7w30-Zzu46-BVRI_0 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1666562113086545502', time=2024-04-17T07:23:26.839883+0000, chain.objs.size()=1 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EjHpPyzWF6wgZQdsqIxdmZIspu040Hl_0 2024-04-17T05:46:01.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3720855437280988788', time=2024-04-17T07:23:29.646713+0000, chain.objs.size()=1 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.S0V_0uoYYCSnRIO85S4-xbfnlkRhDBD_0 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2764568660068001987', time=2024-04-17T07:23:30.342385+0000, chain.objs.size()=1 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YN1hXiERIlGaAaS8rlBhBglMrK4ddjl_0 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6679672475531208050', time=2024-04-17T07:23:32.623315+0000, chain.objs.size()=1 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.p6ksKS9C7cETGQke4LAEIkG0jJ3oHKn_0 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10744982641101088756', time=2024-04-17T07:23:32.868595+0000, chain.objs.size()=1 2024-04-17T05:46:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GoEzz6dshUHvGc_wBUYHSQMAWMFMEPn_0 2024-04-17T05:46:01.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5297760538182044266', time=2024-04-17T07:23:32.974249+0000, chain.objs.size()=1 2024-04-17T05:46:01.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bD-SOqPxFOrNx_J8n1iGTe2dbrDld_D_0 2024-04-17T05:46:01.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7604665648506114949', time=2024-04-17T07:23:33.906034+0000, chain.objs.size()=1 2024-04-17T05:46:01.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MD8-2DMfQwOCsTDuO0Z1zW2C9RRpjP9_0 2024-04-17T05:46:01.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1717400133981074801', time=2024-04-17T07:23:34.407108+0000, chain.objs.size()=1 2024-04-17T05:46:01.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1KZNKJAf9gnGdsboHs8bvQ5VyH--omk_0 2024-04-17T05:46:01.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.612+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6088788720144784471', time=2024-04-17T07:23:34.476898+0000, chain.objs.size()=1 2024-04-17T05:46:01.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.612+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TjViu3wfuKtsg7AOP7Umtu3O4tiQgSs_0 2024-04-17T05:46:01.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.111024175686569398', time=2024-04-17T07:23:37.030928+0000, chain.objs.size()=1 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.M9j-wtmX4z-HtNkEOlkjsIvUD3B8NUi_0 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4215935892450019906', time=2024-04-17T07:23:37.032586+0000, chain.objs.size()=1 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fa09cf-Xz_pvepjmLfsMHuz6uEpkSbY_0 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8085855862436757826', time=2024-04-17T07:23:37.921379+0000, chain.objs.size()=1 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.avoywkNH8K-uKZCt3NsJHoDPUwbUIqt_0 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15998989895528498959', time=2024-04-17T07:23:38.712232+0000, chain.objs.size()=1 2024-04-17T05:46:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vTPdgXilJ5KC_kZ6S_DQ2yvj7cQzD2I_0 2024-04-17T05:46:01.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15989417753711235276', time=2024-04-17T07:23:39.807414+0000, chain.objs.size()=1 2024-04-17T05:46:01.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pVKu4p-UdjgjuI7sGntdojYRX9hpXka_0 2024-04-17T05:46:01.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14880257619808836442', time=2024-04-17T07:23:39.981633+0000, chain.objs.size()=1 2024-04-17T05:46:01.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZwqrqpzDjDVEEaQooagA0f8bBe_u6yK_0 2024-04-17T05:46:01.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12993944038081297724', time=2024-04-17T07:23:40.335148+0000, chain.objs.size()=1 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KPYcO3xMIIbG03yV_b7fbcBrfhPOEg6_0 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16823137160194491646', time=2024-04-17T07:23:40.815875+0000, chain.objs.size()=1 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.je2JC62Ht-uLwAHoKNnqVKCbSCTZpCE_0 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15407311938639476978', time=2024-04-17T07:23:41.682099+0000, chain.objs.size()=1 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wm13GaAsSTfycayyYjwf8uk8Z7xnE7A_0 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14977833050285148057', time=2024-04-17T07:23:53.360230+0000, chain.objs.size()=1 2024-04-17T05:46:01.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.326__shadow_.HM1FOdtBsTMGjZF9CLF9ZfLVGyDIQAa_0 2024-04-17T05:46:01.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9921410717576477011', time=2024-04-17T07:24:34.607277+0000, chain.objs.size()=1 2024-04-17T05:46:01.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.373__shadow_.BYHi8S-IG0sLF3G8qWOFKDjKInBbDiJ_0 2024-04-17T05:46:01.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13870830821475578456', time=2024-04-17T07:25:20.305999+0000, chain.objs.size()=1 2024-04-17T05:46:01.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.424__shadow_.VXsdfD9nh2nHMVQ_cyOshfHgZBQ9DRH_0 2024-04-17T05:46:01.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9376852932160028672', time=2024-04-17T07:27:10.441226+0000, chain.objs.size()=1 2024-04-17T05:46:01.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.508__multipart_mymultipart.2~rhFLe9-nBhImAl-8Hln5EiA9GP4UlBT.1 2024-04-17T05:46:01.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13600695665725911239', time=2024-04-17T07:27:15.646058+0000, chain.objs.size()=1 2024-04-17T05:46:01.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.507__shadow_.3fJaQsUIe7-JWkEVkrTP9z5SNecOzeZ_0 2024-04-17T05:46:01.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5878436538862092516', time=2024-04-17T07:27:43.155413+0000, chain.objs.size()=3 2024-04-17T05:46:01.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~thtZubYd6WLjzcVJjCb127xD5Eg4ruZ.1 2024-04-17T05:46:01.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~thtZubYd6WLjzcVJjCb127xD5Eg4ruZ.2 2024-04-17T05:46:01.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~thtZubYd6WLjzcVJjCb127xD5Eg4ruZ.3 2024-04-17T05:46:01.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A', time=2024-04-17T07:29:02.231632+0000, chain.objs.size()=1 2024-04-17T05:46:01.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A.1 2024-04-17T05:46:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A', time=2024-04-17T07:29:02.241811+0000, chain.objs.size()=1 2024-04-17T05:46:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A.2 2024-04-17T05:46:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A', time=2024-04-17T07:29:02.251392+0000, chain.objs.size()=1 2024-04-17T05:46:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A.3 2024-04-17T05:46:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A', time=2024-04-17T07:29:02.261093+0000, chain.objs.size()=1 2024-04-17T05:46:01.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A.4 2024-04-17T05:46:01.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A', time=2024-04-17T07:29:02.271088+0000, chain.objs.size()=1 2024-04-17T05:46:01.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A.5 2024-04-17T05:46:01.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A', time=2024-04-17T07:29:02.280332+0000, chain.objs.size()=1 2024-04-17T05:46:01.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~L4vhkdPEcvnpIY9mryVsW-YeZg6OC5A.6 2024-04-17T05:46:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~_F5nxyh0ycvfQsCMDSGGX3LjBIG4vB0', time=2024-04-17T07:29:51.775050+0000, chain.objs.size()=2 2024-04-17T05:46:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.519__multipart_mymultipart.2~_F5nxyh0ycvfQsCMDSGGX3LjBIG4vB0.1 2024-04-17T05:46:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.519__multipart_mymultipart.2~_F5nxyh0ycvfQsCMDSGGX3LjBIG4vB0.2 2024-04-17T05:46:01.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11626424534951489843', time=2024-04-17T07:31:20.460552+0000, chain.objs.size()=1 2024-04-17T05:46:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.550__multipart_mymultipart.vvfWTggBfUGBVBTKOJ4CIIUxz3f9mL23.1 2024-04-17T05:46:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16380645840739492729', time=2024-04-17T07:32:10.118502+0000, chain.objs.size()=1 2024-04-17T05:46:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:OZxu0h7Q9iKTqNhKMWi0q.0cmphrL41_.mPYk5oNBb1raZA2nlEP8L27yXHBtPuX_0 2024-04-17T05:46:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18274021920092218608', time=2024-04-17T07:32:46.137755+0000, chain.objs.size()=1 2024-04-17T05:46:01.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:8udtbi5M-LJ3xSKnwLS5v0SukJgRgQm_.QvfGQ82NieNDK29V927_WRzYxVD6p6J_0 2024-04-17T05:46:01.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13054439475888595689', time=2024-04-17T07:33:33.381522+0000, chain.objs.size()=1 2024-04-17T05:46:01.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:emuOQEr.v5f7US0CQbgNiL6oyTvPHBn_.8B0PANt9SXHPvpiPHZO8Id_om9c4vas_0 2024-04-17T05:46:01.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_1_247389e134be789890c85fa067c3fdb8', time=2024-04-17T07:33:33.454074+0000, chain.objs.size()=1 2024-04-17T05:46:01.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_0 2024-04-17T05:46:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_1_1d4956bfc8c8f27c3623136ee67cb255', time=2024-04-17T07:33:34.124440+0000, chain.objs.size()=1 2024-04-17T05:46:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_0 2024-04-17T05:46:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6377121642261733914', time=2024-04-17T07:34:47.541387+0000, chain.objs.size()=6 2024-04-17T05:46:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.602__multipart_multipart_enc.2~JyOtPrGC6Gy2UTUl3kBLegEIZHf9veM.1 2024-04-17T05:46:01.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.602__multipart_multipart_enc.2~JyOtPrGC6Gy2UTUl3kBLegEIZHf9veM.2 2024-04-17T05:46:01.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.602__multipart_multipart_enc.2~JyOtPrGC6Gy2UTUl3kBLegEIZHf9veM.3 2024-04-17T05:46:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.602__multipart_multipart_enc.2~JyOtPrGC6Gy2UTUl3kBLegEIZHf9veM.4 2024-04-17T05:46:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.602__multipart_multipart_enc.2~JyOtPrGC6Gy2UTUl3kBLegEIZHf9veM.5 2024-04-17T05:46:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.602__multipart_multipart_enc.2~JyOtPrGC6Gy2UTUl3kBLegEIZHf9veM.6 2024-04-17T05:46:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2998892959408904464', time=2024-04-17T07:37:33.336946+0000, chain.objs.size()=6 2024-04-17T05:46:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.645__multipart_bar.2~dUDBAoMczXWHE5Dr6Ub_nY1ranDIBP2.1 2024-04-17T05:46:01.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.645__multipart_bar.2~dUDBAoMczXWHE5Dr6Ub_nY1ranDIBP2.2 2024-04-17T05:46:01.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.645__multipart_bar.2~dUDBAoMczXWHE5Dr6Ub_nY1ranDIBP2.3 2024-04-17T05:46:01.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.645__multipart_bar.2~dUDBAoMczXWHE5Dr6Ub_nY1ranDIBP2.4 2024-04-17T05:46:01.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.645__multipart_bar.2~dUDBAoMczXWHE5Dr6Ub_nY1ranDIBP2.5 2024-04-17T05:46:01.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.645__multipart_bar.2~dUDBAoMczXWHE5Dr6Ub_nY1ranDIBP2.6 2024-04-17T05:46:01.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9555995457144848948', time=2024-04-17T07:38:22.165214+0000, chain.objs.size()=1 2024-04-17T05:46:01.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.691__shadow_.SKxpOpfeYn4YVY7XDCdBUD4LcTA-LkF_0 2024-04-17T05:46:01.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:01.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-17T05:46:01.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48038' 2024-04-17T05:46:01.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15476325860051709670', time=2024-04-17T07:10:11.421979+0000, chain.objs.size()=2 2024-04-17T05:46:01.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.PMuwX28i4qJqPvMJzDXI8q0lQSbyhLx_0 2024-04-17T05:46:01.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.PMuwX28i4qJqPvMJzDXI8q0lQSbyhLx_1 2024-04-17T05:46:01.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1035019877651016439', time=2024-04-17T07:14:36.319285+0000, chain.objs.size()=1 2024-04-17T05:46:01.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.226__shadow_.qlU016cUtkuDPye0CViWCHFaZ3l7TFd_0 2024-04-17T05:46:01.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12989087218843983621', time=2024-04-17T07:14:48.895528+0000, chain.objs.size()=1 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.233__shadow_.zS-d17Sq7E_SI3O_Wwelt_vsIJAHSFR_0 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5843888975364911299', time=2024-04-17T07:15:10.753183+0000, chain.objs.size()=1 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.246__shadow_.HuBJoysCiUv1cu8SNDmf2NYpZazq3ew_0 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3318022330819028885', time=2024-04-17T07:17:01.355049+0000, chain.objs.size()=1 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kF81hTb2zr_Zd5ylg8z0tfzhV0LxwcU_0 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8822255642255179388', time=2024-04-17T07:17:02.259087+0000, chain.objs.size()=1 2024-04-17T05:46:01.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LYVuZIIIpROf8mP1Zqg5YbZUkSjMwCx_0 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7191530363890361760', time=2024-04-17T07:17:04.347224+0000, chain.objs.size()=1 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gUhVnG5xct7OUSFq1E5RhBDpTdMpHci_0 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1237507609476771147', time=2024-04-17T07:17:05.501150+0000, chain.objs.size()=1 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-LgtU-Kt9Ja0QuapxtX_fs-vA4LWHGY_0 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7465930777664990210', time=2024-04-17T07:17:05.628883+0000, chain.objs.size()=1 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UQauPAoECtZQlZfMp8HS6g1uaIqqf2J_0 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6922547050073751466', time=2024-04-17T07:17:06.847420+0000, chain.objs.size()=1 2024-04-17T05:46:01.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.R76RmT6fTt-M-Ijt8zJp3JaG3F_5c3A_0 2024-04-17T05:46:01.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8994416806476894136', time=2024-04-17T07:17:07.108125+0000, chain.objs.size()=1 2024-04-17T05:46:01.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FHoRmck8dDtYQP3_Oq2iHUA_O8nCj6L_0 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10797275855547664851', time=2024-04-17T07:17:07.251716+0000, chain.objs.size()=1 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MIGSp7_qOldrL918sR5I789KVXrwoxw_0 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8483691039554137449', time=2024-04-17T07:17:08.415839+0000, chain.objs.size()=1 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4In8ldactv6nVQCiafaXS_oM3W4oAC0_0 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1178857265403800270', time=2024-04-17T07:17:10.328104+0000, chain.objs.size()=1 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hoHPdARMuMynNaqD15rvojrNOI5nnOx_0 2024-04-17T05:46:01.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5140621481082544069', time=2024-04-17T07:17:10.739604+0000, chain.objs.size()=1 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1EZ6QYp4vNN9eLDrKLM7eeQ7ATQVdVM_0 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8965529203763994457', time=2024-04-17T07:17:11.103457+0000, chain.objs.size()=1 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dsjVLmeG2yxi9WhR4dR8FhZlJ7_GLWC_0 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18092156680215392863', time=2024-04-17T07:17:11.437369+0000, chain.objs.size()=1 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ip3ej1e0tEpmciuDwJiBJ4NgGOlhW7Z_0 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14562602692144819509', time=2024-04-17T07:17:12.795486+0000, chain.objs.size()=1 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.belmuH-0RppKCrA0dwFvwCr4GZB7hbB_0 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17677338128909448854', time=2024-04-17T07:17:12.931416+0000, chain.objs.size()=1 2024-04-17T05:46:01.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.E5P61eP8LhLSGUHcMZy8np3dy3m8cml_0 2024-04-17T05:46:01.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1976187641783830258', time=2024-04-17T07:17:13.946254+0000, chain.objs.size()=1 2024-04-17T05:46:01.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YleHKHYcvrBu26FdsvzU-qx5OekyAjE_0 2024-04-17T05:46:01.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12110140046204977546', time=2024-04-17T07:17:14.069998+0000, chain.objs.size()=1 2024-04-17T05:46:01.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gonz338ZVkjiYOjTYKwhP0IN0mHBXu8_0 2024-04-17T05:46:01.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7640015318455045255', time=2024-04-17T07:17:15.421371+0000, chain.objs.size()=1 2024-04-17T05:46:01.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rMp0zHRhytM2kEYdpDjTQ4xzpT1Nsz8_0 2024-04-17T05:46:01.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8027277324336433910', time=2024-04-17T07:17:18.192022+0000, chain.objs.size()=1 2024-04-17T05:46:01.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OJO_BfVT_s3q-qtO7u22hDfgCydQRJJ_0 2024-04-17T05:46:01.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14640136342712249668', time=2024-04-17T07:17:19.211452+0000, chain.objs.size()=1 2024-04-17T05:46:01.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.I5F9VncCqwxkFwYcGDDC0jztIc0hmm3_0 2024-04-17T05:46:01.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.443281666194284184', time=2024-04-17T07:17:19.692334+0000, chain.objs.size()=1 2024-04-17T05:46:01.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uGU8vOT6DszPdrwr0GK4s9pO4Na_9Yb_0 2024-04-17T05:46:01.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8044049689425487422', time=2024-04-17T07:17:19.868557+0000, chain.objs.size()=1 2024-04-17T05:46:01.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZzMtCKVxBltPdbMAU-hQCB2Ii3acPup_0 2024-04-17T05:46:01.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15830191092093752060', time=2024-04-17T07:17:20.101019+0000, chain.objs.size()=1 2024-04-17T05:46:01.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7rh787X1MCwiq0QDuotO_i3-XoQ3fFi_0 2024-04-17T05:46:01.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1265364631017209808', time=2024-04-17T07:17:21.027604+0000, chain.objs.size()=1 2024-04-17T05:46:01.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.d8KJLftBNmSwVvqMXitfoTN86dBFEKh_0 2024-04-17T05:46:01.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16578025807162481213', time=2024-04-17T07:17:22.706078+0000, chain.objs.size()=1 2024-04-17T05:46:01.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TTFluXSoH23evQt36Quidfgk91h_hBp_0 2024-04-17T05:46:01.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12046938678369231566', time=2024-04-17T07:17:23.676749+0000, chain.objs.size()=1 2024-04-17T05:46:01.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZU9t78TCm0Ri8PDPeE7en3PgHyiFdgz_0 2024-04-17T05:46:01.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16870789411411820545', time=2024-04-17T07:17:24.099008+0000, chain.objs.size()=1 2024-04-17T05:46:01.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8JLwJWdUsCpEl95G0pQCZLL16f5zp1r_0 2024-04-17T05:46:01.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4258199869607894462', time=2024-04-17T07:17:24.599792+0000, chain.objs.size()=1 2024-04-17T05:46:01.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7g7qDJKP24vv26jWOl6Vp8cfeMwQDr3_0 2024-04-17T05:46:01.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8344922590395912796', time=2024-04-17T07:17:27.459763+0000, chain.objs.size()=1 2024-04-17T05:46:01.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gdMVKJTWxhUPnvLZ8Db6PnN62Ej8H4g_0 2024-04-17T05:46:01.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15915263019011781949', time=2024-04-17T07:17:27.466944+0000, chain.objs.size()=1 2024-04-17T05:46:01.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pudGnV9-NeGCvIgOLVil7u2Ki5E-g3y_0 2024-04-17T05:46:01.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9336102253473957733', time=2024-04-17T07:17:27.822872+0000, chain.objs.size()=1 2024-04-17T05:46:01.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hheZCMXBWhxBjkBMszW2BY8Zo91WDci_0 2024-04-17T05:46:01.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12869570209297906934', time=2024-04-17T07:17:27.964746+0000, chain.objs.size()=1 2024-04-17T05:46:01.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yyzaOMlGFrI63s3yvd3qcpAZngz9PeU_0 2024-04-17T05:46:01.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14115338795600276773', time=2024-04-17T07:17:28.626274+0000, chain.objs.size()=1 2024-04-17T05:46:01.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ecSxP5m7FBXBw8sy7oHBCX2jhFKDB8c_0 2024-04-17T05:46:01.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18217641260658172873', time=2024-04-17T07:17:30.043692+0000, chain.objs.size()=1 2024-04-17T05:46:01.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nqUVycLMrddZh0-xGwL1VrC9Fr5RweR_0 2024-04-17T05:46:01.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14867591954196330974', time=2024-04-17T07:17:30.338774+0000, chain.objs.size()=1 2024-04-17T05:46:01.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7uMK9jcY8X7cRLPI7yLdENntF0ipQg2_0 2024-04-17T05:46:01.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9593982438040252560', time=2024-04-17T07:17:30.651666+0000, chain.objs.size()=1 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bvxq7pHCXQET_sms2ZLvRpx9XaJMKuE_0 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13031030262796642786', time=2024-04-17T07:17:31.100126+0000, chain.objs.size()=1 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YBNs_wqbdsAnDRYROQa-uPRBfeEQmdE_0 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8275506287528783782', time=2024-04-17T07:17:31.886660+0000, chain.objs.size()=1 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vr7eUS8fm-44e-A_Y3lBmuUIqxmMlat_0 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14963365045524851084', time=2024-04-17T07:17:32.046418+0000, chain.objs.size()=1 2024-04-17T05:46:01.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xbjDciJ0pvQITLTgi58gQnAEg6JHEFc_0 2024-04-17T05:46:01.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.802616291977892539', time=2024-04-17T07:17:32.048446+0000, chain.objs.size()=1 2024-04-17T05:46:01.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SrSrdY0aopu18thEcelbuczSY9oF5mH_0 2024-04-17T05:46:01.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13044924564958404456', time=2024-04-17T07:17:47.369708+0000, chain.objs.size()=1 2024-04-17T05:46:01.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.262__shadow_.imqeVhYTE2EAtszgYHZ2z_rqIkyZwyd_0 2024-04-17T05:46:01.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7302705208208713638', time=2024-04-17T07:17:48.855573+0000, chain.objs.size()=1 2024-04-17T05:46:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.263__shadow_.X0KmJ0ShAjBadAHCj5M4OV5tDJJXpPD_0 2024-04-17T05:46:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5943137781144327216', time=2024-04-17T07:17:56.857166+0000, chain.objs.size()=1 2024-04-17T05:46:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.268__shadow_.YvEHvQDMTsIYGTkNdoOrkKNV1T0jpYt_0 2024-04-17T05:46:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4767526799833009239', time=2024-04-17T07:18:22.975442+0000, chain.objs.size()=1 2024-04-17T05:46:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.Rix4rONjdc_j16Rg8sOmEnhcGtzHwNN_0 2024-04-17T05:46:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9916057026864334586', time=2024-04-17T07:18:23.235998+0000, chain.objs.size()=1 2024-04-17T05:46:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.3HQp3_ZJU4VRZmtEcMy1lc4myX4pT1l_0 2024-04-17T05:46:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13925451544158277171', time=2024-04-17T07:18:28.279607+0000, chain.objs.size()=1 2024-04-17T05:46:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.DoGcIIOKNtDdrJp18nc_elu2LxkiZYm_0 2024-04-17T05:46:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13926473921879007729', time=2024-04-17T07:18:33.116121+0000, chain.objs.size()=1 2024-04-17T05:46:01.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.286__shadow_.9g8BRD0vY4k-eSz7QVUvSYk80iW85sQ_0 2024-04-17T05:46:01.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18044631730677434157', time=2024-04-17T07:18:43.317159+0000, chain.objs.size()=1 2024-04-17T05:46:01.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.292__shadow_.k-04WGf7XXmt8Nms9BkMecOy_PP5vr2_0 2024-04-17T05:46:01.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9705739628646479872', time=2024-04-17T07:21:00.230587+0000, chain.objs.size()=1 2024-04-17T05:46:01.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t1w0Kj4w8Slu3IpL4vJkw4lviT-AWAo_0 2024-04-17T05:46:01.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13890558935320794279', time=2024-04-17T07:21:00.829324+0000, chain.objs.size()=1 2024-04-17T05:46:01.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jrty9OoOb6j-cA7Ksf3n1uy0h7xj3HW_0 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5690454236287046018', time=2024-04-17T07:21:01.284978+0000, chain.objs.size()=1 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._520azZkHvpBua-aJwD0hVxlSQ1n-7Z_0 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4535700301219151375', time=2024-04-17T07:21:01.863623+0000, chain.objs.size()=1 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eu_1ueW5s6T0kXJPjt8n80KbklKb0PX_0 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12095258042012316715', time=2024-04-17T07:21:02.198321+0000, chain.objs.size()=1 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tPBm-CAQg7w72dSBL6LMI4Drw0bMQs4_0 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12166916777737605495', time=2024-04-17T07:21:04.420930+0000, chain.objs.size()=1 2024-04-17T05:46:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iC3QRpWuoxNSHe7mSObRcyP31_kDMli_0 2024-04-17T05:46:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10025273374391937805', time=2024-04-17T07:21:04.970678+0000, chain.objs.size()=1 2024-04-17T05:46:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jp46Luf88yquC0JjsFbZ4AO7z8FSQ7E_0 2024-04-17T05:46:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3239515390096239648', time=2024-04-17T07:21:05.187754+0000, chain.objs.size()=1 2024-04-17T05:46:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Y7riaGfHYJzEhRpBtSnrmDJ3R_HWOHN_0 2024-04-17T05:46:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3863457554855633906', time=2024-04-17T07:21:05.556936+0000, chain.objs.size()=1 2024-04-17T05:46:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9VJgWOgLb6c2H6skb38egzFXU1wakUB_0 2024-04-17T05:46:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1972642273155741396', time=2024-04-17T07:21:05.822687+0000, chain.objs.size()=1 2024-04-17T05:46:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.05oRaxTntlf4FvHcTFeX2K7a3tHdM2c_0 2024-04-17T05:46:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4381589904811210551', time=2024-04-17T07:21:05.947866+0000, chain.objs.size()=1 2024-04-17T05:46:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4zgg0sJj-jeWARO5_-m-mFhWVZ0tsLV_0 2024-04-17T05:46:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1516223217104411246', time=2024-04-17T07:21:07.897976+0000, chain.objs.size()=1 2024-04-17T05:46:01.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DaFeE4EbNznyNG12laRn1RIuP9fUvOv_0 2024-04-17T05:46:01.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6504675070953523635', time=2024-04-17T07:21:08.007353+0000, chain.objs.size()=1 2024-04-17T05:46:01.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iou3UvZ8g21XCTFa_x8_ZYKpX_9Z-kB_0 2024-04-17T05:46:01.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15241625857654038490', time=2024-04-17T07:21:08.513228+0000, chain.objs.size()=1 2024-04-17T05:46:01.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UiVcGSTLnBRtDrTtOHnc5yY9ED7w9Ip_0 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12591513388236749998', time=2024-04-17T07:21:09.415844+0000, chain.objs.size()=1 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Z15csYkIWBWC7CUkRRqWlT3frw3-LmY_0 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9680496460905066398', time=2024-04-17T07:21:10.408750+0000, chain.objs.size()=1 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eSSI7k2o45HeXe31Z3T9Xf_rhO2X69m_0 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7695302938813471726', time=2024-04-17T07:21:10.627118+0000, chain.objs.size()=1 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oSt05Sbas1o_AqJvVV44GoFxKnbyJxL_0 2024-04-17T05:46:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9198098181332006605', time=2024-04-17T07:21:10.857141+0000, chain.objs.size()=1 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mG340GMKQyes_NH62ssO5D-oQsQHLcN_0 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15371865734250183416', time=2024-04-17T07:21:10.860207+0000, chain.objs.size()=1 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.NclHZN9prSx8JHSd1x-g7YWVCC3nGy4_0 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10707244877633255794', time=2024-04-17T07:21:11.881328+0000, chain.objs.size()=1 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DlwLGmjuvLRAQtDKNGGXkXUiLlwbGZt_0 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12817465977043485', time=2024-04-17T07:21:12.292635+0000, chain.objs.size()=1 2024-04-17T05:46:01.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.stikyirhSVriesINf5fnGGyw7d5Lnc2_0 2024-04-17T05:46:01.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12323238543409653440', time=2024-04-17T07:21:12.719319+0000, chain.objs.size()=1 2024-04-17T05:46:01.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aR731uWrxC_GfteQcjLvGxvE5FdwQYF_0 2024-04-17T05:46:01.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5419875241586254121', time=2024-04-17T07:21:14.133348+0000, chain.objs.size()=1 2024-04-17T05:46:01.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DGhjKQxe4Ii2TcCE-nWHm5Zc09c9_qs_0 2024-04-17T05:46:01.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10438396912180229890', time=2024-04-17T07:21:16.469901+0000, chain.objs.size()=1 2024-04-17T05:46:01.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pkUW2wC6ZQCXZ1AGqQPy3d66PCB1uRN_0 2024-04-17T05:46:01.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4897384101370952668', time=2024-04-17T07:21:17.302696+0000, chain.objs.size()=1 2024-04-17T05:46:01.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.E29xBGfYS9hYRNcfwfEMXN6Eygen9cG_0 2024-04-17T05:46:01.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4309347070437676689', time=2024-04-17T07:21:17.767375+0000, chain.objs.size()=1 2024-04-17T05:46:01.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UvIQPWTUrvFCE6CluEGP7OXJvcfDHxT_0 2024-04-17T05:46:01.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7761089914374784584', time=2024-04-17T07:21:23.278739+0000, chain.objs.size()=1 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Yo9eootjvATwVLVbFwEqmcdE3NKp4iB_0 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11433180063838606874', time=2024-04-17T07:21:23.324512+0000, chain.objs.size()=1 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0kcRtbH8Z6fmUx2kvdLs5ARscSSY4Vr_0 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11909911514920919628', time=2024-04-17T07:21:23.726064+0000, chain.objs.size()=1 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xtKw9h9N7jT0DL7M-UT8TvHVSJQyHzX_0 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5736503414923617282', time=2024-04-17T07:21:24.465913+0000, chain.objs.size()=1 2024-04-17T05:46:01.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xrm9TICc6_9VY5V9zgxc92An55gteFR_0 2024-04-17T05:46:01.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9198973624924242450', time=2024-04-17T07:21:26.106048+0000, chain.objs.size()=1 2024-04-17T05:46:01.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UAy24Ywn6PQUm9wn80xpneLn_zmEq8O_0 2024-04-17T05:46:01.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16162698111416603181', time=2024-04-17T07:21:28.559573+0000, chain.objs.size()=1 2024-04-17T05:46:01.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UnSBQKQCo5f3LdvrDY9ogPg_75zx9zn_0 2024-04-17T05:46:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2273934091334567998', time=2024-04-17T07:21:29.419125+0000, chain.objs.size()=1 2024-04-17T05:46:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2_TxzBhq70usw5WzILbwhe7zW7cJHF2_0 2024-04-17T05:46:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10183641332183233117', time=2024-04-17T07:23:14.452683+0000, chain.objs.size()=1 2024-04-17T05:46:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gY2vG5rtwaqYFOhog44d8gErgf5W3kJ_0 2024-04-17T05:46:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4982496976827620361', time=2024-04-17T07:23:14.761526+0000, chain.objs.size()=1 2024-04-17T05:46:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aErrVKw0i8pelE_JSO4G0ajmsPaOsj0_0 2024-04-17T05:46:01.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8396747170851117066', time=2024-04-17T07:23:17.734404+0000, chain.objs.size()=1 2024-04-17T05:46:01.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wRuFrIYwZBmBPduExF3koICU9EJqxdX_0 2024-04-17T05:46:01.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9605919652171562739', time=2024-04-17T07:23:18.128519+0000, chain.objs.size()=1 2024-04-17T05:46:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.W_UWTcNhuCL8-d5Nzft-LWlsEqfjK2-_0 2024-04-17T05:46:01.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9404428193051015115', time=2024-04-17T07:23:21.284371+0000, chain.objs.size()=1 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.W5-hyHezxber1CE0yfsO3-HV1R2vXXK_0 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15936774758685841835', time=2024-04-17T07:23:23.521908+0000, chain.objs.size()=1 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xmu5_slsbX570ESAUJfF7ZlWMMC4lEg_0 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14995000130252480681', time=2024-04-17T07:23:23.836174+0000, chain.objs.size()=1 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.n1dpXEAqZS8ilYW0JiGh1N6fm65sPbJ_0 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17785035968129149478', time=2024-04-17T07:23:26.042548+0000, chain.objs.size()=1 2024-04-17T05:46:01.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ybKDdvHQ-EPYe9bw9N057BgKhZ2sxjj_0 2024-04-17T05:46:01.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6406231006608929444', time=2024-04-17T07:23:27.559686+0000, chain.objs.size()=1 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vqZKE5hztG5IUjnuc7npPjdBZFxDIr__0 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17612039722586213020', time=2024-04-17T07:23:29.587104+0000, chain.objs.size()=1 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6cY7tc0BDG34-ey9BjL3Xe8aL-Ux7YQ_0 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11016504549563332046', time=2024-04-17T07:23:29.809347+0000, chain.objs.size()=1 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nQL1THQH_0rZCwdmtFkGxiQFCzLbhWN_0 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12750560076431269096', time=2024-04-17T07:23:33.712821+0000, chain.objs.size()=1 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yWQfq317H9t5Imbcs2xlmX2f2WHSR_t_0 2024-04-17T05:46:01.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11060758576736127996', time=2024-04-17T07:23:33.805687+0000, chain.objs.size()=1 2024-04-17T05:46:01.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Vg_Jt7kAPsKaEVxgftXcXr2z7dXm7DR_0 2024-04-17T05:46:01.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17892807277634348138', time=2024-04-17T07:23:33.979467+0000, chain.objs.size()=1 2024-04-17T05:46:01.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ysBzQ8umY6xWfAMsK8wfsW7Zo4kxuWJ_0 2024-04-17T05:46:01.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3374131758798343461', time=2024-04-17T07:23:34.138815+0000, chain.objs.size()=1 2024-04-17T05:46:01.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8SnqncI0BwacQbhIA6hMRtIU2XG7F0V_0 2024-04-17T05:46:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/48038 2024-04-17T05:46:01.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=22, truncated=0, next_marker='' 2024-04-17T05:46:01.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10869149826653372815', time=2024-04-17T07:23:34.577143+0000, chain.objs.size()=1 2024-04-17T05:46:01.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cEQ6xs9Sldegkme4s7RnZb3qBEHJ8zm_0 2024-04-17T05:46:01.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12883861975138527556', time=2024-04-17T07:23:34.918848+0000, chain.objs.size()=1 2024-04-17T05:46:01.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nXNuTOx0x8g2slU28eFkGNbVzBo3PxY_0 2024-04-17T05:46:01.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14109049374515681434', time=2024-04-17T07:23:35.057486+0000, chain.objs.size()=1 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.79WbtS_OB_TLdwPwiE7uwSkoYIIDYGu_0 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8524796712950108812', time=2024-04-17T07:23:35.896977+0000, chain.objs.size()=1 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4tPFi0I5jVAFAAXmw1358xzauUPiG_v_0 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5500635056314405610', time=2024-04-17T07:23:36.267863+0000, chain.objs.size()=1 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Xhw1ktCKF-vW4ffLnn6B-PuVYnX5EKO_0 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18204890562617713611', time=2024-04-17T07:23:37.833372+0000, chain.objs.size()=1 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-8ZuX2wuT04ITW_15XW7eicnTngPJWx_0 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2250335412867648717', time=2024-04-17T07:23:37.890059+0000, chain.objs.size()=1 2024-04-17T05:46:01.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iElQ9CpMBZyulttNuPAm29xp0jxWIjw_0 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5802534967164038663', time=2024-04-17T07:23:38.340878+0000, chain.objs.size()=1 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MdCv5ckxuQeKQSB_EuCTpOwlI8r3xCP_0 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8300948794852263334', time=2024-04-17T07:23:38.630907+0000, chain.objs.size()=1 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.HQXIBWkk9bPaPdDoUnv9VNHpuUM1r5F_0 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12758292516887843174', time=2024-04-17T07:23:41.238932+0000, chain.objs.size()=1 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.L1e4UZsE74afFAjuVJfU5nNK55FxsNs_0 2024-04-17T05:46:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17619199862762047604', time=2024-04-17T07:23:57.790591+0000, chain.objs.size()=1 2024-04-17T05:46:01.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.331__shadow_.Zda8ZyQo7_xcDvufZ0Oi52OPZe3sRD5_0 2024-04-17T05:46:01.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7068598231618670661', time=2024-04-17T07:25:29.642254+0000, chain.objs.size()=1 2024-04-17T05:46:01.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.432__shadow_.gEGHCtKihWeKyFOKfIjN5Yl7xzjpg7s_0 2024-04-17T05:46:01.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.728+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.477__shadow_._ZYSMRZ88c_6nOr-v9X-sfixA6Ew1M__1_83a4ebded6e18065dca0c5dcea1b53f1', time=2024-04-17T07:26:25.714071+0000, chain.objs.size()=1 2024-04-17T05:46:01.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.728+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.477__shadow_._ZYSMRZ88c_6nOr-v9X-sfixA6Ew1M__0 2024-04-17T05:46:01.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3644657562060415001', time=2024-04-17T07:29:20.061489+0000, chain.objs.size()=3 2024-04-17T05:46:01.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~2J1tR70lHlFDllT490LghvF1BN3y8D7.1 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~2J1tR70lHlFDllT490LghvF1BN3y8D7.2 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~2J1tR70lHlFDllT490LghvF1BN3y8D7.3 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16924306973849248201', time=2024-04-17T07:31:02.496344+0000, chain.objs.size()=1 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.545__shadow_.g5YpMS7w8K9KivYXNaYzP-jAL8uiZOU_0 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1150926601777810527', time=2024-04-17T07:31:07.329854+0000, chain.objs.size()=1 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.546__shadow_.JmjWfC1eygxl_obPKSl3MSYGLzJ1GvR_0 2024-04-17T05:46:01.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14928773724558422733', time=2024-04-17T07:31:29.924797+0000, chain.objs.size()=1 2024-04-17T05:46:01.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.554__shadow_.WOv6FrMZVlQ3DO-cHtNEfgljhZrYl1-_0 2024-04-17T05:46:01.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5533508520800846670', time=2024-04-17T07:31:50.286052+0000, chain.objs.size()=1 2024-04-17T05:46:01.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:YBGqvZ.lKJtpBKWeSYcTzutwDzTEwey_.8aWSqZERUTIRhv9YJEw7L2oayopDqTn_0 2024-04-17T05:46:01.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:aYkwN-EcPPlwfAR2VHy7nBHocJML3oL_.TDjeRwHRu8PYUf7zcW4M64h1eexc6Ie_1_ba9575b62589ea8b642eff8ede950f40', time=2024-04-17T07:33:34.122890+0000, chain.objs.size()=1 2024-04-17T05:46:01.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:aYkwN-EcPPlwfAR2VHy7nBHocJML3oL_.TDjeRwHRu8PYUf7zcW4M64h1eexc6Ie_0 2024-04-17T05:46:01.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17913184806354087038', time=2024-04-17T07:33:45.945585+0000, chain.objs.size()=1 2024-04-17T05:46:01.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:-qs.FVaCnGQ2QVYtbIJJTHJe9D9LchG_.M0xMNyZoKbETbvg5BMWTJy2ZXZyChNG_0 2024-04-17T05:46:01.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12897458174172372872', time=2024-04-17T07:36:46.465765+0000, chain.objs.size()=1 2024-04-17T05:46:01.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.629__shadow_.8z8tx_koPbNOjI9P8974hclZ8NtUgE8_0 2024-04-17T05:46:01.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16552423736260342855', time=2024-04-17T07:38:26.903475+0000, chain.objs.size()=1 2024-04-17T05:46:01.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.695__shadow:m9MuRk.CjjWmA6PpUZPlyD.xp-A0QFj_.7hdmVLquuDsoxQl-lLgmU6G6tAkAiSZ_0 2024-04-17T05:46:01.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:01.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-17T05:46:01.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44294' 2024-04-17T05:46:01.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14083229456872573044', time=2024-04-17T07:11:10.080728+0000, chain.objs.size()=1 2024-04-17T05:46:01.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.179__shadow_.3QC-_rK8A4F9IabeHcRuVrqIa2Jn69e_0 2024-04-17T05:46:01.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1181325548737118399', time=2024-04-17T07:14:46.604351+0000, chain.objs.size()=1 2024-04-17T05:46:01.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.232__shadow_.1pTC0VE5kfI7KnnO9GriClL9kXRq1uq_0 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.379861579732384254', time=2024-04-17T07:14:59.382744+0000, chain.objs.size()=1 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.239__shadow_._8M4DXvPXI2J2M0l88bQiyop1IUqrAt_0 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7976335145374933205', time=2024-04-17T07:15:00.972223+0000, chain.objs.size()=1 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.240__shadow_.ulqvlZsaax6Lo5NIXTDLSLwym-9CzG7_0 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12139021378216554465', time=2024-04-17T07:17:01.757284+0000, chain.objs.size()=1 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.obYTuzonCoSo1gAn46N6a5CdRwXJiGY_0 2024-04-17T05:46:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8412225366267073822', time=2024-04-17T07:17:06.797225+0000, chain.objs.size()=1 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vFry1NNXgOSEaqXbhkoRyUrxbcpe2vO_0 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11735980926290856506', time=2024-04-17T07:17:07.390619+0000, chain.objs.size()=1 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4yx9jMj7KT0BkI4VsIKIrWJaw7WSDdt_0 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10800791552161994612', time=2024-04-17T07:17:07.392103+0000, chain.objs.size()=1 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uLD3M8c0ZwZUyjz4LkF9P6MDq1N7HvP_0 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6538259087504901035', time=2024-04-17T07:17:08.430879+0000, chain.objs.size()=1 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uc0Xidpp8uB42AFBPWviASRSJBRCFzz_0 2024-04-17T05:46:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16436378388831790358', time=2024-04-17T07:17:10.684974+0000, chain.objs.size()=1 2024-04-17T05:46:01.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bMPrptdOxJ0o_PWz_-daMe1mnBpe0IV_0 2024-04-17T05:46:01.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7957939282828330381', time=2024-04-17T07:17:10.896024+0000, chain.objs.size()=1 2024-04-17T05:46:01.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ls4mP2FgaLgdCsC1DpBjkwQt6E3gQCw_0 2024-04-17T05:46:01.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.130520542020382920', time=2024-04-17T07:17:11.927736+0000, chain.objs.size()=1 2024-04-17T05:46:01.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2Uk1SckXKpFNY5O_YElu1MImHCPZNEy_0 2024-04-17T05:46:01.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5280653336969859447', time=2024-04-17T07:17:12.583747+0000, chain.objs.size()=1 2024-04-17T05:46:01.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.t9W675Rx2oUE7Z58XGn5xQaEz4NMdN9_0 2024-04-17T05:46:01.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2325352738369109878', time=2024-04-17T07:17:13.004022+0000, chain.objs.size()=1 2024-04-17T05:46:01.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-mdhfIMmUwzO3l6JbYlOKWcPF8F-v9x_0 2024-04-17T05:46:01.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7545641353736745017', time=2024-04-17T07:17:14.114663+0000, chain.objs.size()=1 2024-04-17T05:46:01.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Rotplyt0k1Om71EDjWCscd23iD_IUrp_0 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15113075505564370814', time=2024-04-17T07:17:14.572859+0000, chain.objs.size()=1 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZmL1HQsP14yYqzNcoJXlmdKz8RTQpus_0 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.21453812996857470', time=2024-04-17T07:17:15.907995+0000, chain.objs.size()=1 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6uKcfubzTfNipXG4XAn5GuYP7hXX8AI_0 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8916055126979763531', time=2024-04-17T07:17:18.356797+0000, chain.objs.size()=1 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VIcyLtMrXnbpcmLKqH4n5J8EgxUa337_0 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13958981110835651950', time=2024-04-17T07:17:19.209896+0000, chain.objs.size()=1 2024-04-17T05:46:01.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.a5H16gdZI7nc63j1Z-F9xBmMqKx8DMx_0 2024-04-17T05:46:01.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3555451705360109834', time=2024-04-17T07:17:19.308986+0000, chain.objs.size()=1 2024-04-17T05:46:01.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QMmI6VpkogiiQwDvQFFmI9YjcrsbatS_0 2024-04-17T05:46:01.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3966168374033038684', time=2024-04-17T07:17:21.956526+0000, chain.objs.size()=1 2024-04-17T05:46:01.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EumLgYJ0gtlWvSxNdiZcii0osp74qHN_0 2024-04-17T05:46:01.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7795444701051300897', time=2024-04-17T07:17:23.822883+0000, chain.objs.size()=1 2024-04-17T05:46:01.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9k2JcKMDBGcppnjp-9NLmySJctc9JvY_0 2024-04-17T05:46:01.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3366257763701909435', time=2024-04-17T07:17:24.626638+0000, chain.objs.size()=1 2024-04-17T05:46:01.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YDRGG-vlYYK_9UwncsvuiXvEp4KM9LY_0 2024-04-17T05:46:01.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4671135099523964337', time=2024-04-17T07:17:26.069025+0000, chain.objs.size()=1 2024-04-17T05:46:01.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JeukJgQRmzdIz9ZtbFJBB1ZVQF14k64_0 2024-04-17T05:46:01.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7818782726650720497', time=2024-04-17T07:17:26.202014+0000, chain.objs.size()=1 2024-04-17T05:46:01.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tzu4u6UJt9fercIFoGohHWKBk1WaPhP_0 2024-04-17T05:46:01.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2978667750748796401', time=2024-04-17T07:17:29.879770+0000, chain.objs.size()=1 2024-04-17T05:46:01.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wg9FrnJ5Pi-Q5QnlYd90FH6ruIzIKtw_0 2024-04-17T05:46:01.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15614196523022985466', time=2024-04-17T07:17:32.276046+0000, chain.objs.size()=1 2024-04-17T05:46:01.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Mrv-hoodYkheWORALRW30I_su8-ZgMX_0 2024-04-17T05:46:01.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14413508908335416871', time=2024-04-17T07:18:23.250746+0000, chain.objs.size()=1 2024-04-17T05:46:01.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.OFRTwlE1SljWjGkkdJzHou1QHwz-Y_b_0 2024-04-17T05:46:01.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13449507549279589527', time=2024-04-17T07:18:27.920707+0000, chain.objs.size()=1 2024-04-17T05:46:01.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.XZ-nfkHHkSxbrkXk8LJ15nKCIyTy7Z9_0 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17989384889205126509', time=2024-04-17T07:18:31.506157+0000, chain.objs.size()=1 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.285__shadow_.ldSfrfCdyGtebee-9k4cNEM591q_zTo_0 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16766252866806674286', time=2024-04-17T07:18:40.101652+0000, chain.objs.size()=1 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.290__shadow_.ae9jSduLGs365-0sj2qj3K6c8web_gK_0 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9568251624485940046', time=2024-04-17T07:20:59.839868+0000, chain.objs.size()=1 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EWShyuUYNAXA2yJYnVh2KHc1ioJqVCs_0 2024-04-17T05:46:01.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5079117227019893047', time=2024-04-17T07:21:00.493569+0000, chain.objs.size()=1 2024-04-17T05:46:01.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gjYswj0VL-yX5K4d0CiRxRbySc3ER3J_0 2024-04-17T05:46:01.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7486444979756059389', time=2024-04-17T07:21:00.968690+0000, chain.objs.size()=1 2024-04-17T05:46:01.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yG-E38CmRJVpNW2aIaD3Qjx8oAmweaa_0 2024-04-17T05:46:01.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3547419757310339946', time=2024-04-17T07:21:01.115871+0000, chain.objs.size()=1 2024-04-17T05:46:01.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rXK2Esl7hLkHTAbzEhAFLx0V3u4hoMc_0 2024-04-17T05:46:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2633916261519804867', time=2024-04-17T07:21:01.572505+0000, chain.objs.size()=1 2024-04-17T05:46:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rBX4HdbLG9nXUbRwejG7LFd5yFipm6o_0 2024-04-17T05:46:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2610333816008809813', time=2024-04-17T07:21:01.595703+0000, chain.objs.size()=1 2024-04-17T05:46:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WmvuTcMOJvi_wN2cBGJ-iJKlQNhCawn_0 2024-04-17T05:46:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9641460097470735753', time=2024-04-17T07:21:01.632080+0000, chain.objs.size()=1 2024-04-17T05:46:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1b9eWVIsB4G-mDFqzBMYe78ro9XRzVK_0 2024-04-17T05:46:01.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10620487119909511009', time=2024-04-17T07:21:01.839671+0000, chain.objs.size()=1 2024-04-17T05:46:01.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.n087KJYgwYDFBilSWlCEIwSgI5QoJtk_0 2024-04-17T05:46:01.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9900971725246272798', time=2024-04-17T07:21:02.181933+0000, chain.objs.size()=1 2024-04-17T05:46:01.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.A87mLbTUfcnF07g-v2J66E8dgk4nEK4_0 2024-04-17T05:46:01.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10161346978919036023', time=2024-04-17T07:21:02.443280+0000, chain.objs.size()=1 2024-04-17T05:46:01.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.P0t9Y8uCOncgEKc3lllmoU6PBj6DIKp_0 2024-04-17T05:46:01.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10691804980740377840', time=2024-04-17T07:21:04.761101+0000, chain.objs.size()=1 2024-04-17T05:46:01.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.f6FyCYBXK5IRdoKZ43IFhM04PpvGmg0_0 2024-04-17T05:46:01.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10372520070167426729', time=2024-04-17T07:21:05.610335+0000, chain.objs.size()=1 2024-04-17T05:46:01.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6Qouox8G63QOVNnX_vSTxX9NKT3733H_0 2024-04-17T05:46:01.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2749240347298661913', time=2024-04-17T07:21:05.717185+0000, chain.objs.size()=1 2024-04-17T05:46:01.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GapzAqIz8_YTke9PtW9xsCEO9bl7GBb_0 2024-04-17T05:46:01.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14293584691900909165', time=2024-04-17T07:21:06.155737+0000, chain.objs.size()=1 2024-04-17T05:46:01.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0aSrgDLdgeWPRsuS1vdyhvdt235QTvk_0 2024-04-17T05:46:01.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14150090597128057361', time=2024-04-17T07:21:06.840486+0000, chain.objs.size()=1 2024-04-17T05:46:01.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Sim1J6-ImgXbGtGhwHY5xYgstAy1m4d_0 2024-04-17T05:46:01.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10328694346927528876', time=2024-04-17T07:21:07.957950+0000, chain.objs.size()=1 2024-04-17T05:46:01.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.X_O_WdseISsEZ2MRikUoECQ70lZG4LY_0 2024-04-17T05:46:01.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13765074850950602105', time=2024-04-17T07:21:09.689682+0000, chain.objs.size()=1 2024-04-17T05:46:01.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.v3T6dRX16TvFbOSznbMSBF2PoGuS3yb_0 2024-04-17T05:46:01.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2560937117960901383', time=2024-04-17T07:21:09.913334+0000, chain.objs.size()=1 2024-04-17T05:46:01.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EEpJRX-iteMX4Sl4m4U7LX0XI6379gi_0 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6288331726334185311', time=2024-04-17T07:21:10.344368+0000, chain.objs.size()=1 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rjdA25goinhZTlbZ8hBRy8BxFU50tfc_0 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9521579000697823834', time=2024-04-17T07:21:10.523959+0000, chain.objs.size()=1 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6aAz5zHIlubhuqNs7qxvHtBO9plAyHr_0 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11932625721703715458', time=2024-04-17T07:21:13.444845+0000, chain.objs.size()=1 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6-ut8fABemIBJw5h5QuRAQ1vn9_h9iL_0 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11712338251982139808', time=2024-04-17T07:21:13.866986+0000, chain.objs.size()=1 2024-04-17T05:46:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.T8Kob7kVZPWsZO8jKVeA5-2njO0qQwn_0 2024-04-17T05:46:01.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9841000024144486991', time=2024-04-17T07:21:14.834777+0000, chain.objs.size()=1 2024-04-17T05:46:01.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EoD4KTKYnZPk7S0XzyyF-L0xwc7LIi9_0 2024-04-17T05:46:01.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15876345332894985443', time=2024-04-17T07:21:15.748580+0000, chain.objs.size()=1 2024-04-17T05:46:01.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jPlOjNebSp2EP8LKHAPIwbHK7GR9taw_0 2024-04-17T05:46:01.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8905355358444567760', time=2024-04-17T07:21:15.751039+0000, chain.objs.size()=1 2024-04-17T05:46:01.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uExh_tP-pz6FBED0EQ7vV9er0u04xmf_0 2024-04-17T05:46:01.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10752322673094870244', time=2024-04-17T07:21:19.574706+0000, chain.objs.size()=1 2024-04-17T05:46:01.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QA3XEcc7-5X2JWZBBCPGArcZWgo98D2_0 2024-04-17T05:46:01.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1658826829339371652', time=2024-04-17T07:21:19.883105+0000, chain.objs.size()=1 2024-04-17T05:46:01.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KjZCwP0Eb1EOoKCzojIbmQqvJr7zgn__0 2024-04-17T05:46:01.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4568368211257215375', time=2024-04-17T07:21:19.958072+0000, chain.objs.size()=1 2024-04-17T05:46:01.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.A2HF2seY1psmC--pjjlQfe21yiL0TGa_0 2024-04-17T05:46:01.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.413086901430563905', time=2024-04-17T07:21:20.355477+0000, chain.objs.size()=1 2024-04-17T05:46:01.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xfR8xPPF_FH29tgqxm-9Xle-VL5g9yD_0 2024-04-17T05:46:01.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17148804500096657526', time=2024-04-17T07:21:20.658673+0000, chain.objs.size()=1 2024-04-17T05:46:01.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5UfMN8-3er0qb9EmK51Gdd5cx4X9XYv_0 2024-04-17T05:46:01.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4600842343739386567', time=2024-04-17T07:21:20.813506+0000, chain.objs.size()=1 2024-04-17T05:46:01.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jiKf3Uj6XLanOgz_d6jFKHyz9e7XDPu_0 2024-04-17T05:46:01.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3270630290084106349', time=2024-04-17T07:21:21.687479+0000, chain.objs.size()=1 2024-04-17T05:46:01.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LOraItKXuDF9SaFqZZ4lamezxJBSV68_0 2024-04-17T05:46:01.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6209810567010896761', time=2024-04-17T07:21:22.181865+0000, chain.objs.size()=1 2024-04-17T05:46:01.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.snZrMbpWSHm-IcjmLwRpy1JaxlzMZWg_0 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13329950381475041740', time=2024-04-17T07:21:22.415269+0000, chain.objs.size()=1 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.80M-rCvpNYEdU6cCkPebzf2jSSqs0pd_0 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13889049006277184613', time=2024-04-17T07:21:24.171571+0000, chain.objs.size()=1 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Aw45MtlYdXpIl0KvZWHG8Syqidrhpmn_0 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18206119304394492010', time=2024-04-17T07:21:25.680193+0000, chain.objs.size()=1 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.N6cZHjulN7zHsP1bPn0XS1kkOrW8R-4_0 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5777561763852089078', time=2024-04-17T07:21:25.943969+0000, chain.objs.size()=1 2024-04-17T05:46:01.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u0ymsozj6IEjB2KSgTQHg-R7nat4rJQ_0 2024-04-17T05:46:01.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1225725176440627749', time=2024-04-17T07:21:26.963671+0000, chain.objs.size()=1 2024-04-17T05:46:01.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Y52psW_xvJKWRgqrg6OnElCFH_SETwH_0 2024-04-17T05:46:01.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12548011851665253831', time=2024-04-17T07:21:26.981189+0000, chain.objs.size()=1 2024-04-17T05:46:01.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.g5_hzzhoe9dEwy-SylSMsiGv2LsyM7j_0 2024-04-17T05:46:01.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13191633227389548196', time=2024-04-17T07:21:27.921558+0000, chain.objs.size()=1 2024-04-17T05:46:01.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7astIxldRipvnM9y3WmxsSS9TXMD0P4_0 2024-04-17T05:46:01.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6146131010954246753', time=2024-04-17T07:21:28.908026+0000, chain.objs.size()=1 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PEVSery2OYbAj1KvubduUCDubhlvHqg_0 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16342426049319945775', time=2024-04-17T07:23:13.863696+0000, chain.objs.size()=1 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.o9ELyTD5uyUjv34-HGzVh1AAFDEoHat_0 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9870781416545992494', time=2024-04-17T07:23:13.997860+0000, chain.objs.size()=1 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.V4fe-huq25ifNclSTmUuLS2e-2eMTzL_0 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.639869947985221988', time=2024-04-17T07:23:14.229895+0000, chain.objs.size()=1 2024-04-17T05:46:01.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tbgdFIg0XJDHNX5kqyz2zcUWGXvJgYV_0 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8177856190609205004', time=2024-04-17T07:23:14.474327+0000, chain.objs.size()=1 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UeQPS4Rc7QYfXJldA2vAqTiIlewozdg_0 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4382607246643378824', time=2024-04-17T07:23:15.176083+0000, chain.objs.size()=1 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.E3BMyzB2bBvfL_dY6kfMYA3NaNGA5Pc_0 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15838945250988417625', time=2024-04-17T07:23:17.359533+0000, chain.objs.size()=1 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.FphbzruA-O8XYJns_x2sCwUcDoF8wQG_0 2024-04-17T05:46:01.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3405415950541937864', time=2024-04-17T07:23:17.567756+0000, chain.objs.size()=1 2024-04-17T05:46:01.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.whtB7nGPR21loWfIIT4Uvko7WKVMuM7_0 2024-04-17T05:46:01.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9707870342088573219', time=2024-04-17T07:23:19.379179+0000, chain.objs.size()=1 2024-04-17T05:46:01.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vO1ORs6hfPkGbqdgrPwBayDiZrdNsyg_0 2024-04-17T05:46:01.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7743579099042426615', time=2024-04-17T07:23:19.491228+0000, chain.objs.size()=1 2024-04-17T05:46:01.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VkVHWRFAJZpzhWCpBSaCwNSrRNPT3oN_0 2024-04-17T05:46:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5425269543679480035', time=2024-04-17T07:23:19.874021+0000, chain.objs.size()=1 2024-04-17T05:46:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dVSlWyycZgAZ8BiAGGfjCTZRlZFckz8_0 2024-04-17T05:46:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16565356793596769518', time=2024-04-17T07:23:21.335965+0000, chain.objs.size()=1 2024-04-17T05:46:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4gP1gLS87mFqV81V7RIB2tNd9Bh_K39_0 2024-04-17T05:46:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13477028729666684877', time=2024-04-17T07:23:21.771454+0000, chain.objs.size()=1 2024-04-17T05:46:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.laP1E2Uqpqr2RjnwQeK1D-M7sQeXbBl_0 2024-04-17T05:46:01.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3974263469333629037', time=2024-04-17T07:23:22.137031+0000, chain.objs.size()=1 2024-04-17T05:46:01.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Z6q8j8UFELGZsRDU260814dJ5lRZvRR_0 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12887463533115969163', time=2024-04-17T07:23:22.936811+0000, chain.objs.size()=1 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Ates3pFrHiadtNgUYtOZGtTWvuwNPQk_0 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1738265151746179672', time=2024-04-17T07:23:23.477384+0000, chain.objs.size()=1 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fBIXf1qXTOQN67bT_jT1WwwKfI02uvS_0 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14906343576250580637', time=2024-04-17T07:23:24.729226+0000, chain.objs.size()=1 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bqCycq76w725G7GNcTkREbuOl8tDcwk_0 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14103483003948658895', time=2024-04-17T07:23:24.846454+0000, chain.objs.size()=1 2024-04-17T05:46:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xNlKL3z5V1gTxg9fi-AabSLVDSH9-s8_0 2024-04-17T05:46:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6054786525125178481', time=2024-04-17T07:23:25.018458+0000, chain.objs.size()=1 2024-04-17T05:46:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CWUhjF6YVhuyifUBWTM-modxUugb7Pq_0 2024-04-17T05:46:01.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1099519355823173624', time=2024-04-17T07:23:25.134799+0000, chain.objs.size()=1 2024-04-17T05:46:01.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1hFgIsm9aDqJjkh5Jmd7L9CdTZtXyIa_0 2024-04-17T05:46:01.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6733298576936548038', time=2024-04-17T07:23:27.072693+0000, chain.objs.size()=1 2024-04-17T05:46:01.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.HBIPBzgpcP1fXoqzNor_FDjQqYNwf5e_0 2024-04-17T05:46:01.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12203728966362627780', time=2024-04-17T07:23:27.074858+0000, chain.objs.size()=1 2024-04-17T05:46:01.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.adZWPZS4OCor9uZK8h8xnO9ZqE7qn1h_0 2024-04-17T05:46:01.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6219442832398343085', time=2024-04-17T07:23:28.827510+0000, chain.objs.size()=1 2024-04-17T05:46:01.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TucsqN0zp_Pne6hT8F5_R-4Ck-7ylql_0 2024-04-17T05:46:01.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5662655300338860101', time=2024-04-17T07:23:28.939761+0000, chain.objs.size()=1 2024-04-17T05:46:01.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7PEvQOQz4Z1ukIlDe09RPuEhAEhTjwo_0 2024-04-17T05:46:01.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4138705069511502701', time=2024-04-17T07:23:29.412923+0000, chain.objs.size()=1 2024-04-17T05:46:01.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RNFfYE0b3yUBvcVx0BAc8AiWLPQRXBU_0 2024-04-17T05:46:01.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16134537087564520581', time=2024-04-17T07:23:29.414779+0000, chain.objs.size()=1 2024-04-17T05:46:01.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RRzDuviuvr3LW0ihrjUvqttunTThrKv_0 2024-04-17T05:46:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6940808202064438053', time=2024-04-17T07:23:30.078085+0000, chain.objs.size()=1 2024-04-17T05:46:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KCJ4IgfYEhe8S0cQsWyLbMhdzIA-SAN_0 2024-04-17T05:46:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15427469324940278395', time=2024-04-17T07:23:30.342254+0000, chain.objs.size()=1 2024-04-17T05:46:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ynFQkpGerPw48FKzB99p02Js8SKPfQ9_0 2024-04-17T05:46:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16781731730095271925', time=2024-04-17T07:23:31.051875+0000, chain.objs.size()=1 2024-04-17T05:46:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2gvi1PaoW4gWfdOam8eEVR972iJJ4Vr_0 2024-04-17T05:46:01.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/44294 2024-04-17T05:46:01.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=29, truncated=0, next_marker='' 2024-04-17T05:46:01.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8462687398487274169', time=2024-04-17T07:23:31.371258+0000, chain.objs.size()=1 2024-04-17T05:46:01.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VRCOMoai7-O7t-2Cd7L5APmQdDn02L8_0 2024-04-17T05:46:01.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9713560305601683620', time=2024-04-17T07:23:32.196822+0000, chain.objs.size()=1 2024-04-17T05:46:01.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SiksdbtvefYRZsHH7rOnAHi7wjfpkVl_0 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15466534724728234053', time=2024-04-17T07:23:32.198757+0000, chain.objs.size()=1 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.g7JCViaY02o-3mRO3TXsUc5_Ru8YE30_0 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5051815890549205825', time=2024-04-17T07:23:32.254209+0000, chain.objs.size()=1 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0WrOydiqv30mg1M87NPyMPniwMDwa7l_0 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17703893041467649630', time=2024-04-17T07:23:32.623363+0000, chain.objs.size()=1 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J07rRfk7c9vEeTtbOV99tRPtjkrkT7a_0 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10905572971948812361', time=2024-04-17T07:23:32.974258+0000, chain.objs.size()=1 2024-04-17T05:46:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WtzlZdGGlK8uPr7vum6v_HAZMoQ_lQQ_0 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11825051986535210510', time=2024-04-17T07:23:33.352340+0000, chain.objs.size()=1 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7CYM6h-wbDEytg2SVyt8Un3_4g_7Zmp_0 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14085220630030214551', time=2024-04-17T07:23:33.615791+0000, chain.objs.size()=1 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.M-erpDb_-yY4-uYTtnIHpoRNIPimMUi_0 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10511247469854492287', time=2024-04-17T07:23:36.376292+0000, chain.objs.size()=1 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ECvIFN4Dzz5HAj4HqbPjULc0im6qyRk_0 2024-04-17T05:46:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13994909064001600614', time=2024-04-17T07:23:37.031098+0000, chain.objs.size()=1 2024-04-17T05:46:01.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PQoPGPJSNVwNCQDvDbNTyK_xpUoTXRX_0 2024-04-17T05:46:01.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14863322012472848135', time=2024-04-17T07:23:38.110187+0000, chain.objs.size()=1 2024-04-17T05:46:01.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.L7l3hf8TuX9_NlvDGd6Er_w_fBkOCJf_0 2024-04-17T05:46:01.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5007370098028458334', time=2024-04-17T07:23:38.245332+0000, chain.objs.size()=1 2024-04-17T05:46:01.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3VHStlXb8w-uY5xFIbRLqZibrmf0DDD_0 2024-04-17T05:46:01.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8304083093238606230', time=2024-04-17T07:23:39.807806+0000, chain.objs.size()=1 2024-04-17T05:46:01.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.LGBZZQNU8Cs6F434vNUYStti_g_SnXk_0 2024-04-17T05:46:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16025829871591848667', time=2024-04-17T07:23:41.600889+0000, chain.objs.size()=1 2024-04-17T05:46:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0gTDpJobGbKY4u80X0j9Kbz6Ev8lhoy_0 2024-04-17T05:46:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6904808681298392711', time=2024-04-17T07:23:50.445829+0000, chain.objs.size()=1 2024-04-17T05:46:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.323__shadow_.eix1jjz_G5qvMX3Bz0AwmAyrUkjom1g_0 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4613458341961431858', time=2024-04-17T07:24:17.619453+0000, chain.objs.size()=1 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.357__shadow_.vDWXs-51BEYXM3IX988Fqhzr0fNQ5Bg_0 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7921989326372740279', time=2024-04-17T07:26:00.522890+0000, chain.objs.size()=1 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.455__shadow_.vW452PuRfPt0ydeSiPg3fchA4lbaPvL_0 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13663711811647585969', time=2024-04-17T07:27:48.917522+0000, chain.objs.size()=3 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.510__shadow:PTZ90OSNJAwOk3iaCIKJh2bUe5Kb.TN_.auHI8hrS2Y6q-R-P9wnJpAoY5ACNduG_0 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.510__shadow:PTZ90OSNJAwOk3iaCIKJh2bUe5Kb.TN_.auHI8hrS2Y6q-R-P9wnJpAoY5ACNduG_1 2024-04-17T05:46:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.510__shadow:PTZ90OSNJAwOk3iaCIKJh2bUe5Kb.TN_.auHI8hrS2Y6q-R-P9wnJpAoY5ACNduG_2 2024-04-17T05:46:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~cWtzakQnnXxtrpn3Qzu8mhbvJh8Wetj', time=2024-04-17T07:29:57.074074+0000, chain.objs.size()=1 2024-04-17T05:46:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.521__multipart_mymultipart.2~cWtzakQnnXxtrpn3Qzu8mhbvJh8Wetj.1 2024-04-17T05:46:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.263874380173597638', time=2024-04-17T07:30:55.582432+0000, chain.objs.size()=1 2024-04-17T05:46:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.543__shadow_.jQmMLr4xhjpdQTrvhci-5lL6ywGg_QZ_0 2024-04-17T05:46:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17961991508717506448', time=2024-04-17T07:31:06.711564+0000, chain.objs.size()=1 2024-04-17T05:46:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.546__shadow_._fmzOy9uw1-1dRy3532z2mxPz3jLv8r_0 2024-04-17T05:46:01.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.297859277077054713', time=2024-04-17T07:31:49.464302+0000, chain.objs.size()=1 2024-04-17T05:46:01.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:ppmjRC6yh5rzV0jAQ-63LwfuNbjX11u_.0o2Vv10aYj-5hCZxNFnUe5aH19KU4p-_0 2024-04-17T05:46:01.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10418890935781872387', time=2024-04-17T07:31:55.745427+0000, chain.objs.size()=1 2024-04-17T05:46:01.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:gkiudTdtZ6ug56zt.mVa--2bSDJn2bm_.KkR91_sO8a-MvgcOoC-Rm9qFAA1QWEQ_0 2024-04-17T05:46:01.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13038865626507444601', time=2024-04-17T07:32:05.784719+0000, chain.objs.size()=1 2024-04-17T05:46:01.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow_.Ahsbvx8Xkl2EAHTm7hAdZRvhqExM77n_0 2024-04-17T05:46:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18352744886475593733', time=2024-04-17T07:32:37.005203+0000, chain.objs.size()=1 2024-04-17T05:46:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:ZnqGZ6d3gG8OxoKqkvajm.wLJ-EEvSw_.9c9_MZlaa71QQgzKAvCuR0qAFeJfrF7_0 2024-04-17T05:46:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16891401649046086433', time=2024-04-17T07:33:06.595798+0000, chain.objs.size()=1 2024-04-17T05:46:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:IQv-0XSZBhPC8nv3Kt2JtkZ2iuvGCL3_.TSvHA1TFYqXlQzFUz-hGrdlXn0UkwNU_0 2024-04-17T05:46:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17681025666362544028', time=2024-04-17T07:33:45.207270+0000, chain.objs.size()=1 2024-04-17T05:46:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:lRtbHPxfD4zlryHVcqi1KFQ2nBY28E9_.RFSyytSpiF99VtropssBY8mQ0dNoRV9_0 2024-04-17T05:46:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4747302018525508129', time=2024-04-17T07:33:53.929294+0000, chain.objs.size()=1 2024-04-17T05:46:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.585__shadow_.7YoVATbDG2Rd1mPRw-9eFGlZH6-IkJG_0 2024-04-17T05:46:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4612014555931791398', time=2024-04-17T07:36:08.214687+0000, chain.objs.size()=6 2024-04-17T05:46:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.610__multipart_multipart_enc.2~TVXMALUXtLCBMyLuV7C_H7bulO_wLdD.1 2024-04-17T05:46:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.610__multipart_multipart_enc.2~TVXMALUXtLCBMyLuV7C_H7bulO_wLdD.2 2024-04-17T05:46:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.610__multipart_multipart_enc.2~TVXMALUXtLCBMyLuV7C_H7bulO_wLdD.3 2024-04-17T05:46:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.610__multipart_multipart_enc.2~TVXMALUXtLCBMyLuV7C_H7bulO_wLdD.4 2024-04-17T05:46:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.610__multipart_multipart_enc.2~TVXMALUXtLCBMyLuV7C_H7bulO_wLdD.5 2024-04-17T05:46:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.610__multipart_multipart_enc.2~TVXMALUXtLCBMyLuV7C_H7bulO_wLdD.6 2024-04-17T05:46:01.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:01.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-17T05:46:01.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42485' 2024-04-17T05:46:01.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13305533020000376663', time=2024-04-17T07:09:56.527827+0000, chain.objs.size()=2 2024-04-17T05:46:01.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.148__shadow_.8i3g9Jvjcku5pPvHB81PCWsB0I9cjZo_0 2024-04-17T05:46:01.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.148__shadow_.8i3g9Jvjcku5pPvHB81PCWsB0I9cjZo_1 2024-04-17T05:46:01.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10946268865750810011', time=2024-04-17T07:14:34.668877+0000, chain.objs.size()=1 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.225__shadow_.AtsD-pL-qBY0oz1ISGs8ncZY43SNG8R_0 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14205132590679636986', time=2024-04-17T07:14:34.678094+0000, chain.objs.size()=1 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.225__shadow_.3YcMY8uy33ImhEIlD8PYv643PtFFCxN_0 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8136944494486668578', time=2024-04-17T07:14:57.759875+0000, chain.objs.size()=1 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.238__shadow_.uKgrrk1mAmBF1HsGCMvTKQzvku8Tvcz_0 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14449817574676691087', time=2024-04-17T07:15:05.900936+0000, chain.objs.size()=1 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.243__shadow_.MCGc8zquscCWUGv_f_afQCZzXqXDuUc_0 2024-04-17T05:46:01.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10233315364882160132', time=2024-04-17T07:17:01.326785+0000, chain.objs.size()=1 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.p4bzQQMuY8up9KECXOgait-p2okVhAf_0 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10802375636543746821', time=2024-04-17T07:17:01.327852+0000, chain.objs.size()=1 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6d59LrFhm1YuIORNqjpRC8IEeEdJSAt_0 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3859840257847355456', time=2024-04-17T07:17:01.782086+0000, chain.objs.size()=1 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FSccewjU9tSk0VbF7WZ7uLTInUR3uf2_0 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10918337053212094719', time=2024-04-17T07:17:02.569905+0000, chain.objs.size()=1 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CYJkS7GvaVDX2MKc3_LMuxOETyNEAqS_0 2024-04-17T05:46:01.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9200519737202967907', time=2024-04-17T07:17:03.048541+0000, chain.objs.size()=1 2024-04-17T05:46:01.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.I3YrbTN07-rR2CwsJTkFHbXzB9GWwX4_0 2024-04-17T05:46:01.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11820065569940558763', time=2024-04-17T07:17:06.110096+0000, chain.objs.size()=1 2024-04-17T05:46:01.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QJMPqccCP3lFoeIkcD2J6ZT9i_sYqE1_0 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1310235364485129192', time=2024-04-17T07:17:06.112098+0000, chain.objs.size()=1 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HxR5eCc8AkPUJxk5rRXVzWWT5cMSaGV_0 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3142796730695691431', time=2024-04-17T07:17:06.897385+0000, chain.objs.size()=1 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PWiwwljRJET82zqsdFDPGOLwGRHcVRh_0 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10273261020810866808', time=2024-04-17T07:17:09.930088+0000, chain.objs.size()=1 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.k-GZi2DQc2J5NiT0qE0UTdSotBbyqQZ_0 2024-04-17T05:46:01.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2103738819895145235', time=2024-04-17T07:17:10.237454+0000, chain.objs.size()=1 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wmdjxBLSaSgwn2H0RkvBr7tnifHGAEF_0 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16297526876503544118', time=2024-04-17T07:17:12.474987+0000, chain.objs.size()=1 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.a3yV8fD0sgrMQyyhtPyCWwSayCRSIBv_0 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4186684593355384247', time=2024-04-17T07:17:15.362516+0000, chain.objs.size()=1 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OQKaIDjVjomNqna1clmOAZ8qFwgqUmF_0 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17454992290741057327', time=2024-04-17T07:17:15.535313+0000, chain.objs.size()=1 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.a4J4ZWVzXBejSaOePKX3b-nKsNL3Xtd_0 2024-04-17T05:46:01.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10453509834949118164', time=2024-04-17T07:17:15.793974+0000, chain.objs.size()=1 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VxNjr8ZfRJAA7Ezb0Du8LXlSHKkd4AY_0 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.460386125048390522', time=2024-04-17T07:17:16.383431+0000, chain.objs.size()=1 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TGteqRRxtIkVRss3LbFHbcFwB3o9CWq_0 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8469741026935624053', time=2024-04-17T07:17:17.983901+0000, chain.objs.size()=1 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gNIGvMg1MZqMXIFp03siUqtacQ5zp9x_0 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13683041766023779262', time=2024-04-17T07:17:18.084883+0000, chain.objs.size()=1 2024-04-17T05:46:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5VbXUA1kkPWlbfzZkDDQ5RW0yIs1JiY_0 2024-04-17T05:46:01.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9519071987193835850', time=2024-04-17T07:17:18.086416+0000, chain.objs.size()=1 2024-04-17T05:46:01.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fve_E-bzJXyvhp4QUqnC-NUmjoo9XCO_0 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3803425362961087995', time=2024-04-17T07:17:18.867960+0000, chain.objs.size()=1 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Dq2I4Gm7QtXJjjy55uwn_bmgSsnGQzB_0 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16513642019134728044', time=2024-04-17T07:17:19.705908+0000, chain.objs.size()=1 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hVkS37HBdK1whw6uFMBX7DFYysKuoKr_0 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12156411396648431184', time=2024-04-17T07:17:20.673275+0000, chain.objs.size()=1 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.35Z1iSoZGrxY2QdZR4boMbNO_vRE6s4_0 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17676062101320212654', time=2024-04-17T07:17:20.974065+0000, chain.objs.size()=1 2024-04-17T05:46:01.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Wq1rPXE7Xk8NJZr9TNh0yIm_y8nlmOr_0 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11376228379739057115', time=2024-04-17T07:17:21.010295+0000, chain.objs.size()=1 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-Sjc3t7Gyrf9oD4c08onEz4ikg8hn74_0 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1196607093241576233', time=2024-04-17T07:17:23.206208+0000, chain.objs.size()=1 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gRQqikfYZINmRJ3MY9b2a05tN7pIirv_0 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.61141193341049130', time=2024-04-17T07:17:23.568889+0000, chain.objs.size()=1 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9ft_5JjCxLq4sgwzK7WGU91yCdbpE1B_0 2024-04-17T05:46:01.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8416026220876572447', time=2024-04-17T07:17:27.100779+0000, chain.objs.size()=1 2024-04-17T05:46:01.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ixziWm7AjJZSpwcA7xWfuXS7jn_GfDq_0 2024-04-17T05:46:01.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2136381346477854775', time=2024-04-17T07:17:27.598964+0000, chain.objs.size()=1 2024-04-17T05:46:01.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0sowKCoXeLriPm8EpByPLrWumewcQgH_0 2024-04-17T05:46:01.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6992303459030157061', time=2024-04-17T07:17:28.073301+0000, chain.objs.size()=1 2024-04-17T05:46:01.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bDSe6ouDLMDRwKEbBtiH8osxJGSm-3a_0 2024-04-17T05:46:01.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4306846671235083945', time=2024-04-17T07:17:28.075422+0000, chain.objs.size()=1 2024-04-17T05:46:01.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.i5FPHJmxvlZP1i0lELg6euQX3aUDL0Y_0 2024-04-17T05:46:01.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3833769230828760517', time=2024-04-17T07:17:28.819194+0000, chain.objs.size()=1 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Q_puEZoQn3LkgkA3YWrjdNgAlSVqIyG_0 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17530134407493859124', time=2024-04-17T07:17:29.926036+0000, chain.objs.size()=1 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fq2EfuO3AWHj8GdnsdIek97-hWYZPNi_0 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15017492588285983378', time=2024-04-17T07:17:31.499055+0000, chain.objs.size()=1 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YOByXUGYWtTID0PIxkGnJKiy6JWwqRE_0 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8106688777217590800', time=2024-04-17T07:17:31.959275+0000, chain.objs.size()=1 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pzHguV-8WnPAlcP7h2nHcDVhHHivuMH_0 2024-04-17T05:46:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3029276796555025028', time=2024-04-17T07:17:56.830420+0000, chain.objs.size()=1 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.268__shadow_.5AKU1ZX8dPjUcre9aGSv4C_oXe8DDQ6_0 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2675912211293624140', time=2024-04-17T07:18:01.708797+0000, chain.objs.size()=1 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.271__shadow_.WFGHPFufkogWS76evdBQXmGnkd4hFbX_0 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15364034233892713354', time=2024-04-17T07:18:27.918493+0000, chain.objs.size()=1 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.JwkmYcJIMmXL5-muLtXIl2iaOUEnQwO_0 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5990197083864027414', time=2024-04-17T07:18:36.619756+0000, chain.objs.size()=1 2024-04-17T05:46:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.288__shadow_.igHy-IcmcGsqc3A_xVA-LjxF9JTYTB1_0 2024-04-17T05:46:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12790770355955813050', time=2024-04-17T07:19:00.276515+0000, chain.objs.size()=1 2024-04-17T05:46:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.303__shadow_.eZjyxQNT46tug3sjvecvCXha32Q3hE9_0 2024-04-17T05:46:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2178618754760124815', time=2024-04-17T07:21:00.032608+0000, chain.objs.size()=1 2024-04-17T05:46:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4znF-1qpIK21qwCa1pOvWk7XQ5udWuI_0 2024-04-17T05:46:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18080228096888420097', time=2024-04-17T07:21:02.524012+0000, chain.objs.size()=1 2024-04-17T05:46:01.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BZWHH9t3ynhEgHw9104FUp36_7d3KSX_0 2024-04-17T05:46:01.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5899707870155439958', time=2024-04-17T07:21:04.034143+0000, chain.objs.size()=1 2024-04-17T05:46:01.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.N5goTTEeRPFbUz40cxJhnJxbEnGR6US_0 2024-04-17T05:46:01.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4580995015701523355', time=2024-04-17T07:21:04.337940+0000, chain.objs.size()=1 2024-04-17T05:46:01.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VxrhwmiXk1LZcOM3FH3UAprit-1X3pS_0 2024-04-17T05:46:01.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3928998338817497585', time=2024-04-17T07:21:05.394206+0000, chain.objs.size()=1 2024-04-17T05:46:01.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.89tMF3fU7ce5pRtdDsryCRADgAQjPlD_0 2024-04-17T05:46:01.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7172407686257892027', time=2024-04-17T07:21:06.139255+0000, chain.objs.size()=1 2024-04-17T05:46:01.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.k3r7HnYy8IFNlFieQs719TQ1pyw3pE8_0 2024-04-17T05:46:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2727061684178868835', time=2024-04-17T07:21:08.383061+0000, chain.objs.size()=1 2024-04-17T05:46:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._XURPyNl4MzTDzaaZCfvDqlO2R9kOoT_0 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8448980835110131471', time=2024-04-17T07:21:09.246907+0000, chain.objs.size()=1 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1thFwcpvU3TSjK9Kxm6lXbWBIxMM3_-_0 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5091531265709302793', time=2024-04-17T07:21:09.421358+0000, chain.objs.size()=1 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nBbF8grx0ajwtbmX5jEKFyrQ3tMZJjB_0 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11188103113763853501', time=2024-04-17T07:21:10.482168+0000, chain.objs.size()=1 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dMeaFJxn32Tl5FUulqztNqNtwDwl1eA_0 2024-04-17T05:46:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10875432558274057059', time=2024-04-17T07:21:11.887007+0000, chain.objs.size()=1 2024-04-17T05:46:01.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.emASVgANGuKelwOCeibzGX00Yh5M_V7_0 2024-04-17T05:46:01.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12906761640060444466', time=2024-04-17T07:21:12.721583+0000, chain.objs.size()=1 2024-04-17T05:46:01.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.awM5rqtvrTl24mYQS3Nc0Db-JRf26qz_0 2024-04-17T05:46:01.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16571591270882274852', time=2024-04-17T07:21:13.425569+0000, chain.objs.size()=1 2024-04-17T05:46:01.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Qvjy7wfpAZASWFpxxt5rDM4frbcdIAJ_0 2024-04-17T05:46:01.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14679425759009976827', time=2024-04-17T07:21:13.905055+0000, chain.objs.size()=1 2024-04-17T05:46:01.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zFDRemTLrfkeZo1iP_Rmz88jp1Kw1fL_0 2024-04-17T05:46:01.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18210796528168904917', time=2024-04-17T07:21:14.470739+0000, chain.objs.size()=1 2024-04-17T05:46:01.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.F_3azNa3zByITrOLw_VdfepXVbj9rNM_0 2024-04-17T05:46:01.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14808435863029142720', time=2024-04-17T07:21:15.861358+0000, chain.objs.size()=1 2024-04-17T05:46:01.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gb5H6ucSATcEMha4nqeRG0W_fvwnlGZ_0 2024-04-17T05:46:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4655731924888832683', time=2024-04-17T07:21:20.441132+0000, chain.objs.size()=1 2024-04-17T05:46:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eQONWcnpTVASdHhGB0gjvh3LddoyMsm_0 2024-04-17T05:46:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12297339348639805296', time=2024-04-17T07:21:20.995629+0000, chain.objs.size()=1 2024-04-17T05:46:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vIzyj6TRc2J7dHMhOnjGqv5yKAv4fbd_0 2024-04-17T05:46:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7742478026167595872', time=2024-04-17T07:21:21.579753+0000, chain.objs.size()=1 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GTeVZ9mk6l7yXrqlHkz8fIzqOrZFHPq_0 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12407834513359749968', time=2024-04-17T07:21:22.034171+0000, chain.objs.size()=1 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.F_JcM8yFWV8x_KaK-hT403IH1vc_Jv1_0 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16992814798124689677', time=2024-04-17T07:21:23.924228+0000, chain.objs.size()=1 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.d5WO6mXfbvTiwylMfuNhOUtSZVW8KXr_0 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17451045693865815404', time=2024-04-17T07:21:25.134166+0000, chain.objs.size()=1 2024-04-17T05:46:01.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.qJ3JEafKdMvLqK8nb7wl0DeyIFZGjCV_0 2024-04-17T05:46:01.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2114797663391118936', time=2024-04-17T07:21:28.428177+0000, chain.objs.size()=1 2024-04-17T05:46:01.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PUgr-KDg9JCxjKHgraPXB7HdTPnFZnR_0 2024-04-17T05:46:01.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8179273289650716033', time=2024-04-17T07:21:28.655547+0000, chain.objs.size()=1 2024-04-17T05:46:01.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aZpABRFuIoxpkUpaXBsk_wKm-4lDvgU_0 2024-04-17T05:46:01.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14755735299959105708', time=2024-04-17T07:21:28.940611+0000, chain.objs.size()=1 2024-04-17T05:46:01.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zDTTfdrPBV60Y4MQb1rz-0tYxpDYtbf_0 2024-04-17T05:46:01.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2847979484648033275', time=2024-04-17T07:21:29.423164+0000, chain.objs.size()=1 2024-04-17T05:46:01.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nWBJuMLpuMsAZg_QhVPbHK3UefO_Eux_0 2024-04-17T05:46:01.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2976138177752549652', time=2024-04-17T07:23:14.643486+0000, chain.objs.size()=1 2024-04-17T05:46:01.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yw84pAq0i9niwJ8rLyBT-TTud9K5WLK_0 2024-04-17T05:46:01.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15814783289595127385', time=2024-04-17T07:23:14.926231+0000, chain.objs.size()=1 2024-04-17T05:46:01.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gtQ-pxQbOvfyHiC4Xs_ZiXcpw29PU0G_0 2024-04-17T05:46:01.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11268500535643043981', time=2024-04-17T07:23:15.153706+0000, chain.objs.size()=1 2024-04-17T05:46:01.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fkqm2xe2XjGSIjMQM1n0qFYt1CCrN3L_0 2024-04-17T05:46:01.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14925868536552925854', time=2024-04-17T07:23:15.529280+0000, chain.objs.size()=1 2024-04-17T05:46:01.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wKtIQBL4TF-XILBAGX2XOTkrybr42kt_0 2024-04-17T05:46:01.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9581994356065712170', time=2024-04-17T07:23:17.093938+0000, chain.objs.size()=1 2024-04-17T05:46:01.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZgFfRZFUGrace2wjb8RtNLwdkMsSlUt_0 2024-04-17T05:46:01.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4504585046893693123', time=2024-04-17T07:23:17.731013+0000, chain.objs.size()=1 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wgrX0J6aqBWSY5jmwn1JMhg5nxt1ZEz_0 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4304745855294297264', time=2024-04-17T07:23:17.951218+0000, chain.objs.size()=1 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cYqFPfkqDtGUOZD6zda2V4v7EbwQm2o_0 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14139578593147469192', time=2024-04-17T07:23:17.953134+0000, chain.objs.size()=1 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bdmYRiHNPNhLFmMxMbOJ8H6I03kdyfs_0 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9086284930226453728', time=2024-04-17T07:23:19.366061+0000, chain.objs.size()=1 2024-04-17T05:46:01.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5YlQQBlDMKZhqmvBVVMD0fh1sK-VrVA_0 2024-04-17T05:46:01.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2752930910579878558', time=2024-04-17T07:23:20.873928+0000, chain.objs.size()=1 2024-04-17T05:46:01.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uzkYbR-e0fkGdz7EYPaAC9ACuG2iLYT_0 2024-04-17T05:46:01.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6213889041888974220', time=2024-04-17T07:23:21.241920+0000, chain.objs.size()=1 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5i-yP2IgagG6njK0pLKUOeNEveVaHnz_0 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13777198045063660249', time=2024-04-17T07:23:22.317114+0000, chain.objs.size()=1 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eKJh9qMARH_J_NE27c2BBU50Yn7v315_0 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15289947933590195778', time=2024-04-17T07:23:22.922314+0000, chain.objs.size()=1 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SosITtCAvP7y69eDLw74F3zWjfTfEHF_0 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11100367560428782438', time=2024-04-17T07:23:24.726448+0000, chain.objs.size()=1 2024-04-17T05:46:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.E5KgaVubzHh3aloIP6B1XD9gakoof5j_0 2024-04-17T05:46:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3497141572603937735', time=2024-04-17T07:23:25.202019+0000, chain.objs.size()=1 2024-04-17T05:46:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YKtQVr9br1dMf18HRYBmxt3b-v44jX8_0 2024-04-17T05:46:01.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2207287181429290861', time=2024-04-17T07:23:27.321142+0000, chain.objs.size()=1 2024-04-17T05:46:01.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.q9C9xicQprCT9YMDW-q59uo4-WPDBgr_0 2024-04-17T05:46:01.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9976332229910158783', time=2024-04-17T07:23:32.112982+0000, chain.objs.size()=1 2024-04-17T05:46:01.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Cc_NWTlYSXsJP3NZzkL132cDgUJn6dc_0 2024-04-17T05:46:01.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3039791695366207035', time=2024-04-17T07:23:32.672322+0000, chain.objs.size()=1 2024-04-17T05:46:01.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cNTGCc7l9He-ow3iwjZ_Mwz1O_rlMIj_0 2024-04-17T05:46:01.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17113629507834591046', time=2024-04-17T07:23:34.502105+0000, chain.objs.size()=1 2024-04-17T05:46:01.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gOdGUduXqIKaQoz6Kx0VWKXGU2ltXf5_0 2024-04-17T05:46:01.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11777914104626417783', time=2024-04-17T07:23:34.812580+0000, chain.objs.size()=1 2024-04-17T05:46:01.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.888+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JxiI870QG2dKscnqYAy1ROyNoGmUCeZ_0 2024-04-17T05:46:01.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15031415765738836086', time=2024-04-17T07:23:35.788466+0000, chain.objs.size()=1 2024-04-17T05:46:01.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RYmLqpYbgOZ6htgljT_gfT7D5UE2FwH_0 2024-04-17T05:46:01.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2311567262940321242', time=2024-04-17T07:23:36.637097+0000, chain.objs.size()=1 2024-04-17T05:46:01.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9LiIQBnxfpJ4Cp1eHrq4UJg4rxtZxro_0 2024-04-17T05:46:01.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6219596387270734397', time=2024-04-17T07:23:37.474307+0000, chain.objs.size()=1 2024-04-17T05:46:01.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XBnhYKZfQbzHGPd8NmsC5RLPrPsSx_R_0 2024-04-17T05:46:01.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15100127394249768621', time=2024-04-17T07:23:39.384930+0000, chain.objs.size()=1 2024-04-17T05:46:01.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fjFi-7EUBp8QBcsyplrcaADv_iiCgor_0 2024-04-17T05:46:01.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16488751224005178687', time=2024-04-17T07:23:40.195723+0000, chain.objs.size()=1 2024-04-17T05:46:01.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uVP1pgRUQjI3PKIV5VMxBLIQU8YgW4x_0 2024-04-17T05:46:01.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8052734026637907792', time=2024-04-17T07:23:40.264146+0000, chain.objs.size()=1 2024-04-17T05:46:01.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ywo0sy_TICal5PavXJwmhlz6HJexl_s_0 2024-04-17T05:46:01.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11507749663413106618', time=2024-04-17T07:23:40.322661+0000, chain.objs.size()=1 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gkUgiq1qpNNaca7HgcbBkvAzJ9shcX9_0 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11191921751824603419', time=2024-04-17T07:23:41.015403+0000, chain.objs.size()=1 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.roj5rL_btCFxMsQTjjBuTWp5XSiVWUF_0 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11766144767048036779', time=2024-04-17T07:23:41.034921+0000, chain.objs.size()=1 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TbhSBPzIX0h-xi1WHn5nkkXWpxU8-8Q_0 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7309911979209367858', time=2024-04-17T07:23:41.362814+0000, chain.objs.size()=1 2024-04-17T05:46:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.892+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hTM4oQOOjw-oI8luN4aT2HxHqGjJvKd_0 2024-04-17T05:46:01.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.896+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.711798965970058507', time=2024-04-17T07:23:41.434023+0000, chain.objs.size()=1 2024-04-17T05:46:01.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.896+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.keDlVmPnpYC7OCxjizH_7KnXOotEh7l_0 2024-04-17T05:46:01.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/42485 2024-04-17T05:46:01.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=13, truncated=0, next_marker='' 2024-04-17T05:46:01.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9371585122138502170', time=2024-04-17T07:23:41.534348+0000, chain.objs.size()=1 2024-04-17T05:46:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J6dl-W8DfAhhSzxnkVRygvjMjmE0gE8_0 2024-04-17T05:46:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6578914213158298933', time=2024-04-17T07:23:41.578842+0000, chain.objs.size()=1 2024-04-17T05:46:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GFWDWEDi_LTWTtln0Gcpx5Qa-5qsBh-_0 2024-04-17T05:46:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3106343301348863679', time=2024-04-17T07:24:22.955139+0000, chain.objs.size()=1 2024-04-17T05:46:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.363__shadow_.14gwXgwHCcedD2vPWujToKqHaegsSrK_0 2024-04-17T05:46:01.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6568424154484856777', time=2024-04-17T07:25:17.765708+0000, chain.objs.size()=1 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.421__shadow_.RZjQqOH9LRJGLQleHhKe2PjhkcSdV6h_0 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1006394338713397256', time=2024-04-17T07:26:17.130473+0000, chain.objs.size()=1 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.r-hjzpATMaACuEKOl6mc-q6DRUgTwAj_0 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.486__shadow_.WL_H6Uy_YNGNX8uK5DUJ2c6hMVkqF1L_1_09555bc9ef8ebeb18fb4e53a1440a610', time=2024-04-17T07:26:34.438618+0000, chain.objs.size()=1 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.486__shadow_.WL_H6Uy_YNGNX8uK5DUJ2c6hMVkqF1L_0 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15180424746902850207', time=2024-04-17T07:27:49.419123+0000, chain.objs.size()=3 2024-04-17T05:46:01.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~c2wuXKIzVCzJuGRkuuBbXhYv6xYKIGS.1 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~c2wuXKIzVCzJuGRkuuBbXhYv6xYKIGS.2 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~c2wuXKIzVCzJuGRkuuBbXhYv6xYKIGS.3 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6432454914184773453', time=2024-04-17T07:32:14.961483+0000, chain.objs.size()=1 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.564__shadow:XsP2B2zEGILfHTYb1l4CGHuVG82zuvW_.kEagL4cGepTz45dLtlNYkRPCQpJPP9U_0 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10797991862263751811', time=2024-04-17T07:32:24.095350+0000, chain.objs.size()=1 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:wr.zTGjeMMm0YU63c6QU4BIU1o53Jo4_.5tWIOxkQ316i9gV7xdenJB1YlDJgp-0_0 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_1_0b943a78aa4d1eed22a73464e8b37f34', time=2024-04-17T07:33:34.122776+0000, chain.objs.size()=1 2024-04-17T05:46:01.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_0 2024-04-17T05:46:01.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.908+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11126569767512565688', time=2024-04-17T07:33:53.937975+0000, chain.objs.size()=1 2024-04-17T05:46:01.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.908+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.585__shadow_.Gd43a2khidf2NqiiGMUZPlgxBJEygD8_0 2024-04-17T05:46:01.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.908+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11235071379644350123', time=2024-04-17T07:36:28.165276+0000, chain.objs.size()=1 2024-04-17T05:46:01.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.908+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.615__shadow_.P_VhoLV-GYdXGiAXaJYYKlAwLWXLw65_0 2024-04-17T05:46:01.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.908+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15734962258985072976', time=2024-04-17T07:37:35.899360+0000, chain.objs.size()=1 2024-04-17T05:46:01.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.908+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.648__shadow_.ffMoUmSHDkprioCZiRwJj070SyAY8GK_0 2024-04-17T05:46:01.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.916+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:01.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.916+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-17T05:46:01.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:01.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:01.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45015' 2024-04-17T05:46:01.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2643713171471981795', time=2024-04-17T07:10:03.258964+0000, chain.objs.size()=3 2024-04-17T05:46:01.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-STANDARD.2~hr-h943M5A0N3OVq3dRrNOC27lbbeH3.1 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-STANDARD.2~hr-h943M5A0N3OVq3dRrNOC27lbbeH3.2 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-STANDARD.2~hr-h943M5A0N3OVq3dRrNOC27lbbeH3.3 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~Z-FjVudDqxmVFnbsxN6WGs9lTYJ9F7E', time=2024-04-17T07:11:42.728969+0000, chain.objs.size()=1 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.192__multipart_mymultipart.2~Z-FjVudDqxmVFnbsxN6WGs9lTYJ9F7E.1 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.948474245727018088', time=2024-04-17T07:14:26.497503+0000, chain.objs.size()=1 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.220__shadow_.tATYj8ltgIo3uEMJdzMAfunmXJh9gKn_0 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18093001019411050650', time=2024-04-17T07:14:54.517043+0000, chain.objs.size()=1 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.236__shadow_.wQbmeHeV6njq1hKpN4oxv_ZUa2kXsK1_0 2024-04-17T05:46:01.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4173005307528995555', time=2024-04-17T07:14:57.766942+0000, chain.objs.size()=1 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.238__shadow_.WP1A0N-r5YmuvZwBxxRalCmp_cVrm0q_0 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18230511945991425798', time=2024-04-17T07:17:01.349111+0000, chain.objs.size()=1 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mXf2IDzXhv8eay4sWd7zSK71tZqKeaj_0 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8321393553221183852', time=2024-04-17T07:17:01.550681+0000, chain.objs.size()=1 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aXoc4qsCMDvX-ntJHp6vndrFi8S7EOE_0 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3541339242922328800', time=2024-04-17T07:17:01.696499+0000, chain.objs.size()=1 2024-04-17T05:46:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._X3qyr4C6hwR1RqRqyy9Txnm4aDFMDY_0 2024-04-17T05:46:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14907067464293313301', time=2024-04-17T07:17:02.932316+0000, chain.objs.size()=1 2024-04-17T05:46:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.d0cdMi2KzFYTz4gxqA29LYsJcaw6QyQ_0 2024-04-17T05:46:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18311730125363799672', time=2024-04-17T07:17:04.373565+0000, chain.objs.size()=1 2024-04-17T05:46:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qf-BAuOdHEjRrg997USE1-FoeKG0Qj-_0 2024-04-17T05:46:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5801671981221947477', time=2024-04-17T07:17:05.424481+0000, chain.objs.size()=1 2024-04-17T05:46:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BxW0eN1XX_C8z0bniOEnackXRGuOLNv_0 2024-04-17T05:46:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18416735875195455060', time=2024-04-17T07:17:05.476256+0000, chain.objs.size()=1 2024-04-17T05:46:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5GKohopD2AZ2pukOjHi20H-lNRLPoRB_0 2024-04-17T05:46:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11836086178873467418', time=2024-04-17T07:17:07.254636+0000, chain.objs.size()=1 2024-04-17T05:46:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FfNjolG4_PlW4bTMrXamI4807u_uOtP_0 2024-04-17T05:46:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16241694235562615806', time=2024-04-17T07:17:07.359270+0000, chain.objs.size()=1 2024-04-17T05:46:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aZAGnpOPmlWCmnsC90K5zfYpXXmlBuh_0 2024-04-17T05:46:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8539997380440006213', time=2024-04-17T07:17:07.731946+0000, chain.objs.size()=1 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7ov4tt3_7AQ47gNfEB7BLX-0J2GqUv5_0 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.238669848112834087', time=2024-04-17T07:17:08.053449+0000, chain.objs.size()=1 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.j-K6nUIbPkjQ2grUKD7oLsijEOTadVb_0 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5211048429875428902', time=2024-04-17T07:17:08.055391+0000, chain.objs.size()=1 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.g9DarN5nD4TZVmsbTNwaSCoLKcS55Kg_0 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9712803043254394077', time=2024-04-17T07:17:08.654964+0000, chain.objs.size()=1 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ou8PVg2aynyO8Jvor35hZhJdGRRqSsZ_0 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13751315693139864984', time=2024-04-17T07:17:09.845489+0000, chain.objs.size()=1 2024-04-17T05:46:01.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.IUXTQqsiw0X4s51kJXheucSWcJqitUt_0 2024-04-17T05:46:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9875135213867164077', time=2024-04-17T07:17:09.936845+0000, chain.objs.size()=1 2024-04-17T05:46:01.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3ul4cn3TzXjNXQvR_G1it2cmmMMlraF_0 2024-04-17T05:46:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16789774067752250233', time=2024-04-17T07:17:09.938316+0000, chain.objs.size()=1 2024-04-17T05:46:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.a58FF0CeE5oC34iSW42BLpYQ6eeXbGc_0 2024-04-17T05:46:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4772204992130143255', time=2024-04-17T07:17:11.106444+0000, chain.objs.size()=1 2024-04-17T05:46:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KCwy4MG69ykQwWLxkLITNzjbqo9DbIU_0 2024-04-17T05:46:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10761649602572891316', time=2024-04-17T07:17:12.659918+0000, chain.objs.size()=1 2024-04-17T05:46:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zKYgJLL1dJDqlpUk1E4_QCHBI_3nOZe_0 2024-04-17T05:46:01.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4834702576822380751', time=2024-04-17T07:17:14.037538+0000, chain.objs.size()=1 2024-04-17T05:46:01.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Gv0eBvn-fT1gYQfb1DNA4-lYsl-ZCEu_0 2024-04-17T05:46:01.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11627806868108167255', time=2024-04-17T07:17:14.202759+0000, chain.objs.size()=1 2024-04-17T05:46:01.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jwMa0Lav4k_dl3p2njpsxx7BwVKtMWM_0 2024-04-17T05:46:01.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4814970623246354964', time=2024-04-17T07:17:14.319760+0000, chain.objs.size()=1 2024-04-17T05:46:01.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.oKYeFwfmDuwohS9_DPTPThgYPeX-t9p_0 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5250815424503931437', time=2024-04-17T07:17:15.789527+0000, chain.objs.size()=1 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iPd3Z-4YWHivrjuirwMoJh8sA1nOdIZ_0 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.390367465484165020', time=2024-04-17T07:17:15.908064+0000, chain.objs.size()=1 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.puey0uAjTauNNZW6kcMI2oMvmUxh_SM_0 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12282518851007618838', time=2024-04-17T07:17:15.910283+0000, chain.objs.size()=1 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Uc7FtwAXRq3XiMJW86mYfkE9yTD1mYZ_0 2024-04-17T05:46:01.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10856774754359682754', time=2024-04-17T07:17:16.264898+0000, chain.objs.size()=1 2024-04-17T05:46:01.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XGM05hI_IFCEENmM0oew5ImPgfaH0xu_0 2024-04-17T05:46:01.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2259952119614313056', time=2024-04-17T07:17:16.607581+0000, chain.objs.size()=1 2024-04-17T05:46:01.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-cGigNqxM4o3hzbfG7uJSwGZd526uON_0 2024-04-17T05:46:01.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11113553928328659975', time=2024-04-17T07:17:17.042151+0000, chain.objs.size()=1 2024-04-17T05:46:01.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.oC74Is8YdqRxq2k7cDhUjKvLIIz9AHf_0 2024-04-17T05:46:01.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9338206584872985975', time=2024-04-17T07:17:23.253923+0000, chain.objs.size()=1 2024-04-17T05:46:01.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MJ-kkF-EBDgkQHOG6fpWWYq_F78rERL_0 2024-04-17T05:46:01.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17664797370513308484', time=2024-04-17T07:17:23.527419+0000, chain.objs.size()=1 2024-04-17T05:46:01.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mhTBnnvno92vOGbQknvgXmYldz61XL__0 2024-04-17T05:46:01.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2918761526935839370', time=2024-04-17T07:17:27.181842+0000, chain.objs.size()=1 2024-04-17T05:46:01.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.W08KAS8PBzgva6WfGczNpgQi6_TVKXQ_0 2024-04-17T05:46:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4721534401108432101', time=2024-04-17T07:17:28.167434+0000, chain.objs.size()=1 2024-04-17T05:46:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KewHhvAQWFw8uF9Na4Fna2taJqti0Ct_0 2024-04-17T05:46:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9129089544918405552', time=2024-04-17T07:17:28.725501+0000, chain.objs.size()=1 2024-04-17T05:46:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6Rqe0nSTBgXFI__HkIXJVhfkPaHISkK_0 2024-04-17T05:46:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2036884482921112828', time=2024-04-17T07:17:30.209349+0000, chain.objs.size()=1 2024-04-17T05:46:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EButlMjrRkHUGtwlEcGPI6W9d1LQIFm_0 2024-04-17T05:46:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1929064139538070819', time=2024-04-17T07:17:31.011159+0000, chain.objs.size()=1 2024-04-17T05:46:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Lwu3zwY-SOItPyjxAXLXS_ePHJYIRoO_0 2024-04-17T05:46:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2453219206054466568', time=2024-04-17T07:17:31.386984+0000, chain.objs.size()=1 2024-04-17T05:46:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uocuXciQlyx3e-VG45EHH9MoZpCRMJp_0 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12532123732772793011', time=2024-04-17T07:17:31.617396+0000, chain.objs.size()=1 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tMnplPNlvfHMXdAzqZHCkhp_pjjLYAl_0 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6953653521122356948', time=2024-04-17T07:17:32.273928+0000, chain.objs.size()=1 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tP0YtYi18esm25BEIcskROsvC60pquv_0 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.311019020266315956', time=2024-04-17T07:17:32.341236+0000, chain.objs.size()=1 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CuWpQ4Wjeexqy2GKEEi8PGD-PQIHR3N_0 2024-04-17T05:46:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4737462409821807396', time=2024-04-17T07:17:36.923018+0000, chain.objs.size()=1 2024-04-17T05:46:01.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.255__shadow_.qIA9qJQrDS95yAA4mmHANf4p-SCMPsT_0 2024-04-17T05:46:01.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6461984817103212226', time=2024-04-17T07:18:23.026317+0000, chain.objs.size()=1 2024-04-17T05:46:01.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.FsHTYiMRqueBd51eVWXo2_COVUEYVXE_0 2024-04-17T05:46:01.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15353927282857601845', time=2024-04-17T07:18:28.041997+0000, chain.objs.size()=1 2024-04-17T05:46:01.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.GG_dNdHMZOOfnxM1-sMx46M9xikD-ib_0 2024-04-17T05:46:01.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.994707258992384770', time=2024-04-17T07:18:33.098143+0000, chain.objs.size()=1 2024-04-17T05:46:01.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.286__shadow_.IvffVGrWXg_XRNTSWvODxLJgWOn0ac6_0 2024-04-17T05:46:01.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.146287260564735197', time=2024-04-17T07:18:38.394866+0000, chain.objs.size()=1 2024-04-17T05:46:01.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.289__shadow_.3XhWkwrggkrXQt8pxqfwYYV0TeqWhAF_0 2024-04-17T05:46:01.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14522233166871683793', time=2024-04-17T07:18:38.396542+0000, chain.objs.size()=1 2024-04-17T05:46:01.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.289__shadow_.AHNVO0NAEMzGsWLa_GfRDfrirRLa0T-_0 2024-04-17T05:46:01.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2065317449050576530', time=2024-04-17T07:18:41.666518+0000, chain.objs.size()=1 2024-04-17T05:46:01.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.291__shadow_.fFxjmDx8z4wWshvny7-29Y4-QPouqw8_0 2024-04-17T05:46:01.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2212320881688916762', time=2024-04-17T07:18:50.326660+0000, chain.objs.size()=1 2024-04-17T05:46:01.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.296__shadow_.VkAPOHUa4YBxl0t-UUbfz0qOxZPrR9R_0 2024-04-17T05:46:01.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2809524022233773195', time=2024-04-17T07:21:03.912225+0000, chain.objs.size()=1 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JFR11jF3DnlnMBRUHvratl6yS3V7xn0_0 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15078648081832549035', time=2024-04-17T07:21:04.305702+0000, chain.objs.size()=1 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0gziFTRQCT_OPasiR4CEMKdyvMp86Tl_0 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6081963654464665089', time=2024-04-17T07:21:04.983541+0000, chain.objs.size()=1 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-ysgUgwr1dcwfI1ZkXBt-YB3sJYSf8g_0 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11665242892729833740', time=2024-04-17T07:21:04.985277+0000, chain.objs.size()=1 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.l1x3rAMfi3JnDeJdzh4RZP8qZtcDRq1_0 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1266597599074879711', time=2024-04-17T07:21:05.715316+0000, chain.objs.size()=1 2024-04-17T05:46:01.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Jxhgc8XyOq4AMJRKdpLmaeKhOHIrCpl_0 2024-04-17T05:46:01.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.625713369445634975', time=2024-04-17T07:21:06.039056+0000, chain.objs.size()=1 2024-04-17T05:46:01.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.16D0UWfGpoYYerZyqJUufDKfYq9gJen_0 2024-04-17T05:46:01.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8239729449674728628', time=2024-04-17T07:21:06.297491+0000, chain.objs.size()=1 2024-04-17T05:46:01.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Yd9FcVts3x5tUk8vHW7I0DADae9_oou_0 2024-04-17T05:46:01.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4538490582917909583', time=2024-04-17T07:21:06.473362+0000, chain.objs.size()=1 2024-04-17T05:46:01.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Q30QG1fCvO2xRsGJruVFrMI7eF6MG1a_0 2024-04-17T05:46:01.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.336996972467777422', time=2024-04-17T07:21:08.044315+0000, chain.objs.size()=1 2024-04-17T05:46:01.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.M58DwH-hL9RLPuAa1AlxXCM8yNtXGbb_0 2024-04-17T05:46:01.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18182862205202143262', time=2024-04-17T07:21:08.356768+0000, chain.objs.size()=1 2024-04-17T05:46:01.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LHdEsnF6sQDsZiceIwQFdtgOfRJ0RRk_0 2024-04-17T05:46:01.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2883508391427272320', time=2024-04-17T07:21:09.359896+0000, chain.objs.size()=1 2024-04-17T05:46:01.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.drWOkowhJqFLjMzp5XhmTpRMR5Bc30q_0 2024-04-17T05:46:01.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2414453096962160967', time=2024-04-17T07:21:09.979427+0000, chain.objs.size()=1 2024-04-17T05:46:01.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uuiphhU2FuhebOUS_giPmmCUQLFey7X_0 2024-04-17T05:46:01.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5074526390746202971', time=2024-04-17T07:21:10.214853+0000, chain.objs.size()=1 2024-04-17T05:46:01.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WQaQnPruznUhSods2bPwA-ykT5gBchU_0 2024-04-17T05:46:01.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8152782744123609832', time=2024-04-17T07:21:11.950747+0000, chain.objs.size()=1 2024-04-17T05:46:01.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uAnQQ3XLt-5QzNHThMO99P_azMcG9Kn_0 2024-04-17T05:46:01.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11926432639636958958', time=2024-04-17T07:21:12.517378+0000, chain.objs.size()=1 2024-04-17T05:46:01.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.NG5CTbv2GT0JkoJrvgRGMV4W5pdYgCD_0 2024-04-17T05:46:01.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12334253940823945692', time=2024-04-17T07:21:13.148224+0000, chain.objs.size()=1 2024-04-17T05:46:01.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5rItvIzBGNEDWaGUl6aQtVQbXW8K2m5_0 2024-04-17T05:46:01.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2306156590451142109', time=2024-04-17T07:21:13.825116+0000, chain.objs.size()=1 2024-04-17T05:46:01.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._tBZYQIp_L-JsYK3GusrqefLCbskEZn_0 2024-04-17T05:46:01.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1552046144613205492', time=2024-04-17T07:21:13.826986+0000, chain.objs.size()=1 2024-04-17T05:46:01.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.seKP-o02p0BnMPOpf4vGZHL5okUaEzK_0 2024-04-17T05:46:01.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6676599267090251645', time=2024-04-17T07:21:14.004285+0000, chain.objs.size()=1 2024-04-17T05:46:01.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oKiyqunhvBUa3OAn8WzT6WFaqWm2szp_0 2024-04-17T05:46:01.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7031135764823926387', time=2024-04-17T07:21:14.252725+0000, chain.objs.size()=1 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9uBqg4AB_WcNGDvCw02_2Mds3QOfI0U_0 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14789492980577116312', time=2024-04-17T07:21:14.255032+0000, chain.objs.size()=1 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jbuhLjrvbHvDV0_CP1-E5DEMlAyKlS7_0 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9324625639534858318', time=2024-04-17T07:21:14.834879+0000, chain.objs.size()=1 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mWPkL2aTSteJdoQjVnR4pyDlROEp0MQ_0 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15582640163514462396', time=2024-04-17T07:21:16.924612+0000, chain.objs.size()=1 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MFIPhvyencXus5bS8_N3EFXnIm6GWMB_0 2024-04-17T05:46:01.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14929333758316608303', time=2024-04-17T07:21:17.512647+0000, chain.objs.size()=1 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.L3ZDnJjrE7KX3rM-VaMdL_WzFeHjYkh_0 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.555177611099797538', time=2024-04-17T07:21:20.436796+0000, chain.objs.size()=1 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xhrGRF3q7dSP8MbJjEO_Q-QH2_YMzHK_0 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12465023214936796884', time=2024-04-17T07:21:25.009995+0000, chain.objs.size()=1 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.X8neCZgBlAsqajYxiXkY3KpO3_RlExe_0 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12982439244641237237', time=2024-04-17T07:21:25.042918+0000, chain.objs.size()=1 2024-04-17T05:46:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TGm6yVj9l51dFWFUfyTk7OMP3XE8-fZ_0 2024-04-17T05:46:01.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7040091561126142740', time=2024-04-17T07:21:25.592822+0000, chain.objs.size()=1 2024-04-17T05:46:01.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.E7xLX-R1m2aWUKOf7kZhjLLH4_dzvj3_0 2024-04-17T05:46:01.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10684936977936201176', time=2024-04-17T07:21:25.877957+0000, chain.objs.size()=1 2024-04-17T05:46:01.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.p6viKp1NryQLBzLVda6JayabOxl0Ydf_0 2024-04-17T05:46:01.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8758231586323325961', time=2024-04-17T07:21:27.089513+0000, chain.objs.size()=1 2024-04-17T05:46:01.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mC5PxgdPH1hXyLWmU9f2Loz2dY_453x_0 2024-04-17T05:46:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12948364159161255521', time=2024-04-17T07:21:27.520029+0000, chain.objs.size()=1 2024-04-17T05:46:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GeHyDLmEk6MGq5RnpGILu0TCXiDoiiE_0 2024-04-17T05:46:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5340562977456037201', time=2024-04-17T07:21:27.921183+0000, chain.objs.size()=1 2024-04-17T05:46:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-45QkcF7Y53WW0c6SygRZZJkc2GOjZS_0 2024-04-17T05:46:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12032265769853177986', time=2024-04-17T07:23:13.997849+0000, chain.objs.size()=1 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8S6dsSSubF35tyaAlVoOLHtVevPXRHp_0 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16527618138916248561', time=2024-04-17T07:23:14.766095+0000, chain.objs.size()=1 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.z9_fCYZEwX7F6G35zziQFvs3x2GYTzF_0 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15351177756422440671', time=2024-04-17T07:23:14.962061+0000, chain.objs.size()=1 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NK_3RsWk4CStTSPf7KbQBk5j_HCNORP_0 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16450316738375343511', time=2024-04-17T07:23:17.171335+0000, chain.objs.size()=1 2024-04-17T05:46:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yzoAZ7c4CnZFg9JiA5SHMiUHXnR-8MQ_0 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15476089368711806692', time=2024-04-17T07:23:17.567771+0000, chain.objs.size()=1 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iIjoQLlTzzGMEl0E9s9KJwRSC_PArBs_0 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1253077883269651184', time=2024-04-17T07:23:17.852283+0000, chain.objs.size()=1 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.v5XEFoROcfTo9mjRKQ3iyk6AmT3aBFJ_0 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3349667656976318839', time=2024-04-17T07:23:18.275242+0000, chain.objs.size()=1 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.h5yZ1XXnAtJcxMAJf1DIgwgKV0xOXSY_0 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12206280448158512841', time=2024-04-17T07:23:19.759458+0000, chain.objs.size()=1 2024-04-17T05:46:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sKTfpE5W5IcJvDx6Q1s3cyeOaBn5xqp_0 2024-04-17T05:46:01.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9752840483352461973', time=2024-04-17T07:23:22.348657+0000, chain.objs.size()=1 2024-04-17T05:46:01.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Le_XLRFLULXxp8NX0WsvDxuckeSqUqt_0 2024-04-17T05:46:01.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.927875335480563518', time=2024-04-17T07:23:22.647527+0000, chain.objs.size()=1 2024-04-17T05:46:01.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mbKC3w2IQQJvLb3E_9Ud6TPMhzwB9Er_0 2024-04-17T05:46:01.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13509021302352240831', time=2024-04-17T07:23:23.857762+0000, chain.objs.size()=1 2024-04-17T05:46:01.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1dNluJ9nMuQ5OrC-LFhgl11iy39vg-e_0 2024-04-17T05:46:01.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.832384483694096645', time=2024-04-17T07:23:23.860034+0000, chain.objs.size()=1 2024-04-17T05:46:01.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5qX-4uccLvSTc4geEBLEydkCdtYVhgf_0 2024-04-17T05:46:01.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14981588367147540817', time=2024-04-17T07:23:25.018550+0000, chain.objs.size()=1 2024-04-17T05:46:01.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.HpqFUgaTjuZH0G5USQ5Oox0HfeW0fA__0 2024-04-17T05:46:01.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2671239920327751774', time=2024-04-17T07:23:25.506812+0000, chain.objs.size()=1 2024-04-17T05:46:01.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.FrxtdUM9SBFwuxRgHBfxGlMY7MV3RW8_0 2024-04-17T05:46:01.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5059041699141375890', time=2024-04-17T07:23:25.888962+0000, chain.objs.size()=1 2024-04-17T05:46:01.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.AjPzX00DUNot48WWNsucThZP4gsubmI_0 2024-04-17T05:46:01.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14544242175308172612', time=2024-04-17T07:23:26.056151+0000, chain.objs.size()=1 2024-04-17T05:46:01.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.y-2q_GAtUfQuYScVhS0URrvH2PXhxUG_0 2024-04-17T05:46:01.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6000677440130439722', time=2024-04-17T07:23:27.072600+0000, chain.objs.size()=1 2024-04-17T05:46:01.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iq6ELTyUTHTe4lI7qNgwFYHlUPYUvea_0 2024-04-17T05:46:01.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45015 2024-04-17T05:46:01.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=40, truncated=0, next_marker='' 2024-04-17T05:46:01.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15070367736700364737', time=2024-04-17T07:23:27.518704+0000, chain.objs.size()=1 2024-04-17T05:46:01.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.STfEW6ieDwL6yitdV6uWKMIsHiV6fwR_0 2024-04-17T05:46:01.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8011356827579643209', time=2024-04-17T07:23:28.565386+0000, chain.objs.size()=1 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.t-ZjRmA5a_RvdfFD5uDJWxhal4bu0N8_0 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.579438509986441135', time=2024-04-17T07:23:28.939866+0000, chain.objs.size()=1 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iN8FY2JpyeVWma426vCS6Fw3lvPxmlL_0 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13675555691114191056', time=2024-04-17T07:23:29.413095+0000, chain.objs.size()=1 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.b7ogNvYiE4aM_8CFed5ge_5C-QuV-Nm_0 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11116351672591492358', time=2024-04-17T07:23:32.623345+0000, chain.objs.size()=1 2024-04-17T05:46:01.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GQ-YlDjhndJkUH8FImDXYO4SOlgVSsv_0 2024-04-17T05:46:01.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7518058406941637287', time=2024-04-17T07:23:32.868632+0000, chain.objs.size()=1 2024-04-17T05:46:01.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uxUExPU7OVQ0fw_OqlXZCwunSjKy063_0 2024-04-17T05:46:01.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13209755126860843645', time=2024-04-17T07:23:33.127384+0000, chain.objs.size()=1 2024-04-17T05:46:01.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YOq_DgUOHSQra1_MENsG3efcJzksGpU_0 2024-04-17T05:46:01.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2444735881326948357', time=2024-04-17T07:23:33.227724+0000, chain.objs.size()=1 2024-04-17T05:46:01.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5Oybkpea5PJ2YOc4paAhurpxVQ8X3Yk_0 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3198887361603557704', time=2024-04-17T07:23:33.284233+0000, chain.objs.size()=1 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eU4_rLImCmYNvkHpRNyBbAQQeswqtp0_0 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8186936904865271914', time=2024-04-17T07:23:33.905976+0000, chain.objs.size()=1 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Hr86UXl0Y0_9l2JrsqiigJCqUiXgLIP_0 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10745499870880228324', time=2024-04-17T07:23:34.687201+0000, chain.objs.size()=1 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eshvXTJL0zOVNSqQf7cyTseH67On4Mk_0 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5557943629219255339', time=2024-04-17T07:23:35.903137+0000, chain.objs.size()=1 2024-04-17T05:46:01.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fBj8nRS9GJP0WghUK_RKYnZcP962yMu_0 2024-04-17T05:46:01.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8024740602311179124', time=2024-04-17T07:23:35.905130+0000, chain.objs.size()=1 2024-04-17T05:46:01.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Omj88EH4DW1XFLpiBARpo6MOgEYITZw_0 2024-04-17T05:46:01.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12804514869960961164', time=2024-04-17T07:23:36.190320+0000, chain.objs.size()=1 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bUxYMqwfNSlaujPRfZwDqcwKpx2l38H_0 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12963757850434386727', time=2024-04-17T07:23:36.376437+0000, chain.objs.size()=1 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BZHMZr9nRSCZtEtp9Fou2dlRb3nezlS_0 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17172726811176861619', time=2024-04-17T07:23:36.378178+0000, chain.objs.size()=1 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0rRJtgo3Ce5BizNP94y3PoU859Q1j1r_0 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2068211407776221478', time=2024-04-17T07:23:36.891814+0000, chain.objs.size()=1 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Oyt-rZi8SF_Ej9Fx4R1FbapduwBQHu-_0 2024-04-17T05:46:01.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17700632319729621818', time=2024-04-17T07:23:37.162048+0000, chain.objs.size()=1 2024-04-17T05:46:01.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Jkt0oIBi5k-fXGxCzb1rUJfQhMgMTTf_0 2024-04-17T05:46:01.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.855301676514727747', time=2024-04-17T07:23:37.348384+0000, chain.objs.size()=1 2024-04-17T05:46:01.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WItY4DvJCxomURYgbFO3p0bfXorM6nq_0 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13448907561275492698', time=2024-04-17T07:23:37.430296+0000, chain.objs.size()=1 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hV8dBZMfldHuBir8ecUcIof3nZkZqXm_0 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17779724858055989137', time=2024-04-17T07:23:38.284888+0000, chain.objs.size()=1 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vsfBV-H-fY6KQhsjiBoJaUxFInqCgrd_0 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14582406988568605507', time=2024-04-17T07:23:38.700830+0000, chain.objs.size()=1 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.O0FX0hVAeJBWpbTvWAd7-k_AhHZ82xG_0 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9765762719434105969', time=2024-04-17T07:23:39.482349+0000, chain.objs.size()=1 2024-04-17T05:46:01.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RBtqAp1yF4a17l1Ph_scaWtPCLWDeVi_0 2024-04-17T05:46:01.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7374381355867675342', time=2024-04-17T07:23:39.807211+0000, chain.objs.size()=1 2024-04-17T05:46:01.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hAtWh7BemfuxnapqiPMBljsBtEl-Llx_0 2024-04-17T05:46:01.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8923490048428561277', time=2024-04-17T07:23:40.706992+0000, chain.objs.size()=1 2024-04-17T05:46:01.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OafANUx_CJWEfjq7toJv1uev7kNDhUb_0 2024-04-17T05:46:01.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14700934992551008062', time=2024-04-17T07:23:40.779046+0000, chain.objs.size()=1 2024-04-17T05:46:01.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8kW-c4iMO1wK7NXdGgj3HrizecysaDO_0 2024-04-17T05:46:01.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.301133663360464964', time=2024-04-17T07:23:41.275912+0000, chain.objs.size()=1 2024-04-17T05:46:01.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gDKJsig6NU1bc5oFzItM0yfc7qsWNz0_0 2024-04-17T05:46:01.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12479213608018533641', time=2024-04-17T07:23:41.320863+0000, chain.objs.size()=1 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SmYKwsJPiBxj96e9q_CaLez-_6LPouX_0 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18308417913007410279', time=2024-04-17T07:24:29.989195+0000, chain.objs.size()=1 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.369__shadow_.8gA2M3k_0FpWjkNhpBrsQf8EqX4MU3V_0 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14469801174503316673', time=2024-04-17T07:24:35.786324+0000, chain.objs.size()=1 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.374__shadow_.Io2Ws29rAJFAr6qsQjMu5WEtu14s6_Z_0 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15703271783778242344', time=2024-04-17T07:25:45.720000+0000, chain.objs.size()=1 2024-04-17T05:46:01.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.446__shadow_.uCl2eFnDwIAK9mPSPzSo8SCkrc1rR0F_0 2024-04-17T05:46:01.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10431634501973327236', time=2024-04-17T07:25:47.401892+0000, chain.objs.size()=1 2024-04-17T05:46:01.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.447__shadow_.-je_2QjaMmd5_YfqIbv8oI8T8lc6b6S_0 2024-04-17T05:46:01.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.490__shadow_.5GS2FmjrPZ43B2Fv3pYCq33y54MwvgL_1_f17f53c130309531b85c3184fd7090cd', time=2024-04-17T07:26:38.843049+0000, chain.objs.size()=1 2024-04-17T05:46:01.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.490__shadow_.5GS2FmjrPZ43B2Fv3pYCq33y54MwvgL_0 2024-04-17T05:46:01.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16954220918568714676', time=2024-04-17T07:27:16.101764+0000, chain.objs.size()=1 2024-04-17T05:46:01.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.508__multipart_mymultipart.2~7zJHU1eM_mUSAfL4NjckIWy-hp0qI4x.1 2024-04-17T05:46:01.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd', time=2024-04-17T07:28:08.636816+0000, chain.objs.size()=1 2024-04-17T05:46:01.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd.2 2024-04-17T05:46:01.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15891810488688191583', time=2024-04-17T07:31:58.306579+0000, chain.objs.size()=1 2024-04-17T05:46:01.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.559__shadow:ee3zXQqCeEIPddJXweuq3g2AZg0dDgS_.jXugfzblzssqu1Bk8WIUIlq27RR5UGO_0 2024-04-17T05:46:01.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1444618196807953355', time=2024-04-17T07:32:55.634237+0000, chain.objs.size()=1 2024-04-17T05:46:01.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:NQpgm-CEhp4iDdPrPzWXy5KM9uzId1M_.kS-NgYT6bKtJUMAbfgvD4FnvKEBd9Ld_0 2024-04-17T05:46:01.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17285445908038563462', time=2024-04-17T07:32:56.333569+0000, chain.objs.size()=1 2024-04-17T05:46:01.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:aAHs4l6l-v4OI2U8evhH-DqdWHPXP8F_.YwEpDvDBdvih2KN7FyaRBZtxPulRM9V_0 2024-04-17T05:46:02.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7706648688369966227', time=2024-04-17T07:33:07.661237+0000, chain.objs.size()=1 2024-04-17T05:46:02.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:FoqmhD-erlm90uH9kw83Y0swG6e96dj_.5DopM9Sr5mHeVTklqYu1-RUe_QAo_Ce_0 2024-04-17T05:46:02.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2904732022440557726', time=2024-04-17T07:38:16.838575+0000, chain.objs.size()=1 2024-04-17T05:46:02.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:01.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.687__multipart_file1.2~8g41ntdQXQqeH0fUhCZG0DDtN2CYouI.1 2024-04-17T05:46:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-17T05:46:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46192' 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17433684754737572758', time=2024-04-17T07:10:15.891073+0000, chain.objs.size()=3 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-STANDARD.2~dReDpbqv6AB5V9O8c1VTENxeCZ8M9sH.1 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-STANDARD.2~dReDpbqv6AB5V9O8c1VTENxeCZ8M9sH.2 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-STANDARD.2~dReDpbqv6AB5V9O8c1VTENxeCZ8M9sH.3 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_1xkNoB5yUmjBP2dkEkYMoQcVVN63oU_', time=2024-04-17T07:10:16.604305+0000, chain.objs.size()=2 2024-04-17T05:46:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.-CFW_GaT3R3MjjrZkvg5qKtBzBfKeTq_0 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.-CFW_GaT3R3MjjrZkvg5qKtBzBfKeTq_1 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16334503232948213247', time=2024-04-17T07:14:34.701145+0000, chain.objs.size()=1 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.225__shadow_.igVgQ8F1KoE8PqwuwpUQDH35mxj0Kw__0 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15285642802924290081', time=2024-04-17T07:14:36.306118+0000, chain.objs.size()=1 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.226__shadow_._6EJn8hgKq0_Kt6LvMH3145zVQvuouy_0 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9751738626639965885', time=2024-04-17T07:14:45.015163+0000, chain.objs.size()=1 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.231__shadow_.NNBRNqYQV6CuCJAN-ludOtd02UBOqn7_0 2024-04-17T05:46:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10997151213202206489', time=2024-04-17T07:14:56.124812+0000, chain.objs.size()=1 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.237__shadow_.dUBWZJ0m2o88TNtdvurB1DEj1T_5toS_0 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14991892001306658043', time=2024-04-17T07:15:02.638456+0000, chain.objs.size()=1 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.241__shadow_.wBmMfoVMWuZ4t4eNM3-MS2RXi7RfG3S_0 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18276910224449170807', time=2024-04-17T07:15:09.135101+0000, chain.objs.size()=1 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.245__shadow_.KdLuAWStxAeJj8mWglES7D3_M3KmgFD_0 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2930453462155376089', time=2024-04-17T07:15:09.169076+0000, chain.objs.size()=1 2024-04-17T05:46:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.245__shadow_.eHgwoMp5lmzSyx2edg4Fpv_fE64Ehqr_0 2024-04-17T05:46:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3625556095289936355', time=2024-04-17T07:15:10.737176+0000, chain.objs.size()=1 2024-04-17T05:46:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.246__shadow_.9CqV4jERUJowrFTNpOQfm7Pf_AUfp2K_0 2024-04-17T05:46:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16246659939515398368', time=2024-04-17T07:15:15.234890+0000, chain.objs.size()=1 2024-04-17T05:46:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.249__shadow_.R1OJiEsXTB5Ovyma3IGcWrQoDNHHD4x_0 2024-04-17T05:46:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16343173237742610295', time=2024-04-17T07:15:16.838050+0000, chain.objs.size()=1 2024-04-17T05:46:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.250__shadow_.Q-2r5ZaQyTxhAbNCBgjmYUKivlD-zle_0 2024-04-17T05:46:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4003772975292765216', time=2024-04-17T07:15:18.477249+0000, chain.objs.size()=1 2024-04-17T05:46:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.251__shadow_.g9D-FAfU7cqGNgp3uAzyt3IsvtAhnpA_0 2024-04-17T05:46:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8689814721577088559', time=2024-04-17T07:17:01.987302+0000, chain.objs.size()=1 2024-04-17T05:46:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.t2khJcYVSDbvBk893JUE7SJxK1MtsXk_0 2024-04-17T05:46:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10164217083209632241', time=2024-04-17T07:17:02.535824+0000, chain.objs.size()=1 2024-04-17T05:46:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.S2FrDwubaZOIJABfy-pxRqcLnRRanze_0 2024-04-17T05:46:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1930755755710006097', time=2024-04-17T07:17:02.632949+0000, chain.objs.size()=1 2024-04-17T05:46:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UJE6_slt0fTXcKLdMtOxWkoQjXl7m5a_0 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16242441610817819737', time=2024-04-17T07:17:04.291585+0000, chain.objs.size()=1 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nGe4xv4mHehX_mdE3hTwWqv8telBUZS_0 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2367463048984569566', time=2024-04-17T07:17:08.034693+0000, chain.objs.size()=1 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lkd0rD_jwd2U45snB4Q3UT9kzJ8XeQG_0 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2159486482095647884', time=2024-04-17T07:17:09.839084+0000, chain.objs.size()=1 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4LUxH3Kt6_USuxRqdBNyGcP1QN8k2pB_0 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3441313334038938073', time=2024-04-17T07:17:10.265539+0000, chain.objs.size()=1 2024-04-17T05:46:02.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rK-KJYkHT1zFq6Aqjv56c79eF5Cfq73_0 2024-04-17T05:46:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16949005833978310230', time=2024-04-17T07:17:14.198567+0000, chain.objs.size()=1 2024-04-17T05:46:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SQJ12yPo3WSLGbq-sli5CcAGZbyxxmx_0 2024-04-17T05:46:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6626544409545975522', time=2024-04-17T07:17:17.122080+0000, chain.objs.size()=1 2024-04-17T05:46:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bB05rUygFfoJ0dnvevy1gm-AUYmt0vF_0 2024-04-17T05:46:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12475691578425367983', time=2024-04-17T07:17:18.083444+0000, chain.objs.size()=1 2024-04-17T05:46:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KG0p7Nrdx89NvQ6BPVfAdp5rtPDjOa8_0 2024-04-17T05:46:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10737509126921650147', time=2024-04-17T07:17:18.708507+0000, chain.objs.size()=1 2024-04-17T05:46:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1hHJ3M9t6xoB457Uz-6xyE99cT0Nq8Z_0 2024-04-17T05:46:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15455691885417858516', time=2024-04-17T07:17:18.956456+0000, chain.objs.size()=1 2024-04-17T05:46:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yHSsc-Cu-zCcYCxmtSq--i4FM58EtfB_0 2024-04-17T05:46:02.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8911061862006150361', time=2024-04-17T07:17:23.897213+0000, chain.objs.size()=1 2024-04-17T05:46:02.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.WUTqxAHo0cqBSxdv1FP0WTzC5l_WX7a_0 2024-04-17T05:46:02.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3410585057147643580', time=2024-04-17T07:17:25.835181+0000, chain.objs.size()=1 2024-04-17T05:46:02.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DiV-OX29bUC1KDegkBnaUl4uHQHjmFr_0 2024-04-17T05:46:02.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2944027574147079106', time=2024-04-17T07:17:26.000170+0000, chain.objs.size()=1 2024-04-17T05:46:02.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UL0lQL24WDo1GYLkRukeCcm6at5i3Ga_0 2024-04-17T05:46:02.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9269792194047270429', time=2024-04-17T07:17:26.469284+0000, chain.objs.size()=1 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EeuH3-f1oKht-VEomq6OtZcVjwzFX7l_0 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17830596635510139143', time=2024-04-17T07:17:27.252871+0000, chain.objs.size()=1 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PlmEiRKZ98Sp18Yu2-hdzpH7-C3x5lX_0 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3956582174677809663', time=2024-04-17T07:17:28.165432+0000, chain.objs.size()=1 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9Dbe0jpaeR_42mh33FTRV6HJNDxNkH1_0 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9226463861977310524', time=2024-04-17T07:17:28.360119+0000, chain.objs.size()=1 2024-04-17T05:46:02.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Wgd-3nQc7i6J6pn1eCr2JaBKoCBLF96_0 2024-04-17T05:46:02.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15052170816448227380', time=2024-04-17T07:17:28.415079+0000, chain.objs.size()=1 2024-04-17T05:46:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SfCFRFtHHP2QQlGbWhcyTL1Lo9Y5Qh3_0 2024-04-17T05:46:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17104015575658511983', time=2024-04-17T07:17:29.941011+0000, chain.objs.size()=1 2024-04-17T05:46:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5K-Ei8cr4V7IeYMFPnsi-SEdtY0-6x5_0 2024-04-17T05:46:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.742438121592536694', time=2024-04-17T07:17:31.102360+0000, chain.objs.size()=1 2024-04-17T05:46:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FcF-5F3UDVXAMCwS6_zWpr2Xzr7TEA7_0 2024-04-17T05:46:02.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13269989190240472915', time=2024-04-17T07:17:31.143018+0000, chain.objs.size()=1 2024-04-17T05:46:02.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VI5dSkbMKrVXTZ0nAXrngw1Wmm-bGyL_0 2024-04-17T05:46:02.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.991704591646308509', time=2024-04-17T07:17:31.567650+0000, chain.objs.size()=1 2024-04-17T05:46:02.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xQ_rwrgRAhgMUOKq7Shoin6N-rIL7JH_0 2024-04-17T05:46:02.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.806720474583892861', time=2024-04-17T07:17:38.433368+0000, chain.objs.size()=1 2024-04-17T05:46:02.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.256__shadow_.bhgH2TRz422WRqhQgGUEFfhW8gg0rP3_0 2024-04-17T05:46:02.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12503404676632947037', time=2024-04-17T07:17:42.945645+0000, chain.objs.size()=1 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.259__shadow_.Pn8_kgS09TCC7L7hNA-y9qZ_0x-xZx7_0 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14017499491005769171', time=2024-04-17T07:17:50.342235+0000, chain.objs.size()=1 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.264__shadow_.A-b5k2z8GmIu_z-S39_1iLYLiuVkUqf_0 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6760854138805879112', time=2024-04-17T07:17:55.234839+0000, chain.objs.size()=1 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.267__shadow_.VgpQNzaGRMxbMR4RjFOLGljdUrpJ9wB_0 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17730223611096010049', time=2024-04-17T07:18:11.622207+0000, chain.objs.size()=1 2024-04-17T05:46:02.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.277__shadow_.WXTeq4NAYigQFRiP-XtLx0NJ9C0beL1_0 2024-04-17T05:46:02.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13479553509441920530', time=2024-04-17T07:18:23.253239+0000, chain.objs.size()=1 2024-04-17T05:46:02.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.gI3Z8ZFVZNQ9Jkzs1qZqN6UISDEG9tC_0 2024-04-17T05:46:02.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8002150237584523704', time=2024-04-17T07:18:28.219147+0000, chain.objs.size()=1 2024-04-17T05:46:02.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.JXSzuJdaHGSRGvbOUXnIkKPaMjQlEuD_0 2024-04-17T05:46:02.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17462460880701957207', time=2024-04-17T07:20:59.873282+0000, chain.objs.size()=1 2024-04-17T05:46:02.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.d0rJE0iDqEtk7uIM23peCfhFWRG7Vgg_0 2024-04-17T05:46:02.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11141604940027536291', time=2024-04-17T07:21:01.288910+0000, chain.objs.size()=1 2024-04-17T05:46:02.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZG86spqxOfypAzz3LUZBSCLblRcrjw1_0 2024-04-17T05:46:02.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12298459682446198996', time=2024-04-17T07:21:01.339928+0000, chain.objs.size()=1 2024-04-17T05:46:02.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZtrTxsUKbdPHzSRqvcVgBSqEANZGxxO_0 2024-04-17T05:46:02.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.462890052594227545', time=2024-04-17T07:21:01.508937+0000, chain.objs.size()=1 2024-04-17T05:46:02.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4vTCud7JfwDqrq-T4jVqh5u7pdXJlqS_0 2024-04-17T05:46:02.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.756318827928930421', time=2024-04-17T07:21:01.765636+0000, chain.objs.size()=1 2024-04-17T05:46:02.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9xMIQtHxHXejo8PCadv3kp4dbGdOlIH_0 2024-04-17T05:46:02.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14752912490070985909', time=2024-04-17T07:21:01.878001+0000, chain.objs.size()=1 2024-04-17T05:46:02.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.E_dQBDOgYexGBuvjZ7j2LlPOlvnDZMw_0 2024-04-17T05:46:02.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10384334960525433575', time=2024-04-17T07:21:02.003880+0000, chain.objs.size()=1 2024-04-17T05:46:02.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Po1safl0RZ7rlB1zU-A3l4OD93GIqES_0 2024-04-17T05:46:02.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8153366630169310067', time=2024-04-17T07:21:02.253793+0000, chain.objs.size()=1 2024-04-17T05:46:02.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yv8CJFmgIVWqwJ2SdHV45JdcfvtZaPW_0 2024-04-17T05:46:02.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9721906385722734248', time=2024-04-17T07:21:02.443310+0000, chain.objs.size()=1 2024-04-17T05:46:02.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EppVcZMji2rezYTitzrT_qz68gUXLCA_0 2024-04-17T05:46:02.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6175901502643436022', time=2024-04-17T07:21:03.987133+0000, chain.objs.size()=1 2024-04-17T05:46:02.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SEU3w4uS8HbIIKj1XmHqJqJ60QqA2e-_0 2024-04-17T05:46:02.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10667527982042838102', time=2024-04-17T07:21:04.803010+0000, chain.objs.size()=1 2024-04-17T05:46:02.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3rlo3F27MV0wXhFytjkScRU-avCjEie_0 2024-04-17T05:46:02.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10729544552403592228', time=2024-04-17T07:21:05.479301+0000, chain.objs.size()=1 2024-04-17T05:46:02.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gfFxIh-Z_fweAvcfdhjRUOssXO2Ccpn_0 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9341483818741996904', time=2024-04-17T07:21:06.694756+0000, chain.objs.size()=1 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Mo3In83vyNCON7wQtt892GYRcksMt8q_0 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17406899645163313553', time=2024-04-17T07:21:08.041839+0000, chain.objs.size()=1 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ee077R-b9RmoJykX5ApmPshxosO9upS_0 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6930238492599177321', time=2024-04-17T07:21:08.393352+0000, chain.objs.size()=1 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HWLMiEX6BiDnygLWclM8nFqTpVZo3Yz_0 2024-04-17T05:46:02.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5229233876032629074', time=2024-04-17T07:21:08.509179+0000, chain.objs.size()=1 2024-04-17T05:46:02.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HcRjMEnhiIcPyjSaKtM6fn7a3Dw7SxY_0 2024-04-17T05:46:02.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6582977745728224418', time=2024-04-17T07:21:08.510822+0000, chain.objs.size()=1 2024-04-17T05:46:02.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.036+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rjGpI1JMnIDEqHdinkbLatkMCGkClF-_0 2024-04-17T05:46:02.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12085293465962255006', time=2024-04-17T07:21:08.802247+0000, chain.objs.size()=1 2024-04-17T05:46:02.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mk3Tzngy6LFhEmxpUJSb8qyrZzXOkuA_0 2024-04-17T05:46:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7805288557510880858', time=2024-04-17T07:21:08.888121+0000, chain.objs.size()=1 2024-04-17T05:46:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.U8SftNqDxb0Hyt1JrojglR39pf43fNV_0 2024-04-17T05:46:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14342071786066875879', time=2024-04-17T07:21:10.951572+0000, chain.objs.size()=1 2024-04-17T05:46:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UnHydW4hfnlPHUcNCmtr2NVWd9gtBqA_0 2024-04-17T05:46:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6751997380929571019', time=2024-04-17T07:21:11.879953+0000, chain.objs.size()=1 2024-04-17T05:46:02.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-0YyXY1TfK13gLquVUy8pNj9oEIcXsK_0 2024-04-17T05:46:02.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16125875619084407402', time=2024-04-17T07:21:12.292077+0000, chain.objs.size()=1 2024-04-17T05:46:02.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.040+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZUJFSoxM_8D8MhL3PLjPgrJybO1yP4R_0 2024-04-17T05:46:02.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16435485307818350830', time=2024-04-17T07:21:12.503258+0000, chain.objs.size()=1 2024-04-17T05:46:02.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.S9qqtAdr3gq9BcYlVFlKRxBwbgnf5v0_0 2024-04-17T05:46:02.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18092938671030398701', time=2024-04-17T07:21:12.623719+0000, chain.objs.size()=1 2024-04-17T05:46:02.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uGQjsicB29dXnBl8nZP1Fv7HpikvwsB_0 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3300226263001482120', time=2024-04-17T07:21:12.952163+0000, chain.objs.size()=1 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oDwFtPAcsICZNBjKQwwO6gyGxWVDbAD_0 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10368879906574341541', time=2024-04-17T07:21:14.414315+0000, chain.objs.size()=1 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-7XovoNh2WV4oM8OUnaQkd_n1xehw8C_0 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1863341528860318471', time=2024-04-17T07:21:16.161413+0000, chain.objs.size()=1 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lf53LK0gZ9fDloPbhX4qeXNHPh_4aMt_0 2024-04-17T05:46:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17821553858997074931', time=2024-04-17T07:21:17.945319+0000, chain.objs.size()=1 2024-04-17T05:46:02.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.e1lU0G38nKN0izSjV9vGzxTBbQttyt-_0 2024-04-17T05:46:02.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6538874597642272148', time=2024-04-17T07:21:18.419272+0000, chain.objs.size()=1 2024-04-17T05:46:02.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LmxO2EtQECZ9aZ8D0y393DrLgbAXlP0_0 2024-04-17T05:46:02.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2714273316832963500', time=2024-04-17T07:21:18.716351+0000, chain.objs.size()=1 2024-04-17T05:46:02.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.e6mDaYW1w-FSL6CJtwBjI8sRjchvSnr_0 2024-04-17T05:46:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6729805652417021889', time=2024-04-17T07:21:19.956691+0000, chain.objs.size()=1 2024-04-17T05:46:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZItDX2f_FzNdyAjrlOqhwFqLbfsKt24_0 2024-04-17T05:46:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7224970261686304718', time=2024-04-17T07:21:21.354611+0000, chain.objs.size()=1 2024-04-17T05:46:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FWUGcrwpHSEnMUYZ060FN3zqW4DJCMn_0 2024-04-17T05:46:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11089174904032081854', time=2024-04-17T07:21:21.675143+0000, chain.objs.size()=1 2024-04-17T05:46:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wSChvVLw5_jlODsyHCtBWRHwkJwAGY__0 2024-04-17T05:46:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5202684336520561704', time=2024-04-17T07:21:22.192078+0000, chain.objs.size()=1 2024-04-17T05:46:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1tZC6wYa4FdpSJiLRI0FD7l6Ji6_j_4_0 2024-04-17T05:46:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4866128782770948850', time=2024-04-17T07:21:23.683480+0000, chain.objs.size()=1 2024-04-17T05:46:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1adRkQiNabWB2GbSihGKb5TNbikC54P_0 2024-04-17T05:46:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9557450021479263015', time=2024-04-17T07:21:27.336836+0000, chain.objs.size()=1 2024-04-17T05:46:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.44Y7hvlBfcw09W2Jdd7n6EUJhZ6yq2w_0 2024-04-17T05:46:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9398233191118853633', time=2024-04-17T07:21:27.884587+0000, chain.objs.size()=1 2024-04-17T05:46:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YhchVkywMK2wzyMFOBlahiVxEDriSIY_0 2024-04-17T05:46:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4091830576385786777', time=2024-04-17T07:21:29.022019+0000, chain.objs.size()=1 2024-04-17T05:46:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RxQZz5b7yQ5M5d2s41o2oZ9LYUYEP3l_0 2024-04-17T05:46:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11755318727633705061', time=2024-04-17T07:21:29.184267+0000, chain.objs.size()=1 2024-04-17T05:46:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3nYPtJXfvfoYWBZkn_IGu9JAJ5d7jeP_0 2024-04-17T05:46:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12016561960187112697', time=2024-04-17T07:23:13.953983+0000, chain.objs.size()=1 2024-04-17T05:46:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vbrrVact8nXjidq6phycWa5vMQM6vjB_0 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8409263540084629521', time=2024-04-17T07:23:15.531978+0000, chain.objs.size()=1 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pZqQRGwzP7n7keWkvh57848mWNp8qxm_0 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17481282648423435321', time=2024-04-17T07:23:15.597413+0000, chain.objs.size()=1 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uWihwR1N9eJvCPY9oce6qVeg93U-d8I_0 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5307029594670357441', time=2024-04-17T07:23:18.236214+0000, chain.objs.size()=1 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.86bhF-WAHqJFit0IEcACL17twG1yLsF_0 2024-04-17T05:46:02.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17008030745080310173', time=2024-04-17T07:23:18.584251+0000, chain.objs.size()=1 2024-04-17T05:46:02.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gTRPFc1FQMgLqyPtJx-P5lFSPRPniy9_0 2024-04-17T05:46:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17825028014362641860', time=2024-04-17T07:23:21.460698+0000, chain.objs.size()=1 2024-04-17T05:46:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SRxH-syZlk4cWO_tjc7SJ7rRhFVwOwO_0 2024-04-17T05:46:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15906548457877969251', time=2024-04-17T07:23:21.462716+0000, chain.objs.size()=1 2024-04-17T05:46:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YdW0x_xFnsOY0byCdnouD-c9-7dVkf2_0 2024-04-17T05:46:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1892190377780043312', time=2024-04-17T07:23:21.466770+0000, chain.objs.size()=1 2024-04-17T05:46:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.h5aQt7wq8hazrBKsg3dsG9MrQQJWX6s_0 2024-04-17T05:46:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15732462572752613822', time=2024-04-17T07:23:21.780299+0000, chain.objs.size()=1 2024-04-17T05:46:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.q2NLJR8xloKnv-qcGVM78g6Brq-Rbsc_0 2024-04-17T05:46:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4293660324810194132', time=2024-04-17T07:23:22.332067+0000, chain.objs.size()=1 2024-04-17T05:46:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.B-AtGQpo237nRDrGcPB0ZPNGmY1D-AQ_0 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12693190256813499807', time=2024-04-17T07:23:22.335053+0000, chain.objs.size()=1 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qEU0WTgLnEmZI6637LzXiTYPbGlFsrR_0 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16882990241961238739', time=2024-04-17T07:23:23.760772+0000, chain.objs.size()=1 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vxx5EvMudz7Dyq_3EAVGOepTZs9Dfdc_0 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.791935329104163689', time=2024-04-17T07:23:25.607758+0000, chain.objs.size()=1 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vSMr5mBePyUa5dJg5-Ye1-2orO3WfsT_0 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7411461683546776466', time=2024-04-17T07:23:25.623948+0000, chain.objs.size()=1 2024-04-17T05:46:02.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZXNELvRUheRY84-GNMSMgmseykEGg8R_0 2024-04-17T05:46:02.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5829997033306164282', time=2024-04-17T07:23:25.818034+0000, chain.objs.size()=1 2024-04-17T05:46:02.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EwWYU_ad7aic4eQvEhIvyStNcqDOEh__0 2024-04-17T05:46:02.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8987101597440848875', time=2024-04-17T07:23:26.038991+0000, chain.objs.size()=1 2024-04-17T05:46:02.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iNWfouZRuaINDPSbP-gPBqPRCquF3Qd_0 2024-04-17T05:46:02.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7258279849154776442', time=2024-04-17T07:23:26.130884+0000, chain.objs.size()=1 2024-04-17T05:46:02.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VB4129x4RG3AwyhlZ0wEwR-E62E6kJ3_0 2024-04-17T05:46:02.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/46192 2024-04-17T05:46:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=30, truncated=0, next_marker='' 2024-04-17T05:46:02.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13214457832875631850', time=2024-04-17T07:23:27.421516+0000, chain.objs.size()=1 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4BSi1oHeFvkebnOtdS6EPncAx6LyjpY_0 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6668538551124413369', time=2024-04-17T07:23:30.044893+0000, chain.objs.size()=1 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Uaa9paE6CZLKuxZ8fQBs5NY-GbVSL18_0 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18171755865438650560', time=2024-04-17T07:23:30.596970+0000, chain.objs.size()=1 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xVkmplVsPFzDwXCZT_Qz7l8LTPMu652_0 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2710488076699275444', time=2024-04-17T07:23:30.918975+0000, chain.objs.size()=1 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TntPToamlc4dSS5W3KxpXfhYpfO7IYM_0 2024-04-17T05:46:02.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14511346370374518993', time=2024-04-17T07:23:32.862945+0000, chain.objs.size()=1 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.coyrnGNXNW5ymkuPiK9kqWn3oC8kpVv_0 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1416280182411019048', time=2024-04-17T07:23:33.013201+0000, chain.objs.size()=1 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iIF_j2f6mkaHeR8rVQNsFIGNMHYFtd4_0 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18218997838841900650', time=2024-04-17T07:23:33.193364+0000, chain.objs.size()=1 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OVRjgWJ5CrgYZB4hkRmIUdQDOJ5RieN_0 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7365068906985208849', time=2024-04-17T07:23:33.478750+0000, chain.objs.size()=1 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oGP5asKTkex5QHdEcsgS3Jx-zTk8Y4a_0 2024-04-17T05:46:02.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16114300733377061448', time=2024-04-17T07:23:33.842556+0000, chain.objs.size()=1 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cx3tiVOH79Mjbp79QqmRkAlch_NSY_x_0 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.39685406791292159', time=2024-04-17T07:23:34.206955+0000, chain.objs.size()=1 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5DMY9mRUqGpqyLfDbg7c78DJjAaHh9Q_0 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17019140911689610121', time=2024-04-17T07:23:36.871284+0000, chain.objs.size()=1 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GF7CI-0dJnq58tsFXq9HFoQZYzZCf7__0 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18127336321815940088', time=2024-04-17T07:23:37.705602+0000, chain.objs.size()=1 2024-04-17T05:46:02.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uf_jEbwtpg1n9hYB6xQo1oH2XK5f-Dt_0 2024-04-17T05:46:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11078138473998352920', time=2024-04-17T07:23:37.768601+0000, chain.objs.size()=1 2024-04-17T05:46:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RJDCcrdu82QeNhzfTBBef0c3cLoA0Cv_0 2024-04-17T05:46:02.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11471321325461571718', time=2024-04-17T07:23:39.792209+0000, chain.objs.size()=1 2024-04-17T05:46:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.zzJvM0fHesgWXza0KBg_DuXzjXLKivK_0 2024-04-17T05:46:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150964651844183438', time=2024-04-17T07:23:41.227393+0000, chain.objs.size()=1 2024-04-17T05:46:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nK3uAiNJfPoXBRJ1Y-0w3POIcwTMwZ9_0 2024-04-17T05:46:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15496101549621961255', time=2024-04-17T07:23:41.564552+0000, chain.objs.size()=1 2024-04-17T05:46:02.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.u7Hd6Rxncg6Ey1IJPcInoal13N7-jve_0 2024-04-17T05:46:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15435702794432574609', time=2024-04-17T07:23:41.666179+0000, chain.objs.size()=1 2024-04-17T05:46:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.280OWXOFCZRBwiS4McecIMiJdliNt8S_0 2024-04-17T05:46:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5420229063760907207', time=2024-04-17T07:23:46.044540+0000, chain.objs.size()=1 2024-04-17T05:46:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.319__shadow_.VqmJf9ANEj16VZzwLzOVtkRHRpVyJ5y_0 2024-04-17T05:46:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2899069194743171433', time=2024-04-17T07:25:50.692326+0000, chain.objs.size()=1 2024-04-17T05:46:02.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.449__shadow_.0up4KDhG8qFd5pr9quo4X3L5k5XoBDu_0 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2768017794905774207', time=2024-04-17T07:31:52.052746+0000, chain.objs.size()=1 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:reepL34BTuRWfwf0HgJXJ0O43AT.2CF_.7S5IS4LgAtkiZnjDUGBTtJ7OlR447bb_0 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16405616862190982066', time=2024-04-17T07:33:41.049092+0000, chain.objs.size()=1 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.574__shadow:Zf4YpNI.5JiN9jfISuPm39sTsqayUcb_.mMs64DII60jSvG1y-XYibfxIXNtGI6l_0 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6203875116278485765', time=2024-04-17T07:33:45.135619+0000, chain.objs.size()=1 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:PKOZSFTwp.gzxjzHSflLVpyjxKrMgKM_.Cfm4l7W3ivLDrEZOLUblWeaKuuVK3mc_0 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13722248263034015185', time=2024-04-17T07:34:04.843881+0000, chain.objs.size()=1 2024-04-17T05:46:02.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.597__shadow_.U0sYfWEwxO2BkOvn4CZhjyWCog3_Y89_0 2024-04-17T05:46:02.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7026595482858400384', time=2024-04-17T07:37:07.050729+0000, chain.objs.size()=1 2024-04-17T05:46:02.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.640__shadow_.SL2qMq5YJSeMbafmpYbYcOidEXesOnk_0 2024-04-17T05:46:02.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16627239398584247792', time=2024-04-17T07:37:41.877925+0000, chain.objs.size()=1 2024-04-17T05:46:02.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.uqS6ilJ7LY2EpBPAG5cpUnIji_2RErQ_0 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17369755206005671761', time=2024-04-17T07:37:45.473330+0000, chain.objs.size()=1 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.655__shadow_.x8eqO7z3VlSYySgLGH4U76-Nxx-dKBk_0 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17938617040035399438', time=2024-04-17T07:38:02.527576+0000, chain.objs.size()=1 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.675__shadow:rqdwNZKS-XUXOfDoeapRZlasZ7IKE8L_.o6sBquQJF6fULK2e9ZvhOxSAfYPVzVX_0 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15696522825514177282', time=2024-04-17T07:38:03.772918+0000, chain.objs.size()=1 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.676__shadow_.wx6JLvlYPFgDJXUex4wZDbWUIUCCvcK_0 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14021007774448535176', time=2024-04-17T07:38:31.606034+0000, chain.objs.size()=1 2024-04-17T05:46:02.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.699__shadow:qjHj54aOCHdofXDN.5DegDeF8HNXOAt_.jT9bWjgZh7SmoHZHw7HXjb-4EiJISBS_0 2024-04-17T05:46:02.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13461019137411083306', time=2024-04-17T07:39:17.391000+0000, chain.objs.size()=10 2024-04-17T05:46:02.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.1 2024-04-17T05:46:02.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.2 2024-04-17T05:46:02.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.3 2024-04-17T05:46:02.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.4 2024-04-17T05:46:02.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.5 2024-04-17T05:46:02.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.6 2024-04-17T05:46:02.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.7 2024-04-17T05:46:02.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.8 2024-04-17T05:46:02.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.9 2024-04-17T05:46:02.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.718__multipart_foo.2~PJ_riw52nI7nBrGtKxjezgqrR04l6h_.10 2024-04-17T05:46:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-17T05:46:02.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/40628' 2024-04-17T05:46:02.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_So5bhVQHkr8HeU7NRg2o8E8o__WS8rF', time=2024-04-17T07:10:26.033985+0000, chain.objs.size()=2 2024-04-17T05:46:02.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.pwO2YrQ2TjRxJZi_13hzlOwuVWsZx-P_0 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.pwO2YrQ2TjRxJZi_13hzlOwuVWsZx-P_1 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17480880423736055262', time=2024-04-17T07:11:44.402816+0000, chain.objs.size()=1 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.193__shadow_.bULI8qTuM0s61rcF7iMFV1fwexX1So5_0 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14140864323143768852', time=2024-04-17T07:14:24.666198+0000, chain.objs.size()=1 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.218__shadow_.Ow1FYzAFGUlpcRwLD9RuhZnFlYpTHV8_0 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15955352460833319240', time=2024-04-17T07:14:40.959253+0000, chain.objs.size()=1 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.228__shadow_.DXxJY8U5_BeUFtBxfIItLVFA2bsf16n_0 2024-04-17T05:46:02.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16823347245868824536', time=2024-04-17T07:14:45.014369+0000, chain.objs.size()=1 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.231__shadow_.UFAfUmayilz90-0vUpli3nQzgZelviz_0 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12357786824171985959', time=2024-04-17T07:14:52.926472+0000, chain.objs.size()=1 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.235__shadow_.-bmF5Hsv6SgJOgGjmGcAb-6Ul3lv411_0 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13191566566827969817', time=2024-04-17T07:15:10.721059+0000, chain.objs.size()=1 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.246__shadow_.NgZn76eDaB7bonffZ2Sz-w_41DCh0OT_0 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8379067729488171825', time=2024-04-17T07:17:01.354490+0000, chain.objs.size()=1 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ljNB78RgOVPuHeWXMYIx2YNFJj-iw3t_0 2024-04-17T05:46:02.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4341591201326313869', time=2024-04-17T07:17:02.217225+0000, chain.objs.size()=1 2024-04-17T05:46:02.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c8eounp_uaqhcHos5iq6AWdcLua-bYO_0 2024-04-17T05:46:02.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17796393885423210677', time=2024-04-17T07:17:03.178506+0000, chain.objs.size()=1 2024-04-17T05:46:02.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-3f1-uI0H2naMEQVSpANmKvOKsRyEZT_0 2024-04-17T05:46:02.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16658404565115234670', time=2024-04-17T07:17:05.567755+0000, chain.objs.size()=1 2024-04-17T05:46:02.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mg-T6nh0tEkqv69mgh7WoLHd1t4m-2h_0 2024-04-17T05:46:02.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10344270341661632511', time=2024-04-17T07:17:06.653453+0000, chain.objs.size()=1 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Q7X5QTpZwg--c-TA54G1-D9U_qGXjlr_0 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10374306835641949884', time=2024-04-17T07:17:07.584690+0000, chain.objs.size()=1 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UTknI9TW5C5Nyetq29BIIJ9cXyIm96f_0 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14728024589772030390', time=2024-04-17T07:17:08.076969+0000, chain.objs.size()=1 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ULxDrb3lz85p_v0chnGDFPM8sQluG-B_0 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11017303477714410695', time=2024-04-17T07:17:08.467996+0000, chain.objs.size()=1 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qII39Ruo2yQ90f0imOdLTQUqqAoL3-m_0 2024-04-17T05:46:02.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9068739692609334622', time=2024-04-17T07:17:11.414683+0000, chain.objs.size()=1 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XzgCqRjqdu-K5ixsk69hetzEF77Di9C_0 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14543735199843696119', time=2024-04-17T07:17:12.200425+0000, chain.objs.size()=1 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nvQ6xwToQS_kjucs5eknM9qJ_iBDwiD_0 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3995297280634356514', time=2024-04-17T07:17:12.291452+0000, chain.objs.size()=1 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.39m6kvXO8A6tiWkFcCrUZ4IEf2mDipJ_0 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5659669426446326300', time=2024-04-17T07:17:14.111979+0000, chain.objs.size()=1 2024-04-17T05:46:02.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aDODN2gC5tomyduJ_6ZVm236VIp-dHU_0 2024-04-17T05:46:02.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1671433786584493006', time=2024-04-17T07:17:14.257581+0000, chain.objs.size()=1 2024-04-17T05:46:02.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5B97nOePkK2yR4ISAqxNyfo3D48f4hm_0 2024-04-17T05:46:02.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9594999754102415397', time=2024-04-17T07:17:15.164641+0000, chain.objs.size()=1 2024-04-17T05:46:02.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dm-nkPV5Xm9XXbi5J-8Fk5nV-TcVgUp_0 2024-04-17T05:46:02.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3501954222051917728', time=2024-04-17T07:17:16.167713+0000, chain.objs.size()=1 2024-04-17T05:46:02.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.P6utzplIVJNGnyjIz-Rk9LaWJUvXPwo_0 2024-04-17T05:46:02.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13633729271468649768', time=2024-04-17T07:17:18.045227+0000, chain.objs.size()=1 2024-04-17T05:46:02.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CHj7BO1Q9K7HKntPdpU0mFTUMCnYLqp_0 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16437903380443560560', time=2024-04-17T07:17:18.985106+0000, chain.objs.size()=1 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GmTsoCCEOWFlWyMsgl2_m8uqQf7eetw_0 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18084550214282909024', time=2024-04-17T07:17:20.639036+0000, chain.objs.size()=1 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sSCyiMxs7QfW6tfV9nZmC-OpZmB1Csg_0 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1743393146603213156', time=2024-04-17T07:17:20.723937+0000, chain.objs.size()=1 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-3psjpFmwKdhdNNJJwE7SpN5Tguh8aw_0 2024-04-17T05:46:02.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8519798147945377638', time=2024-04-17T07:17:22.660831+0000, chain.objs.size()=1 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MBdH1a7pj3JdBX2Sob6fVwZuZC7pGSC_0 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8106697111666848893', time=2024-04-17T07:17:24.363056+0000, chain.objs.size()=1 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-CHEYFyhkgAEJLaooKVrimiADwMsDP5_0 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13808492392336896889', time=2024-04-17T07:17:26.486195+0000, chain.objs.size()=1 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZxUHugE1SD-cIY1lSR0IxUMgG-16cwn_0 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13849213248684177018', time=2024-04-17T07:17:27.125737+0000, chain.objs.size()=1 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.64-UUAmK8zaXP0xLWf7XL6GaWoPaaCM_0 2024-04-17T05:46:02.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3977591631175899667', time=2024-04-17T07:17:27.976073+0000, chain.objs.size()=1 2024-04-17T05:46:02.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Stzz76bMsWiSUbjqnmQa7ip5Lrol5sD_0 2024-04-17T05:46:02.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14160507490461065807', time=2024-04-17T07:17:30.046016+0000, chain.objs.size()=1 2024-04-17T05:46:02.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Smqv-4FQvAW0gOfznkw7q6R9sjbV_5F_0 2024-04-17T05:46:02.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4634799677474546194', time=2024-04-17T07:17:30.262436+0000, chain.objs.size()=1 2024-04-17T05:46:02.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._ekRsTef1H0TdZpbTGFFrT3Dxzm4hCR_0 2024-04-17T05:46:02.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16377360259536220989', time=2024-04-17T07:17:30.539535+0000, chain.objs.size()=1 2024-04-17T05:46:02.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.o5uvyK-Sl8wmcSLxxMKu9Wg8VRKjHVR_0 2024-04-17T05:46:02.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1461452156504999813', time=2024-04-17T07:17:31.108815+0000, chain.objs.size()=1 2024-04-17T05:46:02.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Xju945sta8lpxRVeMqref0ojuc3TPtu_0 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18275096449861450271', time=2024-04-17T07:17:44.447598+0000, chain.objs.size()=1 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.260__shadow_.IKs-rg_gDO0VGAMxpb0jo5kX7OgL1cc_0 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.122995651040106739', time=2024-04-17T07:18:14.864703+0000, chain.objs.size()=1 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.279__shadow_.niTFoVEcRemXNoKMWaub5EMz5a0vpwR_0 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14729507398032402313', time=2024-04-17T07:18:23.237057+0000, chain.objs.size()=1 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.KZvsQw_4hyPJmufzeQtHPI2MvAu-W8z_0 2024-04-17T05:46:02.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12091284822573942145', time=2024-04-17T07:18:29.868271+0000, chain.objs.size()=1 2024-04-17T05:46:02.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.284__shadow_.zJaWz3DlgCKc3QVIWRn55vnkSNDb--v_0 2024-04-17T05:46:02.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16932350535581467913', time=2024-04-17T07:19:10.857236+0000, chain.objs.size()=1 2024-04-17T05:46:02.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.309__shadow:RjzpUQaATuF86fJ6S6AlF72oyuAZAv9_.VnHBF2PupNP9zql3aDDncNpmdM1u8P__0 2024-04-17T05:46:02.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5035537884359467431', time=2024-04-17T07:19:11.135153+0000, chain.objs.size()=1 2024-04-17T05:46:02.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.309__shadow:dj5Nqz2kVo-H7IAXLxnRFwVJECJntUJ_.aq06170gjKOCyMwcYQ4aF0Cy0BqA6TP_0 2024-04-17T05:46:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17341350958399766028', time=2024-04-17T07:20:59.784378+0000, chain.objs.size()=1 2024-04-17T05:46:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3fBTCk3m37VVlNUIo5r11_wqgHyVbvs_0 2024-04-17T05:46:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9949135167471450623', time=2024-04-17T07:21:00.507983+0000, chain.objs.size()=1 2024-04-17T05:46:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.qsbay9HqFR0Dxkth5QFy3zCxCKa68dq_0 2024-04-17T05:46:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1265076599666434830', time=2024-04-17T07:21:01.125049+0000, chain.objs.size()=1 2024-04-17T05:46:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UI02XFxeKBAPvWuHNwTBhvqFiBXy3o2_0 2024-04-17T05:46:02.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15815814637425250124', time=2024-04-17T07:21:03.925858+0000, chain.objs.size()=1 2024-04-17T05:46:02.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Prx7yf8ORKRJA4IfMdrcImfgPg-u_j-_0 2024-04-17T05:46:02.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.116+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9575596069593152449', time=2024-04-17T07:21:05.074613+0000, chain.objs.size()=1 2024-04-17T05:46:02.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.116+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tVnC6_-ALfD2PfNoSO5A6C1ebKvSFT8_0 2024-04-17T05:46:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12980889243239191397', time=2024-04-17T07:21:05.660782+0000, chain.objs.size()=1 2024-04-17T05:46:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UmjupHO0j2b_koOQp2IdoNNYBaR-5Eb_0 2024-04-17T05:46:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5514237343512051987', time=2024-04-17T07:21:06.425362+0000, chain.objs.size()=1 2024-04-17T05:46:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gQ4qbqn3nKXtnpQmeG0uljIRPQze9Ga_0 2024-04-17T05:46:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17433383582831525873', time=2024-04-17T07:21:06.843202+0000, chain.objs.size()=1 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZEx5Rg3haWlpA91cqhg6SxrNZTvEEPI_0 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17113808057464771631', time=2024-04-17T07:21:10.851937+0000, chain.objs.size()=1 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HuNhJMWBJHOwLwe2sKDHSihelfC_zar_0 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12896213652562282635', time=2024-04-17T07:21:12.958225+0000, chain.objs.size()=1 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-GZ6No-gKAk-WMo8ULZJLag0TZ2VfKW_0 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15139076678688230569', time=2024-04-17T07:21:14.411803+0000, chain.objs.size()=1 2024-04-17T05:46:02.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xVtmERrhY9ZsODYNayZaxL1gRhodQRa_0 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9688725223076369636', time=2024-04-17T07:21:14.515527+0000, chain.objs.size()=1 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gNsNsO_8OPvZLzFLeGatZ-8qDFaFKLX_0 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6558037257438175835', time=2024-04-17T07:21:14.517278+0000, chain.objs.size()=1 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u_w6rf8eoX1EdAAeKVz4Bftbt91MP2F_0 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.271811464538356704', time=2024-04-17T07:21:16.949356+0000, chain.objs.size()=1 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._pNK9MUFsYfq4jEUTEskXPkZmzZP-8A_0 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9114777575393523106', time=2024-04-17T07:21:18.143397+0000, chain.objs.size()=1 2024-04-17T05:46:02.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-YKeZTC5aJSSJbxvuVeHHAWTEI_5Eas_0 2024-04-17T05:46:02.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1937131751127863678', time=2024-04-17T07:21:19.575396+0000, chain.objs.size()=1 2024-04-17T05:46:02.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9Z_ofsjnmlB9n4pderh4rx6H6M9uQNT_0 2024-04-17T05:46:02.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14782606127013963214', time=2024-04-17T07:21:19.579510+0000, chain.objs.size()=1 2024-04-17T05:46:02.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.O3_MhV7CkpsS6N9EZ0o-15d_jhlrRgU_0 2024-04-17T05:46:02.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15850348693144581971', time=2024-04-17T07:21:19.950887+0000, chain.objs.size()=1 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GzHRtvcnCJ2rjs42MiG1y0lUDKJDDYU_0 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3512200133546341961', time=2024-04-17T07:21:25.540871+0000, chain.objs.size()=1 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._Jo3MshisYT0NmJBBiKGwoZaYolBYzy_0 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13304492925400908855', time=2024-04-17T07:21:25.943478+0000, chain.objs.size()=1 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.g4j-8QOnjOZjFIF2kh2JS4hMD0M8brP_0 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4456842669377352720', time=2024-04-17T07:21:26.240628+0000, chain.objs.size()=1 2024-04-17T05:46:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SSr-jHljqFOS3izjKe0CAuEOcMBx8Mj_0 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9656671685000453465', time=2024-04-17T07:21:26.267045+0000, chain.objs.size()=1 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0UI6RpP8fh92LOzoWzlGm-MMzWdTWe4_0 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1238083865453361873', time=2024-04-17T07:21:26.268771+0000, chain.objs.size()=1 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jW_I_1CCgkK7sHLHzF25pkFhB7H-IIA_0 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2109557480962920279', time=2024-04-17T07:21:27.818353+0000, chain.objs.size()=1 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MXCPa9SUx_Eh6KfPQv2fS7DAmx9nLvk_0 2024-04-17T05:46:02.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10841177297506051837', time=2024-04-17T07:21:28.189613+0000, chain.objs.size()=1 2024-04-17T05:46:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Moa236ZSt9n6Atsz5DV2oH6uX9b0zbz_0 2024-04-17T05:46:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16926159223055084685', time=2024-04-17T07:21:28.191060+0000, chain.objs.size()=1 2024-04-17T05:46:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.absL5yhKztH3uCRUZ4lKNyv-l4wOUAF_0 2024-04-17T05:46:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6039394285562353503', time=2024-04-17T07:21:28.857893+0000, chain.objs.size()=1 2024-04-17T05:46:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CWXgJbiEEbXA9AJdzyema6rUMpXM-5f_0 2024-04-17T05:46:02.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3520350245661054527', time=2024-04-17T07:21:29.205945+0000, chain.objs.size()=1 2024-04-17T05:46:02.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xj8te-wcyBwAflJQQ7wrLhEWuti2pr-_0 2024-04-17T05:46:02.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14866692297536487554', time=2024-04-17T07:23:12.640252+0000, chain.objs.size()=1 2024-04-17T05:46:02.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lgWL6AoSDRVvXQucRrIjKY2SKaNmktS_0 2024-04-17T05:46:02.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12656349169274881501', time=2024-04-17T07:23:12.740245+0000, chain.objs.size()=1 2024-04-17T05:46:02.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.d4r_fLnluH7YEv4TSLFA8tyVyUTrS6t_0 2024-04-17T05:46:02.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.690117940982772278', time=2024-04-17T07:23:13.083205+0000, chain.objs.size()=1 2024-04-17T05:46:02.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wBfz0oAl_2MNPnRClcoJNkGpI_s_PIq_0 2024-04-17T05:46:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15650770422850242418', time=2024-04-17T07:23:14.451198+0000, chain.objs.size()=1 2024-04-17T05:46:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lD4PDo5KkA6VZYCpYMT-McBdGX6X-Do_0 2024-04-17T05:46:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.192265120711245525', time=2024-04-17T07:23:14.612836+0000, chain.objs.size()=1 2024-04-17T05:46:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lOCS4w8uFmfPoSVP4SBUUE-X_zFHliA_0 2024-04-17T05:46:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1034243017214592818', time=2024-04-17T07:23:14.804619+0000, chain.objs.size()=1 2024-04-17T05:46:02.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Aessgwxafyqzv2Sstyw9MI-g8vOHUgL_0 2024-04-17T05:46:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17912310425774592234', time=2024-04-17T07:23:15.722102+0000, chain.objs.size()=1 2024-04-17T05:46:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GxNiNBj5E_tzeo5ea8XB9F1YGOns4FJ_0 2024-04-17T05:46:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17219161313672247832', time=2024-04-17T07:23:15.852462+0000, chain.objs.size()=1 2024-04-17T05:46:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.s2bAlxOSjGquoswClvXnA7RK42B1I3q_0 2024-04-17T05:46:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4851215139454097164', time=2024-04-17T07:23:17.500557+0000, chain.objs.size()=1 2024-04-17T05:46:02.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9XktTNUR2PEKJ_0kmH676myGT_XAXcJ_0 2024-04-17T05:46:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6261655525097899548', time=2024-04-17T07:23:18.166307+0000, chain.objs.size()=1 2024-04-17T05:46:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xnF4oGPby5ixtse9J_9ldD5FgS1pgzY_0 2024-04-17T05:46:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.391491340518805717', time=2024-04-17T07:23:18.167799+0000, chain.objs.size()=1 2024-04-17T05:46:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-kt3kNJVmDKwODL5wmHQR3E_YWG8ah6_0 2024-04-17T05:46:02.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9912637241915236711', time=2024-04-17T07:23:19.012900+0000, chain.objs.size()=1 2024-04-17T05:46:02.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cfNjbXeL3hjfLHMDCIpUpUZwm4UGCvO_0 2024-04-17T05:46:02.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11948158324043665775', time=2024-04-17T07:23:23.013911+0000, chain.objs.size()=1 2024-04-17T05:46:02.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.b17tlWMdKs5Inm6OwoZXULys6VXQGBH_0 2024-04-17T05:46:02.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12135351523282206099', time=2024-04-17T07:23:23.389659+0000, chain.objs.size()=1 2024-04-17T05:46:02.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NmqTmllHQV65ioK9ADicXGY8yM4-9fd_0 2024-04-17T05:46:02.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7455993663850704659', time=2024-04-17T07:23:23.807665+0000, chain.objs.size()=1 2024-04-17T05:46:02.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.puRYayus_ldj0essR179FeWVaPGZ9hw_0 2024-04-17T05:46:02.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.319554477714831195', time=2024-04-17T07:23:24.941142+0000, chain.objs.size()=1 2024-04-17T05:46:02.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tepO_6cg_CGVV8irn2NgGOl8sNM4eMQ_0 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1179598034947593967', time=2024-04-17T07:23:25.945529+0000, chain.objs.size()=1 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.509tEpL9OIhxhUWlAHVjMQGoUcbcuD8_0 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5254359152441674207', time=2024-04-17T07:23:26.496867+0000, chain.objs.size()=1 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qAZiRy0OYW4F2TsoX_TNsFkjHld-dvj_0 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12560286717292806006', time=2024-04-17T07:23:26.988982+0000, chain.objs.size()=1 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.H6woNysJG3DOjaRCBXai-HXvOcpg5IH_0 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4285203845985855680', time=2024-04-17T07:23:27.386708+0000, chain.objs.size()=1 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Z1NP44vZa0aCTWDs0rMj7YH6Dv0SoHu_0 2024-04-17T05:46:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17308742205556680819', time=2024-04-17T07:23:28.597203+0000, chain.objs.size()=1 2024-04-17T05:46:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xHm2LgqeCv34RMW1_cZs0t9KfL2GsC6_0 2024-04-17T05:46:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12704744266668942148', time=2024-04-17T07:23:29.804096+0000, chain.objs.size()=1 2024-04-17T05:46:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.olmFK4uKj4xjTtfdnatqQohUuEjm_Mr_0 2024-04-17T05:46:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14568775564084812826', time=2024-04-17T07:23:30.314742+0000, chain.objs.size()=1 2024-04-17T05:46:02.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.X376UaAHjvCyOUBlWj4kFdRj2KLYA1O_0 2024-04-17T05:46:02.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10563828545999324379', time=2024-04-17T07:23:33.298191+0000, chain.objs.size()=1 2024-04-17T05:46:02.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.B4Vzams9TX1Qzm5qzYBKcAGnlf2WW79_0 2024-04-17T05:46:02.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17873708617646944757', time=2024-04-17T07:23:33.460663+0000, chain.objs.size()=1 2024-04-17T05:46:02.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.37r7S1aiXXzDQ2qKoC678Zsf3_m_CpQ_0 2024-04-17T05:46:02.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.286403249077280950', time=2024-04-17T07:23:33.477609+0000, chain.objs.size()=1 2024-04-17T05:46:02.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.22GdQJj5AeH_yPMc5geGLZUhsEz_sQf_0 2024-04-17T05:46:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10641587282531734827', time=2024-04-17T07:23:36.626089+0000, chain.objs.size()=1 2024-04-17T05:46:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Zhs1xYQ1_zB5zd3b3HU839EYVB6URRp_0 2024-04-17T05:46:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15611505099360097466', time=2024-04-17T07:23:37.209562+0000, chain.objs.size()=1 2024-04-17T05:46:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.e7jMeogOyTTht0RTNlvJdCkS0u4mqfr_0 2024-04-17T05:46:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12051653945114169368', time=2024-04-17T07:23:37.403415+0000, chain.objs.size()=1 2024-04-17T05:46:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mk0txKhwU0NRrX9PRf4zfqo2opmUTnd_0 2024-04-17T05:46:02.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.507756989281346025', time=2024-04-17T07:23:39.438184+0000, chain.objs.size()=1 2024-04-17T05:46:02.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IQdJSnvmdr_mTqFI1biX7NUMERb-sz4_0 2024-04-17T05:46:02.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12559185283637899159', time=2024-04-17T07:23:39.790342+0000, chain.objs.size()=1 2024-04-17T05:46:02.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cEYkt3rL0u5lMWVu9qLdvswwXtwBehJ_0 2024-04-17T05:46:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/40628 2024-04-17T05:46:02.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=23, truncated=0, next_marker='' 2024-04-17T05:46:02.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13911718268022999606', time=2024-04-17T07:23:40.142565+0000, chain.objs.size()=1 2024-04-17T05:46:02.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4Aq1xlsFXNtqod6Rtb0KmI0Gn9roN-w_0 2024-04-17T05:46:02.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7471040995513256954', time=2024-04-17T07:24:00.312011+0000, chain.objs.size()=1 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.334__shadow_.BHkCPRWK_1licpuP9QQF16ymEKrxVR3_0 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6073665150068650223', time=2024-04-17T07:24:29.279754+0000, chain.objs.size()=1 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.368__shadow_.ZhmB341BBo4vET0JkoUdLtRvHC6aN_7_0 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12601121947366695655', time=2024-04-17T07:25:23.072533+0000, chain.objs.size()=1 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.426__shadow_.-ckr4M4YjfBG6ch1O9Q8qIthzWKqdQa_0 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17038690377119624521', time=2024-04-17T07:25:39.536365+0000, chain.objs.size()=1 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.441__shadow_.EAmOW6e9xSN4QJIYf81_43UXzcIpj-Y_0 2024-04-17T05:46:02.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7582835619199897315', time=2024-04-17T07:25:58.660231+0000, chain.objs.size()=1 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.454__shadow_.pJDcDkEtBgjpftHo7u7ypUpfIWPVZMS_0 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9241568774569509108', time=2024-04-17T07:26:12.397702+0000, chain.objs.size()=1 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.471__shadow_.jYjrAz2O8THfBsiHjpiW2JxymbE4r50_0 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3267339241344681271', time=2024-04-17T07:26:17.122639+0000, chain.objs.size()=1 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_._D-A8fOTExLUJvyFUC0sdcqPFqnXFcN_0 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9917194560001856363', time=2024-04-17T07:26:25.713004+0000, chain.objs.size()=1 2024-04-17T05:46:02.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.477__shadow_._ZYSMRZ88c_6nOr-v9X-sfixA6Ew1M__0 2024-04-17T05:46:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.489__shadow_.D_72jMNIE_4K0Bv0Zr0ku-SZdPGCyJL_1_482510dcbc48d99b75eb38e8375a260a', time=2024-04-17T07:26:38.464965+0000, chain.objs.size()=1 2024-04-17T05:46:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.489__shadow_.D_72jMNIE_4K0Bv0Zr0ku-SZdPGCyJL_0 2024-04-17T05:46:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18156340087123360249', time=2024-04-17T07:26:43.464399+0000, chain.objs.size()=1 2024-04-17T05:46:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_0 2024-04-17T05:46:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2181402192374939312', time=2024-04-17T07:27:48.986878+0000, chain.objs.size()=1 2024-04-17T05:46:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.510__shadow:mgMo45WkFlpSInSWfPwXAMXscUzWEK9_.ePpYjePVWMGwQUqvAwrYhfkglt1Zito_0 2024-04-17T05:46:02.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5271869415866398949', time=2024-04-17T07:30:52.938138+0000, chain.objs.size()=1 2024-04-17T05:46:02.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.542__shadow_.5IQnSMZcA-ABGlTgJLHcDRld-WlmC5Y_0 2024-04-17T05:46:02.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8679133677098342495', time=2024-04-17T07:30:56.134698+0000, chain.objs.size()=1 2024-04-17T05:46:02.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.543__shadow_.z5wA1WUbwjWPGGH2M9bg_yBPIHxiaVP_0 2024-04-17T05:46:02.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11407710482976916351', time=2024-04-17T07:31:53.988420+0000, chain.objs.size()=1 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:4k6RL7Ljnds7xYTtGdsZByb9wZvqNjN_.ANJclvGFvRdtjl3Ca2CqxkRwu3qRlkM_0 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4487331637408149899', time=2024-04-17T07:31:55.275051+0000, chain.objs.size()=1 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:yNKrsGmDEImwyVcG1PFcn.FEENcDmt-_.DQF4GeVvvXn2iuuKG8CYCHXZwEDMWbh_0 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13430540423694949202', time=2024-04-17T07:32:01.416191+0000, chain.objs.size()=1 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.561__shadow_.6V7ubSISm19Xu4WzkxA5ppXPYRrAa-W_0 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10381247067476362750', time=2024-04-17T07:32:21.284431+0000, chain.objs.size()=1 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:BLZPD3ez3qJdaSi7mRM6i0ms3QnJeWg_.wtxad3jtOofQbLalsX3aFMNVqQhs4PL_0 2024-04-17T05:46:02.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7624183345961928120', time=2024-04-17T07:32:29.901345+0000, chain.objs.size()=1 2024-04-17T05:46:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:JNyKrTmI3xzlg6I6BCBDpFAWd0L-lTj_.G-T4YimnjhrjEckDkrcszMtwBFWI1Rv_0 2024-04-17T05:46:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3199667765415711969', time=2024-04-17T07:32:38.769769+0000, chain.objs.size()=1 2024-04-17T05:46:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:xy33ZvGk9rV3Kf3IL7HUrXCx1cU0TvM_.oZkwzxSud_P6PIo5TDthZfb8xdrCkAW_0 2024-04-17T05:46:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12691802855555753315', time=2024-04-17T07:33:40.976705+0000, chain.objs.size()=1 2024-04-17T05:46:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.574__shadow:HGchFjUZ6PDxcv-WVpmtpf0A0S7-s1e_.m5hA3GoUjsFQkhU-VIH7QAlWVebqabD_0 2024-04-17T05:46:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2705759907771599212', time=2024-04-17T07:34:03.006806+0000, chain.objs.size()=1 2024-04-17T05:46:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.595__shadow_.7c_p0mLc_yuHT4er0mryw_6uEA38ZU-_0 2024-04-17T05:46:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.298673708033903569', time=2024-04-17T07:37:41.875985+0000, chain.objs.size()=1 2024-04-17T05:46:02.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.nkv5fDUv7YDEmrvqdHF7khPn4g4y8KP_0 2024-04-17T05:46:02.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.168+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-17T05:46:02.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.168+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45771' 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_qtg67a6B73f-r8PHFqc3nSOxIs3qXI8', time=2024-04-17T07:10:29.903275+0000, chain.objs.size()=1 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.153__shadow_.rR5WMmM0sSDqTQTZSYrj_wfzu-l4QFb_0 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16795931892156569379', time=2024-04-17T07:11:00.917102+0000, chain.objs.size()=1 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.171__shadow_.esRlQKvAC4BeZPje_SaxM0zguthsvkV_0 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5466723752103989198', time=2024-04-17T07:14:40.948302+0000, chain.objs.size()=1 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.228__shadow_.LqwXbSGxdOk60CtYPmelFwiS-GH1x5I_0 2024-04-17T05:46:02.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2191506319446955126', time=2024-04-17T07:14:48.919485+0000, chain.objs.size()=1 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.233__shadow_.aAVMDUGe6YlH435ZZ1w2dMmBjSxdJfC_0 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15968519578539136085', time=2024-04-17T07:14:51.249710+0000, chain.objs.size()=1 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.234__shadow_.FxHoVFqpf7U3_LR_kijkcnXCxuj6s2v_0 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17243492665606253000', time=2024-04-17T07:14:52.909497+0000, chain.objs.size()=1 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.235__shadow_.AyXcGYZK0zvmOIIj2zYR5S88IqFvoec_0 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5759052199746501716', time=2024-04-17T07:17:01.291951+0000, chain.objs.size()=1 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OleLECIiF07FGUkZrvF_2fHCJeEQNN1_0 2024-04-17T05:46:02.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13904993789116598291', time=2024-04-17T07:17:01.295116+0000, chain.objs.size()=1 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.37KwCqQi-U-5qvdYemwBb7u2Xv0FMti_0 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15395231112562872877', time=2024-04-17T07:17:04.375656+0000, chain.objs.size()=1 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9TLJInXe5JSKklR2kFn1_BE09UP90di_0 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13562954502764877984', time=2024-04-17T07:17:05.591083+0000, chain.objs.size()=1 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.g79tgDoGCnSjOKbmkR8d03HV8qVkGHg_0 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3947030990317466807', time=2024-04-17T07:17:06.040855+0000, chain.objs.size()=1 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DjSCbfmZoTD55xL367hUjsoIBffb3v5_0 2024-04-17T05:46:02.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16698940626901016889', time=2024-04-17T07:17:06.179171+0000, chain.objs.size()=1 2024-04-17T05:46:02.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TiQ6og2q7VPq0Jf2zzXfqAYTYxHhARh_0 2024-04-17T05:46:02.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10635176173333970243', time=2024-04-17T07:17:07.719767+0000, chain.objs.size()=1 2024-04-17T05:46:02.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2cNeONHlI2GYBAz3ce4Uaxn061GSHCj_0 2024-04-17T05:46:02.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13091686941380453350', time=2024-04-17T07:17:08.030439+0000, chain.objs.size()=1 2024-04-17T05:46:02.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ASztdKpF0n3QakExAvq6jafMkI8nM0c_0 2024-04-17T05:46:02.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12377513791251959880', time=2024-04-17T07:17:08.300785+0000, chain.objs.size()=1 2024-04-17T05:46:02.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.M3kCaA4XZJJMGZHhIxikMb0l3tYjJ0M_0 2024-04-17T05:46:02.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14253666427340480504', time=2024-04-17T07:17:11.384498+0000, chain.objs.size()=1 2024-04-17T05:46:02.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dpwEyv76Fp_vTmkedGdSQpfYxwbpepH_0 2024-04-17T05:46:02.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14712183110294407824', time=2024-04-17T07:17:15.097553+0000, chain.objs.size()=1 2024-04-17T05:46:02.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dbE70ka_Bd9wqE-auEZHjQoY3Xa2bDO_0 2024-04-17T05:46:02.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10741879953921414020', time=2024-04-17T07:17:18.498929+0000, chain.objs.size()=1 2024-04-17T05:46:02.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.itYBHb3O_0HPThm6XCa7gYt4Uq-CJhC_0 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7757363176448437154', time=2024-04-17T07:17:18.716722+0000, chain.objs.size()=1 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CcJzlCIZC6m3byyQYOTzoGGze-Oc7Vd_0 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2077844734028891584', time=2024-04-17T07:17:18.867929+0000, chain.objs.size()=1 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.j8I983Hqzi0lQffwT8NkZGOvaSs9RI5_0 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2834597966939473463', time=2024-04-17T07:17:19.198087+0000, chain.objs.size()=1 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dsgSm1GNkXKBpWBG0-as8_WWEP5vCz__0 2024-04-17T05:46:02.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17363670557679288590', time=2024-04-17T07:17:19.685050+0000, chain.objs.size()=1 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RRlhHsPiPbeSGwBKocr-fzgUG4FMYxw_0 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1646777041086814760', time=2024-04-17T07:17:20.846634+0000, chain.objs.size()=1 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vOw7_ZBjYMCJ6kF2CatHwEr8InLUcam_0 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12964120891828484542', time=2024-04-17T07:17:21.086788+0000, chain.objs.size()=1 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.M63BsdIqgVBmQI7zKBew1UyuxsG-g-P_0 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6387004855962132812', time=2024-04-17T07:17:22.192305+0000, chain.objs.size()=1 2024-04-17T05:46:02.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aD2ipzJq5kjzywxO4wuB1B-tU9iIvAD_0 2024-04-17T05:46:02.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12779093663694474919', time=2024-04-17T07:17:23.095015+0000, chain.objs.size()=1 2024-04-17T05:46:02.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VHJ1wg-4bQj_cILl-55N7OenKfcnG_N_0 2024-04-17T05:46:02.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16889845802147790721', time=2024-04-17T07:17:23.106220+0000, chain.objs.size()=1 2024-04-17T05:46:02.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9TeDr2krcW6KnEPtWWsCXP_k7OsyS_P_0 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15533850658334687416', time=2024-04-17T07:17:23.211572+0000, chain.objs.size()=1 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jOu8tm0B9rdkWzuTtesG0hAGKPVKMnd_0 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6818592858198982000', time=2024-04-17T07:17:26.067536+0000, chain.objs.size()=1 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0NDWmbYeJMmZGh1OOifwwjsgCZvkHYN_0 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2678675090886602068', time=2024-04-17T07:17:26.628304+0000, chain.objs.size()=1 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GZPBnekCOVPiRDO7v2_63lbs9jHVEr3_0 2024-04-17T05:46:02.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9022372411173145021', time=2024-04-17T07:17:26.782544+0000, chain.objs.size()=1 2024-04-17T05:46:02.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HD-yELkAkGjHSdv_Zooh4sd8rv6BJxs_0 2024-04-17T05:46:02.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1665391628702072893', time=2024-04-17T07:17:27.565665+0000, chain.objs.size()=1 2024-04-17T05:46:02.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fLAlgDgN2vkCyWKRkP6luYQ42J04ddx_0 2024-04-17T05:46:02.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5166928606448459087', time=2024-04-17T07:17:28.159336+0000, chain.objs.size()=1 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QufgpzfidvajO4qcqyv1RtEvuoQof-9_0 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9447767647025821107', time=2024-04-17T07:17:28.923126+0000, chain.objs.size()=1 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.O7LIUZmjdRw_792wGPAzwMIaf0q9lox_0 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7882128934802019515', time=2024-04-17T07:17:29.213765+0000, chain.objs.size()=1 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.apw6XoYU8A975_rNIz0g6ehOnwWjGSX_0 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8446692097847204119', time=2024-04-17T07:17:30.735875+0000, chain.objs.size()=1 2024-04-17T05:46:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Hgc0n7MZ1sD9KzLKPuWX1JVZxgkHFpS_0 2024-04-17T05:46:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16431913838469756206', time=2024-04-17T07:17:51.948349+0000, chain.objs.size()=1 2024-04-17T05:46:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.265__shadow_.ZNCZhLG6fv0DhVLYlLhQ6Yf-7ir9bKx_0 2024-04-17T05:46:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4210554367984506148', time=2024-04-17T07:17:55.259911+0000, chain.objs.size()=1 2024-04-17T05:46:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.267__shadow_.nFru46YVdJcBhYkifdzarGK3giJuoVd_0 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12220480468484763591', time=2024-04-17T07:18:14.885778+0000, chain.objs.size()=1 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.279__shadow_.7MZHjM-ZRGM_6lJpmmADD2R6QrUWvnp_0 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18179838610464152516', time=2024-04-17T07:18:27.982140+0000, chain.objs.size()=1 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.LrDe7SgsNXsuLTe3hZ9J0nuDQSYROLP_0 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14785579751563913103', time=2024-04-17T07:18:40.079639+0000, chain.objs.size()=1 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.290__shadow_.3lfsMPlDDnP67s-Q-EUwHw4kPn9qEKI_0 2024-04-17T05:46:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4803452588318501279', time=2024-04-17T07:21:01.383186+0000, chain.objs.size()=1 2024-04-17T05:46:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.X5nWlXESQYhxLawN9-WyrmGl_MGQl4q_0 2024-04-17T05:46:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15119278004420630525', time=2024-04-17T07:21:05.265332+0000, chain.objs.size()=1 2024-04-17T05:46:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yoQzTblPi-4zZ94cFFAG7bm8buIOdaK_0 2024-04-17T05:46:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16196046734000809086', time=2024-04-17T07:21:06.583463+0000, chain.objs.size()=1 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Z8m2sczKmAw-gS5CkKTx6WjWPlmBg1j_0 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5806203254423073145', time=2024-04-17T07:21:08.845767+0000, chain.objs.size()=1 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Oe2KKm91G0PEd9XNzyb2jKZyDfGh7hO_0 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13450145286046665996', time=2024-04-17T07:21:08.928358+0000, chain.objs.size()=1 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZNw-NdPCWaeeHrsRDVkH8mgWqxQ3CVG_0 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7082481789835649166', time=2024-04-17T07:21:09.286279+0000, chain.objs.size()=1 2024-04-17T05:46:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.cIr35LanOGUHkYn5_rLUh8eTgbgmd0a_0 2024-04-17T05:46:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6235360655517069801', time=2024-04-17T07:21:11.824949+0000, chain.objs.size()=1 2024-04-17T05:46:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.cwHayJkiFhHZGRhWlbf5KgVHwVwlMUR_0 2024-04-17T05:46:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8839054952032399885', time=2024-04-17T07:21:12.087771+0000, chain.objs.size()=1 2024-04-17T05:46:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EQqIuJ0dD7m0rHmgB3U2myfb-cKW3j1_0 2024-04-17T05:46:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.912848007784730918', time=2024-04-17T07:21:13.995198+0000, chain.objs.size()=1 2024-04-17T05:46:02.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3rVs-JstEIhEAf7xwPpUkSHcPxw0fFw_0 2024-04-17T05:46:02.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1164230212464919599', time=2024-04-17T07:21:14.471386+0000, chain.objs.size()=1 2024-04-17T05:46:02.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IORD94A4Sto9mOfZWRZkfqsz19a4SQt_0 2024-04-17T05:46:02.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14627384890899131403', time=2024-04-17T07:21:14.825676+0000, chain.objs.size()=1 2024-04-17T05:46:02.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.T2AwYLc_9jxTfrkUj4cM0CixqfyYHKq_0 2024-04-17T05:46:02.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9677929995998537562', time=2024-04-17T07:21:15.869034+0000, chain.objs.size()=1 2024-04-17T05:46:02.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4NLZU0en50mAYblZmnc-5acFhkH5hl__0 2024-04-17T05:46:02.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3813535629736898936', time=2024-04-17T07:21:16.040376+0000, chain.objs.size()=1 2024-04-17T05:46:02.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LAsDVHfnYokkEnEoZILm0HBR0kF1mg6_0 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14246399758257191208', time=2024-04-17T07:21:16.855843+0000, chain.objs.size()=1 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.83NlLcmvb-OUSqzyJ7U6uixJzT3t4x2_0 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6725711661848171505', time=2024-04-17T07:21:17.585039+0000, chain.objs.size()=1 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._06jmtr5Q7G1RoP5SJdV4PMonmAjx8A_0 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1945120867281098481', time=2024-04-17T07:21:17.744194+0000, chain.objs.size()=1 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.C2qXH1SFoKi0cuN9xKPIM4Zt7SvKk-v_0 2024-04-17T05:46:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6769953547053389387', time=2024-04-17T07:21:17.745747+0000, chain.objs.size()=1 2024-04-17T05:46:02.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YgA1e_SlhmFEJ4Xof8iwUUsezk5svv__0 2024-04-17T05:46:02.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14887260893666547576', time=2024-04-17T07:21:17.948117+0000, chain.objs.size()=1 2024-04-17T05:46:02.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.A4GEdNr4Y1tm9wkm3zyBbO-8VjRZHCf_0 2024-04-17T05:46:02.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4077605292864190129', time=2024-04-17T07:21:18.726403+0000, chain.objs.size()=1 2024-04-17T05:46:02.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GidGZ5JRGb-qbGM4u6352J5wgR2xu1V_0 2024-04-17T05:46:02.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9509356875533913692', time=2024-04-17T07:21:19.492615+0000, chain.objs.size()=1 2024-04-17T05:46:02.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5DXsTvU2fLU9vYNKveWTtqFw0s1pjzo_0 2024-04-17T05:46:02.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7128077171561461576', time=2024-04-17T07:21:23.289609+0000, chain.objs.size()=1 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.83wSb2Pk9-PXxjlvkEtkiY0lZ_auq_Z_0 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6212044734139812930', time=2024-04-17T07:21:23.296486+0000, chain.objs.size()=1 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dqcprrVSKkH_YouiiknmiGH0vlOtGGO_0 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2982281652558841852', time=2024-04-17T07:21:23.404856+0000, chain.objs.size()=1 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.09vXEdFUttXNEGvL5xE1fIaNB7plEsp_0 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4350482198949861400', time=2024-04-17T07:21:25.548051+0000, chain.objs.size()=1 2024-04-17T05:46:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.20PKJ8pLH8pBP9D9kldJZ33GeUgOiuX_0 2024-04-17T05:46:02.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10986630011477194927', time=2024-04-17T07:21:26.137888+0000, chain.objs.size()=1 2024-04-17T05:46:02.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.j6lrehVLxgf_pSxVw4HjyWhwpf9mMfg_0 2024-04-17T05:46:02.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2071709151286764159', time=2024-04-17T07:21:28.949292+0000, chain.objs.size()=1 2024-04-17T05:46:02.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Dee-37D8xIC4nKuMvi-sOPuVTjXBMd-_0 2024-04-17T05:46:02.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7426635528756015897', time=2024-04-17T07:21:29.444850+0000, chain.objs.size()=1 2024-04-17T05:46:02.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rhuRK2j-bFnqocPuh5D4VpL722vIT5E_0 2024-04-17T05:46:02.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9780305463652685507', time=2024-04-17T07:23:12.640351+0000, chain.objs.size()=1 2024-04-17T05:46:02.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rlgvC3Q3VMF9M2h4iHRpASowa0HjXCb_0 2024-04-17T05:46:02.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4680334705847627492', time=2024-04-17T07:23:13.057076+0000, chain.objs.size()=1 2024-04-17T05:46:02.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Njghx7Q5E1DTGXokeh7r4_rHABdQiSq_0 2024-04-17T05:46:02.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3696410336542972310', time=2024-04-17T07:23:13.087710+0000, chain.objs.size()=1 2024-04-17T05:46:02.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.204+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WfewFZkhPMRuOtsr8ReijBRcGoStLgp_0 2024-04-17T05:46:02.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9767236332097554952', time=2024-04-17T07:23:13.862046+0000, chain.objs.size()=1 2024-04-17T05:46:02.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ebyPSQY_j_pFC2GvLRIoy-kYhdDA-G3_0 2024-04-17T05:46:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12175273234580861825', time=2024-04-17T07:23:14.244754+0000, chain.objs.size()=1 2024-04-17T05:46:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XGN29mCg9QWQiA4jc5ARuGgXqVBpVsb_0 2024-04-17T05:46:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15315351575450460130', time=2024-04-17T07:23:18.490508+0000, chain.objs.size()=1 2024-04-17T05:46:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.swheqRm2gbHUYFQ-tj4jvBnrcpeGfsv_0 2024-04-17T05:46:02.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16567570514373872674', time=2024-04-17T07:23:18.566662+0000, chain.objs.size()=1 2024-04-17T05:46:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OobblP0WWjPB4DnkmSbQBSn2XW67Z5J_0 2024-04-17T05:46:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15753518370162979113', time=2024-04-17T07:23:18.568085+0000, chain.objs.size()=1 2024-04-17T05:46:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8GaXkDDVj6nq8oG8_-QEj21bMkvW9x__0 2024-04-17T05:46:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14327435604103586311', time=2024-04-17T07:23:19.020860+0000, chain.objs.size()=1 2024-04-17T05:46:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QYb5zHSyWrIfsj4mrdMRpOnQCSEcyp8_0 2024-04-17T05:46:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15836351435516102575', time=2024-04-17T07:23:19.562017+0000, chain.objs.size()=1 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Cul2-LNK4eWRFdh6K5yTVmjV6MdRt1d_0 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15179179615402614512', time=2024-04-17T07:23:21.136567+0000, chain.objs.size()=1 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sj3G9pgZYj6mSadb2x02G6OSS4eTBOr_0 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1192138123323609807', time=2024-04-17T07:23:21.402039+0000, chain.objs.size()=1 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bYzpeNno8--JjxyZK5l_WawtOs_ZDyy_0 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7077953119405089340', time=2024-04-17T07:23:23.384689+0000, chain.objs.size()=1 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WuZBohXSd1xZE2ampuMG7f7ci5cR1EJ_0 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15710242168666515783', time=2024-04-17T07:23:23.739807+0000, chain.objs.size()=1 2024-04-17T05:46:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qR4B3mFWih94ZumbJxCsHbcT5G4sFaA_0 2024-04-17T05:46:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4516038437762910602', time=2024-04-17T07:23:25.146263+0000, chain.objs.size()=1 2024-04-17T05:46:02.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ULs7g3k44OUDKK0UHzWYea9BU9mjQsJ_0 2024-04-17T05:46:02.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2649421431688951756', time=2024-04-17T07:23:25.240047+0000, chain.objs.size()=1 2024-04-17T05:46:02.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7bkWEe-XGl1v1utDQ5YLWCgiBCsIMJX_0 2024-04-17T05:46:02.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5318184689136617796', time=2024-04-17T07:23:25.812068+0000, chain.objs.size()=1 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rq1BytYlmhjc0CYgKoPmVHa3bzADMCR_0 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13965369224598565810', time=2024-04-17T07:23:26.452000+0000, chain.objs.size()=1 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4TN4Wkr7A4QqPuegRwyJQFjpOnkCKbL_0 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13246348639809721040', time=2024-04-17T07:23:26.759683+0000, chain.objs.size()=1 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dTlc31G_G7YvD1imEEIs3_uri54v2a__0 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10122737036059672582', time=2024-04-17T07:23:28.928060+0000, chain.objs.size()=1 2024-04-17T05:46:02.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4czQj_-9z6CcAfUdEbPgbeRbXhgmSgs_0 2024-04-17T05:46:02.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13954248247472002266', time=2024-04-17T07:23:29.748370+0000, chain.objs.size()=1 2024-04-17T05:46:02.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3ZROBIUueApq-fv14K8x6mp9pRjj8yS_0 2024-04-17T05:46:02.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8034577560999320855', time=2024-04-17T07:23:29.895665+0000, chain.objs.size()=1 2024-04-17T05:46:02.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wRsveYwUCLxXxV10DpgjIGvO0xKN0k9_0 2024-04-17T05:46:02.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5724009343187308326', time=2024-04-17T07:23:29.897997+0000, chain.objs.size()=1 2024-04-17T05:46:02.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.L9VUBLvZ1WfTxh9XdBVXrD5CvIM87eL_0 2024-04-17T05:46:02.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7416917193697913786', time=2024-04-17T07:23:30.721906+0000, chain.objs.size()=1 2024-04-17T05:46:02.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XEtuB7fF5ItDL5bF15_7zlUjl_0MLbs_0 2024-04-17T05:46:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5629047109374883280', time=2024-04-17T07:23:31.157566+0000, chain.objs.size()=1 2024-04-17T05:46:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.f_PbJz03gtot555o0JI7-X9iGUkuJLo_0 2024-04-17T05:46:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4145041331875978876', time=2024-04-17T07:23:32.567161+0000, chain.objs.size()=1 2024-04-17T05:46:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XX-CM9-XfoQ_VidhPa41EhdxeMrADKD_0 2024-04-17T05:46:02.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1159082716156289228', time=2024-04-17T07:23:33.724494+0000, chain.objs.size()=1 2024-04-17T05:46:02.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.h33b582SIDG7TVoyTmAwrN3jnnTzQzb_0 2024-04-17T05:46:02.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10794838146178312889', time=2024-04-17T07:23:34.184809+0000, chain.objs.size()=1 2024-04-17T05:46:02.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xxsm4VLh1-Ow_hpHFKXhAmhgbwQy7Vo_0 2024-04-17T05:46:02.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11933407394688654252', time=2024-04-17T07:23:34.286551+0000, chain.objs.size()=1 2024-04-17T05:46:02.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fhKG66rBRpyYFLIQunSkirVUg8kYEx4_0 2024-04-17T05:46:02.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6064847030020475700', time=2024-04-17T07:23:34.865545+0000, chain.objs.size()=1 2024-04-17T05:46:02.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kqVgoXJ4WGWKFqDZlminZjloCM-7UX4_0 2024-04-17T05:46:02.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11772507308375573121', time=2024-04-17T07:23:34.867866+0000, chain.objs.size()=1 2024-04-17T05:46:02.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VNxgrbNPdXHY3yuZJkUViHz1_cTMrC7_0 2024-04-17T05:46:02.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13253973125418522808', time=2024-04-17T07:23:36.876299+0000, chain.objs.size()=1 2024-04-17T05:46:02.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PoaK5IhysxfGTRw2h6QqyreTQx6DGkE_0 2024-04-17T05:46:02.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.228+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45771 2024-04-17T05:46:02.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=13, truncated=0, next_marker='' 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4624334952863948079', time=2024-04-17T07:23:38.395148+0000, chain.objs.size()=1 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Mun174JSScvjnIBzqmgs-HvV0r24JlN_0 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12051431311594689599', time=2024-04-17T07:23:39.387407+0000, chain.objs.size()=1 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7tP7kTKOcG9a7wjTxnmFMh6sCo5AI0d_0 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14007524293418466924', time=2024-04-17T07:23:39.525270+0000, chain.objs.size()=1 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dk3S1MUHFW_V2fTWtzPyEC5FjkVWddL_0 2024-04-17T05:46:02.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18343751893318030967', time=2024-04-17T07:23:39.529587+0000, chain.objs.size()=1 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N4bPqMa3WX3KxGBJjRayM-N-ZXJrFpf_0 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5320276279415458389', time=2024-04-17T07:23:40.402699+0000, chain.objs.size()=1 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2hz9B2NUJAvfViIcklBF2-LSrpy-XNz_0 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7245878089872956950', time=2024-04-17T07:23:41.265999+0000, chain.objs.size()=1 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WXxoWpd79VVc1G1EtTvJsZOQXTlSIbi_0 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16475720674443169924', time=2024-04-17T07:24:07.236941+0000, chain.objs.size()=1 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.343__shadow_.bqOskjSXEo7LqiZNLjIFfo-mE8RY133_0 2024-04-17T05:46:02.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1137232442363284357', time=2024-04-17T07:24:24.927523+0000, chain.objs.size()=1 2024-04-17T05:46:02.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.364__shadow_.u4OE0PTrjVRy2gtPPJ8FUOkTqGTFz7e_0 2024-04-17T05:46:02.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16905083627838043187', time=2024-04-17T07:29:10.733101+0000, chain.objs.size()=6 2024-04-17T05:46:02.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.vQvzAgt5s_6-wh-6UKGDcaChER4FOnR0.1 2024-04-17T05:46:02.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.j8I8HXOo0D9oiWGV3LqbEk80jj8JnY3w.2 2024-04-17T05:46:02.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.WlU_PeFkTKlWLlrTwWYvw_b0MInm7c-8.3 2024-04-17T05:46:02.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.QyTZJ650htoBpiT4srcwJMQx4F6B11c9.4 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.vWMfF7XeytoOIoH6zdfurnpQhcQ2op5d.5 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.T4cpvWTZlCWRR9R-EYcbp6Rp6CGhnyqp.6 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3833012254099048029', time=2024-04-17T07:30:44.352733+0000, chain.objs.size()=1 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.539__shadow_._aZyh5_34XW2dNmSd50xzBEzuKnsdtE_0 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3265376393718014834', time=2024-04-17T07:33:33.514411+0000, chain.objs.size()=1 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:U-sVSqAJ5Rfq-lXz2tQYTn1AOTSQFd4_.PdHTHTt03vB5gtz2LTafhKObNMZqKbJ_0 2024-04-17T05:46:02.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7967230187986252378', time=2024-04-17T07:33:36.932801+0000, chain.objs.size()=1 2024-04-17T05:46:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.572__shadow:I5AoElFBzF6Hnxr2Sjf4tuoIBa2w1eG_.IP5eKJOV7dMmx5OfbjNlh_0nbaLY7A2_0 2024-04-17T05:46:02.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12876003117772769921', time=2024-04-17T07:37:41.880855+0000, chain.objs.size()=1 2024-04-17T05:46:02.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.6hBO39PF6JsaAkfDL425-ZjG2tBZ_Ol_0 2024-04-17T05:46:02.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-17T05:46:02.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/41376' 2024-04-17T05:46:02.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15508177694147577816', time=2024-04-17T07:09:40.173007+0000, chain.objs.size()=1 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.138__shadow_.XXTQVOjrxWT5LRiReXEZhXGx-bFO6Ra_0 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4551244860841607190', time=2024-04-17T07:11:03.211980+0000, chain.objs.size()=1 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.173__shadow_.-Asjp9LC1fjd63MfLtldsu9T1m0FI9__0 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6160574841301944305', time=2024-04-17T07:14:59.380801+0000, chain.objs.size()=1 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.239__shadow_.MxwtMYUZWql6YHGjo61ny3E6J13B8dp_0 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12065071038315233578', time=2024-04-17T07:15:05.880447+0000, chain.objs.size()=1 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.243__shadow_.bowrIMZsyrJDqSFmnr5pM4U_Qnhavd6_0 2024-04-17T05:46:02.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10688978730556532335', time=2024-04-17T07:15:07.508320+0000, chain.objs.size()=1 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.244__shadow_.qMFgnz6W3gqZZI8_iQtk-R3DuzGYqMS_0 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12142238806345449797', time=2024-04-17T07:17:03.733233+0000, chain.objs.size()=1 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UF_pff3m3SDcBjEwhcQXLSQeMvGwWf0_0 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11295186099981304824', time=2024-04-17T07:17:04.328266+0000, chain.objs.size()=1 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FlqlC8ow5G7k2SqKF1vTupkFmRbhrQ-_0 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3954333608558356129', time=2024-04-17T07:17:08.226391+0000, chain.objs.size()=1 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2Nq4SUyWsSaM4aLbSq5adYp1-LFf1jZ_0 2024-04-17T05:46:02.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9730152315012658768', time=2024-04-17T07:17:08.578539+0000, chain.objs.size()=1 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nZWGK6mCw2FvKtLTRAA8yo36Qk9ps07_0 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5455762538273626589', time=2024-04-17T07:17:09.930082+0000, chain.objs.size()=1 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PJo5HH4UeTI8yZdac_3ju2r-OmL2IJL_0 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11112588860952535434', time=2024-04-17T07:17:10.666786+0000, chain.objs.size()=1 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nHs9cm4zclHAz7RwESPrFezvKy-IN2m_0 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4453555660086279735', time=2024-04-17T07:17:11.386836+0000, chain.objs.size()=1 2024-04-17T05:46:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.56--NlO-5bfjG4olH1de_OOKLVIGhRg_0 2024-04-17T05:46:02.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9204359332720638652', time=2024-04-17T07:17:16.511423+0000, chain.objs.size()=1 2024-04-17T05:46:02.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rslLoYLzKT_5FeBcvCGQLHGtNHxFesB_0 2024-04-17T05:46:02.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6823700026596622290', time=2024-04-17T07:17:17.983712+0000, chain.objs.size()=1 2024-04-17T05:46:02.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5k8sIL-fbRqdbjZtwX70juPtL93nxYm_0 2024-04-17T05:46:02.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4253007580734798611', time=2024-04-17T07:17:18.385103+0000, chain.objs.size()=1 2024-04-17T05:46:02.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kwixwgtqqjmw-x3B6vpMRlpvjAONfM9_0 2024-04-17T05:46:02.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10212424177676535003', time=2024-04-17T07:17:19.705926+0000, chain.objs.size()=1 2024-04-17T05:46:02.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GcwyrZBNGlmZ8V9DtwjxPrgfEqxOCRF_0 2024-04-17T05:46:02.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9500128750404234697', time=2024-04-17T07:17:19.707920+0000, chain.objs.size()=1 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XlTS4vi4wc3d5Exqo3tgJ-vv15zXhHg_0 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7714630262501610094', time=2024-04-17T07:17:20.673553+0000, chain.objs.size()=1 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vfg7nN6vzQfpP0oqiOP4PLvGbzwEEKH_0 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9280762914862138469', time=2024-04-17T07:17:20.971024+0000, chain.objs.size()=1 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ttXYVil5F7LNrvuR1ctIWYLr1xB8ykK_0 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3156175411556152830', time=2024-04-17T07:17:24.246407+0000, chain.objs.size()=1 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FoRjquWq5KYk8PiQB4Rb6R2ljbgWcWp_0 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6316745317714562047', time=2024-04-17T07:17:24.943538+0000, chain.objs.size()=1 2024-04-17T05:46:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xbpttNNErTFjHCOnjm9IyblFKuRNxcG_0 2024-04-17T05:46:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15891186912539119002', time=2024-04-17T07:17:25.082242+0000, chain.objs.size()=1 2024-04-17T05:46:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HtRi787FJ4h6oJankpkRu-oeCX5MnPl_0 2024-04-17T05:46:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16556672815016907993', time=2024-04-17T07:17:27.054762+0000, chain.objs.size()=1 2024-04-17T05:46:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.IefdAFqExfV2p7DoI2BVuY_k5-V-fUM_0 2024-04-17T05:46:02.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10935391636368180132', time=2024-04-17T07:17:29.004201+0000, chain.objs.size()=1 2024-04-17T05:46:02.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.G3aE7srSeRliq89Vjj2-pPOJfHR1s39_0 2024-04-17T05:46:02.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9385279406108054614', time=2024-04-17T07:17:29.924163+0000, chain.objs.size()=1 2024-04-17T05:46:02.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.a0r1iyNhaAjtrhyhYy-dpicGcjfd-ku_0 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4855874931967333070', time=2024-04-17T07:17:30.717128+0000, chain.objs.size()=1 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MbAwtkJvIzMf5WwNpbw3_RBH2M6tvHE_0 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5917415626311841198', time=2024-04-17T07:17:30.912016+0000, chain.objs.size()=1 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pw1KTlpvjj5pdrjPUsK7jF3Sbe_4qGL_0 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10837286909307666736', time=2024-04-17T07:17:31.073428+0000, chain.objs.size()=1 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.H73K50Ad95TfJFGg2q4BMv9WlDpJ1Gj_0 2024-04-17T05:46:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7665083677512067270', time=2024-04-17T07:17:31.576141+0000, chain.objs.size()=1 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uP740tuvWfUauskbIl9vACGRjsHFS4c_0 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5398637323252454906', time=2024-04-17T07:18:11.611014+0000, chain.objs.size()=1 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.277__shadow_.1Y9WiNPWzkGfv-ASMc_eTr-so-k64Le_0 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10391661723113899170', time=2024-04-17T07:18:16.488629+0000, chain.objs.size()=1 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.280__shadow_.0G6Mpaq-6W_X7Du492FwGyZIigdCYqV_0 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.206519410521460844', time=2024-04-17T07:18:46.594059+0000, chain.objs.size()=1 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.294__shadow_.5poDC2DRYwYofQ2m7AhSXiiNWdDXjwR_0 2024-04-17T05:46:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11718004857132822928', time=2024-04-17T07:18:46.596029+0000, chain.objs.size()=1 2024-04-17T05:46:02.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.294__shadow_.zwcYaPPz4JCL0VNFLt02cH4AbE2T_RK_0 2024-04-17T05:46:02.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12322079410712575389', time=2024-04-17T07:18:50.335201+0000, chain.objs.size()=1 2024-04-17T05:46:02.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.296__shadow_.cNQcDSZXkmbFpBMH6jWvWDXuaQuEh-w_0 2024-04-17T05:46:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.260+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13265693287384013759', time=2024-04-17T07:18:50.336833+0000, chain.objs.size()=1 2024-04-17T05:46:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.260+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.296__shadow_.8HCT7V80IIYEnqRdokglvHPjvO9MOjD_0 2024-04-17T05:46:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.260+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8817965043724839142', time=2024-04-17T07:19:13.966443+0000, chain.objs.size()=1 2024-04-17T05:46:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.260+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.311__shadow_.umYLRdapXc4-9Qfm0JeJ8reWP2dt0H0_0 2024-04-17T05:46:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5618618385461879257', time=2024-04-17T07:20:59.795614+0000, chain.objs.size()=1 2024-04-17T05:46:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TUsoQUx7m2JC6LHAjDe6ldiuKwJMwWe_0 2024-04-17T05:46:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7125626031724259746', time=2024-04-17T07:21:00.614154+0000, chain.objs.size()=1 2024-04-17T05:46:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pQ4CMjqivk4u9YGNLWakV1eSsIKwgoY_0 2024-04-17T05:46:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13302808054527896450', time=2024-04-17T07:21:04.831753+0000, chain.objs.size()=1 2024-04-17T05:46:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MKj-rJzzS-q-nuqqyLf0YLNT5xX3Rc7_0 2024-04-17T05:46:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5246700899535749467', time=2024-04-17T07:21:05.196392+0000, chain.objs.size()=1 2024-04-17T05:46:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dQQKvGOgtOlQPjTRIzl8inquFtRHIf9_0 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9864356075862725879', time=2024-04-17T07:21:05.454862+0000, chain.objs.size()=1 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YxDeddGOBJaO1pa1S2HlM0pzKHcV6C0_0 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6696146045997199342', time=2024-04-17T07:21:06.407488+0000, chain.objs.size()=1 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6FmD4Vsr-rSkeD7iW4j0gXDX9BpGgMx_0 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13124638102433283950', time=2024-04-17T07:21:06.849302+0000, chain.objs.size()=1 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.qaqI326XgGGHHLFq7aEGy5H4eFmPsoC_0 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6325000440453942666', time=2024-04-17T07:21:07.972881+0000, chain.objs.size()=1 2024-04-17T05:46:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Bqwm74ROXXx6jEqevAipy_f7Z3aKKc0_0 2024-04-17T05:46:02.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7565639283833893472', time=2024-04-17T07:21:08.773151+0000, chain.objs.size()=1 2024-04-17T05:46:02.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fYlZnfcFHFlJdw9F2lo-wHgzhBGrA7c_0 2024-04-17T05:46:02.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10116587464499594315', time=2024-04-17T07:21:10.482259+0000, chain.objs.size()=1 2024-04-17T05:46:02.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eIkeu6RrQBzyRbvofnnbnBsg_1-zCYT_0 2024-04-17T05:46:02.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15708662819523279245', time=2024-04-17T07:21:12.317097+0000, chain.objs.size()=1 2024-04-17T05:46:02.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._a3jGuxPtzK4dLelEpBQ9so5XlXqRYM_0 2024-04-17T05:46:02.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4765232489806406352', time=2024-04-17T07:21:12.517798+0000, chain.objs.size()=1 2024-04-17T05:46:02.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HlBfw1idyxWP_jcnarRdbKycyU8LT0O_0 2024-04-17T05:46:02.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1072217330004824183', time=2024-04-17T07:21:12.721437+0000, chain.objs.size()=1 2024-04-17T05:46:02.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LTI5YtDU9J4WKP0qvF6UzKyjYQE7FSo_0 2024-04-17T05:46:02.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15653295045857945002', time=2024-04-17T07:21:13.425496+0000, chain.objs.size()=1 2024-04-17T05:46:02.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5ZGNaenrdatogcuKVX-DKncTHY7GqaI_0 2024-04-17T05:46:02.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8568887439893424124', time=2024-04-17T07:21:14.653420+0000, chain.objs.size()=1 2024-04-17T05:46:02.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t7X8teJ4zficqoP-JTXvdRf_DEELVou_0 2024-04-17T05:46:02.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5240409103315653110', time=2024-04-17T07:21:16.696836+0000, chain.objs.size()=1 2024-04-17T05:46:02.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1yhZtB7fR7MYdJ5XSZy81lx_AkfSdcb_0 2024-04-17T05:46:02.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9395601300463506893', time=2024-04-17T07:21:18.168226+0000, chain.objs.size()=1 2024-04-17T05:46:02.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.d7wQ4N00U10LSRVBa4efB7HQmkRvy-6_0 2024-04-17T05:46:02.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8686291394893649289', time=2024-04-17T07:21:19.938972+0000, chain.objs.size()=1 2024-04-17T05:46:02.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Wv3fGUTNRmh-pn5GaJjYsZL6KWaueC2_0 2024-04-17T05:46:02.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16572032165610295276', time=2024-04-17T07:21:19.941221+0000, chain.objs.size()=1 2024-04-17T05:46:02.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JT5CLd-Z5Q9Zn341AFYnQDvtuP1k-ms_0 2024-04-17T05:46:02.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8078409565678060539', time=2024-04-17T07:21:20.995540+0000, chain.objs.size()=1 2024-04-17T05:46:02.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mLVdHnHZd_w6qwvu7fA9cTYXX0hQ0tZ_0 2024-04-17T05:46:02.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.272+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16984368457649973032', time=2024-04-17T07:21:24.055289+0000, chain.objs.size()=1 2024-04-17T05:46:02.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.272+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.r7K2GMSvnkmBlL8vjd-tM0eL9JfwUrx_0 2024-04-17T05:46:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7937658183321432540', time=2024-04-17T07:21:24.194836+0000, chain.objs.size()=1 2024-04-17T05:46:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4o7cYzNdPhF0FWCR5Pt8THestRNCTdS_0 2024-04-17T05:46:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5235673757441479758', time=2024-04-17T07:21:25.361365+0000, chain.objs.size()=1 2024-04-17T05:46:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YVc2XwmYa_Yu6uzttMoSk4VRWMa0Z-h_0 2024-04-17T05:46:02.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12196363905251066577', time=2024-04-17T07:21:26.216639+0000, chain.objs.size()=1 2024-04-17T05:46:02.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4C3uQU7WFnbS-2T3euV-AtiMQZ82XX0_0 2024-04-17T05:46:02.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.929166689780549585', time=2024-04-17T07:21:27.566995+0000, chain.objs.size()=1 2024-04-17T05:46:02.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BSMFPwo7BxQpCYNWcLJO2uA2Qyr3DpJ_0 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1092788088667840376', time=2024-04-17T07:21:29.256165+0000, chain.objs.size()=1 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.G8nDTosUkeKpy957uwK-cB8bTeGm4GM_0 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7016963872458592225', time=2024-04-17T07:21:29.340962+0000, chain.objs.size()=1 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4w1s3J__H8yKRbxskOry-Tj-rabDw20_0 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4865661323462308010', time=2024-04-17T07:23:12.752168+0000, chain.objs.size()=1 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J3JpyEhMRbxSlItIlPmuoI0wYmD6eVI_0 2024-04-17T05:46:02.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12126650401206228275', time=2024-04-17T07:23:14.549233+0000, chain.objs.size()=1 2024-04-17T05:46:02.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uzZN3CPTTWZIOOK2xyzL3cLZyZyzOLO_0 2024-04-17T05:46:02.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8920694962771272902', time=2024-04-17T07:23:15.107442+0000, chain.objs.size()=1 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lW0G-TgLUM2VH80wHdsBg2ajHY9wdhx_0 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5438310961741264625', time=2024-04-17T07:23:15.529359+0000, chain.objs.size()=1 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TatewZrCZnwrgbMeascRCVqP1VZF8vX_0 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18160908737569870382', time=2024-04-17T07:23:15.726469+0000, chain.objs.size()=1 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OkVd82CFLxZNP05vXN9B8cMcFfSpRzl_0 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11177459833080573442', time=2024-04-17T07:23:15.779484+0000, chain.objs.size()=1 2024-04-17T05:46:02.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.i3mZR7PwE0bYs73bA6OoXRS1lqZZTR2_0 2024-04-17T05:46:02.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13156192771858950509', time=2024-04-17T07:23:17.092617+0000, chain.objs.size()=1 2024-04-17T05:46:02.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.A9rQMk0u5HiHv3elinvVMB9GNMavP86_0 2024-04-17T05:46:02.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13245600779219414731', time=2024-04-17T07:23:17.094611+0000, chain.objs.size()=1 2024-04-17T05:46:02.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RVcgiBRwAU34048zaV8lUYKkya59bKL_0 2024-04-17T05:46:02.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17437460026174174794', time=2024-04-17T07:23:17.412508+0000, chain.objs.size()=1 2024-04-17T05:46:02.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8fY6DYwuYYYsndQkpKmyuWmytkTuW8C_0 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7843006877509612358', time=2024-04-17T07:23:17.730834+0000, chain.objs.size()=1 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hRQc5AcCzgzhTaF9ta8FFduKSVk75bg_0 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16730817146514592270', time=2024-04-17T07:23:18.155722+0000, chain.objs.size()=1 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3KL3W2PzdalwSqhCqprra_ISeibQQ6t_0 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2661669941043444262', time=2024-04-17T07:23:18.157721+0000, chain.objs.size()=1 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yhg4q1JE10JlzwEO5K-TlMAx_CxxonS_0 2024-04-17T05:46:02.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5609830016884997402', time=2024-04-17T07:23:18.757203+0000, chain.objs.size()=1 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8C-S5EwIZ0OOQ_mRbrUSWMPqtsyViBH_0 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9212306048624929423', time=2024-04-17T07:23:19.568067+0000, chain.objs.size()=1 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UJH115RO4GZKzq3PjtcTBQbmzwaizph_0 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4060440068337265703', time=2024-04-17T07:23:21.696628+0000, chain.objs.size()=1 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VuRTH1Fzz-87XcZkkU962U6WVlot_aV_0 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13300916937711361119', time=2024-04-17T07:23:23.043380+0000, chain.objs.size()=1 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QabuP4sh8u3aBqYYS5s1AOA2NHrnAbg_0 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1631734721690319767', time=2024-04-17T07:23:26.841603+0000, chain.objs.size()=1 2024-04-17T05:46:02.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._jiryRwKCw5bh5TlW23hmmH5aMf2XrK_0 2024-04-17T05:46:02.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.284+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3717936930477665597', time=2024-04-17T07:23:26.981268+0000, chain.objs.size()=1 2024-04-17T05:46:02.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.284+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EO2dCM1SZV6A_NzTXbzVsfPQ39HVxgG_0 2024-04-17T05:46:02.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12576176395270595704', time=2024-04-17T07:23:27.192803+0000, chain.objs.size()=1 2024-04-17T05:46:02.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.epY-k8wjQgzlHCCvoHnX97VAkvoxZzR_0 2024-04-17T05:46:02.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8889435879602974975', time=2024-04-17T07:23:28.455829+0000, chain.objs.size()=1 2024-04-17T05:46:02.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DhcqnNzSl2YLBj5oo1Mq4W0Tkn9ASmL_0 2024-04-17T05:46:02.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7449615374902363274', time=2024-04-17T07:23:30.629209+0000, chain.objs.size()=1 2024-04-17T05:46:02.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xKQBZN8SykmUOy6ywQC8uL2lImDUf4O_0 2024-04-17T05:46:02.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5007170689726377760', time=2024-04-17T07:23:31.276247+0000, chain.objs.size()=1 2024-04-17T05:46:02.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.AMlGVYUGpGFm5k9vXsaTMyJU4ZQKqll_0 2024-04-17T05:46:02.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5178362815843630370', time=2024-04-17T07:23:31.277987+0000, chain.objs.size()=1 2024-04-17T05:46:02.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GqdHR3mCsRm5a3-C65vcErEbRLH7gwT_0 2024-04-17T05:46:02.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14644853192165021331', time=2024-04-17T07:23:34.502110+0000, chain.objs.size()=1 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CP97qT8i9zr82OwHUDOejQMZOlK9G56_0 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8803867268042656175', time=2024-04-17T07:23:35.052847+0000, chain.objs.size()=1 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iXZw0fiGH9IWDjZWQe-xm6NNP__g_UI_0 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6335818918519179371', time=2024-04-17T07:23:36.113651+0000, chain.objs.size()=1 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jVFHlvxy7p-JwFyPIINpzy3qwVVtubv_0 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8641781151589306362', time=2024-04-17T07:23:37.428793+0000, chain.objs.size()=1 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NhlZN_QTrq82b_4hqi2imSrnAL0zym0_0 2024-04-17T05:46:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13729215061596548920', time=2024-04-17T07:23:39.442887+0000, chain.objs.size()=1 2024-04-17T05:46:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.41zcz5lnGiYmWNWu7PCFOXrVBSso9np_0 2024-04-17T05:46:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15788328244398009727', time=2024-04-17T07:23:40.477175+0000, chain.objs.size()=1 2024-04-17T05:46:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.M2I_cXwHni70HExAaH2-TQnk4k02piN_0 2024-04-17T05:46:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12162401192027305798', time=2024-04-17T07:23:40.478686+0000, chain.objs.size()=1 2024-04-17T05:46:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OSxLReUOWr9szR2ZmbaZVaEro4zbLdb_0 2024-04-17T05:46:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7563460328941623933', time=2024-04-17T07:23:40.597194+0000, chain.objs.size()=1 2024-04-17T05:46:02.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-J9lLpqYrNouN4WtdlDP-levyvkuYTu_0 2024-04-17T05:46:02.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15787035457715221509', time=2024-04-17T07:23:40.736069+0000, chain.objs.size()=1 2024-04-17T05:46:02.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uSo9hS-_s2CYNQMGUkge8n4LMqYrbYK_0 2024-04-17T05:46:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3039462889424330946', time=2024-04-17T07:23:41.578672+0000, chain.objs.size()=1 2024-04-17T05:46:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IiEeNtgcmhaiPE7xNhRbsDRhzuKGpC-_0 2024-04-17T05:46:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10789111123249885067', time=2024-04-17T07:24:26.866139+0000, chain.objs.size()=1 2024-04-17T05:46:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.366__shadow_.mfnmc9UiHVaapS_EqM5g3HtV3yor_X3_0 2024-04-17T05:46:02.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16184698160481218890', time=2024-04-17T07:24:58.572433+0000, chain.objs.size()=1 2024-04-17T05:46:02.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.398__shadow_.bx9kIHZr6v6mZ10hoCGsywpbro-2Etq_0 2024-04-17T05:46:02.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10935415903123299032', time=2024-04-17T07:25:21.398119+0000, chain.objs.size()=1 2024-04-17T05:46:02.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.425__shadow_.rxwFDtkCegVH0Q920Z1fx1hoi3hgpa__0 2024-04-17T05:46:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2402955465436111235', time=2024-04-17T07:25:59.286304+0000, chain.objs.size()=1 2024-04-17T05:46:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.454__shadow_.hL7HWvawJUZLKZlrAx-0vkka9JJjyr7_0 2024-04-17T05:46:02.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/41376 2024-04-17T05:46:02.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-17T05:46:02.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4066783685890793403', time=2024-04-17T07:26:01.141909+0000, chain.objs.size()=1 2024-04-17T05:46:02.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.455__shadow_.z1ULApb_tP_ZvSOHr9LSf46GwdWp6L8_0 2024-04-17T05:46:02.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13779723385336514135', time=2024-04-17T07:26:01.146360+0000, chain.objs.size()=1 2024-04-17T05:46:02.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.455__shadow_.xTOAGuYGnM4Y4grjI_3g460Qwje5oNU_0 2024-04-17T05:46:02.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3094099595952901340', time=2024-04-17T07:26:18.775995+0000, chain.objs.size()=1 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.474__shadow_._5zZ25PZawstteRbqhFcNYn1JGffTIy_0 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11222144543616735801', time=2024-04-17T07:27:06.185909+0000, chain.objs.size()=1 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.506__multipart_mymultipartcopy.2~pwATmP9eO-4i6UU5UHJ6xHgdf1tplV6.1 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6807102087936896409', time=2024-04-17T07:32:01.160617+0000, chain.objs.size()=1 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.561__shadow:wZDY4VDV.WOua6RrV5P3elhLGCXdtAY_.lPkWF3sBeCZZ2guzFeebHcsVmNgaxDP_0 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:emuOQEr.v5f7US0CQbgNiL6oyTvPHBn_.8B0PANt9SXHPvpiPHZO8Id_om9c4vas_1_24a55237e5a03dadaf596b2675dc326f', time=2024-04-17T07:33:34.122799+0000, chain.objs.size()=1 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:emuOQEr.v5f7US0CQbgNiL6oyTvPHBn_.8B0PANt9SXHPvpiPHZO8Id_om9c4vas_0 2024-04-17T05:46:02.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3345803487610964415', time=2024-04-17T07:33:45.294635+0000, chain.objs.size()=1 2024-04-17T05:46:02.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:iD6Tjg2Jc8oraWlvi8F1Y0cgw7C6Fcw_.I4Rf7yeZSg8cQPVSvFGf_cYLUqFtsiF_0 2024-04-17T05:46:02.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18334895771101856962', time=2024-04-17T07:39:23.384665+0000, chain.objs.size()=1 2024-04-17T05:46:02.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.723__shadow_.iHztpuNChBFTNxdOAhKQBX0-rmEx9N0_0 2024-04-17T05:46:02.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-17T05:46:02.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47207' 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13644283611273151573', time=2024-04-17T07:14:34.696361+0000, chain.objs.size()=1 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.225__shadow_.Gh0Wgx-G3-vV0uYr8db9xVrJjnWlwIP_0 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1086854347345789968', time=2024-04-17T07:14:56.142528+0000, chain.objs.size()=1 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.237__shadow_.aYs5y043U2HwXuDiLSJaMZ0YlkNmRKp_0 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5176303285599052614', time=2024-04-17T07:14:59.383664+0000, chain.objs.size()=1 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.239__shadow_.daVhViIAUmrvqwfKivObK8hDs_sfNic_0 2024-04-17T05:46:02.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16644449995500760008', time=2024-04-17T07:17:01.779256+0000, chain.objs.size()=1 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3Z0cM9Yf9OgLVF7R4A84naKRGam3Uag_0 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12900611707720290095', time=2024-04-17T07:17:02.098381+0000, chain.objs.size()=1 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.cP77BCyUx5rHE__KTtnIxyMQW7KP0db_0 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12370762245249848953', time=2024-04-17T07:17:03.237185+0000, chain.objs.size()=1 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0nM7qrEYV4KXI3ORs3rwIJ3X9vHeWkO_0 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12249323925030850949', time=2024-04-17T07:17:03.733861+0000, chain.objs.size()=1 2024-04-17T05:46:02.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.G0D23B5tairEJjKvWLdd1hhGxcgEAW__0 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4206794412521134908', time=2024-04-17T07:17:04.378198+0000, chain.objs.size()=1 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.b3SHFmvIb4jJnpmcjKTFFo_ocUWSuda_0 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17569850360331594755', time=2024-04-17T07:17:05.496432+0000, chain.objs.size()=1 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Fx2PNZU45s6eC4RR48FYpd2K4qZQLNq_0 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4448917162187535777', time=2024-04-17T07:17:05.498380+0000, chain.objs.size()=1 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Bm8o_kS7MeTocScPkCEDdXCI5wfYfsh_0 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9143302759487456340', time=2024-04-17T07:17:07.395124+0000, chain.objs.size()=1 2024-04-17T05:46:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sFBsHFKKE7mpjLJW2ZGGjuHnNCislsZ_0 2024-04-17T05:46:02.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3850584745644416158', time=2024-04-17T07:17:08.267045+0000, chain.objs.size()=1 2024-04-17T05:46:02.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sTbcJCzIPD8lH4Japvgv3QmDL8VYvEZ_0 2024-04-17T05:46:02.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4441460510181654124', time=2024-04-17T07:17:10.633318+0000, chain.objs.size()=1 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RxUiV7o_prj3B8N37Q10a747OcBzlvU_0 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12614178679254227394', time=2024-04-17T07:17:11.730756+0000, chain.objs.size()=1 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GxLLmM0Ctnu50CZm2gay-n5LeRh2l8R_0 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17296093919665341480', time=2024-04-17T07:17:11.733095+0000, chain.objs.size()=1 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iQP3uc4U3fgf5Ht5s0PFs03c8iiJ1W8_0 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10341057887143786327', time=2024-04-17T07:17:12.362944+0000, chain.objs.size()=1 2024-04-17T05:46:02.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xgHiNJTxI0ByaDla7Lm7wpBsO8XbzkA_0 2024-04-17T05:46:02.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11642551825264418985', time=2024-04-17T07:17:12.871748+0000, chain.objs.size()=1 2024-04-17T05:46:02.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QYVdAi99QHAs2sw8FwoDVdGK9coD1Va_0 2024-04-17T05:46:02.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11162218122562018305', time=2024-04-17T07:17:14.488135+0000, chain.objs.size()=1 2024-04-17T05:46:02.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BBdY_Q7q942vaOdShLrcWJ4LislcwF4_0 2024-04-17T05:46:02.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14784166432123182548', time=2024-04-17T07:17:15.050945+0000, chain.objs.size()=1 2024-04-17T05:46:02.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VxwN5O-s5WWleUVLffCqCjfIl_H29vx_0 2024-04-17T05:46:02.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16576246882212128767', time=2024-04-17T07:17:16.345651+0000, chain.objs.size()=1 2024-04-17T05:46:02.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yWKJDRMIUbF-tTItRWcRAsET0WqHrVi_0 2024-04-17T05:46:02.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10627746886926361169', time=2024-04-17T07:17:18.516850+0000, chain.objs.size()=1 2024-04-17T05:46:02.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.e5J1WH6ndNicJO-onK026-EgL41ZH09_0 2024-04-17T05:46:02.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13314276948202884136', time=2024-04-17T07:17:19.401831+0000, chain.objs.size()=1 2024-04-17T05:46:02.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Edvi2LkU2AHgL3kpJlqSicW5la3tsQY_0 2024-04-17T05:46:02.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8425584062488319603', time=2024-04-17T07:17:20.476273+0000, chain.objs.size()=1 2024-04-17T05:46:02.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6z_vLQfUotCgwlywBtEMe7llVIcWNHM_0 2024-04-17T05:46:02.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.58068158847629418', time=2024-04-17T07:17:20.972338+0000, chain.objs.size()=1 2024-04-17T05:46:02.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c1ChZ-Ch9Thj4cOCHMd7muPhGGO2VfV_0 2024-04-17T05:46:02.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13484276193653422994', time=2024-04-17T07:17:22.086288+0000, chain.objs.size()=1 2024-04-17T05:46:02.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.h-pol5HGqW9gBgr4xBCabRrBaCmmwGN_0 2024-04-17T05:46:02.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4033917297418344978', time=2024-04-17T07:17:23.136052+0000, chain.objs.size()=1 2024-04-17T05:46:02.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qyeULdh6DomVgWahmlBFoA96aoS9QrN_0 2024-04-17T05:46:02.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10170613791269375813', time=2024-04-17T07:17:23.212168+0000, chain.objs.size()=1 2024-04-17T05:46:02.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BIEwrJGsgdw-kWlGx870pt1aIgfyXn6_0 2024-04-17T05:46:02.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16796278264652151008', time=2024-04-17T07:17:23.623242+0000, chain.objs.size()=1 2024-04-17T05:46:02.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CQQP0l7xJMcW8CrWOBGzWYtoRycAVPo_0 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.824960009835002744', time=2024-04-17T07:17:23.723581+0000, chain.objs.size()=1 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-oaV98OClTBJj0Jsv8A_PvoCXh64PyC_0 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7521462328304872045', time=2024-04-17T07:17:23.818964+0000, chain.objs.size()=1 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.78vlGnjCL8ERohDJuwxvgVLgAYJZRQb_0 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13400225641573412112', time=2024-04-17T07:17:24.300645+0000, chain.objs.size()=1 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Jw8BW12xj27OtM1WIVHDeG3ApW9eZIh_0 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3334760194165787693', time=2024-04-17T07:17:26.338201+0000, chain.objs.size()=1 2024-04-17T05:46:02.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UL85JyGcXwP-ILaIdglEXrAJjMy5RvK_0 2024-04-17T05:46:02.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4807218228215571610', time=2024-04-17T07:17:27.222493+0000, chain.objs.size()=1 2024-04-17T05:46:02.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ulfeSNeCzBA5Ijt6DlaGMOaw3J3i-B8_0 2024-04-17T05:46:02.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13330460092855095355', time=2024-04-17T07:17:27.656269+0000, chain.objs.size()=1 2024-04-17T05:46:02.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9cQ3rfxdRVrPWTLDtMGNNdLy-ihcoTk_0 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1976228921037537824', time=2024-04-17T07:17:27.782811+0000, chain.objs.size()=1 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hjO6vv3QRY2zSqjcUlyf6gpR9i0eoJ7_0 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.668936991540990247', time=2024-04-17T07:17:28.392925+0000, chain.objs.size()=1 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dvKWGACHDtWutaUTuvTDmBl9G8SClZW_0 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5901258376220092253', time=2024-04-17T07:17:28.395078+0000, chain.objs.size()=1 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.r4ZzXz7pc8KWT8JCMIOhlWHTqOYou6R_0 2024-04-17T05:46:02.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16218892801606420022', time=2024-04-17T07:17:29.929549+0000, chain.objs.size()=1 2024-04-17T05:46:02.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OpgHUtIBJCSC944aDbHRbFNq6Dmf4mb_0 2024-04-17T05:46:02.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1205548533503264281', time=2024-04-17T07:17:35.453471+0000, chain.objs.size()=1 2024-04-17T05:46:02.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.254__shadow_.KzIO-LL0-IqZJShmjmipGl0pEwykq9A_0 2024-04-17T05:46:02.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8500617116283098536', time=2024-04-17T07:17:39.920688+0000, chain.objs.size()=1 2024-04-17T05:46:02.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.257__shadow_.ouH-Ye6rR0jZXTyhqn1OJdm11SUxzti_0 2024-04-17T05:46:02.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11404366472581319387', time=2024-04-17T07:17:41.389813+0000, chain.objs.size()=1 2024-04-17T05:46:02.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.258__shadow_.mIoDD1lh6jwn7dlsw7vlgmVKduktBuq_0 2024-04-17T05:46:02.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7550784140576898765', time=2024-04-17T07:17:48.842238+0000, chain.objs.size()=1 2024-04-17T05:46:02.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.263__shadow_.RI3cv188X3vx_Q02CXDEbXsgJ3cjtJq_0 2024-04-17T05:46:02.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13813451630165815468', time=2024-04-17T07:17:58.480355+0000, chain.objs.size()=1 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.269__shadow_.quLuBlWF7KQaKhj0_nPq-Te4TfSLfiJ_0 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16256586330859922090', time=2024-04-17T07:18:04.966960+0000, chain.objs.size()=1 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.273__shadow_.GrteGvjBMi4rSbREZP_j55LmM9js-eH_0 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7814495974293691574', time=2024-04-17T07:18:48.194911+0000, chain.objs.size()=1 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.295__shadow_.byDswtoatv08_7uSR2dlAZE5a0qiU1d_0 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5883329388862007053', time=2024-04-17T07:21:00.593394+0000, chain.objs.size()=1 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.R8tQwJYoV3SsVNUxzvVQ0i64zSdkKKe_0 2024-04-17T05:46:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7338576810962758776', time=2024-04-17T07:21:01.296695+0000, chain.objs.size()=1 2024-04-17T05:46:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mChxmhK6i3M-3UXJoo4rNIKTPsTpYcs_0 2024-04-17T05:46:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11702917878316907347', time=2024-04-17T07:21:04.020607+0000, chain.objs.size()=1 2024-04-17T05:46:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EG5pe47iwTH6KukCeyjccBAAWwE-GWE_0 2024-04-17T05:46:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11909698566628499695', time=2024-04-17T07:21:04.231459+0000, chain.objs.size()=1 2024-04-17T05:46:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eMdaP0agEbVq3L9ByWKXrRaryz4aASQ_0 2024-04-17T05:46:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11992619651582690268', time=2024-04-17T07:21:04.331255+0000, chain.objs.size()=1 2024-04-17T05:46:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ki8-RzuIo65KS-BaLyFArxoQoSzOQQw_0 2024-04-17T05:46:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10046471499542326124', time=2024-04-17T07:21:04.435320+0000, chain.objs.size()=1 2024-04-17T05:46:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u2_c_0tEeCgt9UA16eRFdtz9fvvDi0q_0 2024-04-17T05:46:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10587571933784269065', time=2024-04-17T07:21:04.436983+0000, chain.objs.size()=1 2024-04-17T05:46:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ru882rIKDaq4O5H7UM1ySCRTtwfEnGW_0 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7242022785389472510', time=2024-04-17T07:21:05.721649+0000, chain.objs.size()=1 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8WqkuFwR4hcLhcq_tss5P-3817IO7a2_0 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8898497577742547376', time=2024-04-17T07:21:07.907511+0000, chain.objs.size()=1 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t5w2i9WD_6ZaIz-tkDC4wo6eAnuP292_0 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15515479998853401000', time=2024-04-17T07:21:09.613915+0000, chain.objs.size()=1 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QWhQO1SebTMKoSE48LvLui0v0hztZiy_0 2024-04-17T05:46:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6763355566661188843', time=2024-04-17T07:21:12.716739+0000, chain.objs.size()=1 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.k0RvtD6uwK7f4_RxHU333COxOSO98qH_0 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16283183089611950024', time=2024-04-17T07:21:12.972676+0000, chain.objs.size()=1 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YT5h2nyuD713cNWd5v9M1dE96tud32a_0 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8276771040740658538', time=2024-04-17T07:21:14.241255+0000, chain.objs.size()=1 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rKXspMVfhOUOvgucDa1bGOzEA6Zrhvo_0 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15588341809430037117', time=2024-04-17T07:21:18.138563+0000, chain.objs.size()=1 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zK3fz5i5hq8Ua41AFWd7oFNzhGuvYNz_0 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4258016075781624457', time=2024-04-17T07:21:18.370439+0000, chain.objs.size()=1 2024-04-17T05:46:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Dnxa8UppTpuA7MPxPbmgNfOzk0XjWZ7_0 2024-04-17T05:46:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8117735854558565708', time=2024-04-17T07:21:18.775213+0000, chain.objs.size()=1 2024-04-17T05:46:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CIQUfZoZIg87SPZ07XIzeZrM0XrEwh6_0 2024-04-17T05:46:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9408208886926260976', time=2024-04-17T07:21:19.574532+0000, chain.objs.size()=1 2024-04-17T05:46:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BCWjgg7zsLopgBrl7ayC3afYROE_2Yz_0 2024-04-17T05:46:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11809407765711715065', time=2024-04-17T07:21:24.190198+0000, chain.objs.size()=1 2024-04-17T05:46:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tz1gmc4f07op18E-nbcUP2OJGrurPuP_0 2024-04-17T05:46:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12303978537418215263', time=2024-04-17T07:21:24.584480+0000, chain.objs.size()=1 2024-04-17T05:46:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YCj29yg__CrlCEGJmON80YO038bvP7D_0 2024-04-17T05:46:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2004642525302917278', time=2024-04-17T07:21:24.691915+0000, chain.objs.size()=1 2024-04-17T05:46:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RnhFLpcVlsqTMuEiU3RQvDeIdZtHbbN_0 2024-04-17T05:46:02.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16594348197891582211', time=2024-04-17T07:21:25.126260+0000, chain.objs.size()=1 2024-04-17T05:46:02.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.OBRw3Zh3sZM3qPZJ-gggzHE0djT2PWV_0 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12926897869186782927', time=2024-04-17T07:21:26.179184+0000, chain.objs.size()=1 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gB3OGwqmIMej4PSfalDiZtfeWzD9yzr_0 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13531688169635050729', time=2024-04-17T07:21:27.084660+0000, chain.objs.size()=1 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JyTFe_mzx1GzJJa-Dl_m8oLTt8AU5yn_0 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17098538876120404912', time=2024-04-17T07:21:27.089788+0000, chain.objs.size()=1 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SS44CCKOV7PUGdaZPkSPooHGivQ1wIw_0 2024-04-17T05:46:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2692719986517630363', time=2024-04-17T07:21:28.783556+0000, chain.objs.size()=1 2024-04-17T05:46:02.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Z5rI1xKG-ogbncXbn7tFYbvqH-cg36i_0 2024-04-17T05:46:02.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16358702348246824034', time=2024-04-17T07:23:12.638248+0000, chain.objs.size()=1 2024-04-17T05:46:02.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DMmD-45NG1fHQ5HS8MQ8FQnKVjLc-Il_0 2024-04-17T05:46:02.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.866015489702886138', time=2024-04-17T07:23:15.720133+0000, chain.objs.size()=1 2024-04-17T05:46:02.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3b0EJmnbmIcC-v-fktJpjOLh0_rakpP_0 2024-04-17T05:46:02.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1505432179799962975', time=2024-04-17T07:23:17.135716+0000, chain.objs.size()=1 2024-04-17T05:46:02.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-7urnQU3Ht00pxRF8H9OsuALE_vFcdo_0 2024-04-17T05:46:02.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17328396157621252740', time=2024-04-17T07:23:18.588278+0000, chain.objs.size()=1 2024-04-17T05:46:02.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N8K5WgXb5vNhFIWc_vSMD44ha56R_VK_0 2024-04-17T05:46:02.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14352174079173504831', time=2024-04-17T07:23:19.731487+0000, chain.objs.size()=1 2024-04-17T05:46:02.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kC55aB5UEwNQuwn1EO99kpwmlnLgN0o_0 2024-04-17T05:46:02.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15987524940369342075', time=2024-04-17T07:23:19.733395+0000, chain.objs.size()=1 2024-04-17T05:46:02.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2bE5PEqAEkSFDOmRPgmZQuDaQcQ6jGQ_0 2024-04-17T05:46:02.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10795902791002942587', time=2024-04-17T07:23:21.000113+0000, chain.objs.size()=1 2024-04-17T05:46:02.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZJbhFxPRgMvXhvSBD5qcGA9p7dfucfV_0 2024-04-17T05:46:02.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3876896110176225398', time=2024-04-17T07:23:26.132647+0000, chain.objs.size()=1 2024-04-17T05:46:02.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rkBRGxac_PrKWI31mHmFDm5vxmHZK2R_0 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5940966462763668545', time=2024-04-17T07:23:28.545147+0000, chain.objs.size()=1 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MZMXVtOnVT2drd6DHPShFnm-pxsdnmv_0 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13017487411969050896', time=2024-04-17T07:23:29.347931+0000, chain.objs.size()=1 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.p7sgjAErPRLpkp6KCnPRoCHpJesXBtu_0 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12413732387129234370', time=2024-04-17T07:23:29.742654+0000, chain.objs.size()=1 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1O01TZpFnJcsVc2TCiKSaxU6jjisgcE_0 2024-04-17T05:46:02.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11003433858922872613', time=2024-04-17T07:23:30.628139+0000, chain.objs.size()=1 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rsCMaepkeygA5nAepY0550iDX79S9BQ_0 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.43978960282255733', time=2024-04-17T07:23:31.156207+0000, chain.objs.size()=1 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1uhP6Y-qa1qHGlwxI4zVN5McZcgQi8z_0 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7958683157657779116', time=2024-04-17T07:23:32.165681+0000, chain.objs.size()=1 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XAr6LZwquj8-IqfQhBN3lXBbtElF1hJ_0 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3884441460393317310', time=2024-04-17T07:23:32.545662+0000, chain.objs.size()=1 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.c-X8FJCnQt3DacIfqk6CQn4YiYiQVJQ_0 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11612218027026353774', time=2024-04-17T07:23:34.867648+0000, chain.objs.size()=1 2024-04-17T05:46:02.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Xw_2uwovbN5PKh3hSaFikemvQwlhrWs_0 2024-04-17T05:46:02.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.710995525920528105', time=2024-04-17T07:23:35.008236+0000, chain.objs.size()=1 2024-04-17T05:46:02.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Pf2SQ2mtmEw-YBtpqBlWmgXXa2eOguJ_0 2024-04-17T05:46:02.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15075631189443472102', time=2024-04-17T07:23:36.237781+0000, chain.objs.size()=1 2024-04-17T05:46:02.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tci3vXmg82JM-6_amhPfoZg9F_0leJK_0 2024-04-17T05:46:02.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3029753191613033995', time=2024-04-17T07:23:36.877985+0000, chain.objs.size()=1 2024-04-17T05:46:02.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yq9pbI-_SUSD5sWUwkVGPPxGwhA6X9H_0 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3424028723519313212', time=2024-04-17T07:23:36.905295+0000, chain.objs.size()=1 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3ihe0P1lqT0IfHwJe3Ed2exVIbkLM95_0 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6340885593648261087', time=2024-04-17T07:23:37.254567+0000, chain.objs.size()=1 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sHzOWADytkEm3H_7TqfF7AXbXh5xB3d_0 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.936938670015464299', time=2024-04-17T07:23:38.513472+0000, chain.objs.size()=1 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TdD-YQjqm4wCygLcxA0EL_MlziOrOQ3_0 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15693406935501717502', time=2024-04-17T07:23:38.660642+0000, chain.objs.size()=1 2024-04-17T05:46:02.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dEyftpzApynLT52M1xL4BcZx1yz809X_0 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14762795428891326823', time=2024-04-17T07:23:40.889536+0000, chain.objs.size()=1 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qNMxGV2oasqthB2LgKpLLdo1Bq1SG0f_0 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13639625090080183843', time=2024-04-17T07:23:41.234173+0000, chain.objs.size()=1 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6-EgV9OhfyuZ_1Ec51_VLzJOutxUyMF_0 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4507032192380052373', time=2024-04-17T07:23:48.329329+0000, chain.objs.size()=1 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.321__shadow_.-SVKxZmiRvzsg5AcksLpPMnIHRQltbo_0 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9850378828643467537', time=2024-04-17T07:25:26.119414+0000, chain.objs.size()=1 2024-04-17T05:46:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.429__shadow_.tyekzRdohAHCq1fkaHPVmN011cVZxMB_0 2024-04-17T05:46:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17388194364308924324', time=2024-04-17T07:25:44.028226+0000, chain.objs.size()=1 2024-04-17T05:46:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.356+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.445__shadow_.ljVJLVyz601GGfzHjbHnJNctHrFEUbj_0 2024-04-17T05:46:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.360+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15814739759760945075', time=2024-04-17T07:25:45.712107+0000, chain.objs.size()=1 2024-04-17T05:46:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.360+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.446__shadow_.cVH_xiciHHblFEPc40WSJOG_KzKWda5_0 2024-04-17T05:46:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.360+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10294585820570476238', time=2024-04-17T07:25:55.808980+0000, chain.objs.size()=1 2024-04-17T05:46:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.360+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.452__shadow_.KdJBQ_q4dFcGEKS7OEiQAIWm_1dvcUw_0 2024-04-17T05:46:02.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/47207 2024-04-17T05:46:02.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T05:46:02.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13019764099606936465', time=2024-04-17T07:26:38.851026+0000, chain.objs.size()=1 2024-04-17T05:46:02.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.490__shadow_.5GS2FmjrPZ43B2Fv3pYCq33y54MwvgL_0 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.869496248632329236', time=2024-04-17T07:29:33.621841+0000, chain.objs.size()=3 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~oWBZdFQ3aowG9e5xBfhCLNvgZbMtFxh.1 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~oWBZdFQ3aowG9e5xBfhCLNvgZbMtFxh.2 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~oWBZdFQ3aowG9e5xBfhCLNvgZbMtFxh.3 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11198961056444914987', time=2024-04-17T07:31:35.494011+0000, chain.objs.size()=1 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:LkKZlseYyUG8FdeAl.WYyApX2UvLcJG_.9PStn3BXa2xQFKTqI76aze0xLcmmq6W_0 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7193543080772547763', time=2024-04-17T07:31:48.392205+0000, chain.objs.size()=1 2024-04-17T05:46:02.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:vEWkomsDZZRLrhAbnRr.gt-w4hb.f.L_.qabcRJRieVeilOAoCL9dBgXu8GNtshF_0 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.704562774213181381', time=2024-04-17T07:33:46.246240+0000, chain.objs.size()=1 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:7JfFNGemUkAiaeZFNK6k1eqQaUCTkzh_.gXqbYJDoNNu5YEbLINA3ELDskHK1wkQ_0 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15452855320789320446', time=2024-04-17T07:37:47.441990+0000, chain.objs.size()=1 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.657__shadow_.y2Arh5ecFQi4SfoWIICXsqeC2tn1A6b_0 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.77659461881485970', time=2024-04-17T07:38:06.937467+0000, chain.objs.size()=1 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.679__shadow:4gZotOKm-fU2gL3rx5GN.FOh8tJlu7f_.dx8ARhtMyFbxRUSwvWyLSiEEXxJvpct_0 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.704__shadow_.NcrAv0bkypiqbnf2Tn4bStwPPWVUk9m_1_9d657ebf6a88861c4c4a15cda153844d', time=2024-04-17T07:38:55.431492+0000, chain.objs.size()=1 2024-04-17T05:46:02.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.704__shadow_.NcrAv0bkypiqbnf2Tn4bStwPPWVUk9m_0 2024-04-17T05:46:02.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3599960864176723996', time=2024-04-17T07:39:50.169475+0000, chain.objs.size()=1 2024-04-17T05:46:02.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.724__shadow_.kVMGujrt1VHUGHTpvyjthhFMhfwosrG_0 2024-04-17T05:46:02.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-17T05:46:02.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44869' 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1343817132603515495', time=2024-04-17T07:10:20.088250+0000, chain.objs.size()=3 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-LUKEWARM.2~2t5tYxpgymWgADi0-f_iYqnB0M7Cz9r.1 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-LUKEWARM.2~2t5tYxpgymWgADi0-f_iYqnB0M7Cz9r.2 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-LUKEWARM.2~2t5tYxpgymWgADi0-f_iYqnB0M7Cz9r.3 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11800337836202923121', time=2024-04-17T07:14:33.071222+0000, chain.objs.size()=1 2024-04-17T05:46:02.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.224__shadow_.DAEi-1Dmyyp6w5eER3zdeqUwIezup28_0 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6431652870010537501', time=2024-04-17T07:14:46.586372+0000, chain.objs.size()=1 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.232__shadow_.WojGSokL98ehMnOonP2w-HRsitbAFK3_0 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18160861679392443793', time=2024-04-17T07:15:07.477047+0000, chain.objs.size()=1 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.244__shadow_.DnzYTadHTUig-g7vEOFG1sZA09LeFzi_0 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1733359427481066206', time=2024-04-17T07:17:02.265999+0000, chain.objs.size()=1 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hA0PBeKzMED-UrAe-dnjClf-qUQ4_VO_0 2024-04-17T05:46:02.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15475859803060864639', time=2024-04-17T07:17:02.861146+0000, chain.objs.size()=1 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-OiL_eNMurH7sy34B2iNFDGHWOpQTZt_0 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16597641926711572633', time=2024-04-17T07:17:03.485599+0000, chain.objs.size()=1 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1ITmrPFAsQOu-hH25S6vl-UVEmZ4hVr_0 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.97360274553425998', time=2024-04-17T07:17:06.775270+0000, chain.objs.size()=1 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CIh9t2OY3jK3Y0dpUjR9vnwNsyzP1Dx_0 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13568337337205668072', time=2024-04-17T07:17:07.377544+0000, chain.objs.size()=1 2024-04-17T05:46:02.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aBJLz8VeslO94uFEze4Bgi0BGJvV8gy_0 2024-04-17T05:46:02.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16723767894672165600', time=2024-04-17T07:17:07.392973+0000, chain.objs.size()=1 2024-04-17T05:46:02.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dVMNWZ3of6_5Hhl5cD1Tfw1xm1BC-ok_0 2024-04-17T05:46:02.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11333161946086859631', time=2024-04-17T07:17:08.693080+0000, chain.objs.size()=1 2024-04-17T05:46:02.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8enf7GZ6x790px91t7CxOxfjI3HMGbG_0 2024-04-17T05:46:02.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7982653939970724678', time=2024-04-17T07:17:11.384580+0000, chain.objs.size()=1 2024-04-17T05:46:02.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OZETA1EfGg7tNZMVBCZxDImR0odyqK7_0 2024-04-17T05:46:02.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13986838186063618742', time=2024-04-17T07:17:11.883761+0000, chain.objs.size()=1 2024-04-17T05:46:02.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6YqGPcG-vRrnS266dsx6PJ5oXh1Rx4N_0 2024-04-17T05:46:02.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6771579473333967848', time=2024-04-17T07:17:12.998522+0000, chain.objs.size()=1 2024-04-17T05:46:02.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ygBJ_TXMoLgNmfwYaJc1sP80fg1ozNy_0 2024-04-17T05:46:02.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.32442244247961401', time=2024-04-17T07:17:14.564313+0000, chain.objs.size()=1 2024-04-17T05:46:02.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RFG-exuDPHn8uGFc9EtY3ge_BDp4Jri_0 2024-04-17T05:46:02.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15449778577562618403', time=2024-04-17T07:17:15.893213+0000, chain.objs.size()=1 2024-04-17T05:46:02.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NAtPpTAl4X8bvJ03eztsC0YC_nmotpc_0 2024-04-17T05:46:02.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9702689497245391405', time=2024-04-17T07:17:17.066586+0000, chain.objs.size()=1 2024-04-17T05:46:02.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qMf3vqy8bVc-qNAuHK44jWixn2Z-4Yx_0 2024-04-17T05:46:02.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6296596283043039153', time=2024-04-17T07:17:18.362321+0000, chain.objs.size()=1 2024-04-17T05:46:02.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BHli3b7k_XYGheeIVLAg6LX-Fmj5njp_0 2024-04-17T05:46:02.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1682814046473267217', time=2024-04-17T07:17:18.425187+0000, chain.objs.size()=1 2024-04-17T05:46:02.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JCb86K43bN0zR_LtD-frsBuIscbmdfk_0 2024-04-17T05:46:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16411899570189161327', time=2024-04-17T07:17:18.498897+0000, chain.objs.size()=1 2024-04-17T05:46:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ckNhEQvOgrloy55H249ofJn9ui_CRKn_0 2024-04-17T05:46:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5585451254583474547', time=2024-04-17T07:17:19.198275+0000, chain.objs.size()=1 2024-04-17T05:46:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.T_5Ri6B9v6zieRwbrjf9cSgJ9-AnqN9_0 2024-04-17T05:46:02.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3410741133185583520', time=2024-04-17T07:17:22.195769+0000, chain.objs.size()=1 2024-04-17T05:46:02.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3tM9axWRKnLFXuireD8dUnrE0O4bZ9W_0 2024-04-17T05:46:02.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8939415170901327740', time=2024-04-17T07:17:22.575382+0000, chain.objs.size()=1 2024-04-17T05:46:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iudEI9IEwzVtzw6ztO4kOORfJrUohmn_0 2024-04-17T05:46:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13440869041874336630', time=2024-04-17T07:17:23.369470+0000, chain.objs.size()=1 2024-04-17T05:46:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PKKnFqbK6m1u-PmrGoiEKeH7SW0AKV3_0 2024-04-17T05:46:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1091307150880363016', time=2024-04-17T07:17:24.226608+0000, chain.objs.size()=1 2024-04-17T05:46:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.peWnx8NIhPsDrBtzCFeD85_PJjo2_eL_0 2024-04-17T05:46:02.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12114135687695581889', time=2024-04-17T07:17:26.031046+0000, chain.objs.size()=1 2024-04-17T05:46:02.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.26wUfyPwrnbTagB3UClzJEeNGgL-UT-_0 2024-04-17T05:46:02.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2134210962259893159', time=2024-04-17T07:17:26.358647+0000, chain.objs.size()=1 2024-04-17T05:46:02.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wI6_aphXcXk2lu-WzjGBwcyv0jdsgXB_0 2024-04-17T05:46:02.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.406736888620687171', time=2024-04-17T07:17:26.472794+0000, chain.objs.size()=1 2024-04-17T05:46:02.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BB_jIDSuVKgAv_E1k8Q3padu4r6R7_F_0 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5987295119256710488', time=2024-04-17T07:17:26.574358+0000, chain.objs.size()=1 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FWxUrz1mX1hquFSD63e7NCRHCi7Phus_0 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15584112816245208888', time=2024-04-17T07:17:27.175190+0000, chain.objs.size()=1 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Pqwe5MvL3qFux3WZ1qW02O89MPhQ-8a_0 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1377933585384723350', time=2024-04-17T07:17:32.375235+0000, chain.objs.size()=1 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Z6-r9FykHud-WVpPGazelreKhyjwmZv_0 2024-04-17T05:46:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.751043787273378668', time=2024-04-17T07:17:45.916006+0000, chain.objs.size()=1 2024-04-17T05:46:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.261__shadow_.Tcb72RHu8yD28La38_fmaLUo82BJQGa_0 2024-04-17T05:46:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11849335049774046326', time=2024-04-17T07:18:04.986755+0000, chain.objs.size()=1 2024-04-17T05:46:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.273__shadow_.mbGo4Ev2ZClfVO_PhmSIV8McjdZm-Jf_0 2024-04-17T05:46:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13901926559316903038', time=2024-04-17T07:18:06.585331+0000, chain.objs.size()=1 2024-04-17T05:46:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.274__shadow_.x_fxDebgCNiXVxiWkKxcRrA311WYgnF_0 2024-04-17T05:46:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4760288355348173722', time=2024-04-17T07:18:10.029269+0000, chain.objs.size()=1 2024-04-17T05:46:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.276__shadow_.EPMOEKEmtpJlPeFYcJK20pg51uNHgeW_0 2024-04-17T05:46:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2026928273532816007', time=2024-04-17T07:18:11.646378+0000, chain.objs.size()=1 2024-04-17T05:46:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.277__shadow_.DHlHTon0Jaqus8dkMaEiglu3KVKoODo_0 2024-04-17T05:46:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4072246059453378511', time=2024-04-17T07:18:23.238918+0000, chain.objs.size()=1 2024-04-17T05:46:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.tm7jwRb23XxCrUrjFf5DoxVVPGGtvCp_0 2024-04-17T05:46:02.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14679335398633399140', time=2024-04-17T07:18:31.485906+0000, chain.objs.size()=1 2024-04-17T05:46:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.285__shadow_.z_QbQajTxzhPSW72QcpP2QwaTM5FizZ_0 2024-04-17T05:46:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15639714862882461354', time=2024-04-17T07:18:41.701656+0000, chain.objs.size()=1 2024-04-17T05:46:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.291__shadow_.IwiFcpsqCcOKkLJwirEFZYdq54ftTV8_0 2024-04-17T05:46:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11473696714367794426', time=2024-04-17T07:18:41.713936+0000, chain.objs.size()=1 2024-04-17T05:46:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.291__shadow_.7kpbeLWf0JtsBufcTVcIwaohycibn1v_0 2024-04-17T05:46:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6407171270354821835', time=2024-04-17T07:18:52.953930+0000, chain.objs.size()=1 2024-04-17T05:46:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.297__shadow:6ebT8VCRMZYaX6FBmM0JjF9LOjVsQgk_.7eV-LwgjHYjvUnNtHsKf6QE0nmdbOQu_0 2024-04-17T05:46:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18351144369420375168', time=2024-04-17T07:21:01.367430+0000, chain.objs.size()=1 2024-04-17T05:46:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.G36QIml-rwfC0IJ13l4SRwJbF0V3R2g_0 2024-04-17T05:46:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1312857750303454148', time=2024-04-17T07:21:01.867915+0000, chain.objs.size()=1 2024-04-17T05:46:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ESB9T-pMnOQTBp2hagXeGBRoPb2-rI7_0 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2091803959424348350', time=2024-04-17T07:21:02.287931+0000, chain.objs.size()=1 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.useNRt1iut52JegDoEZGku8dGrs0JTk_0 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5964287737859554745', time=2024-04-17T07:21:02.499103+0000, chain.objs.size()=1 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.J6SwULKmpAZyHTA6OUo1AvXBr6fmns0_0 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10253254730494454824', time=2024-04-17T07:21:04.006181+0000, chain.objs.size()=1 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.z-6Liufwr40vXJSxLDlfw8aABwiFhRB_0 2024-04-17T05:46:02.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11500890643603324853', time=2024-04-17T07:21:06.766399+0000, chain.objs.size()=1 2024-04-17T05:46:02.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PPxtjPEcG8wmB45QsHAs6rGbmYx-6K0_0 2024-04-17T05:46:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10965689743148699677', time=2024-04-17T07:21:07.912711+0000, chain.objs.size()=1 2024-04-17T05:46:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Hd8uarMZ9ERW9U8nI_RzmLM7niet0at_0 2024-04-17T05:46:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.959331183377884376', time=2024-04-17T07:21:08.892334+0000, chain.objs.size()=1 2024-04-17T05:46:02.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nwUeztT7fHZHvSwQwTmPLc_j6FT52OX_0 2024-04-17T05:46:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16772713531724909104', time=2024-04-17T07:21:09.954342+0000, chain.objs.size()=1 2024-04-17T05:46:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IJhWHy_xsX8PFP-IrSHu0NgyK1uLEkx_0 2024-04-17T05:46:02.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10421872534001004265', time=2024-04-17T07:21:09.977483+0000, chain.objs.size()=1 2024-04-17T05:46:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DQ3qqE5LkY3CqBKXLGbXzyByFJZ6srS_0 2024-04-17T05:46:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8324475555676512098', time=2024-04-17T07:21:10.501363+0000, chain.objs.size()=1 2024-04-17T05:46:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.z_BATv_YFTRj2R4x15DKWuJUqsuCaB2_0 2024-04-17T05:46:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2672572783663993878', time=2024-04-17T07:21:10.923006+0000, chain.objs.size()=1 2024-04-17T05:46:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8yJfgXsSzCEvUfWjxti3mcJe7uByNZ5_0 2024-04-17T05:46:02.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4014663368853188791', time=2024-04-17T07:21:12.614338+0000, chain.objs.size()=1 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CXmMkXbS-hVEOerX-PGLbylaSTm6UN-_0 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.956910394164584795', time=2024-04-17T07:21:12.724912+0000, chain.objs.size()=1 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MOE7nVTG1Ft4eL9tx-dgu8lH1o4smKu_0 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15496759498663284554', time=2024-04-17T07:21:13.837748+0000, chain.objs.size()=1 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Y62NJ4FnsgoVFz2nhK695RIkBgs0rrb_0 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2593402192286161821', time=2024-04-17T07:21:16.521972+0000, chain.objs.size()=1 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZZgr5E1gVa5P_ETXwDBoM1hBOkC-gJ1_0 2024-04-17T05:46:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2671401957826932436', time=2024-04-17T07:21:17.154522+0000, chain.objs.size()=1 2024-04-17T05:46:02.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fzQ_qi00nGNtvr9gsJIaqdbYL-lSP3I_0 2024-04-17T05:46:02.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14817730532755358826', time=2024-04-17T07:21:17.948057+0000, chain.objs.size()=1 2024-04-17T05:46:02.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CWOPksbgd7iTMmSgCIlp7xMQiJQgqIK_0 2024-04-17T05:46:02.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15727677344737918186', time=2024-04-17T07:21:18.726260+0000, chain.objs.size()=1 2024-04-17T05:46:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Wb9mWFMMpRAGMoXNZ-JNt6L0302dEkC_0 2024-04-17T05:46:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18239002084737881378', time=2024-04-17T07:21:19.577075+0000, chain.objs.size()=1 2024-04-17T05:46:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sk11dlmfEE00VO1kQbOtiLRk8BLjtPt_0 2024-04-17T05:46:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6191734110796687245', time=2024-04-17T07:21:19.834236+0000, chain.objs.size()=1 2024-04-17T05:46:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iWJd7-1HppUUOMWZvlg0KMOykVbfqT4_0 2024-04-17T05:46:02.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13981391590186861825', time=2024-04-17T07:21:20.287634+0000, chain.objs.size()=1 2024-04-17T05:46:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.v3uV4GJ8I8eMLl2ZjbX6xe-btr7cHx2_0 2024-04-17T05:46:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13198220118089409448', time=2024-04-17T07:21:21.216457+0000, chain.objs.size()=1 2024-04-17T05:46:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._zk6QBPcYlpU9ndfncEQbDwQEpGdyDk_0 2024-04-17T05:46:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17345556133657070034', time=2024-04-17T07:21:21.548687+0000, chain.objs.size()=1 2024-04-17T05:46:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Fpq_QqrT6obd3PxLm9mNMNk4mAyjDUC_0 2024-04-17T05:46:02.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15164926966282745285', time=2024-04-17T07:21:22.175434+0000, chain.objs.size()=1 2024-04-17T05:46:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.80zqP2pkm26YSWssXkvGiEWsvLPachy_0 2024-04-17T05:46:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5615644217594314759', time=2024-04-17T07:21:22.177510+0000, chain.objs.size()=1 2024-04-17T05:46:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0qa1wAxmGxOroSQ-9vqbOwOOBfk1gKr_0 2024-04-17T05:46:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1293665392723785924', time=2024-04-17T07:21:22.564919+0000, chain.objs.size()=1 2024-04-17T05:46:02.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iV0wqvi_xM5pzNMjRYZthkU6o5Cr4Qo_0 2024-04-17T05:46:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16043703492093254621', time=2024-04-17T07:21:23.294223+0000, chain.objs.size()=1 2024-04-17T05:46:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ikn9sEzNdAE7qdf03CmfHkQQccjsrzx_0 2024-04-17T05:46:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17899362793352591060', time=2024-04-17T07:21:24.200052+0000, chain.objs.size()=1 2024-04-17T05:46:02.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QVkNytd8ZsZKoZmIItqfdfX3RhxFN4L_0 2024-04-17T05:46:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11337557754959410059', time=2024-04-17T07:21:25.975452+0000, chain.objs.size()=1 2024-04-17T05:46:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RHcG7I26TJrXbVxr8pWuRj7gh5uiac4_0 2024-04-17T05:46:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15115397758866884943', time=2024-04-17T07:21:26.267075+0000, chain.objs.size()=1 2024-04-17T05:46:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SiKZt4vVWoOnpOMbaC-IzzDqzpBQujj_0 2024-04-17T05:46:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1083271133317217887', time=2024-04-17T07:21:26.989018+0000, chain.objs.size()=1 2024-04-17T05:46:02.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lxg55OIbFd_Y7pApFQjc2X5XG5Pg7_H_0 2024-04-17T05:46:02.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2664750579835968047', time=2024-04-17T07:21:27.007184+0000, chain.objs.size()=1 2024-04-17T05:46:02.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3UmUWcCm2e63XivjaXQIQ2W0lBvZRIq_0 2024-04-17T05:46:02.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1873727403696783785', time=2024-04-17T07:21:28.223431+0000, chain.objs.size()=1 2024-04-17T05:46:02.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wPv062QEDPgdulZCXgWpVTSs4cI1jQQ_0 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3004334457119585959', time=2024-04-17T07:23:14.922651+0000, chain.objs.size()=1 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.njeEysjDUT7aGZbNmurTl9bclBiH8Oc_0 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6531106811475742143', time=2024-04-17T07:23:15.628761+0000, chain.objs.size()=1 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.LgEYs8kju5Otg3xXjut0dczI8c0R18Z_0 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7988362217367698263', time=2024-04-17T07:23:15.949725+0000, chain.objs.size()=1 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xABaj6UKgyEqfp13qvsCkqE43b1lufO_0 2024-04-17T05:46:02.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11267043727170656529', time=2024-04-17T07:23:18.164485+0000, chain.objs.size()=1 2024-04-17T05:46:02.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZsUTTS8biR35mwWW4oBEsnMO_OQk81X_0 2024-04-17T05:46:02.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12421725611450835762', time=2024-04-17T07:23:18.843650+0000, chain.objs.size()=1 2024-04-17T05:46:02.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.v1WOlm_QeU9LWWuokUrlBQSjwv8kU-K_0 2024-04-17T05:46:02.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3862591143650303379', time=2024-04-17T07:23:21.901452+0000, chain.objs.size()=1 2024-04-17T05:46:02.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Bntr4FzWhjShkMOjivpY-9r0vWNNS0m_0 2024-04-17T05:46:02.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16342014828753184905', time=2024-04-17T07:23:23.045172+0000, chain.objs.size()=1 2024-04-17T05:46:02.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.FC3S4qAekK738myi8BOP1B78P28DkIa_0 2024-04-17T05:46:02.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3771538825257899369', time=2024-04-17T07:23:23.384599+0000, chain.objs.size()=1 2024-04-17T05:46:02.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tqNC07UeCH4G1f3kgYJ45yiUlnYqvog_0 2024-04-17T05:46:02.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16754519697430922708', time=2024-04-17T07:23:23.817196+0000, chain.objs.size()=1 2024-04-17T05:46:02.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.416+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DTSTr4EjCOezXwE_Zg_L17wTcUhSiLM_0 2024-04-17T05:46:02.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1369188310699431846', time=2024-04-17T07:23:23.819416+0000, chain.objs.size()=1 2024-04-17T05:46:02.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2ZGf0M0xfttmp_KXc40dkSkfS_wsg_g_0 2024-04-17T05:46:02.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7838070690950556281', time=2024-04-17T07:23:24.767471+0000, chain.objs.size()=1 2024-04-17T05:46:02.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EiI16RAVqWegngkHVfoIBn6hGw3WSqq_0 2024-04-17T05:46:02.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15004066482025517927', time=2024-04-17T07:23:24.769411+0000, chain.objs.size()=1 2024-04-17T05:46:02.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vBnhpu5dJACH2XFsoyovz0X0QOi-9Gy_0 2024-04-17T05:46:02.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5885901276207179714', time=2024-04-17T07:23:25.570829+0000, chain.objs.size()=1 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KyYtkao9fh_N2iRwUcYpzoJFTuWbS5v_0 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17190484634158840907', time=2024-04-17T07:23:25.696532+0000, chain.objs.size()=1 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1CR4AnxjW7UIIzuJ9QpDidJFusDe1Bb_0 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12066721929288578164', time=2024-04-17T07:23:26.231939+0000, chain.objs.size()=1 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._Eg2twMeiUsfC_OGGwxdBT9MOekccMP_0 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10576137466689060228', time=2024-04-17T07:23:28.543858+0000, chain.objs.size()=1 2024-04-17T05:46:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kIKbi43W8uhTkekWjS0-RoRxnFvo5U9_0 2024-04-17T05:46:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11660821492094401717', time=2024-04-17T07:23:29.107054+0000, chain.objs.size()=1 2024-04-17T05:46:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UC1C7QxX4acXTSQ-4qF8FYLFebeAkA5_0 2024-04-17T05:46:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4071419155813898631', time=2024-04-17T07:23:30.822057+0000, chain.objs.size()=1 2024-04-17T05:46:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pOXEo_ivxdK29MQ9rgzxBPJubxx1x7F_0 2024-04-17T05:46:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2991260240283365894', time=2024-04-17T07:23:31.073185+0000, chain.objs.size()=1 2024-04-17T05:46:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vAGmbDIzodAhNBqRlYIVAoc6l7-ICWz_0 2024-04-17T05:46:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15521686833513058512', time=2024-04-17T07:23:33.145822+0000, chain.objs.size()=1 2024-04-17T05:46:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RCBF5RfJQAnjgGE5ZJwPWqtnGTWfV32_0 2024-04-17T05:46:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16707986208049672719', time=2024-04-17T07:23:34.020398+0000, chain.objs.size()=1 2024-04-17T05:46:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2xJEaDuczbqqvzJZkvh8Sty5cSe9qY4_0 2024-04-17T05:46:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9358709950676786122', time=2024-04-17T07:23:35.886125+0000, chain.objs.size()=1 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ikQW4hTlC9hLEWWRbYKbZ4UdSTcFf57_0 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6425360431687805697', time=2024-04-17T07:23:37.014965+0000, chain.objs.size()=1 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kG1HA4QwAXzzZ3H2ZgGR3VTfxrLuDoq_0 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9672528358043764200', time=2024-04-17T07:23:37.880527+0000, chain.objs.size()=1 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PLQ1eWEUq6dgtmecOa7yoaFqvGXvYrU_0 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6483491675508742443', time=2024-04-17T07:23:38.220589+0000, chain.objs.size()=1 2024-04-17T05:46:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MFrXbHTGbUxtNCOPZckx1jNAIGaIZwl_0 2024-04-17T05:46:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/44869 2024-04-17T05:46:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2024-04-17T05:46:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9627538678820754052', time=2024-04-17T07:23:55.683556+0000, chain.objs.size()=1 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.329__shadow_.Pz_laQ6NXCmt1qAG_GEAgISpzxTnge8_0 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12169913112104845859', time=2024-04-17T07:24:31.941577+0000, chain.objs.size()=1 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.371__shadow_.TbuO0ZqzhEpBSzZM5NJv1ePYQxIsD1E_0 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7231973726671572975', time=2024-04-17T07:24:41.229564+0000, chain.objs.size()=1 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.380__shadow_.O37fFLyadQ89Xpq1f6D3sZxflTflxgL_0 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9512573826917691652', time=2024-04-17T07:25:44.015425+0000, chain.objs.size()=1 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.445__shadow_.xbhrDmO8RjWZ2w4jpMzZ5vVlVXOB46x_0 2024-04-17T05:46:02.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.338365860124118601', time=2024-04-17T07:26:01.139152+0000, chain.objs.size()=1 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.455__shadow_.F-lrOto7Cb1NDjZjn9zA7BuDF5EdryX_0 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12726588243990618823', time=2024-04-17T07:30:50.660323+0000, chain.objs.size()=1 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.541__shadow_.hpHQvEsfT0GitrLUqah3qmJcb3Ti4Ae_0 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15262830977213402230', time=2024-04-17T07:31:28.102471+0000, chain.objs.size()=2 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.552__shadow_.SjcPwqanf5os5G2nFHWkBonG1A7aoM8_0 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.552__shadow_.SjcPwqanf5os5G2nFHWkBonG1A7aoM8_1 2024-04-17T05:46:02.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8009345856615325495', time=2024-04-17T07:31:40.976636+0000, chain.objs.size()=1 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:2ijbj-YRBuNpz76Q3qdVClr5s2WpyUL_.DdJ8qSn5uZVHL2_Sy09qyWILv4b8qwn_0 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14568412578525726789', time=2024-04-17T07:31:42.051132+0000, chain.objs.size()=1 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:J13GfdGAGL..k50oxiKdQBeD0z8Z-3J_.4buOMuEnNdtTOP3rG_NOXaqBMixa-KR_0 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8502367513837450136', time=2024-04-17T07:31:50.984289+0000, chain.objs.size()=1 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:46SXzFA.J8MscKXlggtkzHto21tN51o_.Ig2vfhGe16flNvet_FojGlA3nI-H-nt_0 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17180208507333042964', time=2024-04-17T07:33:46.180125+0000, chain.objs.size()=1 2024-04-17T05:46:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:PrGqiFC2OTV9mYWeLemLThrYEquEFuF_.zia7JkIiZxneamP-U7LjmsQ2_BWC8MD_0 2024-04-17T05:46:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5745145087744537901', time=2024-04-17T07:36:37.499567+0000, chain.objs.size()=1 2024-04-17T05:46:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.623__shadow_.jWhqwF8oiPegJYXBznzL7koBPasdXsp_0 2024-04-17T05:46:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10811360308137054212', time=2024-04-17T07:36:54.696696+0000, chain.objs.size()=1 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.633__shadow_.jPIf-kZ_YxYp_NxUR4QeVOOAW_KMcO6_0 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17846692950276534213', time=2024-04-17T07:38:04.906265+0000, chain.objs.size()=1 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.677__shadow:T.p1MJVR0z7ihisI6pCF6rBubj5UT3K_.ldUQYS0qbQ4xLLNabl34XVRspFL9SAc_0 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17127681951929711636', time=2024-04-17T07:38:24.639737+0000, chain.objs.size()=1 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.693__shadow:jEuqupqKbd5hGLWGlq9w11vrgtoo2SH_.kzIC-AaV3DTPt8ySnDbQ5aqYP25xLAj_0 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5954277808747409323', time=2024-04-17T07:38:28.408977+0000, chain.objs.size()=1 2024-04-17T05:46:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.696__multipart_file1.2~7kjYYElb-Qa91ijChu60MdYyZHoVsUA.1 2024-04-17T05:46:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.703__shadow_.07ZfuOPrCcLoDAnzjKXxI9e6z1x2Ysd_1_f029a993024c8e076bcdfe6dcfac3624', time=2024-04-17T07:38:54.339575+0000, chain.objs.size()=1 2024-04-17T05:46:02.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.436+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.703__shadow_.07ZfuOPrCcLoDAnzjKXxI9e6z1x2Ysd_0 2024-04-17T05:46:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-17T05:46:02.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46669' 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_v8NVKzICNVvsNKUS4LYtZGmQBEVa5eu', time=2024-04-17T07:10:26.108115+0000, chain.objs.size()=2 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.cFQSu7emHIGuL0rxN7WYGHY6-e8C_Md_0 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.cFQSu7emHIGuL0rxN7WYGHY6-e8C_Md_1 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1728846543326319577', time=2024-04-17T07:14:31.411494+0000, chain.objs.size()=1 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.223__shadow_.xNs8tGz3DeDnLrw6bwPwQRCofAdLyVw_0 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13036476124228244777', time=2024-04-17T07:14:54.524623+0000, chain.objs.size()=1 2024-04-17T05:46:02.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.236__shadow_.Se3LlNi_VMUnhvX_Jk2l-hmH48P5Ei6_0 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15495412942864170928', time=2024-04-17T07:14:54.526905+0000, chain.objs.size()=1 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.236__shadow_.04AuvoflYY8o6jDmD7SntsQMieyQeQ0_0 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9098199007187866479', time=2024-04-17T07:15:04.260596+0000, chain.objs.size()=1 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.242__shadow_.3puXTWfyDaU_q5gbFqgJ-HJMEqap3in_0 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8941926335523691373', time=2024-04-17T07:17:01.544890+0000, chain.objs.size()=1 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sk6GIr20LDxZdCOlYdm-_BA91isPoPb_0 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13560677054911832053', time=2024-04-17T07:17:01.600016+0000, chain.objs.size()=1 2024-04-17T05:46:02.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.k2LJe_OHsNDI1mx2CdCHKiDbwj0xsIu_0 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13151131319450970489', time=2024-04-17T07:17:03.958263+0000, chain.objs.size()=1 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zxz-mQahw_wFD9H-W6lo2xTjnTR42R3_0 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16253765823092271101', time=2024-04-17T07:17:04.063474+0000, chain.objs.size()=1 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rXwMF_n05ZLki84e75afIC06dUElRPB_0 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16419287912353133503', time=2024-04-17T07:17:05.491021+0000, chain.objs.size()=1 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Trj8VS4lYI1Btb7R7nVpkTGivV0-Urq_0 2024-04-17T05:46:02.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8066107620069796034', time=2024-04-17T07:17:05.702585+0000, chain.objs.size()=1 2024-04-17T05:46:02.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tLGtnp0y2SbdO7nWM58ApAkDhk7yg-D_0 2024-04-17T05:46:02.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11743114263513520391', time=2024-04-17T07:17:06.426954+0000, chain.objs.size()=1 2024-04-17T05:46:02.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VRrITUjihkfNkGCv2-cdXaXpedAeLiJ_0 2024-04-17T05:46:02.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3484262430413884090', time=2024-04-17T07:17:07.711425+0000, chain.objs.size()=1 2024-04-17T05:46:02.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZRxADkCsg2lefGHtvshgmm366SjF6sy_0 2024-04-17T05:46:02.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3596451723881294050', time=2024-04-17T07:17:12.333023+0000, chain.objs.size()=1 2024-04-17T05:46:02.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.l_PmaycFAbyas4bJPE2MHEpM7ufwjDv_0 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3775044063545006214', time=2024-04-17T07:17:12.914888+0000, chain.objs.size()=1 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HLX-AWZs9qjOWUiXV8E5TIOiqUeH0LO_0 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13870951772996092322', time=2024-04-17T07:17:13.894428+0000, chain.objs.size()=1 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bBjnGPuCCooYKoUjo8LMEyd9xdk9Zin_0 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1720014464525217360', time=2024-04-17T07:17:14.225004+0000, chain.objs.size()=1 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KxGEth8Q9M8VslDHvxd7FaA28UnOqjs_0 2024-04-17T05:46:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16722363568963349015', time=2024-04-17T07:17:14.388228+0000, chain.objs.size()=1 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1iGcQprbgx_NutGjLN0EdCNkbyonYp7_0 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6969153977528009502', time=2024-04-17T07:17:15.058001+0000, chain.objs.size()=1 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yHJ_0wwJJi4vStHSRfu2M78BQSOC3Uq_0 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6120259555447750010', time=2024-04-17T07:17:15.639682+0000, chain.objs.size()=1 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c_FW8LJPbctplxbrq37wnSRGRSMOEFV_0 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17590653826438715740', time=2024-04-17T07:17:15.971369+0000, chain.objs.size()=1 2024-04-17T05:46:02.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BURi-IN5tOYs4aKcSa58ojWeS1ffUbh_0 2024-04-17T05:46:02.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9200812724984788409', time=2024-04-17T07:17:17.107873+0000, chain.objs.size()=1 2024-04-17T05:46:02.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rNz2fYDdyHqQX-FqfVBxAtMhWf2Bc6O_0 2024-04-17T05:46:02.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5519815521036565120', time=2024-04-17T07:17:17.109523+0000, chain.objs.size()=1 2024-04-17T05:46:02.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CX_vB5Op8qnl_j8bPbdV9-P9kLn7FHe_0 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17842336380532277807', time=2024-04-17T07:17:18.512392+0000, chain.objs.size()=1 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.X6wZ7YAT4aFMdk0V9FhrxP2FZMOXw3Z_0 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14472615244064592762', time=2024-04-17T07:17:19.583282+0000, chain.objs.size()=1 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.v91V-r20cORRsfwGlY5ZSkxTODW2UHE_0 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1282369898972892529', time=2024-04-17T07:17:20.033641+0000, chain.objs.size()=1 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EwIpyGfGL7ennz7WxMu81340BwQuAJ1_0 2024-04-17T05:46:02.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.804284631392417750', time=2024-04-17T07:17:20.197959+0000, chain.objs.size()=1 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bUNCQuq5mLWe4NiENq2BjmUT7qPGwrs_0 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12704137415253397531', time=2024-04-17T07:17:20.199313+0000, chain.objs.size()=1 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.C6UC0cl1kC1ByGYmmTwqle4BUhDGaPk_0 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8747199107690430547', time=2024-04-17T07:17:22.389796+0000, chain.objs.size()=1 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lH47rkwCD_4bUjoT5Cs-bmybEPgpQTM_0 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13134079137200391530', time=2024-04-17T07:17:24.059786+0000, chain.objs.size()=1 2024-04-17T05:46:02.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KoO4Jn5AuRbXcFdkFvr265_JepbyOvI_0 2024-04-17T05:46:02.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6918942350303147120', time=2024-04-17T07:17:24.323367+0000, chain.objs.size()=1 2024-04-17T05:46:02.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mF9VgJtXdWoD-G6WCQ9iROyp3-o5j-c_0 2024-04-17T05:46:02.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.684673675144006866', time=2024-04-17T07:17:24.325364+0000, chain.objs.size()=1 2024-04-17T05:46:02.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jrP-Qa8Nv1mEZ5XXjoLnLVhmy2tCEMp_0 2024-04-17T05:46:02.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8701412871156841862', time=2024-04-17T07:17:24.517872+0000, chain.objs.size()=1 2024-04-17T05:46:02.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6kise74SvzGamKwipEcW3WFax19Pb17_0 2024-04-17T05:46:02.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9388287871705353975', time=2024-04-17T07:17:25.804244+0000, chain.objs.size()=1 2024-04-17T05:46:02.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._IExIRpiGKn_0-qkXI9Mck3SUcQT0eK_0 2024-04-17T05:46:02.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4468213128074076245', time=2024-04-17T07:17:26.030281+0000, chain.objs.size()=1 2024-04-17T05:46:02.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jbua29avsyc7yL-0V3KiI1Uw5dlt8ex_0 2024-04-17T05:46:02.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3372679307604375146', time=2024-04-17T07:17:28.158603+0000, chain.objs.size()=1 2024-04-17T05:46:02.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.eiEUa21IncpwUVUniNHYDV8rzYkuFsE_0 2024-04-17T05:46:02.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17188154312697141663', time=2024-04-17T07:17:30.514616+0000, chain.objs.size()=1 2024-04-17T05:46:02.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.d0gfMl8z5Vo1Zy72xhgsFWbUvU5x76M_0 2024-04-17T05:46:02.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8147222302805338026', time=2024-04-17T07:17:30.662918+0000, chain.objs.size()=1 2024-04-17T05:46:02.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.R7QRdWUMHGJC_-kyJupO3SRREyZQMWU_0 2024-04-17T05:46:02.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16651972167745479497', time=2024-04-17T07:17:31.465135+0000, chain.objs.size()=1 2024-04-17T05:46:02.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hXKxxEvzt6lWD-8FcpFCuV-BV9Zmo-8_0 2024-04-17T05:46:02.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4153494860034711712', time=2024-04-17T07:17:31.822817+0000, chain.objs.size()=1 2024-04-17T05:46:02.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.S58x01PX6-SIo4EKcdNCcIKHU4EwwWc_0 2024-04-17T05:46:02.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.258328157455825145', time=2024-04-17T07:18:16.511125+0000, chain.objs.size()=1 2024-04-17T05:46:02.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.280__shadow_.8l-LrUMoxWT7dhBal8b_irHsyc50teU_0 2024-04-17T05:46:02.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14590021849080376929', time=2024-04-17T07:18:27.916060+0000, chain.objs.size()=1 2024-04-17T05:46:02.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.Qr2qPt-z8B69AbGE1_xAvvY9JzDZG5E_0 2024-04-17T05:46:02.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16172851016813908437', time=2024-04-17T07:21:00.614217+0000, chain.objs.size()=1 2024-04-17T05:46:02.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xy1bQcAXChwwZk4ssT7l9bx8xLXGi-1_0 2024-04-17T05:46:02.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9055023416087643005', time=2024-04-17T07:21:01.387699+0000, chain.objs.size()=1 2024-04-17T05:46:02.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3n9LeSt52TjKBuxnN1zIU3AHe9OIhUz_0 2024-04-17T05:46:02.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2737022656723170410', time=2024-04-17T07:21:01.519283+0000, chain.objs.size()=1 2024-04-17T05:46:02.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1cJnqVPpASuZ0_5UhB48j5RtxEg08bt_0 2024-04-17T05:46:02.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3378155987814978140', time=2024-04-17T07:21:01.692883+0000, chain.objs.size()=1 2024-04-17T05:46:02.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gwldT8yU_r32GJjoMIAsiJ9F5jdDyDl_0 2024-04-17T05:46:02.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16632196493207714277', time=2024-04-17T07:21:01.902387+0000, chain.objs.size()=1 2024-04-17T05:46:02.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tZqfO4KiLQSJAxYyXTKztZhjz6hZPKL_0 2024-04-17T05:46:02.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15922957964110446647', time=2024-04-17T07:21:02.311599+0000, chain.objs.size()=1 2024-04-17T05:46:02.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5HDiMR9nqZahVFR_jX5vurKO2h0GZ0z_0 2024-04-17T05:46:02.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5123515413750598434', time=2024-04-17T07:21:02.777733+0000, chain.objs.size()=1 2024-04-17T05:46:02.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FgdFayTHQ0TAvr84I6M6xuoQPagWNI2_0 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4813447557942491103', time=2024-04-17T07:21:05.196406+0000, chain.objs.size()=1 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4_zVz5IsjSYYmuuNGmKeu4erGr5QNjr_0 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7761588991332594943', time=2024-04-17T07:21:05.452924+0000, chain.objs.size()=1 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vQh0t4jXWBNOwki9qiapxn2aVYPQF7v_0 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6023686189485367220', time=2024-04-17T07:21:05.455121+0000, chain.objs.size()=1 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SeqJp9guMkyfKxHRsMhD5PJbW4ELWXh_0 2024-04-17T05:46:02.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16511812432501188875', time=2024-04-17T07:21:06.139188+0000, chain.objs.size()=1 2024-04-17T05:46:02.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._bv5rme52jNLLgf-9x-_suh4LUsPmXJ_0 2024-04-17T05:46:02.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8043264949809901744', time=2024-04-17T07:21:07.972954+0000, chain.objs.size()=1 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lDhmIG0V4dti6rkhooHsPKKegH228hs_0 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3316697320891943402', time=2024-04-17T07:21:08.383144+0000, chain.objs.size()=1 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vVV2Dg79dxZMNxupRWQZxgtEgHIkwDp_0 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10145780759606994444', time=2024-04-17T07:21:08.772940+0000, chain.objs.size()=1 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-FlrtY_V4rkHCcYSFOrXfVF3gn1fBM4_0 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12710542378304865629', time=2024-04-17T07:21:09.186632+0000, chain.objs.size()=1 2024-04-17T05:46:02.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6ck_zuoXKCPNaKBHVL_qi9LsfcpSXlI_0 2024-04-17T05:46:02.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14553541807907935601', time=2024-04-17T07:21:09.628415+0000, chain.objs.size()=1 2024-04-17T05:46:02.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.my9SaxX2DIRoEgvWmaLPjuNdpr8euXS_0 2024-04-17T05:46:02.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8123573469465322124', time=2024-04-17T07:21:10.570556+0000, chain.objs.size()=1 2024-04-17T05:46:02.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XhE-2dptbFBypxyafXLEYfO4L__khnR_0 2024-04-17T05:46:02.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12741382621574795442', time=2024-04-17T07:21:10.881648+0000, chain.objs.size()=1 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3qplIfT4F7nOz8vHrkZGU76nQIYt2NR_0 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15225172765752367006', time=2024-04-17T07:21:12.079182+0000, chain.objs.size()=1 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u6uPSEXs3kqZ_FbL7t8GGsp7ec3ZBCf_0 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1935285252928257134', time=2024-04-17T07:21:12.616722+0000, chain.objs.size()=1 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8V2R0Cq6A_RFReOs6w2SFaEowEqsWTN_0 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8078448538516281583', time=2024-04-17T07:21:14.780728+0000, chain.objs.size()=1 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.J91RDaQWYSa7vC-chlIR8Tn29vReHYb_0 2024-04-17T05:46:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6733967521180190697', time=2024-04-17T07:21:15.861544+0000, chain.objs.size()=1 2024-04-17T05:46:02.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.K3RuFE3vHRKuPYJCBEpBLZjiyw-BST1_0 2024-04-17T05:46:02.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6443044211001665998', time=2024-04-17T07:21:17.251477+0000, chain.objs.size()=1 2024-04-17T05:46:02.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9Ui1kmEmhT2q4srDIzP47CWBMUpYuHQ_0 2024-04-17T05:46:02.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11039975662208342221', time=2024-04-17T07:21:19.586724+0000, chain.objs.size()=1 2024-04-17T05:46:02.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._hr0dCIiNtY0EcgwgfYbOBtKgwSwZbQ_0 2024-04-17T05:46:02.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7136566406210226443', time=2024-04-17T07:21:19.939195+0000, chain.objs.size()=1 2024-04-17T05:46:02.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FkipRV1eqK054PRo-hqY2e4Fm04vWCU_0 2024-04-17T05:46:02.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11092377808615413547', time=2024-04-17T07:21:20.625281+0000, chain.objs.size()=1 2024-04-17T05:46:02.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QUOAfkeT0RaMeOhwsGXqH9bNY-Eps4p_0 2024-04-17T05:46:02.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11133924756846985515', time=2024-04-17T07:21:20.830764+0000, chain.objs.size()=1 2024-04-17T05:46:02.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.R2j3Y2usakcF5Yrl_Qfjx33HcxCYr9A_0 2024-04-17T05:46:02.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14357152786241530308', time=2024-04-17T07:21:22.353163+0000, chain.objs.size()=1 2024-04-17T05:46:02.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.meS59x0ugA5kKXjvWdYCW6pu-1B2o8D_0 2024-04-17T05:46:02.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2793147192368188647', time=2024-04-17T07:21:22.537787+0000, chain.objs.size()=1 2024-04-17T05:46:02.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zE2iH2Ge1Aham4z6HRT7rZTGp3M8uPv_0 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14719407946068872680', time=2024-04-17T07:21:24.283262+0000, chain.objs.size()=1 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.bqE5reQVKqtcTB7ukPXj9-oV1w1XoI5_0 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2044414985320072698', time=2024-04-17T07:21:24.951153+0000, chain.objs.size()=1 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EHlDiYycUOZzVLrEE9g_ByLRDFNiHg__0 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16655645501630561723', time=2024-04-17T07:21:28.727383+0000, chain.objs.size()=1 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.NXdr5osh6_1KjtFp2pRrcC4DZ8IdFkn_0 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4685377419166654634', time=2024-04-17T07:21:29.139457+0000, chain.objs.size()=1 2024-04-17T05:46:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HSFiWZfcR_EkwXzXqW1VQZSNzom6H0Q_0 2024-04-17T05:46:02.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13635004057802624134', time=2024-04-17T07:21:29.262635+0000, chain.objs.size()=1 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-HoJimZMkyOETtTYvKGrNYn39SEXFiZ_0 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1153706268381829556', time=2024-04-17T07:23:12.678785+0000, chain.objs.size()=1 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TBq8ct2Sw1EVVZrxeQsTyzaUqPDWsGp_0 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2083548274039865851', time=2024-04-17T07:23:13.594149+0000, chain.objs.size()=1 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SJuXD7vwIpAb07UUzDMdrDqrk11ID4k_0 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11938230869686434624', time=2024-04-17T07:23:14.237333+0000, chain.objs.size()=1 2024-04-17T05:46:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mXVU7NTDSYVntBcQnbqtVa-ZF24grdU_0 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4725398763748827295', time=2024-04-17T07:23:15.933898+0000, chain.objs.size()=1 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8UC0hwdpUmKMpdipaCXdPGPkhoNc6B5_0 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6482083097749028918', time=2024-04-17T07:23:17.000777+0000, chain.objs.size()=1 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Oa7nYsKV8IeWJr6N_EhJVMFbiXDW02g_0 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11993718542597060763', time=2024-04-17T07:23:19.157410+0000, chain.objs.size()=1 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Heh3Qj9t9LBid6fa1BEpEms-wPjmASV_0 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16250135608868784662', time=2024-04-17T07:23:19.657276+0000, chain.objs.size()=1 2024-04-17T05:46:02.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ObS8nQBaNp0r6WDPPS4w_hsDhelR5Qn_0 2024-04-17T05:46:02.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10813491320081710526', time=2024-04-17T07:23:21.391375+0000, chain.objs.size()=1 2024-04-17T05:46:02.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8Ur1YTDD3YyaE42ekTpt82jq2eGRoQ1_0 2024-04-17T05:46:02.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5127546774616237526', time=2024-04-17T07:23:21.779805+0000, chain.objs.size()=1 2024-04-17T05:46:02.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.i9GCV6m1R7WhKn8brVXvZ-JAPVElzNL_0 2024-04-17T05:46:02.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11158431656205554780', time=2024-04-17T07:23:22.208556+0000, chain.objs.size()=1 2024-04-17T05:46:02.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dXMozEEIQEGNc1lHGlWaWT525lomY5v_0 2024-04-17T05:46:02.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3703400731840466803', time=2024-04-17T07:23:25.143639+0000, chain.objs.size()=1 2024-04-17T05:46:02.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kRbSqQ07UyFATyLvfeWHXXM-ictskRQ_0 2024-04-17T05:46:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10280719884742215051', time=2024-04-17T07:23:28.675619+0000, chain.objs.size()=1 2024-04-17T05:46:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9_zY_GisjTmudP7qhQ1qs1BSsRhIwfV_0 2024-04-17T05:46:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18141119658334294342', time=2024-04-17T07:23:30.682380+0000, chain.objs.size()=1 2024-04-17T05:46:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SWhT5bEFzxxO8viPK332MkvGZ57utcs_0 2024-04-17T05:46:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13909225230451959746', time=2024-04-17T07:23:33.130045+0000, chain.objs.size()=1 2024-04-17T05:46:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.R2S0S5Ah2_IVSmIiMdHeYHhjQ1hByYo_0 2024-04-17T05:46:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13060935718446818552', time=2024-04-17T07:23:33.553666+0000, chain.objs.size()=1 2024-04-17T05:46:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bjlevB1UODy_3w8SJg6H6w903CFmWgj_0 2024-04-17T05:46:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15051199639433314054', time=2024-04-17T07:23:34.153822+0000, chain.objs.size()=1 2024-04-17T05:46:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tyxZus8nhE4KGA0l47PiH9IX7J86n4F_0 2024-04-17T05:46:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9098193097266619823', time=2024-04-17T07:23:34.155755+0000, chain.objs.size()=1 2024-04-17T05:46:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J6l6FjI5eknYSe8SADElaJMqJgRaSX6_0 2024-04-17T05:46:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17926724053675918794', time=2024-04-17T07:23:37.809902+0000, chain.objs.size()=1 2024-04-17T05:46:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J4RdeQJyafqm5J8U3-7QQJMWsxouEp2_0 2024-04-17T05:46:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1192010501666688923', time=2024-04-17T07:23:38.539698+0000, chain.objs.size()=1 2024-04-17T05:46:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0_-FKPkegBUGSF6y8_VZqT_IZJn01j-_0 2024-04-17T05:46:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6671582566444211692', time=2024-04-17T07:23:40.262086+0000, chain.objs.size()=1 2024-04-17T05:46:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._9gzH34TRfzf3p52ios0elR40Ja9K1e_0 2024-04-17T05:46:02.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9497473726874008241', time=2024-04-17T07:23:40.847935+0000, chain.objs.size()=1 2024-04-17T05:46:02.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N7vZaD6ITJhBI3XBnDN-NOe9baoHWAc_0 2024-04-17T05:46:02.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2833402728317279123', time=2024-04-17T07:23:41.034956+0000, chain.objs.size()=1 2024-04-17T05:46:02.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0eKF284JLJekVD13XVUQNS-8UU86Ijn_0 2024-04-17T05:46:02.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2593082431616193768', time=2024-04-17T07:23:41.178489+0000, chain.objs.size()=1 2024-04-17T05:46:02.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UWM4cNCq8oCQM_Y5q18PoFKg8DeMtR__0 2024-04-17T05:46:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.500+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12355709196941090772', time=2024-04-17T07:23:49.469954+0000, chain.objs.size()=1 2024-04-17T05:46:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.322__shadow_.ILcQdtBCmN1B6iWSsHA9ceDy5JtQYaN_0 2024-04-17T05:46:02.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/46669 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=18, truncated=0, next_marker='' 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11294875801198332689', time=2024-04-17T07:24:25.923240+0000, chain.objs.size()=1 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.365__shadow_.n8oNCAcTIgnhnXp2OyrhdiCKY3MafXK_0 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13322715405203421012', time=2024-04-17T07:24:41.217736+0000, chain.objs.size()=1 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.380__shadow_.QVVV8hwr7LfbiBaTDWfaA9TNJ2jZJNT_0 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4203167851195919022', time=2024-04-17T07:25:42.380655+0000, chain.objs.size()=1 2024-04-17T05:46:02.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.444__shadow_.jEVhU30d_W5wtJnDWpzg71tuUW-JtM__0 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4125609508668280269', time=2024-04-17T07:25:58.425207+0000, chain.objs.size()=1 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.454__shadow_.5JFUJPWQY0ITJ0xAhTT9MQ_xqBVNRKD_0 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15701410227305442558', time=2024-04-17T07:26:17.168592+0000, chain.objs.size()=1 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.xXv5ZNn40iSKnXV60Pf08DTFu13Bt66_0 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_1_3d44d528807391581fcf545e6f7b3b94', time=2024-04-17T07:26:24.460739+0000, chain.objs.size()=3 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_0 2024-04-17T05:46:02.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_1 2024-04-17T05:46:02.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_2 2024-04-17T05:46:02.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1584710887088101317', time=2024-04-17T07:28:43.861210+0000, chain.objs.size()=6 2024-04-17T05:46:02.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc.1 2024-04-17T05:46:02.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc.2 2024-04-17T05:46:02.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.nOFIHcJlf_96SIhO0spTnhCQU8aB7nZa.3 2024-04-17T05:46:02.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc.4 2024-04-17T05:46:02.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc.5 2024-04-17T05:46:02.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~tANGBxm8R7vsdAxIZndoHyp8RIG7JBc.6 2024-04-17T05:46:02.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4805448367864631277', time=2024-04-17T07:29:22.609482+0000, chain.objs.size()=2 2024-04-17T05:46:02.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~OiQQkIoOa4ZynEViBTJDOOSnz9fP-tn.1 2024-04-17T05:46:02.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~OiQQkIoOa4ZynEViBTJDOOSnz9fP-tn.2 2024-04-17T05:46:02.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6567291902050948927', time=2024-04-17T07:30:47.097917+0000, chain.objs.size()=1 2024-04-17T05:46:02.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.540__shadow_.BBTpLuOv44hn9HSTBnjZke1HExg52iv_0 2024-04-17T05:46:02.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16950503045524237230', time=2024-04-17T07:31:04.910670+0000, chain.objs.size()=1 2024-04-17T05:46:02.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.546__shadow_.ckwtpXhuJPzdIogo4esZ2oJTfCYjd_P_0 2024-04-17T05:46:02.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7248889821818445324', time=2024-04-17T07:31:14.447001+0000, chain.objs.size()=1 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.547__shadow_.RjbF17tjdiYZ9YUZpAPDZHcge4CwRK__0 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7962229709558617581', time=2024-04-17T07:31:21.479169+0000, chain.objs.size()=1 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.551__shadow_.AaEFNfQQ0Fh9rbHxMYqDRONVxDhuQ9G_0 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2285219032268235351', time=2024-04-17T07:32:03.260608+0000, chain.objs.size()=1 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.562__shadow_.VCj8qCJv88noVjt3wrgYYwfvxB4k3Rw_0 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10302194253613391831', time=2024-04-17T07:33:29.533424+0000, chain.objs.size()=1 2024-04-17T05:46:02.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:OtXHI20WPYlIdjEermhn7N5QVfrYynA_.qOviki3YD6lmqqex8QjMwg1ZlQHpXcy_0 2024-04-17T05:46:02.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2111452076730189470', time=2024-04-17T07:34:00.932185+0000, chain.objs.size()=1 2024-04-17T05:46:02.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.593__shadow_.msJCrvDC61go6eARWWE3xQS_K6gGxw0_0 2024-04-17T05:46:02.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2545491525896805267', time=2024-04-17T07:35:35.174145+0000, chain.objs.size()=6 2024-04-17T05:46:02.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604__multipart_multipart_enc.2~cG65yPPPqh36acXwdzWwcUvQPXUAEZe.1 2024-04-17T05:46:02.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604__multipart_multipart_enc.2~cG65yPPPqh36acXwdzWwcUvQPXUAEZe.2 2024-04-17T05:46:02.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604__multipart_multipart_enc.2~cG65yPPPqh36acXwdzWwcUvQPXUAEZe.3 2024-04-17T05:46:02.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604__multipart_multipart_enc.2~cG65yPPPqh36acXwdzWwcUvQPXUAEZe.4 2024-04-17T05:46:02.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604__multipart_multipart_enc.2~cG65yPPPqh36acXwdzWwcUvQPXUAEZe.5 2024-04-17T05:46:02.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604__multipart_multipart_enc.2~cG65yPPPqh36acXwdzWwcUvQPXUAEZe.6 2024-04-17T05:46:02.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12582395758447896018', time=2024-04-17T07:37:39.737624+0000, chain.objs.size()=1 2024-04-17T05:46:02.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.650__shadow_.AM3uOk0w-tM6JH53HsuECdasdMO-v9C_0 2024-04-17T05:46:02.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13956227826274698026', time=2024-04-17T07:38:09.218970+0000, chain.objs.size()=1 2024-04-17T05:46:02.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.681__shadow:K6UVy7JoN3NHhVfXn0rk.JbPNaIgEdX_.XQGB_NOiFBNQvOCrPAXtXt7PyRI-SV7_0 2024-04-17T05:46:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.524+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-17T05:46:02.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.524+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.524+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42027' 2024-04-17T05:46:02.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_fSh2aAaWwqTcAAlHAIK53jOl8jEJ11s', time=2024-04-17T07:10:12.746488+0000, chain.objs.size()=1 2024-04-17T05:46:02.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.aecFzM619cvDhrVW3H-Ftkn1g5W03MM_0 2024-04-17T05:46:02.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_q5ZdkZ3Y4mHZnPMd_ZI5U7G808s9vjn', time=2024-04-17T07:10:21.011962+0000, chain.objs.size()=2 2024-04-17T05:46:02.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.XCmi98KXMobNc7b6i9PEs_qCCFI5PaH_0 2024-04-17T05:46:02.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.XCmi98KXMobNc7b6i9PEs_qCCFI5PaH_1 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~m9B2bRIluxtNN3XNbHLJga1h9Fv8ibc', time=2024-04-17T07:11:40.360380+0000, chain.objs.size()=1 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.191__multipart_mymultipart.2~m9B2bRIluxtNN3XNbHLJga1h9Fv8ibc.1 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15618383500361467556', time=2024-04-17T07:14:33.040799+0000, chain.objs.size()=1 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.224__shadow_.RPVvjFqpNKu11VZE6pbLbsHtfNj_Isi_0 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6500816356239319831', time=2024-04-17T07:14:51.256576+0000, chain.objs.size()=1 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.234__shadow_.yuGQRzo2BbmGdhT2jDf1tIJKp303dg0_0 2024-04-17T05:46:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9134955423078623901', time=2024-04-17T07:14:52.924163+0000, chain.objs.size()=1 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.235__shadow_.By0_egwYiARTsVn7hcp8OdyBWWtEdXJ_0 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15040894085408791846', time=2024-04-17T07:15:12.224124+0000, chain.objs.size()=1 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.247__shadow_.a26dFo0PFe4iq_Xqh0siiz8XfLMXuf9_0 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6360039528149824235', time=2024-04-17T07:17:01.981521+0000, chain.objs.size()=1 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ktCnrSPnuRRat7Kl22ldamRprl_pNIM_0 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8372577577241233670', time=2024-04-17T07:17:02.536505+0000, chain.objs.size()=1 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EAthPlBgRuzsVhJSQVnpTGmKUyLqN6A_0 2024-04-17T05:46:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16335802567359865856', time=2024-04-17T07:17:03.454117+0000, chain.objs.size()=1 2024-04-17T05:46:02.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.AC16p0yaQchV4xIvV8QK9w7VinJa3CV_0 2024-04-17T05:46:02.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7857137885761305500', time=2024-04-17T07:17:03.996184+0000, chain.objs.size()=1 2024-04-17T05:46:02.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yg245CESI2UllV849w7z4XThBNflpGX_0 2024-04-17T05:46:02.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16569584571296341349', time=2024-04-17T07:17:07.359360+0000, chain.objs.size()=1 2024-04-17T05:46:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dnszgfHxU4JGHS4sd8-xEmDNf3Dteum_0 2024-04-17T05:46:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6709501849535334932', time=2024-04-17T07:17:07.625742+0000, chain.objs.size()=1 2024-04-17T05:46:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rnL5vy4qfs52aizuq-UGYlTE_Qm6s_P_0 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1935280627212281372', time=2024-04-17T07:17:08.323509+0000, chain.objs.size()=1 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YWbICVEmAagr685raf5UACFoUteCXYy_0 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11363171720866557584', time=2024-04-17T07:17:08.430631+0000, chain.objs.size()=1 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1pPUqKFv8DsIrte6BfDekerfnOKpfeA_0 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5511435828238424136', time=2024-04-17T07:17:11.850064+0000, chain.objs.size()=1 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.USDr1fh8yjG1wkzpSMqIsAZxvGxX_9G_0 2024-04-17T05:46:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8000034659725623101', time=2024-04-17T07:17:14.202671+0000, chain.objs.size()=1 2024-04-17T05:46:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EdJ9ber2u27J4lpMSMAzFfy91ccSWla_0 2024-04-17T05:46:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15059671494360912792', time=2024-04-17T07:17:14.685885+0000, chain.objs.size()=1 2024-04-17T05:46:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xzgozInMFMY5UCEJKUuffKbrLuTeuuV_0 2024-04-17T05:46:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3177903625718393281', time=2024-04-17T07:17:17.977899+0000, chain.objs.size()=1 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.J1PCLAP98uiNu-cICQ5C18tBjMdGf8i_0 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5772168320759061170', time=2024-04-17T07:17:18.089681+0000, chain.objs.size()=1 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Y-X4ocSaUVJBZ1FqLBzUV5IEez6Pose_0 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14117849655277576080', time=2024-04-17T07:17:19.201475+0000, chain.objs.size()=1 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ce4Q7PlPxsTW0YEy-vF22xM0EEg4HUb_0 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.766840622364067011', time=2024-04-17T07:17:19.658966+0000, chain.objs.size()=1 2024-04-17T05:46:02.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.O6F7Vu10tlzC55CMIa3y6kzB-mSodWX_0 2024-04-17T05:46:02.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5450667569826059792', time=2024-04-17T07:17:20.044148+0000, chain.objs.size()=1 2024-04-17T05:46:02.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iS0ZxHneRolTH9tcdGjLI0aPf_z1gNW_0 2024-04-17T05:46:02.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9593939999136300374', time=2024-04-17T07:17:20.680079+0000, chain.objs.size()=1 2024-04-17T05:46:02.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.532+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LleP1E9KLWbGlvFIpkgwvxcZuOCrdeJ_0 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10474136497343926853', time=2024-04-17T07:17:23.615851+0000, chain.objs.size()=1 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DpagNyg119ByO7FgTU72tvQ04_E2LpH_0 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11452162142468901438', time=2024-04-17T07:17:24.061290+0000, chain.objs.size()=1 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.duTuuObw0sbS52BkOBEaaPHUa618OXH_0 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16752601146296100302', time=2024-04-17T07:17:24.420785+0000, chain.objs.size()=1 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5eXkEN4Rl69gTKPBE-37LBK47XycoCv_0 2024-04-17T05:46:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9605594917890528732', time=2024-04-17T07:17:24.422658+0000, chain.objs.size()=1 2024-04-17T05:46:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.536+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QA6RYXzyFZMogQTJnI94VecPD0PWFjs_0 2024-04-17T05:46:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9653256794204887977', time=2024-04-17T07:17:25.118410+0000, chain.objs.size()=1 2024-04-17T05:46:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._Kzl5muJ0yoR6zAeetFrN0Wd23GMfz1_0 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9405156337439662431', time=2024-04-17T07:17:27.275557+0000, chain.objs.size()=1 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XQwkySPRFmBtI3AcLTQGqc1M_omlsUJ_0 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9811305371433795417', time=2024-04-17T07:17:28.385328+0000, chain.objs.size()=1 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bcgW7nTdI9T7D6r7cVd7frPXNsqkJ88_0 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7305298589885920376', time=2024-04-17T07:17:28.697207+0000, chain.objs.size()=1 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FdmE9bCEUGu7FzY5G1k946g7oj8Zy3w_0 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4240683911011951558', time=2024-04-17T07:17:30.242548+0000, chain.objs.size()=1 2024-04-17T05:46:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aO34UsKKIbu9BeT7Zr5Qtm_sYPK_fQv_0 2024-04-17T05:46:02.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18200389365160449014', time=2024-04-17T07:17:30.310820+0000, chain.objs.size()=1 2024-04-17T05:46:02.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.619s-fTQmCFdRQV4KAhwIY5h5L8tMRL_0 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3181430343305047838', time=2024-04-17T07:17:31.201531+0000, chain.objs.size()=1 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DfcubWH22nujsbFB7TZYp5DOHlwwgzo_0 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5851796742053486009', time=2024-04-17T07:17:31.281270+0000, chain.objs.size()=1 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.28gWHO_sxdYfTD5favbq0119vhg5XBI_0 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1828319597545555919', time=2024-04-17T07:17:38.434454+0000, chain.objs.size()=1 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.256__shadow_.YppBx2ABaS6VcRsh75ZAE6u0wlrjT4v_0 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4893816952793269480', time=2024-04-17T07:17:39.933391+0000, chain.objs.size()=1 2024-04-17T05:46:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.540+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.257__shadow_.kDrJgNF4zw1c4c36mxno_uK_-IDQMxw_0 2024-04-17T05:46:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6580093025549528276', time=2024-04-17T07:17:58.477188+0000, chain.objs.size()=1 2024-04-17T05:46:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.269__shadow_.P5sJZatKkBphXvYVyvJyUlUg9nJ_OnV_0 2024-04-17T05:46:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16639245850277663144', time=2024-04-17T07:18:22.882603+0000, chain.objs.size()=1 2024-04-17T05:46:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.2DGuWJ9pJHSEU2qfDHeFC68s9YGvKEH_0 2024-04-17T05:46:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1766087123931741726', time=2024-04-17T07:18:27.920931+0000, chain.objs.size()=1 2024-04-17T05:46:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.KWMyTy9861ZL4tvt6o16txQa_biXOdf_0 2024-04-17T05:46:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15767516774556700411', time=2024-04-17T07:18:28.041850+0000, chain.objs.size()=1 2024-04-17T05:46:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.0YIHFnbxZG0WccjeGJz7FncpTza9CCK_0 2024-04-17T05:46:02.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.629258744516225196', time=2024-04-17T07:21:00.182761+0000, chain.objs.size()=1 2024-04-17T05:46:02.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6nhBRnEI_W6pfba8Hq4Px_mC1qohA9F_0 2024-04-17T05:46:02.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1929649397836560521', time=2024-04-17T07:21:00.307251+0000, chain.objs.size()=1 2024-04-17T05:46:02.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5_uR9tQVSPLwRyYpyrcBBcy8FeGYdJZ_0 2024-04-17T05:46:02.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17307160015270328156', time=2024-04-17T07:21:00.414831+0000, chain.objs.size()=1 2024-04-17T05:46:02.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VA6uka0_Nyy_TZLQWfy38XDWX52kyMe_0 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7444567786757307822', time=2024-04-17T07:21:04.230377+0000, chain.objs.size()=1 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MRJuW0xb1GglFAvC1fKpt8LaHyNwlcj_0 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17626104158323879968', time=2024-04-17T07:21:04.622954+0000, chain.objs.size()=1 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wcnnt2dRwxDBhuZobLQSspd-4ucxfdG_0 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7382908266264387641', time=2024-04-17T07:21:05.901752+0000, chain.objs.size()=1 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZrKUIVMCjJP2YAStMGFXP4mUVUTDFL2_0 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18341704423436014065', time=2024-04-17T07:21:06.695102+0000, chain.objs.size()=1 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uS-Vdiirw3IEBF_QXva_O0Iq3lowq1R_0 2024-04-17T05:46:02.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1991301966870514341', time=2024-04-17T07:21:10.215141+0000, chain.objs.size()=1 2024-04-17T05:46:02.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nYO_qbOo4_tWR8AOMN0mFtQzVftwtfh_0 2024-04-17T05:46:02.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15172481385394267022', time=2024-04-17T07:21:10.785199+0000, chain.objs.size()=1 2024-04-17T05:46:02.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XhIMX7g1SjbCwwWHKgKwzQJ7DqU_NLu_0 2024-04-17T05:46:02.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6734033007059004479', time=2024-04-17T07:21:12.574095+0000, chain.objs.size()=1 2024-04-17T05:46:02.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FxHoTVHA91xifduIhVIzkGnS6kbCMlN_0 2024-04-17T05:46:02.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15487405957359649906', time=2024-04-17T07:21:13.733239+0000, chain.objs.size()=1 2024-04-17T05:46:02.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lYFUNvntPGkJuSP5mfQgFSRxC0qyUay_0 2024-04-17T05:46:02.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11580575610732177635', time=2024-04-17T07:21:16.096602+0000, chain.objs.size()=1 2024-04-17T05:46:02.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u5UKgqZXkBGVU0UJzMtSev7YamEYoas_0 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15535351799806426600', time=2024-04-17T07:21:17.459919+0000, chain.objs.size()=1 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zNK5wKEDBQDuvU5UX73zQsHQinMviiR_0 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3957124613964766998', time=2024-04-17T07:21:18.425737+0000, chain.objs.size()=1 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Hs3evYkexGc9xT6o5m_bAooLxLL0s59_0 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12873064965635319492', time=2024-04-17T07:21:20.723318+0000, chain.objs.size()=1 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PgWMwUDNK2uma2TE0hs20CdRIXDVbR1_0 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11157441263356571204', time=2024-04-17T07:21:21.518986+0000, chain.objs.size()=1 2024-04-17T05:46:02.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rGh5WsJPWFxJAfU4u9Myfg_rxKs2ua-_0 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15028068091707449498', time=2024-04-17T07:21:21.687365+0000, chain.objs.size()=1 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9dbfH_ej7jZuTPmQvNNgsec93XG3ilr_0 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17032222891739950975', time=2024-04-17T07:21:22.006073+0000, chain.objs.size()=1 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.e96TtWf_ihB7AJnopUM83Pxsvoe2J5z_0 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17519920087878598974', time=2024-04-17T07:21:23.375859+0000, chain.objs.size()=1 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WvwNSj6KFZizeYLHV8IC5LTTumD-mBu_0 2024-04-17T05:46:02.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2225669222662978177', time=2024-04-17T07:21:23.677233+0000, chain.objs.size()=1 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JYuXp2vpk3vNV-01v-RvvsmYddO3gcP_0 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15235706977035122842', time=2024-04-17T07:21:25.108409+0000, chain.objs.size()=1 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ENID0MHL_z1YwtbPUqJhpLkoFlQKBS2_0 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13457194121273987978', time=2024-04-17T07:21:26.055133+0000, chain.objs.size()=1 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4Jc5WfsFu3oYfMtHV6eRpmYHDg2WPlT_0 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8396357594941405177', time=2024-04-17T07:21:27.043001+0000, chain.objs.size()=1 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.j51bQJdDWN0tx7vDt1FQOxTrf9MM-AO_0 2024-04-17T05:46:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8053636513703331460', time=2024-04-17T07:21:28.314909+0000, chain.objs.size()=1 2024-04-17T05:46:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vvdAYKzxmF3tAqOsF5Qbf2q3IxAO9dZ_0 2024-04-17T05:46:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8224262575881143282', time=2024-04-17T07:21:28.711750+0000, chain.objs.size()=1 2024-04-17T05:46:02.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.c8InTqlwQr6jQrajkgp1shdrNrRtbdz_0 2024-04-17T05:46:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5525350087121878729', time=2024-04-17T07:21:29.210265+0000, chain.objs.size()=1 2024-04-17T05:46:02.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.D-E2_7tB_utzO7s_qfOREtToawT-ET2_0 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3435242307307928281', time=2024-04-17T07:21:29.332553+0000, chain.objs.size()=1 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.P2k5dqGmBBqevZ_qXtPej0fW0snZ05b_0 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3935564602775507772', time=2024-04-17T07:23:13.997902+0000, chain.objs.size()=1 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wB_qKKya4Or3SNW_9v8xtqJ-XPjNP0x_0 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2110807482994166572', time=2024-04-17T07:23:14.000166+0000, chain.objs.size()=1 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CXnfgEkU56JWUPTWIoLN3rJgAmaMQ81_0 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10623057265631833093', time=2024-04-17T07:23:14.561364+0000, chain.objs.size()=1 2024-04-17T05:46:02.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.P5fFOO2l3W4s6jViNM2iRnudrEUPIEu_0 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14729866928263276279', time=2024-04-17T07:23:15.270367+0000, chain.objs.size()=1 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.HhCVNgYWSDueHU-2Yo1PGb1LD68LcnA_0 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4181764839469794526', time=2024-04-17T07:23:15.338814+0000, chain.objs.size()=1 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KKnel9p2ahWOztvQNoYK4FGQBWsgmFf_0 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2446316306367928654', time=2024-04-17T07:23:17.102988+0000, chain.objs.size()=1 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.u8KFLnlMK2DwdYFBjITKms16tGE0G8h_0 2024-04-17T05:46:02.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5725048870846656542', time=2024-04-17T07:23:17.171349+0000, chain.objs.size()=1 2024-04-17T05:46:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.l8iKB8z0qFqEy_bpfhVwCco4CHWqd_S_0 2024-04-17T05:46:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17846758947260455809', time=2024-04-17T07:23:19.938118+0000, chain.objs.size()=1 2024-04-17T05:46:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QJJxPAgBaucB51UIr8WBfkLWhwuBaH7_0 2024-04-17T05:46:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10736953435409397349', time=2024-04-17T07:23:20.909957+0000, chain.objs.size()=1 2024-04-17T05:46:02.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gCrZHSTMV_ZM460kjGjU_OKNB9uRgz4_0 2024-04-17T05:46:02.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.748965605987366607', time=2024-04-17T07:23:23.477305+0000, chain.objs.size()=1 2024-04-17T05:46:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aZ0unx_UhYUCv8p1nhlSxTwA_pxF0WK_0 2024-04-17T05:46:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7374025676105130411', time=2024-04-17T07:23:23.857723+0000, chain.objs.size()=1 2024-04-17T05:46:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8KQUh3mOu2how6mtZFGeGFEm-Ke1grl_0 2024-04-17T05:46:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3706299153072053085', time=2024-04-17T07:23:25.135412+0000, chain.objs.size()=1 2024-04-17T05:46:02.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hLQ1hscYMYV-T0DlUDmu4cNgYES5-dj_0 2024-04-17T05:46:02.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4491981319489253081', time=2024-04-17T07:23:26.131858+0000, chain.objs.size()=1 2024-04-17T05:46:02.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._dQ6519hPntTdaw-xNOPZvA76abP43M_0 2024-04-17T05:46:02.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2867674080306178674', time=2024-04-17T07:23:28.673549+0000, chain.objs.size()=1 2024-04-17T05:46:02.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5p1mX6OO98sf-Y3-UUPctukH2veOU03_0 2024-04-17T05:46:02.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15150250933153549396', time=2024-04-17T07:23:29.124420+0000, chain.objs.size()=1 2024-04-17T05:46:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._X9XnfDhrFzBO8Gp8wmSfFi5NEV_fgt_0 2024-04-17T05:46:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10724401441290870077', time=2024-04-17T07:23:29.413121+0000, chain.objs.size()=1 2024-04-17T05:46:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bPqp5dSwLNDbYfGmhFl95lHg1kuGvyh_0 2024-04-17T05:46:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4545942347212760208', time=2024-04-17T07:23:29.760590+0000, chain.objs.size()=1 2024-04-17T05:46:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7j35f_m2sFlgxMh5BgFK8iZyjJ5liTN_0 2024-04-17T05:46:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8836917499733885024', time=2024-04-17T07:23:30.676135+0000, chain.objs.size()=1 2024-04-17T05:46:02.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6p8LabTioBSgp2tdEZ8gAGwPyxnIWf5_0 2024-04-17T05:46:02.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13333594522238965884', time=2024-04-17T07:23:31.051719+0000, chain.objs.size()=1 2024-04-17T05:46:02.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eFXtwwRCTYlv6NvFluSkERs4ZzsWQjZ_0 2024-04-17T05:46:02.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15973939438035253152', time=2024-04-17T07:23:32.623363+0000, chain.objs.size()=1 2024-04-17T05:46:02.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.44kV39B-iy4cLfR1WF-i-EvEXP-un3d_0 2024-04-17T05:46:02.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7682590859273658763', time=2024-04-17T07:23:34.878507+0000, chain.objs.size()=1 2024-04-17T05:46:02.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZWyZk4lGvAzHodiy_zbLWVBRkA4ig2S_0 2024-04-17T05:46:02.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3659222616661200756', time=2024-04-17T07:23:36.504158+0000, chain.objs.size()=1 2024-04-17T05:46:02.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.B9XDyOzKD_yFHYc12BnxhYC3BnscAR7_0 2024-04-17T05:46:02.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563971951875439186', time=2024-04-17T07:23:38.363477+0000, chain.objs.size()=1 2024-04-17T05:46:02.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.itPhJb684hg7Vyq9sxhMi-8-rMgf2jQ_0 2024-04-17T05:46:02.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16436106184559353990', time=2024-04-17T07:23:40.768061+0000, chain.objs.size()=1 2024-04-17T05:46:02.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XTYH8MFRd7WF9gBDB0wMCY16esYcdhP_0 2024-04-17T05:46:02.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1645347499427679049', time=2024-04-17T07:23:40.887601+0000, chain.objs.size()=1 2024-04-17T05:46:02.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GkrnueuGZ7KyjaZJwpqth32G0hNOro6_0 2024-04-17T05:46:02.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15426495326109753021', time=2024-04-17T07:23:59.544528+0000, chain.objs.size()=1 2024-04-17T05:46:02.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.333__shadow_.8cnbSvTmVT0G0iPr_1YPcDoDafN_KDD_0 2024-04-17T05:46:02.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1921153155893817515', time=2024-04-17T07:25:49.038863+0000, chain.objs.size()=1 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.448__shadow_.cYDehdKadtH3qXXG74n_VK83OXQHLUI_0 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11840664765804573582', time=2024-04-17T07:25:59.278594+0000, chain.objs.size()=1 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.454__shadow_.BfLe_Wzbt6-M821tRztx0smp1dERbZX_0 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16785902124435255949', time=2024-04-17T07:26:31.668261+0000, chain.objs.size()=1 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.483__shadow_.ThbDYy3OrM9D1bShhixGBbvVmP1JJzT_0 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13282759512140071691', time=2024-04-17T07:26:38.457304+0000, chain.objs.size()=1 2024-04-17T05:46:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.489__shadow_.D_72jMNIE_4K0Bv0Zr0ku-SZdPGCyJL_0 2024-04-17T05:46:02.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16625767617738702024', time=2024-04-17T07:28:26.322754+0000, chain.objs.size()=6 2024-04-17T05:46:02.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd.1 2024-04-17T05:46:02.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.Y7MVaFjaeWR6Qk5BqlhkD8sIKtgxgbiH.2 2024-04-17T05:46:02.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd.3 2024-04-17T05:46:02.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd.4 2024-04-17T05:46:02.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd.5 2024-04-17T05:46:02.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~PXu5KcnRwDIXruJLwa9l6J6ZJjfRKcd.6 2024-04-17T05:46:02.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/42027 2024-04-17T05:46:02.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-17T05:46:02.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4529191025480662449', time=2024-04-17T07:30:43.849997+0000, chain.objs.size()=1 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.539__shadow_.ST6Nd138t1UDF2hBrTePS1_K0o0vY_I_0 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12127739205410971029', time=2024-04-17T07:31:36.188675+0000, chain.objs.size()=1 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:E0yRYpmjp03qtcEoszG1IG0tb.Rp9kP_._ra_4SchhfX74tnXFbiJREshEbHXjeM_0 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14125504373397755182', time=2024-04-17T07:32:25.864084+0000, chain.objs.size()=1 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:gp5zlTCP8OQggRXFONWa-ShgqeInnr1_.s-PchGMEKD8EVvkSDg6dImPhoodBRbo_0 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5447960731404903352', time=2024-04-17T07:33:35.113101+0000, chain.objs.size()=1 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.571__shadow:sW02ZhaYb8ewDdVU.QTEi2xroJZlkNA_.edMbUddOTHKQEeEhxSKQu0hV0Lkoq44_0 2024-04-17T05:46:02.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.264944159671827265', time=2024-04-17T07:36:30.615131+0000, chain.objs.size()=1 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.617__shadow_.D4fk-cnTnjmWoCGVcod-Ig9UMt2i7bM_0 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6101457652105478952', time=2024-04-17T07:36:39.381122+0000, chain.objs.size()=1 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.625__shadow_.DeypDeok4ZDVG6wPWZYBV9dN8TXFoNq_0 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1543240769980236330', time=2024-04-17T07:38:29.155413+0000, chain.objs.size()=1 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.697__shadow:gV2kN8glK5de4BxopAgMm1rJ8X-xxAl_.C0jxxbJ4Ol8LXiHXhh-X9XOpDeNzJlF_0 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8012068640227345460', time=2024-04-17T07:38:43.567535+0000, chain.objs.size()=1 2024-04-17T05:46:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.701__shadow:Av5eFlBFPYVs3H5Gnxwe6Bib5nBd3aR_.Kptxb5Nt1xK3fG4bUgu2edSxNghndVL_0 2024-04-17T05:46:02.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13707987142024858873', time=2024-04-17T07:39:07.474341+0000, chain.objs.size()=1 2024-04-17T05:46:02.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.717__shadow_.3kKY3tWqOoWL1WK291RJxrDLIGcgbbb_0 2024-04-17T05:46:02.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-17T05:46:02.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/58927' 2024-04-17T05:46:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16371914941422233607', time=2024-04-17T07:09:56.456135+0000, chain.objs.size()=2 2024-04-17T05:46:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.148__shadow_.OaFgJtMYX001pPu13lDnlnxluCJyjCi_0 2024-04-17T05:46:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.148__shadow_.OaFgJtMYX001pPu13lDnlnxluCJyjCi_1 2024-04-17T05:46:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9299460378814878778', time=2024-04-17T07:10:05.995439+0000, chain.objs.size()=1 2024-04-17T05:46:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.hgy57Ej7pERpHl2uwC3uLpg8Y5fDHiC_0 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_9KCMPS9nqOXb_V-12yiPEofcE24gslV', time=2024-04-17T07:10:09.357552+0000, chain.objs.size()=2 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.XXk2SDjukmM-vicFM7eO6ny7uCZu1r1_0 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.XXk2SDjukmM-vicFM7eO6ny7uCZu1r1_1 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_nT2BgH9dtTKaTZfFes40oF_UG3EJ7Fl', time=2024-04-17T07:10:12.673782+0000, chain.objs.size()=2 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.OQijHI38x4DMQ__ITclQuXGDnyLQkqS_0 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.OQijHI38x4DMQ__ITclQuXGDnyLQkqS_1 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14314286632776597316', time=2024-04-17T07:14:31.436310+0000, chain.objs.size()=1 2024-04-17T05:46:02.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.223__shadow_.MxLckzwxVyGUZNHId6q-gVP2YrX0tuD_0 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13193913898640774080', time=2024-04-17T07:17:01.787762+0000, chain.objs.size()=1 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fl7-p3HGAYKN3HGAkOS95XiNo3xxAI-_0 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16864205580900104818', time=2024-04-17T07:17:03.049284+0000, chain.objs.size()=1 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.x_krYI5U4tpvDa-u5lM3Nuj_spT6X_Z_0 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15032404870087531056', time=2024-04-17T07:17:07.640419+0000, chain.objs.size()=1 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.v2xi1WgyOEx7BBk4rvp3U1FMzT-7mQU_0 2024-04-17T05:46:02.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4457234321920002660', time=2024-04-17T07:17:07.761225+0000, chain.objs.size()=1 2024-04-17T05:46:02.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OQ11WPDrugUcodk5yjxVVlKQjc1G6pD_0 2024-04-17T05:46:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7574734852350593783', time=2024-04-17T07:17:10.296051+0000, chain.objs.size()=1 2024-04-17T05:46:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CaioQckPT4m16Jj4jOpP4bW7TvEQpWE_0 2024-04-17T05:46:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2712752594452093862', time=2024-04-17T07:17:11.692854+0000, chain.objs.size()=1 2024-04-17T05:46:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.83BZiDa0pC3Xe9FtU9BvEJQi6Vrg2KU_0 2024-04-17T05:46:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10317104014249424493', time=2024-04-17T07:17:12.586562+0000, chain.objs.size()=1 2024-04-17T05:46:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ETtycQUASM2LoZ3LzgdvDk7xK1zEJA6_0 2024-04-17T05:46:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.510073609690155129', time=2024-04-17T07:17:14.444102+0000, chain.objs.size()=1 2024-04-17T05:46:02.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dqOm9sVAwIxbMiWmY3K288awlvvscC__0 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7299537262785256332', time=2024-04-17T07:17:16.797668+0000, chain.objs.size()=1 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.90Fil3PwMfdHEsn8t4U451HmwpdyQDU_0 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9618024957605830587', time=2024-04-17T07:17:16.799866+0000, chain.objs.size()=1 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7BrelUgGIyhRIYmFsJHek7XjEWJmC65_0 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3992636160350343991', time=2024-04-17T07:17:17.058150+0000, chain.objs.size()=1 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Vu4CQWXCG9UrL5k868lfK8bv2AbzAG__0 2024-04-17T05:46:02.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7686257507989847606', time=2024-04-17T07:17:18.782551+0000, chain.objs.size()=1 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ObUCYxTvPVoDCBxeshh3R56u-NTcQuf_0 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13134847027527422970', time=2024-04-17T07:17:20.103145+0000, chain.objs.size()=1 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9irG8ndPDEAZiIXmS4Wn-Tipnl_UVoT_0 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17924804817488451881', time=2024-04-17T07:17:20.242327+0000, chain.objs.size()=1 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.eO_vzuH7567cuCzDt-CSu2kUpi_x-SE_0 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.58489532796548878', time=2024-04-17T07:17:22.082306+0000, chain.objs.size()=1 2024-04-17T05:46:02.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pxaq0ofAVZQDe3qFc791hoR_qFzOBb-_0 2024-04-17T05:46:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.601641246586312343', time=2024-04-17T07:17:22.825448+0000, chain.objs.size()=1 2024-04-17T05:46:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.AqNu82UrRfyrPmkjOD4hbyx58Zqn9zu_0 2024-04-17T05:46:02.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16097922320803643039', time=2024-04-17T07:17:23.798427+0000, chain.objs.size()=1 2024-04-17T05:46:02.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.a9AlXxr4K4dq5hLCywu0HNIXQrTrfke_0 2024-04-17T05:46:02.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13604088825964392820', time=2024-04-17T07:17:24.823077+0000, chain.objs.size()=1 2024-04-17T05:46:02.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2Mov4xaIOm7SDdWKMZ0QdpoOof8KZ92_0 2024-04-17T05:46:02.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7788622465776778569', time=2024-04-17T07:17:26.486023+0000, chain.objs.size()=1 2024-04-17T05:46:02.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pQo9eQOCgrUW3o5_WW19mcO5-_bUMwQ_0 2024-04-17T05:46:02.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11638305658518468115', time=2024-04-17T07:17:27.126139+0000, chain.objs.size()=1 2024-04-17T05:46:02.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KOsJfp3BS5eSYtH3TPXNj5WgdUFMpy2_0 2024-04-17T05:46:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10950302021926827811', time=2024-04-17T07:17:27.338453+0000, chain.objs.size()=1 2024-04-17T05:46:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KHHi335qJIqZjcQ5T8XPlAfz8x-rqw3_0 2024-04-17T05:46:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9459170925971967831', time=2024-04-17T07:17:27.663022+0000, chain.objs.size()=1 2024-04-17T05:46:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DDjh1YKjbpZj-pRdb636Gj6KaUQMBxu_0 2024-04-17T05:46:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5330346338540287811', time=2024-04-17T07:17:28.252507+0000, chain.objs.size()=1 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ODud60a0IBJJoDOjymndGd8H7QLdUH5_0 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3763997971906739233', time=2024-04-17T07:17:29.168957+0000, chain.objs.size()=1 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mJeCNU3JW22PEFES699RXBLh5gbmXnS_0 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7330994251005317733', time=2024-04-17T07:17:30.539393+0000, chain.objs.size()=1 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kaECrHt8w0jIFsnA2rZ4B49utn9cyO5_0 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10797607440355143928', time=2024-04-17T07:17:31.422991+0000, chain.objs.size()=1 2024-04-17T05:46:02.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kPJ1nRrCVKPVF1Y3fnGth3qjKFIwmvW_0 2024-04-17T05:46:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14962733785619140863', time=2024-04-17T07:17:32.067695+0000, chain.objs.size()=1 2024-04-17T05:46:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NgxmyESFyA1WscsnsJOQKWJ2LuE9c2I_0 2024-04-17T05:46:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11856001589118208895', time=2024-04-17T07:17:36.893995+0000, chain.objs.size()=1 2024-04-17T05:46:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.255__shadow_.BY2upftU-Wb-xN8FRNFmOx6NEbmnd75_0 2024-04-17T05:46:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13337301043596048772', time=2024-04-17T07:17:44.467706+0000, chain.objs.size()=1 2024-04-17T05:46:02.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.608+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.260__shadow_.0Mxuzi-BTCxRYgZ4oXjz8_tpScZnfi4_0 2024-04-17T05:46:02.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13375805151190587014', time=2024-04-17T07:18:06.585363+0000, chain.objs.size()=1 2024-04-17T05:46:02.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.274__shadow_.7zM4FofCjg0N8N9X8iHlnRMIprcO-jj_0 2024-04-17T05:46:02.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1275510928555621183', time=2024-04-17T07:18:22.985842+0000, chain.objs.size()=1 2024-04-17T05:46:02.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.ZOtmn3A4n_nXK0kRQrOcYg6qP6l9UBu_0 2024-04-17T05:46:02.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16926654254122041592', time=2024-04-17T07:18:28.218802+0000, chain.objs.size()=1 2024-04-17T05:46:02.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.YI79w2H4mE8GSrVczqBohDmd9yMOwHA_0 2024-04-17T05:46:02.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11902920378735768850', time=2024-04-17T07:18:28.220865+0000, chain.objs.size()=1 2024-04-17T05:46:02.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.x1gDXt0Wj-FIZgXyNGPPX9rKNapTIzb_0 2024-04-17T05:46:02.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6939934551635097269', time=2024-04-17T07:18:28.280668+0000, chain.objs.size()=1 2024-04-17T05:46:02.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.x8SFNBO7UOEby0Lq91EJZxhSKfyZ0Yy_0 2024-04-17T05:46:02.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12693057897770579605', time=2024-04-17T07:18:29.868362+0000, chain.objs.size()=1 2024-04-17T05:46:02.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.284__shadow_.2OM3m5mQkhcZkYqL0DJ1ZrtDSIV6ce__0 2024-04-17T05:46:02.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4133228906848508880', time=2024-04-17T07:18:38.401947+0000, chain.objs.size()=1 2024-04-17T05:46:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.289__shadow_.GABL0WgV-on_pTj2EV_HaHBupisT3gt_0 2024-04-17T05:46:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9814780778792126445', time=2024-04-17T07:18:48.176305+0000, chain.objs.size()=1 2024-04-17T05:46:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.295__shadow_.yaJAe4Ie5voRH0KkB0tlqLRIO7bVsio_0 2024-04-17T05:46:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8507898053359414370', time=2024-04-17T07:19:15.440443+0000, chain.objs.size()=1 2024-04-17T05:46:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.312__shadow_.Dt0yJSRbWE1l_8DWqDFkN1mLx5tGPtU_0 2024-04-17T05:46:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10604442111076455368', time=2024-04-17T07:20:59.852709+0000, chain.objs.size()=1 2024-04-17T05:46:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.hD40wIFjVnFlM_NQZujASui_2jYMWg-_0 2024-04-17T05:46:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18146870095460161223', time=2024-04-17T07:21:00.811470+0000, chain.objs.size()=1 2024-04-17T05:46:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pNSwuC97Ovi_BAt1yED2eeURq1pMSeY_0 2024-04-17T05:46:02.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5242810275112314536', time=2024-04-17T07:21:01.033314+0000, chain.objs.size()=1 2024-04-17T05:46:02.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.77tpeyhI4r_qEcTCLH100AFbo4-7ErG_0 2024-04-17T05:46:02.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3653759250953406362', time=2024-04-17T07:21:02.789110+0000, chain.objs.size()=1 2024-04-17T05:46:02.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Q-NlHcaokkbKXa31XfOyBmb5APIRB-Q_0 2024-04-17T05:46:02.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.899734478805235041', time=2024-04-17T07:21:04.228824+0000, chain.objs.size()=1 2024-04-17T05:46:02.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.r57Y4qKqsHHLIAU4OCmWy30BlHVzJJb_0 2024-04-17T05:46:02.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7917325842306225666', time=2024-04-17T07:21:06.604663+0000, chain.objs.size()=1 2024-04-17T05:46:02.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yqAeoyLDlDJkVTLEU9bFNzcGYFlOn3M_0 2024-04-17T05:46:02.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18125745545822793137', time=2024-04-17T07:21:07.975121+0000, chain.objs.size()=1 2024-04-17T05:46:02.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dKoAc1qwELSBoPoMqZ1Jlz-ibVEA5fk_0 2024-04-17T05:46:02.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.249800192636390423', time=2024-04-17T07:21:08.348214+0000, chain.objs.size()=1 2024-04-17T05:46:02.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jnjZOQc-XpYEGmA2_-uabjd002dbawp_0 2024-04-17T05:46:02.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10707231868075511520', time=2024-04-17T07:21:08.783494+0000, chain.objs.size()=1 2024-04-17T05:46:02.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uLTgOncx6rqWtETDNlz2ARdzVo26UFI_0 2024-04-17T05:46:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14823769480124601704', time=2024-04-17T07:21:09.120360+0000, chain.objs.size()=1 2024-04-17T05:46:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lHUSQ8-VdkujMFHlzF9pgcgi9b6QOBG_0 2024-04-17T05:46:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11937917469912791551', time=2024-04-17T07:21:10.716714+0000, chain.objs.size()=1 2024-04-17T05:46:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2iVBkqxotTUeLhi-Ie6EneYgj6HPqoQ_0 2024-04-17T05:46:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1728195464138945425', time=2024-04-17T07:21:11.859036+0000, chain.objs.size()=1 2024-04-17T05:46:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8cVqovHHFtKWcRb5UlAoOyFWd9Iu-JQ_0 2024-04-17T05:46:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11746543132389517650', time=2024-04-17T07:21:14.017451+0000, chain.objs.size()=1 2024-04-17T05:46:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SKgKoCpxxgXy9peQD9L2XhmBDE6QfyE_0 2024-04-17T05:46:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.446697407400012254', time=2024-04-17T07:21:14.212823+0000, chain.objs.size()=1 2024-04-17T05:46:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.07JKaLO2Q_3t52ciVjQM64a9SLT2Syg_0 2024-04-17T05:46:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1197397779285916268', time=2024-04-17T07:21:16.581639+0000, chain.objs.size()=1 2024-04-17T05:46:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TfF3O60sOWkAabocj40jW_ONI79B4k7_0 2024-04-17T05:46:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6966296558522589155', time=2024-04-17T07:21:21.312944+0000, chain.objs.size()=1 2024-04-17T05:46:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uXG-4xR9MGne54LO7tiuVjiwhl25L9z_0 2024-04-17T05:46:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1299534479249697861', time=2024-04-17T07:21:22.138146+0000, chain.objs.size()=1 2024-04-17T05:46:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BJFtpNnamYqOCHBqOZA7eCeISDbenYs_0 2024-04-17T05:46:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9848572465079810304', time=2024-04-17T07:21:23.329603+0000, chain.objs.size()=1 2024-04-17T05:46:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.O5barTgBQAtwvU6LKEOvzWTYMW9Cjmz_0 2024-04-17T05:46:02.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15625407613563574602', time=2024-04-17T07:21:23.684541+0000, chain.objs.size()=1 2024-04-17T05:46:02.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.swhHwOKpt7rsC5f-OQy17UtODb0RzUg_0 2024-04-17T05:46:02.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13119449782141042049', time=2024-04-17T07:21:24.088363+0000, chain.objs.size()=1 2024-04-17T05:46:02.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.d-Md6B8W742ZjIlZaTDfQ-hEiRxJTof_0 2024-04-17T05:46:02.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18065743320149568388', time=2024-04-17T07:21:25.735875+0000, chain.objs.size()=1 2024-04-17T05:46:02.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HpLvZwcdqEs6IQ6dtzq-ib-hFIMezBi_0 2024-04-17T05:46:02.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1592533554776208190', time=2024-04-17T07:21:25.910513+0000, chain.objs.size()=1 2024-04-17T05:46:02.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kzaSLJQ4gANJKDCcHvNuqbdAByBvDJ2_0 2024-04-17T05:46:02.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13248445147864599003', time=2024-04-17T07:21:27.808921+0000, chain.objs.size()=1 2024-04-17T05:46:02.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.T2y6WiFgRxJOFZq8FTiq4CN3bcddchV_0 2024-04-17T05:46:02.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10492857792109194103', time=2024-04-17T07:21:27.919264+0000, chain.objs.size()=1 2024-04-17T05:46:02.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dHa8QWmNlaPLZAu1vg2HS8-PjKRyBiK_0 2024-04-17T05:46:02.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2025339887707656439', time=2024-04-17T07:21:28.302896+0000, chain.objs.size()=1 2024-04-17T05:46:02.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4tgfEzM3Ttdb72EmXQhOAZelF--mfmm_0 2024-04-17T05:46:02.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5125141657861243939', time=2024-04-17T07:23:12.640389+0000, chain.objs.size()=1 2024-04-17T05:46:02.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eI4OqVRlFenOhtUpzc9aTUqz0-vlXe4_0 2024-04-17T05:46:02.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11784499455568454234', time=2024-04-17T07:23:13.572254+0000, chain.objs.size()=1 2024-04-17T05:46:02.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WWOanT_eTnmadhvaUzsAVsz1YNaTWXS_0 2024-04-17T05:46:02.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5702140478482898640', time=2024-04-17T07:23:15.904202+0000, chain.objs.size()=1 2024-04-17T05:46:02.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gdf8_Tgr9ZxbWL7MUNtrY2nZ_fdjJe7_0 2024-04-17T05:46:02.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.94649216293879413', time=2024-04-17T07:23:17.111722+0000, chain.objs.size()=1 2024-04-17T05:46:02.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mp6tYZo_TF12oeFgajQdmqpfbBLWXc5_0 2024-04-17T05:46:02.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8806401634259914020', time=2024-04-17T07:23:18.548334+0000, chain.objs.size()=1 2024-04-17T05:46:02.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RPX5a9SPp4fZCePWfgVuuimopGfjmBg_0 2024-04-17T05:46:02.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2126142216789861116', time=2024-04-17T07:23:18.837632+0000, chain.objs.size()=1 2024-04-17T05:46:02.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xTMcwlMWATnMfWA8rhepqIaQCRfSY98_0 2024-04-17T05:46:02.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9926840745487239033', time=2024-04-17T07:23:23.389743+0000, chain.objs.size()=1 2024-04-17T05:46:02.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pA6Bz6HyewExiHU7X00rmHB6CPTn7c8_0 2024-04-17T05:46:02.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7585991847860866247', time=2024-04-17T07:23:23.391377+0000, chain.objs.size()=1 2024-04-17T05:46:02.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gD9KA33vRBIq3Rb_wvJ0usdPCovz9hP_0 2024-04-17T05:46:02.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17809760130108548753', time=2024-04-17T07:23:25.314479+0000, chain.objs.size()=1 2024-04-17T05:46:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kybaeqC20eHoMA1s3rq7oRbt3fkO8TO_0 2024-04-17T05:46:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10927747196185169088', time=2024-04-17T07:23:25.428208+0000, chain.objs.size()=1 2024-04-17T05:46:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.v-5pS7cKPazxQfD5vVnkByJ5QnvTeYw_0 2024-04-17T05:46:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8188486691517289260', time=2024-04-17T07:23:25.519730+0000, chain.objs.size()=1 2024-04-17T05:46:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QcH2JQJJtc-DAALv9nqFdIc5_PRv8mc_0 2024-04-17T05:46:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17321776627062006341', time=2024-04-17T07:23:25.802687+0000, chain.objs.size()=1 2024-04-17T05:46:02.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.w7lvP4oy2Xbkl6ewi1saAJM8EAVG4Y4_0 2024-04-17T05:46:02.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6601454676416927992', time=2024-04-17T07:23:26.148695+0000, chain.objs.size()=1 2024-04-17T05:46:02.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8MW12JysDYdAYAfboU4xmpb1VLhutzo_0 2024-04-17T05:46:02.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15440160917325752932', time=2024-04-17T07:23:26.413914+0000, chain.objs.size()=1 2024-04-17T05:46:02.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KyVVlp2RuhIQoPWCUn5q64ZnyW7c2Dt_0 2024-04-17T05:46:02.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1147020766608666997', time=2024-04-17T07:23:26.496735+0000, chain.objs.size()=1 2024-04-17T05:46:02.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wR4nPbI17ssZf1SwNio2RsMo_sItk-T_0 2024-04-17T05:46:02.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10803234426269952845', time=2024-04-17T07:23:26.702590+0000, chain.objs.size()=1 2024-04-17T05:46:02.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Tcx-WuOPKcEFLw8Oy0cDdflHWH9ETSY_0 2024-04-17T05:46:02.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.229093673289719670', time=2024-04-17T07:23:26.987604+0000, chain.objs.size()=1 2024-04-17T05:46:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.y-CwUyaKYHe1Qg640WRMPU2zV-zdAJp_0 2024-04-17T05:46:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12724354463411784194', time=2024-04-17T07:23:27.521728+0000, chain.objs.size()=1 2024-04-17T05:46:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.luEfuX3hzzaprgjgyoX0cQ5t67_PtrB_0 2024-04-17T05:46:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14731136109621068315', time=2024-04-17T07:23:28.471045+0000, chain.objs.size()=1 2024-04-17T05:46:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qxqrQaPhLsNb_nkrvn00j8Bg5-pHGMk_0 2024-04-17T05:46:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3348474525052068892', time=2024-04-17T07:23:29.131548+0000, chain.objs.size()=1 2024-04-17T05:46:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.arDqVt0P10AF5z3sQdhw8mZK1XZjE20_0 2024-04-17T05:46:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5622063160873748693', time=2024-04-17T07:23:29.284326+0000, chain.objs.size()=1 2024-04-17T05:46:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vOwD7UTSt8i7o_mn0CJFQFAPXvMuKk7_0 2024-04-17T05:46:02.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17262586126495059668', time=2024-04-17T07:23:30.046432+0000, chain.objs.size()=1 2024-04-17T05:46:02.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.V1vj-KAKBQ4-COHOiyC_SsYpQflJVL4_0 2024-04-17T05:46:02.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3535995664789802746', time=2024-04-17T07:23:30.316855+0000, chain.objs.size()=1 2024-04-17T05:46:02.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.opVoQQpqq_N9By1lq1rNVLhPPr2TpaL_0 2024-04-17T05:46:02.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5751925601380259345', time=2024-04-17T07:23:30.583146+0000, chain.objs.size()=1 2024-04-17T05:46:02.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mEE_02WhAdRfZVZUCSM8tN_qA8XNRDw_0 2024-04-17T05:46:02.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4683578153092011372', time=2024-04-17T07:23:32.156779+0000, chain.objs.size()=1 2024-04-17T05:46:02.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JOSqv7g3xFpRujcWe2njMEj0BQ5Euij_0 2024-04-17T05:46:02.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5976990462887377634', time=2024-04-17T07:23:33.460726+0000, chain.objs.size()=1 2024-04-17T05:46:02.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qK7TzKClCr5OaUxt6qp6uIRwSZXm4ZX_0 2024-04-17T05:46:02.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5860532499016769817', time=2024-04-17T07:23:33.597334+0000, chain.objs.size()=1 2024-04-17T05:46:02.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iIKkncaeZww8wdFKxD4H8AL7wRMKKl8_0 2024-04-17T05:46:02.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3161188963966528685', time=2024-04-17T07:23:34.739372+0000, chain.objs.size()=1 2024-04-17T05:46:02.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.zbQLTFf5-c1xL-xO6H6a8PdkolcswdO_0 2024-04-17T05:46:02.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3780570224313563587', time=2024-04-17T07:23:36.242888+0000, chain.objs.size()=1 2024-04-17T05:46:02.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EH9PLq9YZXfr1CbsU13g4ZEmujqhcyA_0 2024-04-17T05:46:02.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13777616796914509790', time=2024-04-17T07:23:36.993844+0000, chain.objs.size()=1 2024-04-17T05:46:02.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Q2_9EyJAy9_J2QOYMdj0xuFf7wnYPy9_0 2024-04-17T05:46:02.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14993190045070538399', time=2024-04-17T07:23:38.364895+0000, chain.objs.size()=1 2024-04-17T05:46:02.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7SXKmyWX3WDVxcqWuBPCgE3yVT6K-89_0 2024-04-17T05:46:02.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3240885922737927825', time=2024-04-17T07:23:38.729022+0000, chain.objs.size()=1 2024-04-17T05:46:02.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.s_Ur3ZdnpFZ_uz9cszxYyFi85fmg1k-_0 2024-04-17T05:46:02.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/58927 2024-04-17T05:46:02.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=24, truncated=0, next_marker='' 2024-04-17T05:46:02.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4930389282797678483', time=2024-04-17T07:23:39.974845+0000, chain.objs.size()=1 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ja3yjzD5S-76rw9akk9oWbFz_wdTSCD_0 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10059838946827917880', time=2024-04-17T07:23:40.551304+0000, chain.objs.size()=1 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DMfAPizHmz2CUYmalYN38CoEqK1xMTJ_0 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9478760884670523999', time=2024-04-17T07:23:41.098688+0000, chain.objs.size()=1 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kzCLdCuDmtcv-4ItOXR4GnKPSHkBMhG_0 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3069327903345959010', time=2024-04-17T07:23:41.540724+0000, chain.objs.size()=1 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.08QViehKljZoYtFvfM7TRSXgaDOmEJi_0 2024-04-17T05:46:02.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4940597007439047470', time=2024-04-17T07:23:41.574895+0000, chain.objs.size()=1 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.j92celkFCNnfdbLT7gshEQ1IQB2FVNG_0 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6560481048528827499', time=2024-04-17T07:24:01.270127+0000, chain.objs.size()=1 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.335__shadow_.SWP6BJsVZIFpQMJbnog7Exn_Fn-1a3m_0 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10552405089138921267', time=2024-04-17T07:24:32.432937+0000, chain.objs.size()=1 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.371__shadow_.Pnwtz5MKTMaHn-z0GFhEPDWKu6PajJL_0 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8527432625873787215', time=2024-04-17T07:25:27.287556+0000, chain.objs.size()=1 2024-04-17T05:46:02.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.430__shadow_.GRhOxLbdSKy9u_SloB4STmC9qgp_D81_0 2024-04-17T05:46:02.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3272606553146102677', time=2024-04-17T07:26:02.981002+0000, chain.objs.size()=1 2024-04-17T05:46:02.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.456__shadow_.YIU2IpYJqJCyfHjUafMBaawXgt9Jn87_0 2024-04-17T05:46:02.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9006205797979420996', time=2024-04-17T07:26:56.148616+0000, chain.objs.size()=6 2024-04-17T05:46:02.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.1 2024-04-17T05:46:02.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2 2024-04-17T05:46:02.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.3 2024-04-17T05:46:02.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.4 2024-04-17T05:46:02.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.5 2024-04-17T05:46:02.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.6 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7971752076610381630', time=2024-04-17T07:27:03.608562+0000, chain.objs.size()=1 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.504__shadow_.9AaQzdRNAGxRvr0Wz-RomO2qHU0DpkJ_0 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13099549975634592070', time=2024-04-17T07:29:27.383854+0000, chain.objs.size()=1 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~j2aQXhTlhH-JqJi3ishGXxk6nACOnr1.1 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5002139854232339355', time=2024-04-17T07:29:37.313031+0000, chain.objs.size()=2 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~dPRLtp3rgyHq15zTWxlSzGNXu9GsoG0.1 2024-04-17T05:46:02.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~dPRLtp3rgyHq15zTWxlSzGNXu9GsoG0.2 2024-04-17T05:46:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14839670447512272823', time=2024-04-17T07:31:03.009541+0000, chain.objs.size()=1 2024-04-17T05:46:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.545__shadow_.mDquc9fAo1v_EdHPCRQX2gZK0qn_HMo_0 2024-04-17T05:46:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13770517269938977937', time=2024-04-17T07:31:29.011765+0000, chain.objs.size()=1 2024-04-17T05:46:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.553__shadow_.40q2GGX1zF1gG4b0_OTrKErb4CMVi1k_0 2024-04-17T05:46:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612559860450188791', time=2024-04-17T07:31:42.907162+0000, chain.objs.size()=1 2024-04-17T05:46:02.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:W3C8X.GdkNp29Jh.k8S3jUgE2SuyHxF_.TIQ98fOrxpMHoNwPwotT5jhEWgkj_Hu_0 2024-04-17T05:46:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5376125621467392876', time=2024-04-17T07:31:54.693128+0000, chain.objs.size()=1 2024-04-17T05:46:02.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:H-h.s.T43-qcttMzZH0sA1KGygLXYje_.TkX7SfRDh78QhFnEhoUGkNc7qQkBmSD_0 2024-04-17T05:46:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1398818027291425606', time=2024-04-17T07:33:03.035247+0000, chain.objs.size()=1 2024-04-17T05:46:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:xZwxmvIA6jW5V1Xn9RC1N6kYedg04Qw_.k5bxO_tYFpXRkxsLrbdT4wEjdXvxumN_0 2024-04-17T05:46:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16329101244950866046', time=2024-04-17T07:33:29.272331+0000, chain.objs.size()=1 2024-04-17T05:46:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:r2OXlSMBY49vgtzBhTFxg2NaKJn4xfs_.Qc-nZCKht2tlL12JwqI9e6v29_wMND2_0 2024-04-17T05:46:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13217002492062560924', time=2024-04-17T07:33:58.571944+0000, chain.objs.size()=1 2024-04-17T05:46:02.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.591__shadow_.hdwzvbR1EWrWHC07XIq7z4MCviebftX_0 2024-04-17T05:46:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9153041809334928089', time=2024-04-17T07:36:31.557818+0000, chain.objs.size()=1 2024-04-17T05:46:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.618__shadow_.qFX0n0qRUd6rJ3kFPdAbdi5p_2dB8Wf_0 2024-04-17T05:46:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9303922828921469896', time=2024-04-17T07:37:44.034552+0000, chain.objs.size()=1 2024-04-17T05:46:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.653__shadow_.8lIH6SHjKWaunUyifD3X0SUS6ektMoi_0 2024-04-17T05:46:02.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17786617814815429661', time=2024-04-17T07:37:47.066969+0000, chain.objs.size()=1 2024-04-17T05:46:02.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.656__shadow_.43LVEO4KO8Jp4yr4oC-icVbNQkhsRdH_0 2024-04-17T05:46:02.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16233988363981646244', time=2024-04-17T07:38:19.543924+0000, chain.objs.size()=1 2024-04-17T05:46:02.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.689__shadow:nLJFEnDkI2apf6vYvlBjV.GMSCpBRZS_.-6D-DYfZG-wp3PRqHCPeX_-qZpbyfFc_0 2024-04-17T05:46:02.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.676+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-17T05:46:02.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47088' 2024-04-17T05:46:02.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5348062510750964358', time=2024-04-17T07:14:26.493547+0000, chain.objs.size()=1 2024-04-17T05:46:02.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.220__shadow_.yi9uOBEr80BhXApNzXr-w8TI_QrcumG_0 2024-04-17T05:46:02.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13013345907952677887', time=2024-04-17T07:14:33.070373+0000, chain.objs.size()=1 2024-04-17T05:46:02.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.224__shadow_.xJTM_YiOrPOya2XZKsgxP0EMpHco943_0 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18069629200744502642', time=2024-04-17T07:14:46.604533+0000, chain.objs.size()=1 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.232__shadow_.O4X_5zMDu67YDYD7QpdkomljJsMHi___0 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18371242551281171022', time=2024-04-17T07:15:05.915657+0000, chain.objs.size()=1 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.243__shadow_.WpKZxKiXiwcObp13U6lJ2b5j9a6CL7H_0 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1188379683995558385', time=2024-04-17T07:17:01.357106+0000, chain.objs.size()=1 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gBKrnvvcviBtgTOslVIGoAwIFVVAyAK_0 2024-04-17T05:46:02.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9123650740173292119', time=2024-04-17T07:17:02.635668+0000, chain.objs.size()=1 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lP8sanSwquWF0KrapuadWjdygjbiUcL_0 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6232038513118585736', time=2024-04-17T07:17:02.765666+0000, chain.objs.size()=1 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aoHQtBxq5tSOkIVhnDuMTzmnD-NeLqR_0 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12997798657512137657', time=2024-04-17T07:17:03.049893+0000, chain.objs.size()=1 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.WtX9nshaCGlmKDMhMQcBtpX-B4CSAIg_0 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9129190836072272203', time=2024-04-17T07:17:07.021358+0000, chain.objs.size()=1 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.laRe3-mPaVi9yhjtl4Eqxc6yp_2xLhd_0 2024-04-17T05:46:02.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5028956186315701533', time=2024-04-17T07:17:07.022712+0000, chain.objs.size()=1 2024-04-17T05:46:02.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2XxbSyfDvyrWuc5qnTqZv36cZs5Ps4l_0 2024-04-17T05:46:02.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1841192786864353848', time=2024-04-17T07:17:07.107932+0000, chain.objs.size()=1 2024-04-17T05:46:02.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.O6oif8dl_rMsMUcLkaWrPzQqwi5EcvQ_0 2024-04-17T05:46:02.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16786800922094901363', time=2024-04-17T07:17:07.404864+0000, chain.objs.size()=1 2024-04-17T05:46:02.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VXA9Agj6TcFGrW6W1--aTHcxLsw25rs_0 2024-04-17T05:46:02.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8188335320652841843', time=2024-04-17T07:17:08.415795+0000, chain.objs.size()=1 2024-04-17T05:46:02.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KDjGoAitFsv5snsTyrBswkkk_BTtujg_0 2024-04-17T05:46:02.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6334696052809751584', time=2024-04-17T07:17:11.104019+0000, chain.objs.size()=1 2024-04-17T05:46:02.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rEIROBPgkSMmnUaCigKgHTAP_Hgy_Y0_0 2024-04-17T05:46:02.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12894375671349455883', time=2024-04-17T07:17:11.438550+0000, chain.objs.size()=1 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Y9t1cBMhQ_XhnR2UUvFkDaqSGnlDs6r_0 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7568941235433118431', time=2024-04-17T07:17:12.203578+0000, chain.objs.size()=1 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.15X7ZOB_nLZWInaXf49qsfhxmurzZ1c_0 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7069520293150723424', time=2024-04-17T07:17:14.484791+0000, chain.objs.size()=1 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rbs5vHajP9_9Splvj_L-S8UfgvLjA8l_0 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11676387988433984927', time=2024-04-17T07:17:16.412081+0000, chain.objs.size()=1 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.m6VNYC1pzKD1AVWfnPAzZ28rZlg2wrh_0 2024-04-17T05:46:02.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9254095138270113019', time=2024-04-17T07:17:16.413913+0000, chain.objs.size()=1 2024-04-17T05:46:02.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7E5FBGeEE98sFuc8VeI2Zj-dmlJc4IM_0 2024-04-17T05:46:02.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13664706344877442929', time=2024-04-17T07:17:16.792143+0000, chain.objs.size()=1 2024-04-17T05:46:02.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vPRBaIMCEoUQXaWdH3IrK7iPJgqACoY_0 2024-04-17T05:46:02.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2864032346890245537', time=2024-04-17T07:17:18.041156+0000, chain.objs.size()=1 2024-04-17T05:46:02.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.huZuEtg6gdyJNMgnO6fr_0nYkHZsdhk_0 2024-04-17T05:46:02.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11634203956275491008', time=2024-04-17T07:17:18.660756+0000, chain.objs.size()=1 2024-04-17T05:46:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FobfDOJisdK_KucW_v30491LuXSjifd_0 2024-04-17T05:46:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13144251866174378429', time=2024-04-17T07:17:19.439513+0000, chain.objs.size()=1 2024-04-17T05:46:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JBY38_b44segRTB7vU_wpjOKUh9-n8G_0 2024-04-17T05:46:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11078063745277589426', time=2024-04-17T07:17:20.099655+0000, chain.objs.size()=1 2024-04-17T05:46:02.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yX5bVMWbLNRpT_HPF8vXxYu1Zacq_T5_0 2024-04-17T05:46:02.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12919032859232227447', time=2024-04-17T07:17:20.867376+0000, chain.objs.size()=1 2024-04-17T05:46:02.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BGrdCOiyWZAfbXCev60N0Ogqe6K8lj8_0 2024-04-17T05:46:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17349023337563670602', time=2024-04-17T07:17:21.062754+0000, chain.objs.size()=1 2024-04-17T05:46:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.eQhP-0LoiewRMfRRsRO4bXKE7F9D3c6_0 2024-04-17T05:46:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3329981982035055202', time=2024-04-17T07:17:23.215102+0000, chain.objs.size()=1 2024-04-17T05:46:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OBN0gVNWR880ruexHFiOTXCCOTVJ8oQ_0 2024-04-17T05:46:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11364866410111641105', time=2024-04-17T07:17:25.821970+0000, chain.objs.size()=1 2024-04-17T05:46:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4-PSU6L2mYTYjiMh_xzSnfYA77ddQsO_0 2024-04-17T05:46:02.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3222484906190237801', time=2024-04-17T07:17:25.823809+0000, chain.objs.size()=1 2024-04-17T05:46:02.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KAZ0GLCyL9NZ0LKrj_T6STIX9wSEz8r_0 2024-04-17T05:46:02.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18198901484503990299', time=2024-04-17T07:17:26.599599+0000, chain.objs.size()=1 2024-04-17T05:46:02.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iIG8EkoFqaKRpLIg6XCtuR5gq3Tje-8_0 2024-04-17T05:46:02.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4085066046595937012', time=2024-04-17T07:17:27.819633+0000, chain.objs.size()=1 2024-04-17T05:46:02.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SSYDfALfj0-R3DTw6YJKLVLqnSVg3UU_0 2024-04-17T05:46:02.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4249736336244551099', time=2024-04-17T07:17:28.255621+0000, chain.objs.size()=1 2024-04-17T05:46:02.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YT-b2YvfxuF4OhWigM5bM7PIkyzU5_j_0 2024-04-17T05:46:02.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3339592529750420337', time=2024-04-17T07:17:29.165698+0000, chain.objs.size()=1 2024-04-17T05:46:02.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ax77FIyos0KJBh74uh-Du_C2Xm2CiQ5_0 2024-04-17T05:46:02.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8385909152679241708', time=2024-04-17T07:17:30.194067+0000, chain.objs.size()=1 2024-04-17T05:46:02.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Z7vEyEelVDaUBaKN0JBZ_ePrbpJCxM7_0 2024-04-17T05:46:02.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4912440697350459205', time=2024-04-17T07:17:30.651539+0000, chain.objs.size()=1 2024-04-17T05:46:02.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6yjCSBlLVxXZr63k4ASTLZ6cfm886OM_0 2024-04-17T05:46:02.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12235190049866053052', time=2024-04-17T07:17:32.205219+0000, chain.objs.size()=1 2024-04-17T05:46:02.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.M-7du5JwxbHX8UIV3uhLWUrB9ayr1et_0 2024-04-17T05:46:02.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10616737841818408891', time=2024-04-17T07:17:51.952003+0000, chain.objs.size()=1 2024-04-17T05:46:02.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.265__shadow_.Mr7CuI_xn8DeIKlUqSpDHMjhPwcQgg9_0 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6176371294137568820', time=2024-04-17T07:18:03.332218+0000, chain.objs.size()=1 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.272__shadow_.-_QMgnuyUKNEiMELmObtXqULVKXbifP_0 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3972180466909116797', time=2024-04-17T07:18:10.023210+0000, chain.objs.size()=1 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.276__shadow_.jtSIQ3KPDtw3usqEvFJ5UO6MBc0Jhn3_0 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15088548033151478615', time=2024-04-17T07:18:23.293871+0000, chain.objs.size()=1 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.LG6izwOLkAQ9mtM9jHLcENeF85JUBtq_0 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5512491758506215560', time=2024-04-17T07:18:27.920339+0000, chain.objs.size()=1 2024-04-17T05:46:02.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.Fn9o3DgaQvC5bJ0LLhgqIVTbOe2vFfI_0 2024-04-17T05:46:02.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17951598963744634116', time=2024-04-17T07:18:27.923087+0000, chain.objs.size()=1 2024-04-17T05:46:02.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.b8488GjCnLF0nnOrl6U8589w0W5RtaH_0 2024-04-17T05:46:02.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1262696778319277130', time=2024-04-17T07:18:31.491084+0000, chain.objs.size()=1 2024-04-17T05:46:02.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.285__shadow_.BO0sITdTQWCd6893Xhu_gB3pI9NhDs-_0 2024-04-17T05:46:02.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14245588643041041784', time=2024-04-17T07:18:38.379281+0000, chain.objs.size()=1 2024-04-17T05:46:02.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.289__shadow_.26g-RIzdLin3rX8tC3vnKPoCbFMFueK_0 2024-04-17T05:46:02.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5372384746545563347', time=2024-04-17T07:18:44.961298+0000, chain.objs.size()=1 2024-04-17T05:46:02.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.293__shadow_.gqoD98eBbWwgz-K7W7KLd3mk_aM88K1_0 2024-04-17T05:46:02.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11258560097498073992', time=2024-04-17T07:18:48.157740+0000, chain.objs.size()=1 2024-04-17T05:46:02.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.295__shadow_.U3ffYWV_y2Uug76OyEtyXAeUrvxUWro_0 2024-04-17T05:46:02.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12606655413931794134', time=2024-04-17T07:21:00.011702+0000, chain.objs.size()=1 2024-04-17T05:46:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xr7VjxcQkXYqEJeDIseNEynV05w1x23_0 2024-04-17T05:46:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15566622539607183411', time=2024-04-17T07:21:00.400141+0000, chain.objs.size()=1 2024-04-17T05:46:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.b37fJFGBKmvKWo2maZ75j0WI-8vcaRu_0 2024-04-17T05:46:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17353250541880621482', time=2024-04-17T07:21:00.994282+0000, chain.objs.size()=1 2024-04-17T05:46:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Bq1dhtDsVOJpzBEOvrrRGzLM6WBfIqo_0 2024-04-17T05:46:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2122964182409292422', time=2024-04-17T07:21:02.718200+0000, chain.objs.size()=1 2024-04-17T05:46:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.o-M__22g5glxi3Dica-fnwUHhloyvzi_0 2024-04-17T05:46:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1446315770146162061', time=2024-04-17T07:21:02.826993+0000, chain.objs.size()=1 2024-04-17T05:46:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4ADUKE_qtQXGX4TA3js00Ab0VzFp7uD_0 2024-04-17T05:46:02.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3077326374011183887', time=2024-04-17T07:21:04.764061+0000, chain.objs.size()=1 2024-04-17T05:46:02.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4AGobWFWk0iLekfUfV0dQNCk2naPbbA_0 2024-04-17T05:46:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9109657272883112268', time=2024-04-17T07:21:06.871313+0000, chain.objs.size()=1 2024-04-17T05:46:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.saHlGSSJvLeEB98rkp4p1wkikgCd5cV_0 2024-04-17T05:46:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13258463763616060004', time=2024-04-17T07:21:08.189938+0000, chain.objs.size()=1 2024-04-17T05:46:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HqZhRgVLL35JWODX5j5EKXsXCuYFLhx_0 2024-04-17T05:46:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12586950308406431753', time=2024-04-17T07:21:09.102499+0000, chain.objs.size()=1 2024-04-17T05:46:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.p-ihXGCG7hYdxAQsM__0nbLXAOkTcI5_0 2024-04-17T05:46:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9513289292463664456', time=2024-04-17T07:21:09.919585+0000, chain.objs.size()=1 2024-04-17T05:46:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.AfyeD7w-jqH24YbeEXKbDtQRdyX1koh_0 2024-04-17T05:46:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15219863809622144218', time=2024-04-17T07:21:09.964084+0000, chain.objs.size()=1 2024-04-17T05:46:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.v3uHgASphw0O2WXrWZBu5-_tqXM8yvo_0 2024-04-17T05:46:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8810061179044587533', time=2024-04-17T07:21:10.363750+0000, chain.objs.size()=1 2024-04-17T05:46:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2lL33ZuguaNsRLR0ywPcu9YXfVLld7T_0 2024-04-17T05:46:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14723577412688661008', time=2024-04-17T07:21:12.995939+0000, chain.objs.size()=1 2024-04-17T05:46:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.z-YIfwx8cZKsEoQvg4nafAUAAnIbdnE_0 2024-04-17T05:46:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12269657219536258955', time=2024-04-17T07:21:15.764995+0000, chain.objs.size()=1 2024-04-17T05:46:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yvYqCpU7DmUNh4DJSWxxVgU0IUE7JC6_0 2024-04-17T05:46:02.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14612675734722931636', time=2024-04-17T07:21:16.164488+0000, chain.objs.size()=1 2024-04-17T05:46:02.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DFACdx8B8krx2XpCc-rXrG-8l7nCFJq_0 2024-04-17T05:46:02.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13412629906988260255', time=2024-04-17T07:21:16.166135+0000, chain.objs.size()=1 2024-04-17T05:46:02.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TAVN_vnNpjL6B60YIzx88eaHW0VbwSg_0 2024-04-17T05:46:02.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5331980891330185973', time=2024-04-17T07:21:20.055068+0000, chain.objs.size()=1 2024-04-17T05:46:02.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pXTYVt3jrfkT49IQfqfpfkVJNQpt0F3_0 2024-04-17T05:46:02.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.789448856442008650', time=2024-04-17T07:21:20.989731+0000, chain.objs.size()=1 2024-04-17T05:46:02.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.opQEB1mvvHN50zThijiMS4JUUaZWJB0_0 2024-04-17T05:46:02.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12510125558476089577', time=2024-04-17T07:21:21.748129+0000, chain.objs.size()=1 2024-04-17T05:46:02.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Z2X620aWk3-0Zdf5dSTF95w48bfZAaP_0 2024-04-17T05:46:02.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9273730475800758336', time=2024-04-17T07:21:21.852626+0000, chain.objs.size()=1 2024-04-17T05:46:02.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Bde9qi0pqx6UBicENPC1L9ax2MRQU4D_0 2024-04-17T05:46:02.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13336773681909788756', time=2024-04-17T07:21:22.472620+0000, chain.objs.size()=1 2024-04-17T05:46:02.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.x91TbLtXRGdUGGTv4pUGgsz5smjXLzb_0 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5612962144148399121', time=2024-04-17T07:21:23.324331+0000, chain.objs.size()=1 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GNrrLih3APpbAv62eFZs_yE0ErmxQY2_0 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4016690791818476938', time=2024-04-17T07:21:23.687699+0000, chain.objs.size()=1 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xicnh-LLF1OeqdUMU0BNXoUTge8KF9m_0 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18283019414410730862', time=2024-04-17T07:21:24.414740+0000, chain.objs.size()=1 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._nd1v4HtUspQUai6Dpsg_8ymEmjQH2Z_0 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3355273964609282625', time=2024-04-17T07:21:24.756797+0000, chain.objs.size()=1 2024-04-17T05:46:02.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1Ai-Mn_uy0F9-ZWt-t61yeZv5N4vW8c_0 2024-04-17T05:46:02.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4093454395968420845', time=2024-04-17T07:21:28.316539+0000, chain.objs.size()=1 2024-04-17T05:46:02.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xcdTarC7LPX0UznLmXnA5Ld2oU1fO_H_0 2024-04-17T05:46:02.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8367265728427900691', time=2024-04-17T07:21:28.422256+0000, chain.objs.size()=1 2024-04-17T05:46:02.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.va7aUPmvAffbPMgigzGAudA0kebM0T4_0 2024-04-17T05:46:02.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8678242640505334223', time=2024-04-17T07:21:29.205896+0000, chain.objs.size()=1 2024-04-17T05:46:02.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HDozhA5nxyasWmwjX9LKtrmsoiUepE-_0 2024-04-17T05:46:02.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14211746024485097318', time=2024-04-17T07:23:13.037087+0000, chain.objs.size()=1 2024-04-17T05:46:02.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.depMDwVtMbmFDpeB7mas8sTKFqFdDNr_0 2024-04-17T05:46:02.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17131013146042605096', time=2024-04-17T07:23:13.592881+0000, chain.objs.size()=1 2024-04-17T05:46:02.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.u447I6SqM7A9auTArWiIEUgwk983MoD_0 2024-04-17T05:46:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7017046690813578976', time=2024-04-17T07:23:15.390029+0000, chain.objs.size()=1 2024-04-17T05:46:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ExJdUacD-TRJLOEBZD7a-lUg2q_Z9GH_0 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1324980828451471024', time=2024-04-17T07:23:15.587926+0000, chain.objs.size()=1 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1Ah8-yT8j3OSBLgMxx5UFDFJW_oS5q1_0 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15870903985509335658', time=2024-04-17T07:23:15.909026+0000, chain.objs.size()=1 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fBploDQ66nNx9D-Hh7S3bI9NTACbDCf_0 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11442242337508283873', time=2024-04-17T07:23:15.938404+0000, chain.objs.size()=1 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Ugc9KxsmGcjnoSfbpuoCQDHzVh1NXWm_0 2024-04-17T05:46:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8461178328443862509', time=2024-04-17T07:23:17.367819+0000, chain.objs.size()=1 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.k5NXflWknU95NIZTxS6MmmPcp5Sd-tZ_0 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14200179211231954191', time=2024-04-17T07:23:17.504016+0000, chain.objs.size()=1 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oaTqKSuO-mDc2_mpcucwSNBQW0tQ5p__0 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9186702175969910998', time=2024-04-17T07:23:17.928440+0000, chain.objs.size()=1 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DsLFX86sVGDS930k_LKfpKVHr9u-r1r_0 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4519526487456408441', time=2024-04-17T07:23:18.573425+0000, chain.objs.size()=1 2024-04-17T05:46:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0gQ3AoPIqtTANB5GFGYLJRNnW1CcC_g_0 2024-04-17T05:46:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13623896002186286291', time=2024-04-17T07:23:19.297388+0000, chain.objs.size()=1 2024-04-17T05:46:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._zXe3tGEU3MjLAq-E6BRVbao6N7lYTA_0 2024-04-17T05:46:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11110512842199479544', time=2024-04-17T07:23:19.621355+0000, chain.objs.size()=1 2024-04-17T05:46:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YbUxCRTRP96zL-se_yValy71aeSG6na_0 2024-04-17T05:46:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14828497326927193180', time=2024-04-17T07:23:19.814211+0000, chain.objs.size()=1 2024-04-17T05:46:02.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JEdVzV__I4sHws_1JXf8woQvuWSwbDf_0 2024-04-17T05:46:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15401226880140507284', time=2024-04-17T07:23:22.426581+0000, chain.objs.size()=1 2024-04-17T05:46:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qINYzSM6a5q-h0fEQSJM9dO9tO6_OvW_0 2024-04-17T05:46:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16671381171613347720', time=2024-04-17T07:23:23.097554+0000, chain.objs.size()=1 2024-04-17T05:46:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Rplg7mgb8yV8IanNzOzL3jMMdXxy1cN_0 2024-04-17T05:46:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5934841943943874976', time=2024-04-17T07:23:25.943076+0000, chain.objs.size()=1 2024-04-17T05:46:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.f2IZ6XbkYG8qyGSZLfSTMt7Bj5T9t_D_0 2024-04-17T05:46:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2541975266637373626', time=2024-04-17T07:23:26.621394+0000, chain.objs.size()=1 2024-04-17T05:46:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WDT09buITFx5g47MSnwEtZOLz2VGDXg_0 2024-04-17T05:46:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1310602058777497560', time=2024-04-17T07:23:27.388528+0000, chain.objs.size()=1 2024-04-17T05:46:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1PJgGsj828wtnXyPRPcE-To8wQ-P8v3_0 2024-04-17T05:46:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.488480084608843191', time=2024-04-17T07:23:30.639939+0000, chain.objs.size()=1 2024-04-17T05:46:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RXXF9kM-psU8neGmqXRbk9gjJ7_lUb__0 2024-04-17T05:46:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12265231841729987976', time=2024-04-17T07:23:30.641972+0000, chain.objs.size()=1 2024-04-17T05:46:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oGv2aP55U4fGiRsL7uRrjAVSwkriGA2_0 2024-04-17T05:46:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9035836543224131743', time=2024-04-17T07:23:30.989717+0000, chain.objs.size()=1 2024-04-17T05:46:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.os8s3aCF08yVZmMvf892KCPZ7wqEACP_0 2024-04-17T05:46:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.731325148200161481', time=2024-04-17T07:23:31.075048+0000, chain.objs.size()=1 2024-04-17T05:46:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.u1ceDRuoZQmG0miSdNxnwClPcmZmGlP_0 2024-04-17T05:46:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7507878891866202102', time=2024-04-17T07:23:31.359991+0000, chain.objs.size()=1 2024-04-17T05:46:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hC9L7-mn3LZ17IIcTwk_aYpMaFOHO03_0 2024-04-17T05:46:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1268639558470734215', time=2024-04-17T07:23:32.155047+0000, chain.objs.size()=1 2024-04-17T05:46:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-IWAilNT2makyjs_lp03D_j4cOp7T-b_0 2024-04-17T05:46:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11509083712169598778', time=2024-04-17T07:23:35.057507+0000, chain.objs.size()=1 2024-04-17T05:46:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uLG5RziAOGEL6-aCAd1n3cOufPxvSPU_0 2024-04-17T05:46:02.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.728+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11084046808242543885', time=2024-04-17T07:23:37.505239+0000, chain.objs.size()=1 2024-04-17T05:46:02.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.728+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.r16se3cwALh29d0ttkr43qcuaSFafki_0 2024-04-17T05:46:02.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.728+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/47088 2024-04-17T05:46:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2024-04-17T05:46:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11034349962646287845', time=2024-04-17T07:23:39.796476+0000, chain.objs.size()=1 2024-04-17T05:46:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ptefpTegxxfyQIRDXxSaO0iAPhwZUuC_0 2024-04-17T05:46:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12230270008215402629', time=2024-04-17T07:23:39.895219+0000, chain.objs.size()=1 2024-04-17T05:46:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OUEV8kzJ7jRhJnwxUGnJPl1IKweMnLT_0 2024-04-17T05:46:02.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6573766968107204757', time=2024-04-17T07:23:40.119844+0000, chain.objs.size()=1 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wyuqfjH8VNOlgGtfemw2NaZSTFiuXVE_0 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10460889514096262938', time=2024-04-17T07:23:41.313471+0000, chain.objs.size()=1 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.npE2-ZloYHjJDk2ybt3lWU5dh0a_Zjy_0 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5897784107037018121', time=2024-04-17T07:23:41.666761+0000, chain.objs.size()=1 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Kiy03uGqrr5cPdVrptFr9VrCdh30H-Q_0 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10919310674082523559', time=2024-04-17T07:24:27.538968+0000, chain.objs.size()=1 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.367__shadow_.SYKn3Fd6sSVnRyhWlef5pWSbA2UYXIq_0 2024-04-17T05:46:02.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10403858125517400084', time=2024-04-17T07:24:46.730990+0000, chain.objs.size()=1 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.384__shadow:eXzzQEnNI3D-7XWDWE.O.cioF79PFIC_.K0w7jyZZm3e1p9zyt5CSFdYR_Qr5N03_0 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12470139926826450306', time=2024-04-17T07:25:16.647763+0000, chain.objs.size()=1 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.420__shadow_.SQKmTFuBjqy0ulDmCshrubLQ--svA5__0 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17242523009021660398', time=2024-04-17T07:31:39.216235+0000, chain.objs.size()=1 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:pr6AVdpZvz-ccWqvOg2sufla25MoLUA_.8_5d8uvrzJmQ1YNI4pt9BvbYWUynvkf_0 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4091674805292911163', time=2024-04-17T07:31:40.502807+0000, chain.objs.size()=1 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:cryPfsesA5DYmyuzGHK9OzHAXxOv5Gw_.HhtjutIDvnL_6XVGiWs87fTVLMljXps_0 2024-04-17T05:46:02.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6181053016925430762', time=2024-04-17T07:32:05.627946+0000, chain.objs.size()=1 2024-04-17T05:46:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow_.--vDt6tuc7FuwHgf3LIQ6iC8ThFYjn7_0 2024-04-17T05:46:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10632903752779467979', time=2024-04-17T07:32:22.342653+0000, chain.objs.size()=1 2024-04-17T05:46:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:BionuhGl.M178EiGSQUSD6I3P6qV9hj_.K6hdO1HHOgCVlZ5TBhgUUPlOWcZHRW8_0 2024-04-17T05:46:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17676621950393342460', time=2024-04-17T07:32:23.278372+0000, chain.objs.size()=1 2024-04-17T05:46:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:DhfiPkUMNfNW6p2CnHAjIYWq8i3Ya.e_.6eEOLgcCRtuxyZfPOJnKQd0M2GaqAKj_0 2024-04-17T05:46:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3426779016612136107', time=2024-04-17T07:32:43.687185+0000, chain.objs.size()=1 2024-04-17T05:46:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:9DM1nQzeut.L7fZvQhXMvXbUItmQQsZ_.5d_2GIfKkmNDHI2FbQcayhcI2-mwmV5_0 2024-04-17T05:46:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18035615087097382502', time=2024-04-17T07:33:29.419069+0000, chain.objs.size()=1 2024-04-17T05:46:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:rueBSOtNlP.ukIzn.7fv.NdGk7DFPtu_.orH9uene7W_wA13hC1aj_QDfCNYwCFY_0 2024-04-17T05:46:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7698547524233078890', time=2024-04-17T07:34:03.929573+0000, chain.objs.size()=1 2024-04-17T05:46:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.596__shadow_.plGs9ngg_H2K-afgF1nbiCjBmoYzJzz_0 2024-04-17T05:46:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17754359611356307166', time=2024-04-17T07:36:50.624354+0000, chain.objs.size()=1 2024-04-17T05:46:02.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.631__shadow_.BbuH8rn2uwD0Aa0rgN0pRHVkn-XLLcE_0 2024-04-17T05:46:02.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.740+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-17T05:46:02.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45284' 2024-04-17T05:46:02.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5171546346607824347', time=2024-04-17T07:10:35.925038+0000, chain.objs.size()=1 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.156__shadow_.-sQd8ZK8UjdA8G7YEVLx5A0e7GabfEb_0 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3852357380565941767', time=2024-04-17T07:14:28.114412+0000, chain.objs.size()=1 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.221__shadow_.g4ULY82Esqix2pXroFtUXkXTnOAkjYX_0 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.689266328203063374', time=2024-04-17T07:14:46.609483+0000, chain.objs.size()=1 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.232__shadow_.oP2GNOCAlKqVoKNOhl9kmY8pRgfm40d_0 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8868516871528981231', time=2024-04-17T07:14:57.793906+0000, chain.objs.size()=1 2024-04-17T05:46:02.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.238__shadow_.IsW8TTXpNXCQdOFdjgoG7TxhgrTsebl_0 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8664059989162730531', time=2024-04-17T07:15:12.249850+0000, chain.objs.size()=1 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.247__shadow_.p4vsenksIMAgaaNpulqwomwkuT47vAF_0 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2217363045247844742', time=2024-04-17T07:15:15.242126+0000, chain.objs.size()=1 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.249__shadow_.OYv5xKMlrHc-T2yPUfh9KslAGZQFeHP_0 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1062384024214953005', time=2024-04-17T07:17:01.809880+0000, chain.objs.size()=1 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lx83jbwUZBoJVbyV5HAdDwCDiOMsmX-_0 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15506588522337252450', time=2024-04-17T07:17:02.542770+0000, chain.objs.size()=1 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QuAcNaoaC5Ajom1lVvbb9jptxCQbreg_0 2024-04-17T05:46:02.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17128217976632815860', time=2024-04-17T07:17:02.792511+0000, chain.objs.size()=1 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Lt01Zk7Nlig17CqhCxIP9iLfw95kGMZ_0 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8815261602649399348', time=2024-04-17T07:17:02.794182+0000, chain.objs.size()=1 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aR2x5Ig4ofCfjyIH5-Fi0qHTR9RgRMK_0 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8301112578269312285', time=2024-04-17T07:17:03.493234+0000, chain.objs.size()=1 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.IsWs0KDvP5jJh23hMw5ofulE0TvgXEy_0 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13360319171085471327', time=2024-04-17T07:17:03.957969+0000, chain.objs.size()=1 2024-04-17T05:46:02.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VbhRsXH6uY5scbvGXyY9dgm7C_O6i9d_0 2024-04-17T05:46:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5466420834280074024', time=2024-04-17T07:17:06.107737+0000, chain.objs.size()=1 2024-04-17T05:46:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3cpNuomZ_G9njg_ERDXznvgIbda1an__0 2024-04-17T05:46:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7096638995445385261', time=2024-04-17T07:17:06.655065+0000, chain.objs.size()=1 2024-04-17T05:46:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.W1d4j-OY2oHuOKAUMgvNPgQdfAGOBY5_0 2024-04-17T05:46:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9893684553976141905', time=2024-04-17T07:17:08.347893+0000, chain.objs.size()=1 2024-04-17T05:46:02.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZxXqtuUvjsHVl0jQUuhZVAh_nphQKu-_0 2024-04-17T05:46:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11588988720784884558', time=2024-04-17T07:17:08.400457+0000, chain.objs.size()=1 2024-04-17T05:46:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.WlMowQV7RhGLwkVWjLe9BwzW1PX1FvE_0 2024-04-17T05:46:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9168872422460258475', time=2024-04-17T07:17:10.764800+0000, chain.objs.size()=1 2024-04-17T05:46:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XJer1S5JZeGW5kpoycwkkzKQiWhmc6a_0 2024-04-17T05:46:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10767755844016329402', time=2024-04-17T07:17:12.476622+0000, chain.objs.size()=1 2024-04-17T05:46:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.j8f1jAddluTKzYgIzBFu25BUu-iMw4K_0 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7446837572328737449', time=2024-04-17T07:17:12.725535+0000, chain.objs.size()=1 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YroQK8PG8ht6Sq8pI8myRvJ-9R69hGd_0 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6794541366376528145', time=2024-04-17T07:17:12.757596+0000, chain.objs.size()=1 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XBjCmuyaY-ww6OgJm2YmDddKCv60HPX_0 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6460758486969442546', time=2024-04-17T07:17:16.179636+0000, chain.objs.size()=1 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ctYtnQdVN2wGkxzDsYXaTy-uxV3Nb4Q_0 2024-04-17T05:46:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1580589618968496927', time=2024-04-17T07:17:16.605834+0000, chain.objs.size()=1 2024-04-17T05:46:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OFuQW98Dux_QpUCpfKhXMLha4SkkbOY_0 2024-04-17T05:46:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1864377191018851287', time=2024-04-17T07:17:16.607935+0000, chain.objs.size()=1 2024-04-17T05:46:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.oOimg8I7BMRJrHHcD3RRj5hgfZnqPy6_0 2024-04-17T05:46:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18445053387405564576', time=2024-04-17T07:17:17.007565+0000, chain.objs.size()=1 2024-04-17T05:46:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CyqYTSyJaQN6Q3posuCf6GnACWkuSJY_0 2024-04-17T05:46:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17238700082402505839', time=2024-04-17T07:17:18.870051+0000, chain.objs.size()=1 2024-04-17T05:46:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dxMyKjhDWN-1tKzbKuG2WdEAd82WJPO_0 2024-04-17T05:46:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.106524661869659953', time=2024-04-17T07:17:19.188255+0000, chain.objs.size()=1 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6kUXX-vSIBhm2PkNZ7w0pR_pcaS8jbB_0 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7491645678240929188', time=2024-04-17T07:17:19.400201+0000, chain.objs.size()=1 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GVZn31au_XG1jlX0-vU3hB540wCIhX__0 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1991968231033591656', time=2024-04-17T07:17:20.033760+0000, chain.objs.size()=1 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lYhfoRMFGX2xtPb6UUTdqRH69Ch3RHv_0 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18242541674442251217', time=2024-04-17T07:17:20.971035+0000, chain.objs.size()=1 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rN8gIesM4DKBS4eLqqS2tObK-0jjX5o_0 2024-04-17T05:46:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14996347356071752449', time=2024-04-17T07:17:24.801522+0000, chain.objs.size()=1 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.V1CuHS9Pyj5jQcFkVDZbyHxaVA5D9BM_0 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14218616271427247367', time=2024-04-17T07:17:26.534316+0000, chain.objs.size()=1 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wxwg7WX-D5PD3E8SoYx76hakaojn4dJ_0 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13215671037486615988', time=2024-04-17T07:17:27.596835+0000, chain.objs.size()=1 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ue4xRvTSk2b-Bs4eQErTdJ2Q2EYljdk_0 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13761810791996887137', time=2024-04-17T07:17:28.124212+0000, chain.objs.size()=1 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Dav-DILioOuel75-NLYRRnRiWZpZCLL_0 2024-04-17T05:46:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17184133910526314230', time=2024-04-17T07:17:28.321292+0000, chain.objs.size()=1 2024-04-17T05:46:02.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qEMCW_dXG37vQHf8PgwiKOOSRwzPojX_0 2024-04-17T05:46:02.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16677456205580202810', time=2024-04-17T07:17:28.426786+0000, chain.objs.size()=1 2024-04-17T05:46:02.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2O3mQtuoP51sOdMILRdm2ErIBQ5wFG__0 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7854102572113342733', time=2024-04-17T07:17:29.158720+0000, chain.objs.size()=1 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vntzUQSPbw48vgoCiycwUqWRNTZ_X_u_0 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12780655506414035407', time=2024-04-17T07:17:29.161049+0000, chain.objs.size()=1 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lhZnL0hmKz-YAvQh8_Rsi-wmAL-ZmN6_0 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8350324385118219227', time=2024-04-17T07:17:29.924157+0000, chain.objs.size()=1 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MzklySQTITzeJ2uIoycVz8X9Mp6xg4J_0 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14483396282606931865', time=2024-04-17T07:17:29.927643+0000, chain.objs.size()=1 2024-04-17T05:46:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qPvQ4XUp9BeGyH4UPu4YLZ5wdqWuhSx_0 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15534308121028702735', time=2024-04-17T07:17:30.662732+0000, chain.objs.size()=1 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iupHDj7Qfv3ftCv8OT64biOuiAZM3qC_0 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15283343499959133969', time=2024-04-17T07:17:31.465135+0000, chain.objs.size()=1 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1-3SbayMmUfKFuYukxlB61jUdItyCfw_0 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18135192168386062617', time=2024-04-17T07:17:31.822812+0000, chain.objs.size()=1 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XPpyyJLhEpifkUH1Dm55mcEdrrLacij_0 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18442376736398292763', time=2024-04-17T07:17:39.912221+0000, chain.objs.size()=1 2024-04-17T05:46:02.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.257__shadow_.q1adknjydo4FBjuQV5VaWrer3d_1mc4_0 2024-04-17T05:46:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16385155922406131387', time=2024-04-17T07:17:53.628486+0000, chain.objs.size()=1 2024-04-17T05:46:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.266__shadow_.RKlbyG69yE7uovtTsB_3TjrBJU7sN-Y_0 2024-04-17T05:46:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10798049885578491418', time=2024-04-17T07:18:03.322319+0000, chain.objs.size()=1 2024-04-17T05:46:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.272__shadow_.ErirnZKsLhR7gGiPInq_gb1fDl8XqQU_0 2024-04-17T05:46:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17184445169241988856', time=2024-04-17T07:18:16.517069+0000, chain.objs.size()=1 2024-04-17T05:46:02.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.280__shadow_.ToZYQ5yBWYZY1Xd80vWkMo6RDp5fsMs_0 2024-04-17T05:46:02.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8041752259080709520', time=2024-04-17T07:18:23.240690+0000, chain.objs.size()=1 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.n5RgrLE1fz7jDre466ORSm8bBcJPT-J_0 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2604732239319343986', time=2024-04-17T07:18:27.916269+0000, chain.objs.size()=1 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.C32Jcn373ccZ3nNAsU-2C-aRy6whiaR_0 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2865674860464322970', time=2024-04-17T07:18:28.248250+0000, chain.objs.size()=1 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.0SpWueNu2IUWqZ5-gLTF_Rer63opZ8e_0 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10395999255266639731', time=2024-04-17T07:18:29.891972+0000, chain.objs.size()=1 2024-04-17T05:46:02.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.284__shadow_.FQgrc6qPV9uzXqyquA3GB5bXZZO8vP4_0 2024-04-17T05:46:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3287549982294237473', time=2024-04-17T07:18:34.915836+0000, chain.objs.size()=1 2024-04-17T05:46:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.287__shadow_.ykjUDxBoMHxckUE70XUVraoQUvLlQWx_0 2024-04-17T05:46:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5549335743022961769', time=2024-04-17T07:18:44.981125+0000, chain.objs.size()=1 2024-04-17T05:46:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.293__shadow_.E8HHdRYFVj6od2w6jPHr67RlvcXa-tB_0 2024-04-17T05:46:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15499542846083096831', time=2024-04-17T07:19:11.284458+0000, chain.objs.size()=1 2024-04-17T05:46:02.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.309__shadow:ntzufNJRRW7uiD90EvHG0VlpI94dK.s_.jk2f2z5FQgv-E8g2c1IlfVMh7-3dMAe_0 2024-04-17T05:46:02.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16481414651634626982', time=2024-04-17T07:19:15.445601+0000, chain.objs.size()=1 2024-04-17T05:46:02.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.312__shadow_.2wWfTRVjM0kyK4qLvLfLe4n2yzPRc6a_0 2024-04-17T05:46:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1826864318093474197', time=2024-04-17T07:20:59.795733+0000, chain.objs.size()=1 2024-04-17T05:46:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UnwvzOySF1uZtCGWqXVhbbLvLCl9J9v_0 2024-04-17T05:46:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16231538787524774198', time=2024-04-17T07:21:02.777595+0000, chain.objs.size()=1 2024-04-17T05:46:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ttcVocW8Pi2ozu03wBV4wY_k7fCBYvi_0 2024-04-17T05:46:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.300163491375746836', time=2024-04-17T07:21:02.832881+0000, chain.objs.size()=1 2024-04-17T05:46:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1FOHNa-boxQqMJ67g191RCxg9_P6ZtT_0 2024-04-17T05:46:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9714886187413953021', time=2024-04-17T07:21:05.005317+0000, chain.objs.size()=1 2024-04-17T05:46:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0ruugHK5EFXcVeBkKT9Gm7MCt0ffZxm_0 2024-04-17T05:46:02.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11242520526136148541', time=2024-04-17T07:21:05.957755+0000, chain.objs.size()=1 2024-04-17T05:46:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fJiBb_Ps24EvZMDs47TwHBfajomqL5K_0 2024-04-17T05:46:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11036357796112460532', time=2024-04-17T07:21:06.768823+0000, chain.objs.size()=1 2024-04-17T05:46:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fdaA9Vlfy4_EjX2P0qFk4dcXq8YJWnD_0 2024-04-17T05:46:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14193930377905920552', time=2024-04-17T07:21:09.186558+0000, chain.objs.size()=1 2024-04-17T05:46:02.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6N15FZmPT82eG0AwXnGRk1gPpuHHT6Q_0 2024-04-17T05:46:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1806212416125841399', time=2024-04-17T07:21:09.248389+0000, chain.objs.size()=1 2024-04-17T05:46:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wT_TlX5FVbAUbrtZTVxR1mjsrwgAxZ2_0 2024-04-17T05:46:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4161398127709820581', time=2024-04-17T07:21:10.902502+0000, chain.objs.size()=1 2024-04-17T05:46:02.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vZNZDu2xn4QZnoe78wiKmSu_juz4Ula_0 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7098717290548904525', time=2024-04-17T07:21:12.079702+0000, chain.objs.size()=1 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9hO0sbVz9bP_Rzb2UyCE_pkhTOAbbKw_0 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1897021607401959467', time=2024-04-17T07:21:12.242091+0000, chain.objs.size()=1 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Bf9pABj5Hge45e_Q-fKiiITQC-pIXcm_0 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9251659722431244861', time=2024-04-17T07:21:12.316817+0000, chain.objs.size()=1 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Aq6p5jhnOy79hxoDG__WGQ4Ggbx179S_0 2024-04-17T05:46:02.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13851297844554042190', time=2024-04-17T07:21:13.554245+0000, chain.objs.size()=1 2024-04-17T05:46:02.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.s9Ql2adAi7Ra5zgsA-5n99v7CYJXoFO_0 2024-04-17T05:46:02.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14116940397143224103', time=2024-04-17T07:21:14.213993+0000, chain.objs.size()=1 2024-04-17T05:46:02.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3cKas-dCtBLJ4LSGKlw59o2MBAnbyk1_0 2024-04-17T05:46:02.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16281915250441924307', time=2024-04-17T07:21:14.828652+0000, chain.objs.size()=1 2024-04-17T05:46:02.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.40NnvM_TblNcpv6qMNr5QNdwRSZCkL__0 2024-04-17T05:46:02.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8231195512656534113', time=2024-04-17T07:21:15.863912+0000, chain.objs.size()=1 2024-04-17T05:46:02.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mCfsU0wEAyQ2s2J45s1khJJ026mHEyz_0 2024-04-17T05:46:02.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.233054568222994473', time=2024-04-17T07:21:18.168406+0000, chain.objs.size()=1 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WmeoeLfojOXe9Yt0_5RJu5VNwrn3wQh_0 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2405828960629922740', time=2024-04-17T07:21:21.843996+0000, chain.objs.size()=1 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Wwa__P3H9p8O8McZ5wSgwiUUhZwsTVr_0 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8493261965886144734', time=2024-04-17T07:21:21.925943+0000, chain.objs.size()=1 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XflpM74y23jKuVmExK5LMyrPNBxHJ_8_0 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5577040025741181258', time=2024-04-17T07:21:23.724343+0000, chain.objs.size()=1 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fKEC85ElndGhdzDHPI1weShvpY7iPR8_0 2024-04-17T05:46:02.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5018195800285218931', time=2024-04-17T07:21:24.068261+0000, chain.objs.size()=1 2024-04-17T05:46:02.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jz0L6aPrW0_d629Jx05nx5VPvNNrWAa_0 2024-04-17T05:46:02.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8986562244112042501', time=2024-04-17T07:21:24.609095+0000, chain.objs.size()=1 2024-04-17T05:46:02.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sA0rPIVivDK5BR4HM5m2VPVlLZA3f6E_0 2024-04-17T05:46:02.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13673407076108321501', time=2024-04-17T07:21:24.728331+0000, chain.objs.size()=1 2024-04-17T05:46:02.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u6VJ09U9JlaXs_Nuo4M0HAqDfksgGGq_0 2024-04-17T05:46:02.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3196688029079489648', time=2024-04-17T07:21:25.436552+0000, chain.objs.size()=1 2024-04-17T05:46:02.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PX7njHqfXQqWKkSJC2TkSvXPJzMLdEs_0 2024-04-17T05:46:02.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11386411509057754699', time=2024-04-17T07:21:25.547042+0000, chain.objs.size()=1 2024-04-17T05:46:02.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mBmESRNiBMeG7692i2MHlpMG3Fq8nGt_0 2024-04-17T05:46:02.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7371188525208968672', time=2024-04-17T07:21:25.870114+0000, chain.objs.size()=1 2024-04-17T05:46:02.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t_OCt909NX8RE-e6M-7owefMj6R6f5b_0 2024-04-17T05:46:02.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5922650553806431364', time=2024-04-17T07:21:28.334167+0000, chain.objs.size()=1 2024-04-17T05:46:02.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.w3fER9hQ0VaiyvW1D2-yNFhN29VnJlt_0 2024-04-17T05:46:02.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17871939822456229112', time=2024-04-17T07:21:28.912321+0000, chain.objs.size()=1 2024-04-17T05:46:02.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GcqUNRcwMZg4bMUiB-Y-CnLp3If4h_r_0 2024-04-17T05:46:02.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3967211024158045562', time=2024-04-17T07:23:13.564769+0000, chain.objs.size()=1 2024-04-17T05:46:02.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WibtZDTUqXJxSLaPdz7J0iPZ2-iWz8A_0 2024-04-17T05:46:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12599491612267703991', time=2024-04-17T07:23:14.924792+0000, chain.objs.size()=1 2024-04-17T05:46:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hkKYzlitAGpWwPkxjZa8q1JJdZnWwvw_0 2024-04-17T05:46:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16776337992794681956', time=2024-04-17T07:23:17.092477+0000, chain.objs.size()=1 2024-04-17T05:46:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.drlzuCoozpqDJjimYV5E3KUVi74seQm_0 2024-04-17T05:46:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15973795735894923398', time=2024-04-17T07:23:17.893068+0000, chain.objs.size()=1 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bXZn8Jat0akYVlruYL8Uw1pjVfbT3gk_0 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12463003179373067057', time=2024-04-17T07:23:19.922018+0000, chain.objs.size()=1 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Fg64-UC-PQ6TnrHvTksXaE1gw0MLyyW_0 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12290780141368283667', time=2024-04-17T07:23:21.905146+0000, chain.objs.size()=1 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._JNUkVVHG8bGDxu_ctQqUtHIESBENz8_0 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14514358282710045419', time=2024-04-17T07:23:22.423580+0000, chain.objs.size()=1 2024-04-17T05:46:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VXCh3zoJmuJ-MPdiU6gnH5JxRATLYEI_0 2024-04-17T05:46:02.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11463996708226041864', time=2024-04-17T07:23:23.385710+0000, chain.objs.size()=1 2024-04-17T05:46:02.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vkSPruR5WHLjkb3T6l0Db1GZfu1lVsq_0 2024-04-17T05:46:02.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9939970243280120224', time=2024-04-17T07:23:24.770656+0000, chain.objs.size()=1 2024-04-17T05:46:02.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.T6uRoHr-FCnTDV7kISWIoIF_42A61Dc_0 2024-04-17T05:46:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3400589088935959917', time=2024-04-17T07:23:25.204059+0000, chain.objs.size()=1 2024-04-17T05:46:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.163owoGz5dR19UmH4EjbgNmGE0geNVZ_0 2024-04-17T05:46:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7237493487888763602', time=2024-04-17T07:23:25.205739+0000, chain.objs.size()=1 2024-04-17T05:46:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2yVh7d3ThjgDdqCs3CrQr0fQvCJeoHU_0 2024-04-17T05:46:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3802533914047992747', time=2024-04-17T07:23:26.487871+0000, chain.objs.size()=1 2024-04-17T05:46:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.I9uaHt7Jqo-hb7TkhteYM3NzSMEVwsJ_0 2024-04-17T05:46:02.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7048374624993900536', time=2024-04-17T07:23:28.403558+0000, chain.objs.size()=1 2024-04-17T05:46:02.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Re7gEIIooFBz8WADSXmgYY-QlGYCO6d_0 2024-04-17T05:46:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18328036987517125027', time=2024-04-17T07:23:28.438931+0000, chain.objs.size()=1 2024-04-17T05:46:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nL9zbyukqbnr0BYH-R_OtC82MBm55g6_0 2024-04-17T05:46:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7883198976295304408', time=2024-04-17T07:23:29.259494+0000, chain.objs.size()=1 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DQGLLAR1W_RQED6Z3tt2fOTVmBAPTTa_0 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14746035841429985467', time=2024-04-17T07:23:34.295786+0000, chain.objs.size()=1 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.90qXlTrhmGtgTjOICgMjwdwh3eIow5O_0 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11527906042536178772', time=2024-04-17T07:23:34.395040+0000, chain.objs.size()=1 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RV4HJRsV1puqkb6R2rgBONyFfsTDdNG_0 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2320141057566923437', time=2024-04-17T07:23:34.811011+0000, chain.objs.size()=1 2024-04-17T05:46:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YVAh8oxx_1az82k5kR4epO23xH8FYaF_0 2024-04-17T05:46:02.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45284 2024-04-17T05:46:02.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=24, truncated=0, next_marker='' 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12204675835819930405', time=2024-04-17T07:23:35.910085+0000, chain.objs.size()=1 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SUxtiD2yRd_885WfqCW_ido-hceTCfG_0 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4362295214539955971', time=2024-04-17T07:23:36.639116+0000, chain.objs.size()=1 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lhPvCeP6GsVAI-I1riRdUgfYTp5klOb_0 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2023216761384663541', time=2024-04-17T07:23:37.096017+0000, chain.objs.size()=1 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ddTUBL-wqeBAB143auUzrVw11Z-OryJ_0 2024-04-17T05:46:02.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14567222876159700283', time=2024-04-17T07:23:37.786330+0000, chain.objs.size()=1 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J2GNhWPG0kVJP4ANaiM4haMST5HpxRM_0 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15829795239040216052', time=2024-04-17T07:23:38.637631+0000, chain.objs.size()=1 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KEcC5KCtaiqdWIGM6esHa_AN_XnhK98_0 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8507592268622394441', time=2024-04-17T07:23:39.443001+0000, chain.objs.size()=1 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fRa-tqckMax-edIoKdpVOgVQf6_tG2D_0 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4459612599214317847', time=2024-04-17T07:23:40.261979+0000, chain.objs.size()=1 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.f_11ScfX8DwQa7KrFHRdqLYp6M-OU07_0 2024-04-17T05:46:02.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9653996876548553520', time=2024-04-17T07:23:41.235756+0000, chain.objs.size()=1 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.l1wGOt0qB1or_lkMB6jKRl6liBdfSJF_0 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2805183024015301492', time=2024-04-17T07:23:51.376746+0000, chain.objs.size()=1 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.324__shadow_.4iguZB1bWt9vgPowIbABgE4DKu8DLiW_0 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14211209039849290603', time=2024-04-17T07:23:54.278223+0000, chain.objs.size()=1 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.327__shadow_.zQz731G1NoDVg821Ya8V8Yoxdx3PyVo_0 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12906377297609353344', time=2024-04-17T07:24:41.231343+0000, chain.objs.size()=1 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.380__shadow_.op3GPww7y5K5G0ENvpACob0doqpKRWG_0 2024-04-17T05:46:02.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.812959412227822270', time=2024-04-17T07:26:17.124815+0000, chain.objs.size()=1 2024-04-17T05:46:02.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.sUHX-nZtiIMTavfvQtcPjecmgmBtp1j_0 2024-04-17T05:46:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2669185750244304706', time=2024-04-17T07:27:12.512919+0000, chain.objs.size()=1 2024-04-17T05:46:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.508__multipart_mymultipart.2~EP3rAeT-So4YeYXWfbl3SEE0UbkOozp.1 2024-04-17T05:46:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A', time=2024-04-17T07:28:43.788332+0000, chain.objs.size()=1 2024-04-17T05:46:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A.2 2024-04-17T05:46:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A', time=2024-04-17T07:28:43.798088+0000, chain.objs.size()=1 2024-04-17T05:46:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A.3 2024-04-17T05:46:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1392210776139769461', time=2024-04-17T07:30:52.421103+0000, chain.objs.size()=1 2024-04-17T05:46:02.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.542__shadow_.ekZkwCyr0_B03SGRMrUHimdeRHi2Ck__0 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8404318638968735170', time=2024-04-17T07:31:00.856178+0000, chain.objs.size()=1 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.544__shadow_.FFuCdoUJPDNtCBnxAQhKMmUGKhY59Ie_0 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.422390698033705261', time=2024-04-17T07:31:34.683569+0000, chain.objs.size()=1 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:BJ7vySVHXlLbw2ottPWWk0Q5OA5bWgk_.xPJilRqx_VLP9IC_l3D0tLdUsiwDOR4_0 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.800502715589414132', time=2024-04-17T07:31:58.448795+0000, chain.objs.size()=1 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.559__shadow:tDV8VNlArAQjYrywuIURv7pfU8WD6qY_.Ja_WEGLXnWjtC10-B370EjEQke_AV93_0 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11177539599837399674', time=2024-04-17T07:32:18.875817+0000, chain.objs.size()=1 2024-04-17T05:46:02.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:F.NScfnmp5J8waCgDq7awLLYeO2At-m_.QW8ORj065ziRyXWIrjGZ0qvZOHQgiCB_0 2024-04-17T05:46:02.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13731539977799607963', time=2024-04-17T07:33:00.586041+0000, chain.objs.size()=1 2024-04-17T05:46:02.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:qtyvRvp9NzAmS5.MXt6WHWP-4P98frc_.tvOz3SuQYUloOsGUL79_wNCsHlU7mtD_0 2024-04-17T05:46:02.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11861974840153593597', time=2024-04-17T07:33:46.450990+0000, chain.objs.size()=1 2024-04-17T05:46:02.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:XDOAInlQT3DuDSAFLbo5VLFRLsEGKlq_.ND8zRLiyxz_F9zyH2A0uJ3v780W1wa4_0 2024-04-17T05:46:02.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14983000761015559323', time=2024-04-17T07:37:35.897565+0000, chain.objs.size()=1 2024-04-17T05:46:02.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.648__shadow_.qxHISpzSrhHLP-cCaYQp7yy1ilEDrpG_0 2024-04-17T05:46:02.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6427421295530164766', time=2024-04-17T07:38:19.710850+0000, chain.objs.size()=1 2024-04-17T05:46:02.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.689__shadow:cndYXidQRFMOiFiVMM5TL-SgYRXGJKj_.5bRv6sUiCIc6_X2JhciKEgqa1gxj3qE_0 2024-04-17T05:46:02.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-17T05:46:02.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44250' 2024-04-17T05:46:02.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_SssBr0GUG3d8s-YJQRHxBc7OWeWYRR5', time=2024-04-17T07:10:06.401586+0000, chain.objs.size()=1 2024-04-17T05:46:02.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.Ra47Y3inrtsk99MKgWz_Z03BwQWPJDe_0 2024-04-17T05:46:02.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8730030560980305330', time=2024-04-17T07:14:26.498721+0000, chain.objs.size()=1 2024-04-17T05:46:02.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.220__shadow_.lncW_pWYRay9GEXFDBc5y5RHlfpDGtC_0 2024-04-17T05:46:02.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14348837862057051804', time=2024-04-17T07:14:31.440952+0000, chain.objs.size()=1 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.223__shadow_.4CUIT9LvTSjb_65pWOEXvGP2Q1oVU64_0 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11845053558793127253', time=2024-04-17T07:14:51.234621+0000, chain.objs.size()=1 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.234__shadow_.fxXfwK9JXhmOdeo13rvtETdnlQvihqW_0 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.396158071606005266', time=2024-04-17T07:14:59.380119+0000, chain.objs.size()=1 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.239__shadow_.NGEqEVM3MZf49wN_ueyuVeRAEmSbtKo_0 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1706187471470170875', time=2024-04-17T07:15:13.790347+0000, chain.objs.size()=1 2024-04-17T05:46:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.248__shadow_.weLxXrbjp_BMrrsMHn-ig-dpwnhPzl2_0 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14272619806774721176', time=2024-04-17T07:17:01.547569+0000, chain.objs.size()=1 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ACExIp5p324iCby3G8E8gD9sTPUqtoi_0 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5393773376461154102', time=2024-04-17T07:17:01.987167+0000, chain.objs.size()=1 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kUMMce70h1MZb9kjPrp0tcrvXw2Syh9_0 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6704157921360235039', time=2024-04-17T07:17:03.201868+0000, chain.objs.size()=1 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fo4iBEn_bq_mjCnj2nPP6fbaku0Rbht_0 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1555792392581168427', time=2024-04-17T07:17:03.957060+0000, chain.objs.size()=1 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gA-cdjaA2ZzzuKAbYlgu2qqZN6Zq_KU_0 2024-04-17T05:46:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16626237858060429499', time=2024-04-17T07:17:03.960311+0000, chain.objs.size()=1 2024-04-17T05:46:02.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.R5sIYP8zeChMOVNuQ0ArOJSJSTIPcYz_0 2024-04-17T05:46:02.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13756915822414025489', time=2024-04-17T07:17:04.195189+0000, chain.objs.size()=1 2024-04-17T05:46:02.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sN1CGmOm79nBvic2ginDDbFwax92hxq_0 2024-04-17T05:46:02.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.824+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8958838343977184857', time=2024-04-17T07:17:06.001117+0000, chain.objs.size()=1 2024-04-17T05:46:02.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.824+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RB6FcZ7wnorshpJcFCJVpZ60i4MW65U_0 2024-04-17T05:46:02.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.824+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2876577758919884364', time=2024-04-17T07:17:06.045499+0000, chain.objs.size()=1 2024-04-17T05:46:02.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.824+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fskj8iiMOJhfYBb-1xkIGe4gc2hz_Mf_0 2024-04-17T05:46:02.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.824+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9155812778562112468', time=2024-04-17T07:17:06.221205+0000, chain.objs.size()=1 2024-04-17T05:46:02.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.824+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Or4DTtOYbWiLhzSmNFtcZaV7t413uSF_0 2024-04-17T05:46:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12254480065028821451', time=2024-04-17T07:17:07.727062+0000, chain.objs.size()=1 2024-04-17T05:46:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OdQpSs1v2HCHGe_sDLa9kxsOBZ0Uoxe_0 2024-04-17T05:46:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4658463691495247547', time=2024-04-17T07:17:07.755169+0000, chain.objs.size()=1 2024-04-17T05:46:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hBSqWcY_zQmLNGLpOTriDoJAmJA156w_0 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12168521901181295745', time=2024-04-17T07:17:08.199234+0000, chain.objs.size()=1 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VSPpqNfoY2UUiV3oBPr7MBvrT9ihcok_0 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13960112298109335112', time=2024-04-17T07:17:10.223723+0000, chain.objs.size()=1 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.r88rYq3g6ofbfNYdDwqQvy12U-DhkZD_0 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5576413191833904670', time=2024-04-17T07:17:11.013603+0000, chain.objs.size()=1 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dntTWt6W_4kEWlvtiyTaVoeuG4t4_4X_0 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2803295179989456533', time=2024-04-17T07:17:11.347024+0000, chain.objs.size()=1 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wHR696LfBZv39peTz-SgCjpRnsnSRYT_0 2024-04-17T05:46:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11564146195023000476', time=2024-04-17T07:17:11.357036+0000, chain.objs.size()=1 2024-04-17T05:46:02.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5lOiU3rw4keERzZEJuB9rt1PfoaWh9U_0 2024-04-17T05:46:02.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10100957138004029780', time=2024-04-17T07:17:11.393535+0000, chain.objs.size()=1 2024-04-17T05:46:02.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-F7VEnRI6wJQYLQQEOzqJWF3hvoCuYw_0 2024-04-17T05:46:02.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6987045102305607859', time=2024-04-17T07:17:11.884647+0000, chain.objs.size()=1 2024-04-17T05:46:02.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sX5U8hcM6iZE9a4ZNMHR8__xR6oDgW5_0 2024-04-17T05:46:02.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4256121630339386976', time=2024-04-17T07:17:12.274334+0000, chain.objs.size()=1 2024-04-17T05:46:02.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5tap6suESTMxr3kl0w3mBbSHQYT4gK0_0 2024-04-17T05:46:02.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8840912007344250183', time=2024-04-17T07:17:14.873896+0000, chain.objs.size()=1 2024-04-17T05:46:02.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JH7-hGTT1QHrULe0cV7GQcFK-JzfNQt_0 2024-04-17T05:46:02.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7722610496592971289', time=2024-04-17T07:17:15.039257+0000, chain.objs.size()=1 2024-04-17T05:46:02.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jiyrFUC1uV7idIxCNje9JszA2F1CAjC_0 2024-04-17T05:46:02.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18333336889785421161', time=2024-04-17T07:17:18.384766+0000, chain.objs.size()=1 2024-04-17T05:46:02.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tGtFy_BTvggvrAlRhHatxEVpzmGZnTn_0 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6167597238651206210', time=2024-04-17T07:17:20.031728+0000, chain.objs.size()=1 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vGH6F7QNubjyQYf70b9mcOZ6Jmy7vWU_0 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16412521986687989288', time=2024-04-17T07:17:20.042696+0000, chain.objs.size()=1 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uKLFhVNwDt5CMH-H-3F1NB03IIHd0Wh_0 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16884057115891355457', time=2024-04-17T07:17:20.580069+0000, chain.objs.size()=1 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pIFPw_XyRpxB8AHkjnQMp8sJNe5PpJ-_0 2024-04-17T05:46:02.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11066235935556874138', time=2024-04-17T07:17:20.774689+0000, chain.objs.size()=1 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OZOprnTOs62ePAjJqGE2A4T3RxORWKJ_0 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6463746843372500015', time=2024-04-17T07:17:21.055179+0000, chain.objs.size()=1 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.76Locgzrcb_aoKtHq_-fCM3lY2uZv6K_0 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2989763417435037693', time=2024-04-17T07:17:22.062524+0000, chain.objs.size()=1 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.N_S3Zn9lKjJsVoK5anWMOsWR9IQKHUl_0 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12382652678231274174', time=2024-04-17T07:17:23.207402+0000, chain.objs.size()=1 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.88aAlDfgb78jpylnzdARh5c6wNDp950_0 2024-04-17T05:46:02.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6983989509810867413', time=2024-04-17T07:17:23.662125+0000, chain.objs.size()=1 2024-04-17T05:46:02.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.T2HQ9EvZf-Moy_o4eMhpZS54My8kPz9_0 2024-04-17T05:46:02.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1675859973998434766', time=2024-04-17T07:17:24.058265+0000, chain.objs.size()=1 2024-04-17T05:46:02.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tLSWVN7ImYOK9tf-yeY8s6DA-Z8oqOV_0 2024-04-17T05:46:02.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1021024582837951085', time=2024-04-17T07:17:25.085502+0000, chain.objs.size()=1 2024-04-17T05:46:02.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TCNPI2ndcuGkuJCsYbEEnf7tmn_8W0N_0 2024-04-17T05:46:02.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17201904113848405063', time=2024-04-17T07:17:26.595951+0000, chain.objs.size()=1 2024-04-17T05:46:02.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wrQVuArIpRxqOuZtTdPncJ7ST4vntmW_0 2024-04-17T05:46:02.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7653268935331608487', time=2024-04-17T07:17:27.854262+0000, chain.objs.size()=1 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.AC2YCH4qlZsDGdCDqJTulmAkSNH7j3L_0 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13961755591226076984', time=2024-04-17T07:17:27.888998+0000, chain.objs.size()=1 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4ofdzNu0DvxNNcOuewEik_gVBSmec7n_0 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8568308278348720489', time=2024-04-17T07:17:28.621230+0000, chain.objs.size()=1 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZHTgZlDerq6TrD9iiRr210-E1VLuAbN_0 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12034881957985212390', time=2024-04-17T07:17:28.698548+0000, chain.objs.size()=1 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KnKVs-OVFL_CzkQiZwPTUhht3v0GCiO_0 2024-04-17T05:46:02.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14099774607823663732', time=2024-04-17T07:17:29.153526+0000, chain.objs.size()=1 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pgAx_viod6QhBvgOwpI2yN7zml-o8nr_0 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3335075535279823472', time=2024-04-17T07:17:30.325318+0000, chain.objs.size()=1 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VXz51QB4T5_WtBYULnl3im9FMEIda2W_0 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15774506139053994270', time=2024-04-17T07:17:30.625234+0000, chain.objs.size()=1 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-DnZr0Ssn7260wHhOR-kUhPbMvWEmmy_0 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1225936460420196607', time=2024-04-17T07:17:31.823215+0000, chain.objs.size()=1 2024-04-17T05:46:02.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ugUFDbrSPcHaxYtWj0fIRCUCGBxzu5M_0 2024-04-17T05:46:02.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13907766648144181310', time=2024-04-17T07:17:32.340001+0000, chain.objs.size()=1 2024-04-17T05:46:02.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.uehOwnTVKiOolOytHimKr3nr7jNZleP_0 2024-04-17T05:46:02.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10063738560688249069', time=2024-04-17T07:17:33.912016+0000, chain.objs.size()=1 2024-04-17T05:46:02.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.253__shadow_.8NWRLTr5t9wSb1-gbs3nc40-zz8-lTz_0 2024-04-17T05:46:02.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6077677737485312383', time=2024-04-17T07:17:45.905266+0000, chain.objs.size()=1 2024-04-17T05:46:02.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.261__shadow_.DxcFWD5A6Ebq4q1dbi7sTz71KJ_-tJQ_0 2024-04-17T05:46:02.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3872120771401099021', time=2024-04-17T07:17:51.954409+0000, chain.objs.size()=1 2024-04-17T05:46:02.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.265__shadow_.ZZBEmB2Ry6MEFOjiY9AfA4DrcufPGUQ_0 2024-04-17T05:46:02.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7111993159825178549', time=2024-04-17T07:17:53.605065+0000, chain.objs.size()=1 2024-04-17T05:46:02.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.266__shadow_.RyAqRqY5OJcde43rl5v6Syuo6-RSNm__0 2024-04-17T05:46:02.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1593233203981344649', time=2024-04-17T07:17:55.245997+0000, chain.objs.size()=1 2024-04-17T05:46:02.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.267__shadow_.56w0g3jGYTX2RyKNd3kc8HQ1gflHkbb_0 2024-04-17T05:46:02.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3315181707414634645', time=2024-04-17T07:18:16.503511+0000, chain.objs.size()=1 2024-04-17T05:46:02.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.280__shadow_._B33JR1XvubgvSpwBGA-aVVz8nFCRAr_0 2024-04-17T05:46:02.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12291281531184081907', time=2024-04-17T07:18:46.559743+0000, chain.objs.size()=1 2024-04-17T05:46:02.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.294__shadow_.cjNzK5lLjdjMiD15OEoo8MHmnCs95P0_0 2024-04-17T05:46:02.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5762339608472658959', time=2024-04-17T07:20:59.851416+0000, chain.objs.size()=1 2024-04-17T05:46:02.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aebEAoQvrrlnVdrpy8-WM2_H8sso69K_0 2024-04-17T05:46:02.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3143365287883158152', time=2024-04-17T07:21:03.849789+0000, chain.objs.size()=1 2024-04-17T05:46:02.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JjOCkQuBLJzci4YqbOicbDDRcEPSOji_0 2024-04-17T05:46:02.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2696644066850622581', time=2024-04-17T07:21:04.237556+0000, chain.objs.size()=1 2024-04-17T05:46:02.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dLBfMYpqGqOAXFIoWdTOMVoh0LJnOOh_0 2024-04-17T05:46:02.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16777593698587131417', time=2024-04-17T07:21:04.345200+0000, chain.objs.size()=1 2024-04-17T05:46:02.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4QcjW2-9wJFR-B-3k7lC4tw2ipyfCJ9_0 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18076720816328365643', time=2024-04-17T07:21:04.667004+0000, chain.objs.size()=1 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.18unegSaGTyC1SAI3ZOgO2eRspmSP_Y_0 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11263025345797755369', time=2024-04-17T07:21:05.317871+0000, chain.objs.size()=1 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gWuPcinFVU6t3WaKdwSkyMl50zJH4qC_0 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17391884197439986949', time=2024-04-17T07:21:06.200827+0000, chain.objs.size()=1 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KZ_LJmLm-J3-DMGcijr_9-vbcCKmOj0_0 2024-04-17T05:46:02.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8202724348936496094', time=2024-04-17T07:21:08.361084+0000, chain.objs.size()=1 2024-04-17T05:46:02.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.A_5VmH_G8bAcYwcHhZMrYc9YnEbNZ1M_0 2024-04-17T05:46:02.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.720221595892256773', time=2024-04-17T07:21:08.770480+0000, chain.objs.size()=1 2024-04-17T05:46:02.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xg4oTXGKqy9f7T02x9tnShjLq4SOptZ_0 2024-04-17T05:46:02.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14433884966017345466', time=2024-04-17T07:21:09.357822+0000, chain.objs.size()=1 2024-04-17T05:46:02.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9RpkeTJHAlWqeXuY6JEBVrQ3tF4IaNl_0 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18383300929535539914', time=2024-04-17T07:21:09.576954+0000, chain.objs.size()=1 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0UhQJdWZKdOsoP7-xlHoCQtdlmFQPwx_0 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4731565384315001929', time=2024-04-17T07:21:09.696460+0000, chain.objs.size()=1 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9y9VzHLZz4YEhbTOe7paXisI9rcnBd6_0 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14908247260239766924', time=2024-04-17T07:21:13.197540+0000, chain.objs.size()=1 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8i0T0fctQipO4w9Aq6OTxYXjxnclbMD_0 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15037696455086572721', time=2024-04-17T07:21:13.217600+0000, chain.objs.size()=1 2024-04-17T05:46:02.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZlEc6fFn6NSQZgxvISi65vhO56s4OPJ_0 2024-04-17T05:46:02.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8966055977035147157', time=2024-04-17T07:21:13.371495+0000, chain.objs.size()=1 2024-04-17T05:46:02.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.T_cvCH98h_CYN14VN8zIBUvMwYm9C2N_0 2024-04-17T05:46:02.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14372490792827915224', time=2024-04-17T07:21:17.030288+0000, chain.objs.size()=1 2024-04-17T05:46:02.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vv-899AQCQLCqFa1OI3YbLcWWDGrABQ_0 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8825723801799464735', time=2024-04-17T07:21:17.754801+0000, chain.objs.size()=1 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.w0gYAEVsXq89hwLTJfjVsy6Z80HVAuw_0 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11424983834151666133', time=2024-04-17T07:21:17.945076+0000, chain.objs.size()=1 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gqunv6xfXK0umePVKOkUs-A7kq4dVFu_0 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3117557295310065271', time=2024-04-17T07:21:18.419039+0000, chain.objs.size()=1 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ic96WJOf56Z98kW-s4yC02_6iKUjhrx_0 2024-04-17T05:46:02.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16638237862957484945', time=2024-04-17T07:21:18.597790+0000, chain.objs.size()=1 2024-04-17T05:46:02.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tFK1XKEZSdAZdJdHQBozKaagZ25z6bN_0 2024-04-17T05:46:02.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1832629753362504830', time=2024-04-17T07:21:20.206128+0000, chain.objs.size()=1 2024-04-17T05:46:02.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6jpujGcbQZp2dUZHbQRtCbMcaRLIrlD_0 2024-04-17T05:46:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8757210776277942899', time=2024-04-17T07:21:20.810119+0000, chain.objs.size()=1 2024-04-17T05:46:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.qH8cIa6t4HhcVmdxG51yJaRRpz9VWlV_0 2024-04-17T05:46:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5321192693021319762', time=2024-04-17T07:21:21.675103+0000, chain.objs.size()=1 2024-04-17T05:46:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.e7LjzqvtNmqPmfEVjlcGcjriQ2K2Dol_0 2024-04-17T05:46:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5053801834543244277', time=2024-04-17T07:21:21.896709+0000, chain.objs.size()=1 2024-04-17T05:46:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pf5X3y9phPeXgD9-phSWlxUpQYxFA89_0 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15078152490267973895', time=2024-04-17T07:21:22.482137+0000, chain.objs.size()=1 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pwKnJ6IgRWaAX6VYH-KMhDroarb-JBa_0 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16314693172633194591', time=2024-04-17T07:21:22.495402+0000, chain.objs.size()=1 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yENH7pZRahMwWlIZ1mkg1wYKMdI0l3q_0 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8004828600149673364', time=2024-04-17T07:21:24.237500+0000, chain.objs.size()=1 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3xbLmsOP18Vykw6jGKsGfOwraDeEiNi_0 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3584775830490124435', time=2024-04-17T07:21:25.106457+0000, chain.objs.size()=1 2024-04-17T05:46:02.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mKtShDXAMfPjBD3I_JIEirygDBwY6Ax_0 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15742701618688106620', time=2024-04-17T07:21:25.308102+0000, chain.objs.size()=1 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.AFKx8M5JvLQv4ixBaqzB0H_C5Wqa20I_0 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.779816780034585384', time=2024-04-17T07:21:25.798727+0000, chain.objs.size()=1 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.OJsQaaShgguNMm0x05Cb2GUO75eAQlx_0 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17003086685784593398', time=2024-04-17T07:21:29.399168+0000, chain.objs.size()=1 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pqkL9ycvLJM04TVlPPOE1p9i62sI0gd_0 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13410794668133511984', time=2024-04-17T07:23:12.885790+0000, chain.objs.size()=1 2024-04-17T05:46:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hj9gNrPIp3tNoklQ91ymuVbH-wYpw8j_0 2024-04-17T05:46:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17502134581468280063', time=2024-04-17T07:23:14.333484+0000, chain.objs.size()=1 2024-04-17T05:46:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.b5t5lnjiua3Wn4kX4u31E2Wu2YINakP_0 2024-04-17T05:46:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.540788277871078961', time=2024-04-17T07:23:14.639110+0000, chain.objs.size()=1 2024-04-17T05:46:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8NNToAX7lM02ht3VE4CdVbIpcsKCFed_0 2024-04-17T05:46:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1107930820890516743', time=2024-04-17T07:23:15.307007+0000, chain.objs.size()=1 2024-04-17T05:46:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TAK_wkyAQ1UmhRm7wgkq7SQOLJQF18p_0 2024-04-17T05:46:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3361798719531019969', time=2024-04-17T07:23:15.712880+0000, chain.objs.size()=1 2024-04-17T05:46:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ggVnCV3ZkdOSIclg9jfweWPugiTMjsg_0 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2924530559268128778', time=2024-04-17T07:23:17.073246+0000, chain.objs.size()=1 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DJm34reJEjkCfk1WxH1Z0PAIk4_62Rc_0 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4020725792697383865', time=2024-04-17T07:23:17.516494+0000, chain.objs.size()=1 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Sv-wFvGLw5RDVbwfzQUv2JrFAqBODW1_0 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1465232521812659559', time=2024-04-17T07:23:17.958880+0000, chain.objs.size()=1 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mrMKgCb_rLGVHUXXbtvXZ6LOQ2_R6a6_0 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5329305346285630649', time=2024-04-17T07:23:18.073784+0000, chain.objs.size()=1 2024-04-17T05:46:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1Mezv1Y61zdHFgx0bVB_7MF_rbUWUWJ_0 2024-04-17T05:46:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2364737273904418572', time=2024-04-17T07:23:20.909008+0000, chain.objs.size()=1 2024-04-17T05:46:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nQn0wXHHIAQ6Ix-jDn5i20hEwekNllt_0 2024-04-17T05:46:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17403153229690298478', time=2024-04-17T07:23:21.205845+0000, chain.objs.size()=1 2024-04-17T05:46:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dEH55P5DZRz4wvvv51eTe-3NfmVoRj-_0 2024-04-17T05:46:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12869392888037985465', time=2024-04-17T07:23:21.844109+0000, chain.objs.size()=1 2024-04-17T05:46:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wwBBKAYK0Ek5Naf7NRO8h-OM0cgyfKS_0 2024-04-17T05:46:02.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1644055591072576108', time=2024-04-17T07:23:24.943507+0000, chain.objs.size()=1 2024-04-17T05:46:02.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.zaKMpG9KzBn8phVG6Mz5FU50jS6-k5s_0 2024-04-17T05:46:02.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6022617891762466969', time=2024-04-17T07:23:26.324725+0000, chain.objs.size()=1 2024-04-17T05:46:02.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.grO2u-YbTAlgVEmAxjmZkBH5GXpHxhY_0 2024-04-17T05:46:02.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/44250 2024-04-17T05:46:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=31, truncated=0, next_marker='' 2024-04-17T05:46:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6290360377669732574', time=2024-04-17T07:23:27.203748+0000, chain.objs.size()=1 2024-04-17T05:46:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OoWGAtXVqL7XO5IwWmGiGPhtyf4RM9d_0 2024-04-17T05:46:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15225945960405517356', time=2024-04-17T07:23:27.480090+0000, chain.objs.size()=1 2024-04-17T05:46:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7OCLvHjiyMotjs_xlRY8yBAmbOdOalq_0 2024-04-17T05:46:02.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4001052317706193721', time=2024-04-17T07:23:29.351313+0000, chain.objs.size()=1 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N2IAft3VQS6s7FbCwOpNM_s6ZVk1tRm_0 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17386895641934884014', time=2024-04-17T07:23:30.197166+0000, chain.objs.size()=1 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.t6fxTjc4gCdJ4DzKj4vZoatZ-cbbKpl_0 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10675597674560569918', time=2024-04-17T07:23:30.727734+0000, chain.objs.size()=1 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.LIiEUEoEWdYzwaIm7xKqcVwS_rSg1DT_0 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17564294576941788700', time=2024-04-17T07:23:33.602655+0000, chain.objs.size()=1 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tpGFQe2C6WTV0Xkqh_CSIYtgBMPOzIM_0 2024-04-17T05:46:02.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5009723758525080782', time=2024-04-17T07:23:34.189482+0000, chain.objs.size()=1 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.FIkUg2Z8cNP5CSBgCsta0LxF4VShm7n_0 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4614915199826329729', time=2024-04-17T07:23:34.555019+0000, chain.objs.size()=1 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jv3DnGye95llnmMenJZLagGKVReJO_y_0 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2634485761459970455', time=2024-04-17T07:23:34.851875+0000, chain.objs.size()=1 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.m_sxcYg7uDNrDQkUhuZ-72DyTvCrZeG_0 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9244322215897996101', time=2024-04-17T07:23:35.885736+0000, chain.objs.size()=1 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.q6v3A-zZD0XpzuqSgBMeONdDv4QjCsY_0 2024-04-17T05:46:02.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1956572619607470321', time=2024-04-17T07:23:35.895122+0000, chain.objs.size()=1 2024-04-17T05:46:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6WDVH4gyzGdnKAG0g55NlhbgCrzeZgl_0 2024-04-17T05:46:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2273993355904569439', time=2024-04-17T07:23:37.771052+0000, chain.objs.size()=1 2024-04-17T05:46:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4eCBc8swbXbXxvkJ3ZY3OBiY2JvK94G_0 2024-04-17T05:46:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13356279984728198309', time=2024-04-17T07:23:37.907665+0000, chain.objs.size()=1 2024-04-17T05:46:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eI8OSzgw07EZldqbhD7cqMB3eLrCSmf_0 2024-04-17T05:46:02.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4791669300577177587', time=2024-04-17T07:23:38.136407+0000, chain.objs.size()=1 2024-04-17T05:46:02.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wFm64LNvqgG6569-XUZCBWbESx9fV8R_0 2024-04-17T05:46:02.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1053503154203558628', time=2024-04-17T07:23:39.889929+0000, chain.objs.size()=1 2024-04-17T05:46:02.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VvHjSoWOEuPnxEbLY4u5Ax9iWcypWbx_0 2024-04-17T05:46:02.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10926278170582704519', time=2024-04-17T07:23:40.435139+0000, chain.objs.size()=1 2024-04-17T05:46:02.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xhFGNcoa6c7WU6JzMmNk0A2U0s3BeVA_0 2024-04-17T05:46:02.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3443811364548531158', time=2024-04-17T07:23:40.935724+0000, chain.objs.size()=1 2024-04-17T05:46:02.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8AUcmNN4ZbLdaf2VZpKAmO97lvPm3rR_0 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10102755798483545296', time=2024-04-17T07:23:41.238133+0000, chain.objs.size()=1 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MG0vA9OuG4q_25jqtFFb9fj7C3O131D_0 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17192980439857964684', time=2024-04-17T07:23:41.575890+0000, chain.objs.size()=1 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eiQmI89mFnzLtvuzb1_q_qIJQnv66Su_0 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.635744365537207521', time=2024-04-17T07:23:50.025456+0000, chain.objs.size()=1 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.323__shadow_.2nF5vdayLq8ky5sICUahnx9GDKnsFZH_0 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16705822431690604984', time=2024-04-17T07:24:20.132970+0000, chain.objs.size()=1 2024-04-17T05:46:02.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.360__shadow_.soksfe9XFgBi9QT2YBRa87hMGISzTH3_0 2024-04-17T05:46:02.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8344601663633198764', time=2024-04-17T07:25:19.702946+0000, chain.objs.size()=1 2024-04-17T05:46:02.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.423__shadow_.hAkwdeo5PfsrMjjbHZRIGpaDvCN3t9q_0 2024-04-17T05:46:02.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11337050238324819410', time=2024-04-17T07:29:36.948080+0000, chain.objs.size()=1 2024-04-17T05:46:02.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.514__shadow_.jH2G5i_AxrcTZgfgprFUSH2AVVPSrtz_0 2024-04-17T05:46:02.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11383874963127520253', time=2024-04-17T07:30:21.056028+0000, chain.objs.size()=4 2024-04-17T05:46:02.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.525__multipart_mymultipart.2~byucKcSDSOV9JhqxWCcILpinMnNgPn5.1 2024-04-17T05:46:02.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.880+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.525__multipart_mymultipart.2~byucKcSDSOV9JhqxWCcILpinMnNgPn5.2 2024-04-17T05:46:02.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.525__multipart_mymultipart.KgTXB0XhCC3OUU7XEP_aNg4FEOwzG_hB.3 2024-04-17T05:46:02.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.525__multipart_mymultipart.2~byucKcSDSOV9JhqxWCcILpinMnNgPn5.4 2024-04-17T05:46:02.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14968028130156281439', time=2024-04-17T07:31:58.031002+0000, chain.objs.size()=1 2024-04-17T05:46:02.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.559__shadow:aw6UMRhpkzPV119V6FSHBusayqxZelj_.dlTQCD4gWfr-r3DNmPJl0E5xJmlcxNf_0 2024-04-17T05:46:02.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11658563945128036931', time=2024-04-17T07:32:53.888938+0000, chain.objs.size()=1 2024-04-17T05:46:02.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:a4D6jMOUqZle1OhsKN9npYeKLc0SbIV_.BzoApHAXuAnf_KO2KppMZYr7uxl9k81_0 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12629159092655163756', time=2024-04-17T07:32:59.177637+0000, chain.objs.size()=1 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:Isk3mEKx3h-CPrjE3kAB-E3uAJ47pAI_.bfMYEizKzNRJpXnFVQTVSsHNCJl6eBH_0 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6309424153659777423', time=2024-04-17T07:33:24.695164+0000, chain.objs.size()=4 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~1Ruoi8yHRRy7ehkIdyS0HaRe10Ne1JF.1 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~1Ruoi8yHRRy7ehkIdyS0HaRe10Ne1JF.2 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~1Ruoi8yHRRy7ehkIdyS0HaRe10Ne1JF.3 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~1Ruoi8yHRRy7ehkIdyS0HaRe10Ne1JF.4 2024-04-17T05:46:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6961245218694225506', time=2024-04-17T07:33:59.609680+0000, chain.objs.size()=1 2024-04-17T05:46:02.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.592__shadow_.KKyGZtLDU3riShrQXoFKQUJJmxHJVkx_0 2024-04-17T05:46:02.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11645663053053395302', time=2024-04-17T07:38:18.357192+0000, chain.objs.size()=1 2024-04-17T05:46:02.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.688__shadow:a2UkSHYp.PudfbgvqB0HYXWuv4eWu1k_.-SfmR2m9VpvCGa1ZCw81ps5lF0T-414_0 2024-04-17T05:46:02.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.888+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16468441822589810759', time=2024-04-17T07:38:54.347181+0000, chain.objs.size()=1 2024-04-17T05:46:02.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.888+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.703__shadow_.07ZfuOPrCcLoDAnzjKXxI9e6z1x2Ysd_0 2024-04-17T05:46:02.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.896+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.896+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-17T05:46:02.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.896+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46466' 2024-04-17T05:46:02.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6405942644543084704', time=2024-04-17T07:10:24.503364+0000, chain.objs.size()=3 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-FROZEN.2~Kr4ET9QehL5H45nyddpKcRoLPPuI48w.1 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-FROZEN.2~Kr4ET9QehL5H45nyddpKcRoLPPuI48w.2 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__multipart_foo-FROZEN.2~Kr4ET9QehL5H45nyddpKcRoLPPuI48w.3 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_gmcf7ZqgGJDOEchtMtAuBZyE1n5spzH', time=2024-04-17T07:10:26.179855+0000, chain.objs.size()=2 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.H2WgfkWDLETbwg77xak9zI_dQs8tW0y_0 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.H2WgfkWDLETbwg77xak9zI_dQs8tW0y_1 2024-04-17T05:46:02.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13775170997910934571', time=2024-04-17T07:11:08.893610+0000, chain.objs.size()=1 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.178__shadow_.MevFSbo9zbgdyO0hqaXQt6ORaF8tI8l_0 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10431344643099666755', time=2024-04-17T07:11:33.248482+0000, chain.objs.size()=1 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.187__shadow_.FpXTSTgyA4_J2tCEwd8kVGcIsoOKeOd_0 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15460175067493822896', time=2024-04-17T07:17:02.760529+0000, chain.objs.size()=1 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.p-_Ujf07VB4jgPntrvG6FsEn_6Rnwu1_0 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13673444968753831492', time=2024-04-17T07:17:03.047150+0000, chain.objs.size()=1 2024-04-17T05:46:02.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._f28bY3KCkAA6liufGySipEkwqcCuYh_0 2024-04-17T05:46:02.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12589424669166031872', time=2024-04-17T07:17:03.176939+0000, chain.objs.size()=1 2024-04-17T05:46:02.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fcRccPjaeDk1AxIHjTMRaeibYkDB1nO_0 2024-04-17T05:46:02.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8049172755768964465', time=2024-04-17T07:17:03.497112+0000, chain.objs.size()=1 2024-04-17T05:46:02.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BgCNRNEQdgQM_FpnQEEfSDVtc1rX_Fu_0 2024-04-17T05:46:02.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2442547537562698302', time=2024-04-17T07:17:03.987115+0000, chain.objs.size()=1 2024-04-17T05:46:02.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.900+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OkSbrHkoO5326q1jjF8P8e-DlOZweof_0 2024-04-17T05:46:02.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5401382148809124463', time=2024-04-17T07:17:04.036587+0000, chain.objs.size()=1 2024-04-17T05:46:02.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.21dRqomeYvNDv-3a4LNnuFBQRkRXkS-_0 2024-04-17T05:46:02.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4829362842715682373', time=2024-04-17T07:17:07.111036+0000, chain.objs.size()=1 2024-04-17T05:46:02.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SMeA49d-YehrMi3hvavCxCQMjYWSfnM_0 2024-04-17T05:46:02.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14012185582739518571', time=2024-04-17T07:17:07.422274+0000, chain.objs.size()=1 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ph1tTjUS3Hucl20BWUeCPKbXgmbYQV1_0 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6006717143419842056', time=2024-04-17T07:17:07.959297+0000, chain.objs.size()=1 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FHZYjw3BeZjx00PVro9vr8bqR2Y2FRF_0 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4823470230322307821', time=2024-04-17T07:17:08.075556+0000, chain.objs.size()=1 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.snnPDMjEDXWYBZ4Ns8tF8SogCUGxZdo_0 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14190058260641389575', time=2024-04-17T07:17:09.846269+0000, chain.objs.size()=1 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kFehRaYaJxMeei3k7F4f2pNP7lzuNgB_0 2024-04-17T05:46:02.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11520752544579262123', time=2024-04-17T07:17:11.416944+0000, chain.objs.size()=1 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.P3T1oe0HtrfMcM1-9tzdkM8VizkFqU5_0 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15345200459496202517', time=2024-04-17T07:17:11.819547+0000, chain.objs.size()=1 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.eOqhGJk3rOIO3AQGS8jHW2rzZZteQ9K_0 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16860847220906480908', time=2024-04-17T07:17:12.829932+0000, chain.objs.size()=1 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8m0ydJcTd6KyiJCB46_Oa6bY2jNqb3V_0 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1573314254655856628', time=2024-04-17T07:17:15.308911+0000, chain.objs.size()=1 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sYthyuSXJwf1SF9NtL0lemMRPmZYAaH_0 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15584597966662581542', time=2024-04-17T07:17:15.782614+0000, chain.objs.size()=1 2024-04-17T05:46:02.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.904+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BsEDJ8jW5oqtsc6_Js_gHommK0-wmUZ_0 2024-04-17T05:46:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16749094692523737059', time=2024-04-17T07:17:19.204125+0000, chain.objs.size()=1 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kZ-_96E1BlMITFXvbv5yEBpwT1YMR-z_0 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10687004642912494896', time=2024-04-17T07:17:24.260743+0000, chain.objs.size()=1 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ikg3g44BjEbl6Zb048WNpa_Xcff1wU__0 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12506427049443180026', time=2024-04-17T07:17:27.793828+0000, chain.objs.size()=1 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6105azSJb-xV6tBiHM7Ewoqy3ixcvLz_0 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15079218024157807941', time=2024-04-17T07:17:28.695729+0000, chain.objs.size()=1 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2BktFHIVfT7Q7ThUHe9AFsalutq_qpZ_0 2024-04-17T05:46:02.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5706653744892709097', time=2024-04-17T07:17:28.764380+0000, chain.objs.size()=1 2024-04-17T05:46:02.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.N2RRpiIYD1_mtvWJUUvkEbGlg9kJ1ec_0 2024-04-17T05:46:02.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3666301141082302269', time=2024-04-17T07:17:31.422999+0000, chain.objs.size()=1 2024-04-17T05:46:02.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.oa2m-ma-aRQ8jjKTjKI18HEcqvU4bJ1_0 2024-04-17T05:46:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9098231503164721275', time=2024-04-17T07:17:31.580276+0000, chain.objs.size()=1 2024-04-17T05:46:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.B1mHyRVyj-QOF5TkuZwjpRwYGXtTL7c_0 2024-04-17T05:46:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5337928511947647314', time=2024-04-17T07:18:00.107020+0000, chain.objs.size()=1 2024-04-17T05:46:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.270__shadow_.tc753ZLo8nv_KswQjG6hQt6WQAdOYm8_0 2024-04-17T05:46:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4393527956467680594', time=2024-04-17T07:18:08.283443+0000, chain.objs.size()=1 2024-04-17T05:46:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.275__shadow_.RftQTnjsJxyQe4fYFmIQsRokDG8FSzr_0 2024-04-17T05:46:02.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8501450451686451957', time=2024-04-17T07:18:13.204009+0000, chain.objs.size()=1 2024-04-17T05:46:02.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.278__shadow_.JyyTe7YKhAamwHbV-Rg561197F90K0o_0 2024-04-17T05:46:02.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1090484260919229793', time=2024-04-17T07:18:22.985842+0000, chain.objs.size()=1 2024-04-17T05:46:02.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.912+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.RFQwtskOnXIfCDj7BN1uLQ1FzAem7jb_0 2024-04-17T05:46:02.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.916+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1031898079864793650', time=2024-04-17T07:18:28.218814+0000, chain.objs.size()=1 2024-04-17T05:46:02.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.916+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.LLMQbN5V7N5f14LaQciaGHfcKOXeE04_0 2024-04-17T05:46:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.916+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14616682148075808227', time=2024-04-17T07:21:00.509739+0000, chain.objs.size()=1 2024-04-17T05:46:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.916+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.F-oVOJR9rcFcHlNdX-f8nMe97PiylCX_0 2024-04-17T05:46:02.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.916+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8916540429869651542', time=2024-04-17T07:21:00.811705+0000, chain.objs.size()=1 2024-04-17T05:46:02.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.916+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tPSfRcZPMjc7bxFuVILqh6N2-6D1Qen_0 2024-04-17T05:46:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12609600098765622589', time=2024-04-17T07:21:02.095787+0000, chain.objs.size()=1 2024-04-17T05:46:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ErFIJyDOO824JYyXA_qjwTlqz6u55ee_0 2024-04-17T05:46:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16996951103042465973', time=2024-04-17T07:21:04.509453+0000, chain.objs.size()=1 2024-04-17T05:46:02.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.l6RSMJim2Kv5N__0yHxdz-9tGi5gWWp_0 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5993329909065076358', time=2024-04-17T07:21:05.121061+0000, chain.objs.size()=1 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GbNVXWJaiKoUD5bw2_0pQLw-z72IpuX_0 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7315490337123919043', time=2024-04-17T07:21:06.638142+0000, chain.objs.size()=1 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fJF_TzKs7VqE6ejQIhVH4JoMKYHUVYP_0 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3574044674088307616', time=2024-04-17T07:21:06.812494+0000, chain.objs.size()=1 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IApQYlAeox987OWhHa0nueWjHO8sU-j_0 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9331017835696133331', time=2024-04-17T07:21:08.348261+0000, chain.objs.size()=1 2024-04-17T05:46:02.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nxxxZAgogJMXJlNWYn-E8lSY-U7SuB7_0 2024-04-17T05:46:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12245275362686600133', time=2024-04-17T07:21:11.834516+0000, chain.objs.size()=1 2024-04-17T05:46:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.cYaJ8AaoWpS7Cnua3qgcOHZjcHajZK__0 2024-04-17T05:46:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3210155807586245564', time=2024-04-17T07:21:12.239012+0000, chain.objs.size()=1 2024-04-17T05:46:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.a30mTrqCr26sGq5NaHQxgJV9Y_rgvSJ_0 2024-04-17T05:46:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13171578547445411957', time=2024-04-17T07:21:12.623658+0000, chain.objs.size()=1 2024-04-17T05:46:02.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PccqCybu6ShOJB95RjVYn76t_jLSvT9_0 2024-04-17T05:46:02.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12148919688845137953', time=2024-04-17T07:21:12.625183+0000, chain.objs.size()=1 2024-04-17T05:46:02.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.z1C9t-e-c05X1VO5YArnLUSp-mwkgVG_0 2024-04-17T05:46:02.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12923832471204974083', time=2024-04-17T07:21:13.575915+0000, chain.objs.size()=1 2024-04-17T05:46:02.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.920+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xjprCaDII3eqdwUDkE61wseNHLBV9Jo_0 2024-04-17T05:46:02.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8904434046390156288', time=2024-04-17T07:21:13.578132+0000, chain.objs.size()=1 2024-04-17T05:46:02.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nOIhhfFznMUUelff2Aappe1xwltSPz4_0 2024-04-17T05:46:02.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7540163644873425036', time=2024-04-17T07:21:14.452121+0000, chain.objs.size()=1 2024-04-17T05:46:02.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GtA4uzRxXw5lzv74RdONR80zxT0_11j_0 2024-04-17T05:46:02.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8153114206523546282', time=2024-04-17T07:21:15.788285+0000, chain.objs.size()=1 2024-04-17T05:46:02.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XfxqURCD87anOhG_MM-BLLbChn0Gjqi_0 2024-04-17T05:46:02.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15019210642335995539', time=2024-04-17T07:21:16.165683+0000, chain.objs.size()=1 2024-04-17T05:46:02.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wQX9x-qSh3eqnTNPrmAGXlNCRnuPGsy_0 2024-04-17T05:46:02.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14602539476749154842', time=2024-04-17T07:21:18.055644+0000, chain.objs.size()=1 2024-04-17T05:46:02.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.924+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MxaWu9aTsaFibmiQ98KwdDocOwhitRf_0 2024-04-17T05:46:02.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15641014987574297473', time=2024-04-17T07:21:18.304513+0000, chain.objs.size()=1 2024-04-17T05:46:02.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tSjRiK6d4GInMd891fO7q6Xw3s4ebu7_0 2024-04-17T05:46:02.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15354495928092587139', time=2024-04-17T07:21:18.517369+0000, chain.objs.size()=1 2024-04-17T05:46:02.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QpToCFAiOwrHrhbDNMk-e8Az3_fBbLu_0 2024-04-17T05:46:02.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16506460588639419224', time=2024-04-17T07:21:18.666451+0000, chain.objs.size()=1 2024-04-17T05:46:02.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jR3q_SuW7s5bPh7cwsNIxoaG6I3nLUx_0 2024-04-17T05:46:02.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3222831547123376055', time=2024-04-17T07:21:19.577451+0000, chain.objs.size()=1 2024-04-17T05:46:02.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.hHc0HKgf6l_F_Alr6_26ni2Pqn4BWdj_0 2024-04-17T05:46:02.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5746299873900715886', time=2024-04-17T07:21:20.350659+0000, chain.objs.size()=1 2024-04-17T05:46:02.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KxPpGJlNRKbKfTeOzcxmZ50-gycDC0a_0 2024-04-17T05:46:02.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14840183886953112846', time=2024-04-17T07:21:21.083557+0000, chain.objs.size()=1 2024-04-17T05:46:02.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.928+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9HOUc3v_KQJ-Og57pg7PyHzP8ZUdU8r_0 2024-04-17T05:46:02.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15891280635901389281', time=2024-04-17T07:21:21.431151+0000, chain.objs.size()=1 2024-04-17T05:46:02.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Fq95W21StMurZN09sFe_oRVELda2vLI_0 2024-04-17T05:46:02.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10510945101790693081', time=2024-04-17T07:21:22.471890+0000, chain.objs.size()=1 2024-04-17T05:46:02.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RuiAiRxxTS9Ureg1w1QhnGG8n0QL4eM_0 2024-04-17T05:46:02.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5873849256701652155', time=2024-04-17T07:21:24.351504+0000, chain.objs.size()=1 2024-04-17T05:46:02.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8v6Gp5icJmsWq8HEc4VRscQY608PB96_0 2024-04-17T05:46:02.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2478026654543951348', time=2024-04-17T07:21:25.668276+0000, chain.objs.size()=1 2024-04-17T05:46:02.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.w69KGLkHbwgMnXqyFfSQ-Pe9pWGfFgP_0 2024-04-17T05:46:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18147092076139486474', time=2024-04-17T07:21:26.108727+0000, chain.objs.size()=1 2024-04-17T05:46:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aVa9hPpT0hFiOEHLAJtlC526E4Z6nUz_0 2024-04-17T05:46:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7707654380012176975', time=2024-04-17T07:21:26.239202+0000, chain.objs.size()=1 2024-04-17T05:46:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.932+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wRvtozunQ2ylh34R9Ynr1UDkd_cUFob_0 2024-04-17T05:46:02.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15226922149686487984', time=2024-04-17T07:21:27.373257+0000, chain.objs.size()=1 2024-04-17T05:46:02.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2heiUQOEBP_8P549ONkHPuWcznluYy9_0 2024-04-17T05:46:02.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4104665883796593786', time=2024-04-17T07:21:27.421673+0000, chain.objs.size()=1 2024-04-17T05:46:02.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eba9Bph2wWiv-JiKlQnlojGVgI1W1sL_0 2024-04-17T05:46:02.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8355045997726995644', time=2024-04-17T07:21:27.454204+0000, chain.objs.size()=1 2024-04-17T05:46:02.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.a68lM8NfZfmKAww1KLXGliPS68Ddqkz_0 2024-04-17T05:46:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10759355943238908882', time=2024-04-17T07:21:27.808958+0000, chain.objs.size()=1 2024-04-17T05:46:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1mFLZEjj32j9T1IwJah9VH7_AM84yyx_0 2024-04-17T05:46:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6808653772979297401', time=2024-04-17T07:23:13.568992+0000, chain.objs.size()=1 2024-04-17T05:46:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CXpOKu3kj5WHtZdVRhxEigldHi-L6IF_0 2024-04-17T05:46:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5552027475925931807', time=2024-04-17T07:23:15.189221+0000, chain.objs.size()=1 2024-04-17T05:46:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ax-982H6ZaFz88WGgka0SYM2jED6maq_0 2024-04-17T05:46:02.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8598021264270779324', time=2024-04-17T07:23:17.361470+0000, chain.objs.size()=1 2024-04-17T05:46:02.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UlVsE6Om_lCnyHuBe5zab_yLqd51uhf_0 2024-04-17T05:46:02.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17493987941861799998', time=2024-04-17T07:23:18.494386+0000, chain.objs.size()=1 2024-04-17T05:46:02.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OX46MdrjS_UEpJ1EIxbqrSByDpydzCP_0 2024-04-17T05:46:02.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1565582462744419356', time=2024-04-17T07:23:18.548433+0000, chain.objs.size()=1 2024-04-17T05:46:02.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bsjvuUYaBv8hTKazRwoJzbR9cLnLNEp_0 2024-04-17T05:46:02.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7772695822873691382', time=2024-04-17T07:23:18.550603+0000, chain.objs.size()=1 2024-04-17T05:46:02.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aetc4lXxT5KxxnybjLj4PZnJrL6YVU6_0 2024-04-17T05:46:02.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2769424066250001210', time=2024-04-17T07:23:18.552849+0000, chain.objs.size()=1 2024-04-17T05:46:02.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.936+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.P-3eSd6IjXmM-pUHX3zJywQUdoCKsXB_0 2024-04-17T05:46:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13370141039287929827', time=2024-04-17T07:23:19.009746+0000, chain.objs.size()=1 2024-04-17T05:46:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sRTN-IpK_sRMvegIy02LRkobcvVrfsb_0 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9694174587081210470', time=2024-04-17T07:23:19.011203+0000, chain.objs.size()=1 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.x96HSfuYhCHEKQI0qWmja4z50BE8akt_0 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1563539062375760889', time=2024-04-17T07:23:19.923585+0000, chain.objs.size()=1 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JR_4HhF_ld83ne2Um34YN3Gieo8-lh7_0 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.608196869679577915', time=2024-04-17T07:23:21.050796+0000, chain.objs.size()=1 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.w_QTAmoJ3UfjsMKXelOzPdHog7E9e_2_0 2024-04-17T05:46:02.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2666063209422941985', time=2024-04-17T07:23:21.554610+0000, chain.objs.size()=1 2024-04-17T05:46:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9gBsDADbmdLuG6vAQyh0el2zJhO_9E2_0 2024-04-17T05:46:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12798503225912477728', time=2024-04-17T07:23:23.012108+0000, chain.objs.size()=1 2024-04-17T05:46:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.940+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N9QZGqAbyCICSLe7Ge5NVwy7VKgzyax_0 2024-04-17T05:46:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2927053318245438158', time=2024-04-17T07:23:23.756133+0000, chain.objs.size()=1 2024-04-17T05:46:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.07IwNIpFg-x_ftEnl5VASarR9-HfEoA_0 2024-04-17T05:46:02.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5978530849929912199', time=2024-04-17T07:23:25.211025+0000, chain.objs.size()=1 2024-04-17T05:46:02.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IbxokKi3sdzeSJDdA6_NF7B-TiSkSP8_0 2024-04-17T05:46:02.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7925292362459694003', time=2024-04-17T07:23:25.595370+0000, chain.objs.size()=1 2024-04-17T05:46:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hacutTKzgW02faR1ia94t2T6mk51cZt_0 2024-04-17T05:46:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9155143683993438099', time=2024-04-17T07:23:26.222356+0000, chain.objs.size()=1 2024-04-17T05:46:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wecyXTCqdrX_I4X4TKjaNC9UicFxc5z_0 2024-04-17T05:46:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12849454349213381984', time=2024-04-17T07:23:26.633693+0000, chain.objs.size()=1 2024-04-17T05:46:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Lf3fi16CxU_0KV-IAnYEqmcpA1Cw6aU_0 2024-04-17T05:46:02.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6456361915734248109', time=2024-04-17T07:23:26.987588+0000, chain.objs.size()=1 2024-04-17T05:46:02.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JG3PQCXn49jiZ-WJJDSFZKXWvsO9TWy_0 2024-04-17T05:46:02.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8328898863292346385', time=2024-04-17T07:23:28.837316+0000, chain.objs.size()=1 2024-04-17T05:46:02.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.944+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4EkfabHCz3thGVenyee5JGGSxPrffpK_0 2024-04-17T05:46:02.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8192402214060976273', time=2024-04-17T07:23:29.129807+0000, chain.objs.size()=1 2024-04-17T05:46:02.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pSONnRN5Conwo5BO4_UaSyEAvf3iTRL_0 2024-04-17T05:46:02.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16062383831877382708', time=2024-04-17T07:23:29.586703+0000, chain.objs.size()=1 2024-04-17T05:46:02.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.O8V9a9FKvn3auQUxeEiXHT7pWXurRDk_0 2024-04-17T05:46:02.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4106135383996658468', time=2024-04-17T07:23:32.156601+0000, chain.objs.size()=1 2024-04-17T05:46:02.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.HRgOJsRD9x6VCYwktTvn3Epq7BHUmK-_0 2024-04-17T05:46:02.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.957195090447156264', time=2024-04-17T07:23:32.511763+0000, chain.objs.size()=1 2024-04-17T05:46:02.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Ap6wMCEEbGmq-rcJayNmuz9Xl9I8URb_0 2024-04-17T05:46:02.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14584874853139335301', time=2024-04-17T07:23:32.982606+0000, chain.objs.size()=1 2024-04-17T05:46:02.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sI9b6loGWqcfPuKxmb_biJChFX5K3Ek_0 2024-04-17T05:46:02.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5554861719106463378', time=2024-04-17T07:23:35.904715+0000, chain.objs.size()=1 2024-04-17T05:46:02.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SxpZkiz59YTHUEuqCkToia70Hw3kPXO_0 2024-04-17T05:46:02.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1362363927924473324', time=2024-04-17T07:23:37.623284+0000, chain.objs.size()=1 2024-04-17T05:46:02.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.esZkx7b9g-RQcN9eTvaXMczL8cxLfGN_0 2024-04-17T05:46:02.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17047992228087355459', time=2024-04-17T07:23:38.108368+0000, chain.objs.size()=1 2024-04-17T05:46:02.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.APFN5LEOcJ1uKcWuIETc3VSttcizBVT_0 2024-04-17T05:46:02.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12659118338896686266', time=2024-04-17T07:23:38.297564+0000, chain.objs.size()=1 2024-04-17T05:46:02.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.948+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hxTVZUGkuCMIe6wm7puKS8PsbIGY_7Z_0 2024-04-17T05:46:02.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14646867736570092837', time=2024-04-17T07:23:38.631822+0000, chain.objs.size()=1 2024-04-17T05:46:02.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TKH1h_UzdtBlaf9WLHuSPRXXAnMQBob_0 2024-04-17T05:46:02.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6285528877462776147', time=2024-04-17T07:23:39.439512+0000, chain.objs.size()=1 2024-04-17T05:46:02.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.S_-3tIeCDiUmfp2NM4yXTbnPjyulkFh_0 2024-04-17T05:46:02.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16457533881803185721', time=2024-04-17T07:23:40.743015+0000, chain.objs.size()=1 2024-04-17T05:46:02.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sck7AT2CAOxBp0FTJ2-s1RC5h-inxCi_0 2024-04-17T05:46:02.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14137108688451309048', time=2024-04-17T07:23:47.414849+0000, chain.objs.size()=1 2024-04-17T05:46:02.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.320__shadow_.qTIV54sNyVePzU1ghngveGX1sIC5bl5_0 2024-04-17T05:46:02.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7133980225102038614', time=2024-04-17T07:24:22.000169+0000, chain.objs.size()=1 2024-04-17T05:46:02.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.952+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.362__shadow_.kt_C_gCQucXW4KCnbZl7xUuJ5XrA8R__0 2024-04-17T05:46:02.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.956+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/46466 2024-04-17T05:46:02.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2024-04-17T05:46:02.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15557673875153971471', time=2024-04-17T07:25:55.805436+0000, chain.objs.size()=1 2024-04-17T05:46:02.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.452__shadow_.0tz7RBw2K49sjVEfe-wPfeCbhYjSvZ2_0 2024-04-17T05:46:02.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13741989717152045044', time=2024-04-17T07:26:02.993141+0000, chain.objs.size()=1 2024-04-17T05:46:02.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.456__shadow_.U_0sRgXNPxw6kyIJXdxYWkah3-eXQHV_0 2024-04-17T05:46:02.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5897231078425327043', time=2024-04-17T07:26:02.995386+0000, chain.objs.size()=1 2024-04-17T05:46:02.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.456__shadow_.GLrUVbzeVEjBTbW-_87LF_zub5gViVt_0 2024-04-17T05:46:02.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9089196966610912030', time=2024-04-17T07:26:17.163421+0000, chain.objs.size()=1 2024-04-17T05:46:02.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.oWkoxb5sqDgGy6Llmlc94ZyWJCoNBTz_0 2024-04-17T05:46:02.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16787498816792150915', time=2024-04-17T07:27:02.088351+0000, chain.objs.size()=1 2024-04-17T05:46:02.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.502__shadow_.MleKZNVRld8VWdVgJ0Wr7y-wIFl-Qyn_0 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10287421613028800478', time=2024-04-17T07:29:29.062224+0000, chain.objs.size()=2 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~bFymopdS5rKabsImzCWmFnpgZYEyNqn.1 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~bFymopdS5rKabsImzCWmFnpgZYEyNqn.2 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7509406767571097489', time=2024-04-17T07:29:49.413352+0000, chain.objs.size()=2 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.518__multipart_mymultipart.2~CC3BMw5zwt7g8NayqjxCrZrHSNaODOl.1 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.518__multipart_mymultipart.2~CC3BMw5zwt7g8NayqjxCrZrHSNaODOl.2 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1122247658798926140', time=2024-04-17T07:32:14.485973+0000, chain.objs.size()=1 2024-04-17T05:46:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.564__shadow_.FbeO3aGn-FJYjgdx3g5_uaYP3F6bQc5_0 2024-04-17T05:46:02.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16060170428930346596', time=2024-04-17T07:32:20.144215+0000, chain.objs.size()=1 2024-04-17T05:46:02.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:qvt9H.weUcWz3yzkGrG8hiqpWY5ZPbi_.brvROw7RCnm9roDqRG2k3cdfDaecUYt_0 2024-04-17T05:46:02.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7571365415549633299', time=2024-04-17T07:32:40.154315+0000, chain.objs.size()=1 2024-04-17T05:46:02.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.960+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:WebMcvrgr97Wg71sOVoI9YyAuXKOT8-_.237r9wPP0FVwanckKhrzVM6Mx4P0LNo_0 2024-04-17T05:46:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2809143996258575797', time=2024-04-17T07:32:54.824228+0000, chain.objs.size()=1 2024-04-17T05:46:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:yXuqPCcqOTY3d8w.9npYUoYlcByp499_.r-SLpaWDIThhn_Bc8pA7KvI2Z2XMHcB_0 2024-04-17T05:46:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7350684495827545354', time=2024-04-17T07:33:29.343757+0000, chain.objs.size()=1 2024-04-17T05:46:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:66SV24gVGKIAPPNzmhQNpFcMeOTb6qq_.eOCBY5347x-EtPKEuREmwU6xRSilHly_0 2024-04-17T05:46:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12640057311707221257', time=2024-04-17T07:33:37.004501+0000, chain.objs.size()=1 2024-04-17T05:46:02.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.572__shadow:U29QrM1VklJr5G23BVCrfmSp31DSbhc_.YnEvi8LA4VzEW10sY9hzIQmcWgkEWZU_0 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10703968590811944638', time=2024-04-17T07:35:37.178450+0000, chain.objs.size()=1 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.606__shadow_.kgXvJClaxKmwmeOWZL1cxxPz2ObBwxD_0 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6623302968067014744', time=2024-04-17T07:36:56.997579+0000, chain.objs.size()=1 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.634__shadow_.W5qsp7H7T59JRpT_X9oSV40mo95H_e1_0 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14749532090280196054', time=2024-04-17T07:37:33.735936+0000, chain.objs.size()=6 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.646__multipart_baz.2~gY0rI8Cf6geI5DBtL8Shqv_DHKxBveT.1 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.646__multipart_baz.2~gY0rI8Cf6geI5DBtL8Shqv_DHKxBveT.2 2024-04-17T05:46:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.646__multipart_baz.2~gY0rI8Cf6geI5DBtL8Shqv_DHKxBveT.3 2024-04-17T05:46:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.646__multipart_baz.2~gY0rI8Cf6geI5DBtL8Shqv_DHKxBveT.4 2024-04-17T05:46:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.646__multipart_baz.2~gY0rI8Cf6geI5DBtL8Shqv_DHKxBveT.5 2024-04-17T05:46:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.964+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.646__multipart_baz.2~gY0rI8Cf6geI5DBtL8Shqv_DHKxBveT.6 2024-04-17T05:46:02.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.968+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16565864136283823474', time=2024-04-17T07:37:44.034552+0000, chain.objs.size()=1 2024-04-17T05:46:02.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.968+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.653__shadow_.fIyq-YVRCiWKiFyX6otswI2odm6RgTN_0 2024-04-17T05:46:02.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.972+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:02.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-17T05:46:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:02.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47325' 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6784225970357429838', time=2024-04-17T07:10:58.704778+0000, chain.objs.size()=1 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.169__shadow_.PbTTJEOTpra3DqqcgvhIDLagfVlz960_0 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5332872464798307495', time=2024-04-17T07:15:18.481396+0000, chain.objs.size()=1 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.251__shadow_.foeJbe4XlhXRhcF_iKgkZlH6t7o5dfm_0 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4990254931737076532', time=2024-04-17T07:17:02.562061+0000, chain.objs.size()=1 2024-04-17T05:46:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.F6YylD7OM59r2NNQc6sjEvYVs-Q006h_0 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3494698810166003058', time=2024-04-17T07:17:03.051520+0000, chain.objs.size()=1 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lkTHSc3Nvi2O_9I4dbyMBFRIhlOqYpO_0 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7655302757848263380', time=2024-04-17T07:17:03.991614+0000, chain.objs.size()=1 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9c2giM9OfVGNtKdvyNDmllPO1MXq8Re_0 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18132769933904958167', time=2024-04-17T07:17:05.500946+0000, chain.objs.size()=1 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.-OE5gnqBGJbhV3kESZG-aZcB_DCnnHs_0 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14463883434207141568', time=2024-04-17T07:17:07.583754+0000, chain.objs.size()=1 2024-04-17T05:46:02.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.m1tfFH0W-ZZWHCRzqyX6IjpNzi7JTv9_0 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11330906653995987301', time=2024-04-17T07:17:10.435140+0000, chain.objs.size()=1 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TrDjCeMdcSqKPpC6DyIGJE1om6hqmBj_0 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10949803773904192310', time=2024-04-17T07:17:14.265522+0000, chain.objs.size()=1 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._0k9ZEoHoh6piv-bn2p338vWz5mrYea_0 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16345262079220871048', time=2024-04-17T07:17:15.424934+0000, chain.objs.size()=1 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EI5VCXci5KGjuvLLEt1twnNHohsj8jQ_0 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11898975053180795103', time=2024-04-17T07:17:15.692442+0000, chain.objs.size()=1 2024-04-17T05:46:02.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.T-gB6FrxVmG_NpiqVbH62rgOBNYL3uo_0 2024-04-17T05:46:02.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7216298972686636776', time=2024-04-17T07:17:18.385881+0000, chain.objs.size()=1 2024-04-17T05:46:02.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.976+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rAT88V4T6W9HSwwzFyayL8BSctId5D7_0 2024-04-17T05:46:02.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8302878671904474862', time=2024-04-17T07:17:18.433814+0000, chain.objs.size()=1 2024-04-17T05:46:02.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QW6CgNDm8YF7JJ62XNnpZu0jEtqD5o6_0 2024-04-17T05:46:02.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8904006337314917284', time=2024-04-17T07:17:19.439804+0000, chain.objs.size()=1 2024-04-17T05:46:02.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LcU24WE--r2Kws78bpvMuJL02Lvags9_0 2024-04-17T05:46:02.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16221979661707756148', time=2024-04-17T07:17:20.703784+0000, chain.objs.size()=1 2024-04-17T05:46:02.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SmBnM7cuWg4p6G5AORUjmi_eD1bX-hx_0 2024-04-17T05:46:02.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16534184950486874263', time=2024-04-17T07:17:21.961058+0000, chain.objs.size()=1 2024-04-17T05:46:02.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jpVmUJCNq0irQDJA6URRsyf41lIxodI_0 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13068865552850048290', time=2024-04-17T07:17:21.963272+0000, chain.objs.size()=1 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wWt8NEcCKQAXa3jBecBJf8OcioBkNjo_0 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17297309267481777712', time=2024-04-17T07:17:27.093180+0000, chain.objs.size()=1 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.x5vkSnG8uI8VB34sniNx4nEFIZJ7y8j_0 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12650715764481201761', time=2024-04-17T07:17:27.824614+0000, chain.objs.size()=1 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.d-X42h2IBcmWaUHiptSsAhPhMMWXxgA_0 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7215866745270427854', time=2024-04-17T07:17:29.165727+0000, chain.objs.size()=1 2024-04-17T05:46:02.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.980+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ttJst_f9FY5M6nfDoBgNH8czanqMOxH_0 2024-04-17T05:46:02.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.134323110335243039', time=2024-04-17T07:17:32.111115+0000, chain.objs.size()=1 2024-04-17T05:46:02.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SumWvbL_1pDB_KFbboVPF3-FdXG5Wms_0 2024-04-17T05:46:02.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8248040826188458776', time=2024-04-17T07:17:41.402605+0000, chain.objs.size()=1 2024-04-17T05:46:02.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.258__shadow_.HHPup-JmlldbhRY-RAdQSoRtA7d_9ls_0 2024-04-17T05:46:02.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17567663070990067123', time=2024-04-17T07:17:41.404681+0000, chain.objs.size()=1 2024-04-17T05:46:02.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.258__shadow_.002JELqF_fsmYsBP5jt3VwXUKNkJKWb_0 2024-04-17T05:46:02.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9884982489147482020', time=2024-04-17T07:17:50.362129+0000, chain.objs.size()=1 2024-04-17T05:46:02.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.984+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.264__shadow_.grPDFXV-HD3j4jFJxcBgt6zZpKxjbtN_0 2024-04-17T05:46:02.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8248715637403339031', time=2024-04-17T07:18:01.680927+0000, chain.objs.size()=1 2024-04-17T05:46:02.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.271__shadow_.OXW5ZvJYNufMdX0ItKOIyTY1Th793Qx_0 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8427883159293432227', time=2024-04-17T07:18:03.351008+0000, chain.objs.size()=1 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.272__shadow_.kY92EHRycSZtjTb4ZoMwxp6srVD4LAD_0 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9618464411361047048', time=2024-04-17T07:18:22.931873+0000, chain.objs.size()=1 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.J4qn9WrFhBwY6-5ay0gK5o7gTZls8VV_0 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9771621882360424593', time=2024-04-17T07:18:22.975419+0000, chain.objs.size()=1 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.pcsCKPYnjzB9R8FWU8grKjXQN2K25lS_0 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14204629205835917909', time=2024-04-17T07:18:27.920449+0000, chain.objs.size()=1 2024-04-17T05:46:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.1qIXzz7giUz8N1Vg-W7t7vO-ncmWaAO_0 2024-04-17T05:46:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13636593085730419149', time=2024-04-17T07:18:46.560980+0000, chain.objs.size()=1 2024-04-17T05:46:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.294__shadow_.fWUYs1M2IyASoPiuxeB145h5XLwI89X_0 2024-04-17T05:46:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16327124913168855222', time=2024-04-17T07:18:52.935165+0000, chain.objs.size()=1 2024-04-17T05:46:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.297__shadow:SOzwVe4NseihZM6kK93jC2vhJ17t.BT_.uiTsuK6HunnWVz27aI1STyXc9Rt3NQi_0 2024-04-17T05:46:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8449390009108977438', time=2024-04-17T07:20:59.782556+0000, chain.objs.size()=1 2024-04-17T05:46:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mUXfzbk16JoR2z91oUQltX_8zZ1t4Ly_0 2024-04-17T05:46:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15423427323209894688', time=2024-04-17T07:20:59.852726+0000, chain.objs.size()=1 2024-04-17T05:46:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1k_B8YQ_QeIM-1ut10AQWzTSKHYDNwF_0 2024-04-17T05:46:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15043612026407290296', time=2024-04-17T07:21:00.851124+0000, chain.objs.size()=1 2024-04-17T05:46:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LAkpSUG9SQJ-ctLiJuM53ixv_RjRHE4_0 2024-04-17T05:46:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5632752307871703926', time=2024-04-17T07:21:03.851540+0000, chain.objs.size()=1 2024-04-17T05:46:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.988+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Fb4D0g3Q2ByScAyHJEohPIknXF1bqpH_0 2024-04-17T05:46:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14738038223322440431', time=2024-04-17T07:21:03.924639+0000, chain.objs.size()=1 2024-04-17T05:46:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2_g197fSpcu-vE0YUDGQWsyfa4n6X-U_0 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2281485514682940994', time=2024-04-17T07:21:04.420952+0000, chain.objs.size()=1 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yybZkpBAK8EbPKskZoDXu230YlR-W-4_0 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7674886177407940163', time=2024-04-17T07:21:04.627533+0000, chain.objs.size()=1 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.858CRE7WpE8-2qgZaFoS9rZfDLmNFrQ_0 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10296810154294834926', time=2024-04-17T07:21:05.259699+0000, chain.objs.size()=1 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wycrgR2cjBNKYEjs-MrBqYj3vHisGFy_0 2024-04-17T05:46:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8057270928437720773', time=2024-04-17T07:21:05.316974+0000, chain.objs.size()=1 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Vo1q59SIR3bj57kGt2i1xHqiOFWHn54_0 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2749258805065055448', time=2024-04-17T07:21:08.789482+0000, chain.objs.size()=1 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Jb300q0DzRlGAaD6HoQSAcq_k-K8xGd_0 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12327748399588744533', time=2024-04-17T07:21:08.791443+0000, chain.objs.size()=1 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FhnB8KDg-B7RPtRRRDZsaf3GkR_QISw_0 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1311164607258696787', time=2024-04-17T07:21:09.809666+0000, chain.objs.size()=1 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WLhlrk3HLTlKVBUYxWydGLJtx7vom-t_0 2024-04-17T05:46:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3447446825115641082', time=2024-04-17T07:21:10.966484+0000, chain.objs.size()=1 2024-04-17T05:46:02.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.992+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.D18PEV75R09he5X0sB9FdqJ5S7Oihu__0 2024-04-17T05:46:02.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4725215614172676384', time=2024-04-17T07:21:13.498751+0000, chain.objs.size()=1 2024-04-17T05:46:02.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.462T5CAVkU68zcS1u0XshBwpFqn0Uy8_0 2024-04-17T05:46:02.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.996+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.374571908156246212', time=2024-04-17T07:21:14.712071+0000, chain.objs.size()=1 2024-04-17T05:46:02.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:02.996+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2VRfzU1pyVyILbCj_sIi0AK46wlS2NP_0 2024-04-17T05:46:03.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10697757355923858929', time=2024-04-17T07:21:15.765074+0000, chain.objs.size()=1 2024-04-17T05:46:03.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.98xWgjV2LP5d8YOV-nUm245Or3t-Cuj_0 2024-04-17T05:46:03.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2632167292053254512', time=2024-04-17T07:21:15.768165+0000, chain.objs.size()=1 2024-04-17T05:46:03.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yIBp-avDCQ3aacMJG_sQaHpZyijgLnF_0 2024-04-17T05:46:03.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5469350707803278138', time=2024-04-17T07:21:16.508509+0000, chain.objs.size()=1 2024-04-17T05:46:03.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._ABdtaFcXTBY-5ahwv6874SwoxCgSky_0 2024-04-17T05:46:03.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8857717119667694686', time=2024-04-17T07:21:16.552997+0000, chain.objs.size()=1 2024-04-17T05:46:03.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8kRtEyclqdfVm3Z0sTBMDUGsn6kEfYM_0 2024-04-17T05:46:03.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6528933092189577018', time=2024-04-17T07:21:16.924769+0000, chain.objs.size()=1 2024-04-17T05:46:03.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.k1E-E2wZfvKPxIFrw8aDufMcNY6CPQJ_0 2024-04-17T05:46:03.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13484394563878409160', time=2024-04-17T07:21:18.139670+0000, chain.objs.size()=1 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._nV_4IUUWHvcSd8zpMgcawxoG_dPO9K_0 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12802778379619543192', time=2024-04-17T07:21:18.681548+0000, chain.objs.size()=1 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sFj8yUd-5LMNmbZztPXw4sAcXRsyrss_0 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13722586112100315663', time=2024-04-17T07:21:19.592765+0000, chain.objs.size()=1 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9c6oqu0j_72H_b4KOql2tuzt_dhtp8t_0 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16153945210957219880', time=2024-04-17T07:21:20.242173+0000, chain.objs.size()=1 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.b6qPoVwVgaR-TMUw1xHjaQw2iHbCp_d_0 2024-04-17T05:46:03.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12271674441874412158', time=2024-04-17T07:21:20.350235+0000, chain.objs.size()=1 2024-04-17T05:46:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.H100ntDK9vPYowuhfNRoFT5fi41gjwu_0 2024-04-17T05:46:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15382295929200997744', time=2024-04-17T07:21:20.497055+0000, chain.objs.size()=1 2024-04-17T05:46:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.000+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2vCzUBEVi8PzODfi48J8xij6gK_0jsE_0 2024-04-17T05:46:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5994293980000990707', time=2024-04-17T07:21:20.582514+0000, chain.objs.size()=1 2024-04-17T05:46:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xlB3R_QxAN2FP_df_8rI0sICtGV5mUt_0 2024-04-17T05:46:03.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12740603723160441924', time=2024-04-17T07:21:20.944550+0000, chain.objs.size()=1 2024-04-17T05:46:03.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZPneLC0oMKGXaAvNgaf5hQ3OjK3MwKe_0 2024-04-17T05:46:03.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16492655263863943509', time=2024-04-17T07:21:22.332479+0000, chain.objs.size()=1 2024-04-17T05:46:03.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Y4HH0djNPG8ANB40Hj9L33xxjuWlaGO_0 2024-04-17T05:46:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13404895097838686518', time=2024-04-17T07:21:23.725957+0000, chain.objs.size()=1 2024-04-17T05:46:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.knGL3u-Ad2tZkqPMoTPv1PYpZUa63RY_0 2024-04-17T05:46:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3037567503376895051', time=2024-04-17T07:21:25.414595+0000, chain.objs.size()=1 2024-04-17T05:46:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RmgFvM8x1_UlLcsxPJe_BqbUYS6wRrg_0 2024-04-17T05:46:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5808387365463930337', time=2024-04-17T07:21:25.786357+0000, chain.objs.size()=1 2024-04-17T05:46:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YluArKsiWHL2eSwjuhFL1kssbj0VTU5_0 2024-04-17T05:46:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4785331717499922817', time=2024-04-17T07:21:27.798836+0000, chain.objs.size()=1 2024-04-17T05:46:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sABEndws2fQ8fvFVms1EfgR343ivtir_0 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9999253395606611129', time=2024-04-17T07:21:27.844676+0000, chain.objs.size()=1 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HQyNYp1Fl1WNUMVou4hNal4Roig_wgB_0 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9051198337673034540', time=2024-04-17T07:21:27.846377+0000, chain.objs.size()=1 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KxIF0Yre0XQZLq2D2zK6CVx1rQqYjq6_0 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11912262349306962734', time=2024-04-17T07:21:28.899564+0000, chain.objs.size()=1 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SxomwijXi0Uyruy-NMkJNhdcCHxqq5N_0 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10164587116865221830', time=2024-04-17T07:21:29.335546+0000, chain.objs.size()=1 2024-04-17T05:46:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.004+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Gohu8LN2Bb_zcW4I_rUg94eiDLbY2Cj_0 2024-04-17T05:46:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13986366798976649365', time=2024-04-17T07:23:14.820138+0000, chain.objs.size()=1 2024-04-17T05:46:03.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.G63un-EUCaFwgkJkD8J1JK2iTHw0-1s_0 2024-04-17T05:46:03.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17606190332934472805', time=2024-04-17T07:23:14.909629+0000, chain.objs.size()=1 2024-04-17T05:46:03.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jPYaKF5OXPWQA6rfgDX955Yr0ZElEcT_0 2024-04-17T05:46:03.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.008+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10308395599261908281', time=2024-04-17T07:23:15.102494+0000, chain.objs.size()=1 2024-04-17T05:46:03.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.008+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Uf75Ro8h5dwZUseqA3EvJr6tcf_vzD6_0 2024-04-17T05:46:03.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17061648692426430120', time=2024-04-17T07:23:15.183504+0000, chain.objs.size()=1 2024-04-17T05:46:03.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TkxK2Jg0hFpRJeqqNIvrtXVT7HIFJIh_0 2024-04-17T05:46:03.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12292133625003875477', time=2024-04-17T07:23:15.476083+0000, chain.objs.size()=1 2024-04-17T05:46:03.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-uiu1uyeccHWmyc22TRQL2jfCQcvseD_0 2024-04-17T05:46:03.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16654402640038304849', time=2024-04-17T07:23:18.208804+0000, chain.objs.size()=1 2024-04-17T05:46:03.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.x2wBtkB9yul9lLcMNl5NudGuh4PzxFu_0 2024-04-17T05:46:03.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3784608594620664465', time=2024-04-17T07:23:21.004958+0000, chain.objs.size()=1 2024-04-17T05:46:03.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.301LgKcnASK50ZCr7-Vq00H9SNKVhus_0 2024-04-17T05:46:03.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16940114683882247342', time=2024-04-17T07:23:22.487085+0000, chain.objs.size()=1 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.huhU4llchncH0_b_2LNjdisMcjTUlPb_0 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9394666553646278259', time=2024-04-17T07:23:23.739688+0000, chain.objs.size()=1 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fPaBUtY9Fy0qk1rmEgBm-CO13OoMSIT_0 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14327011481354087011', time=2024-04-17T07:23:25.944147+0000, chain.objs.size()=1 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.AkopRZyvN3UlbYdmJPHYRSIP0a3WbrR_0 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6031637617005651873', time=2024-04-17T07:23:25.946316+0000, chain.objs.size()=1 2024-04-17T05:46:03.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.012+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ssXEoWltoVd-sUpsjkxubuxpuEmLh2a_0 2024-04-17T05:46:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16471865380697187749', time=2024-04-17T07:23:26.783741+0000, chain.objs.size()=1 2024-04-17T05:46:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MSzO89yND2RV-xNTPMUbAgP1YbzvDL5_0 2024-04-17T05:46:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15290861821735524423', time=2024-04-17T07:23:27.578172+0000, chain.objs.size()=1 2024-04-17T05:46:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pHdVNZrJfdw2TqC1v69uGhRjazXyRhD_0 2024-04-17T05:46:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2905027142690328057', time=2024-04-17T07:23:29.108150+0000, chain.objs.size()=1 2024-04-17T05:46:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2my5HboFL_DEKqsWtrEnKOS21bQfD2X_0 2024-04-17T05:46:03.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6527163869646254420', time=2024-04-17T07:23:29.811405+0000, chain.objs.size()=1 2024-04-17T05:46:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oV61fOTZlWYAPolHACRyoMv64rjkp6__0 2024-04-17T05:46:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1136438588755776147', time=2024-04-17T07:23:30.931344+0000, chain.objs.size()=1 2024-04-17T05:46:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jKfaF6iX5LQgL73KhxuYNwnYpOZ5L1g_0 2024-04-17T05:46:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16335684650257492079', time=2024-04-17T07:23:34.433600+0000, chain.objs.size()=1 2024-04-17T05:46:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PEaN3Sk9fksK9t0goq6mkdmqMkUWqy-_0 2024-04-17T05:46:03.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7989795661035157075', time=2024-04-17T07:23:34.486953+0000, chain.objs.size()=1 2024-04-17T05:46:03.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YNSZ7TA9PcJDrY56_WRhZ6eAyfbsYiH_0 2024-04-17T05:46:03.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13613074859072970607', time=2024-04-17T07:23:37.502781+0000, chain.objs.size()=1 2024-04-17T05:46:03.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.016+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.M8lzVc2xMyE5KV9_3HwMT_MbTSUBNRE_0 2024-04-17T05:46:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7314210407526359742', time=2024-04-17T07:23:38.008010+0000, chain.objs.size()=1 2024-04-17T05:46:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uOMZ-nJJAhfzmgZ2owGZsMOdVLuysIG_0 2024-04-17T05:46:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5314974302661724657', time=2024-04-17T07:23:38.516898+0000, chain.objs.size()=1 2024-04-17T05:46:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2VqDRjeTMqihti54gHLfFpVbhYwrtj6_0 2024-04-17T05:46:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18168024104508743563', time=2024-04-17T07:23:39.793192+0000, chain.objs.size()=1 2024-04-17T05:46:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lcbuo08SrAk9ltXqzv11zcHoS0Ny-wg_0 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.289918477536511957', time=2024-04-17T07:23:41.664610+0000, chain.objs.size()=1 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ekMNHFGbwyAt22hlhWujalSIWbD9NRN_0 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4986244165400798516', time=2024-04-17T07:23:44.402573+0000, chain.objs.size()=1 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.317__shadow_.Ng_kPx21yh7jKEnWCR4gKkRYMFmRsUR_0 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11366981064545346918', time=2024-04-17T07:24:33.612409+0000, chain.objs.size()=1 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.372__shadow_.o3cIoQK4U6UrtFOELm331ug_f5hYcQ-_0 2024-04-17T05:46:03.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2480991586012050687', time=2024-04-17T07:25:30.763038+0000, chain.objs.size()=1 2024-04-17T05:46:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.020+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.433__shadow_.MsQpwjhWn62hkkn8f2p4AVX4qcQ8yca_0 2024-04-17T05:46:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11546072113944291978', time=2024-04-17T07:25:50.693056+0000, chain.objs.size()=1 2024-04-17T05:46:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.449__shadow_.pV7fPRwHLcJZTrmRM7eBXNw4WNw4GP5_0 2024-04-17T05:46:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3504972362138757854', time=2024-04-17T07:26:36.737441+0000, chain.objs.size()=1 2024-04-17T05:46:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.488__shadow_.iG18N--mlQ4Drns8sXiYqtuo0gsJaIT_0 2024-04-17T05:46:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3786341543733835440', time=2024-04-17T07:27:05.707589+0000, chain.objs.size()=1 2024-04-17T05:46:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.505__shadow_.WGR4Gh11iTAPXMMNijSm3v_L_x1uo9I_0 2024-04-17T05:46:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16118658973876670288', time=2024-04-17T07:29:22.204029+0000, chain.objs.size()=3 2024-04-17T05:46:03.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~9_mS7w44mnaITtBpyDUoWsH7Hr6N0Fo.1 2024-04-17T05:46:03.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~9_mS7w44mnaITtBpyDUoWsH7Hr6N0Fo.2 2024-04-17T05:46:03.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~9_mS7w44mnaITtBpyDUoWsH7Hr6N0Fo.3 2024-04-17T05:46:03.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7811093966353360439', time=2024-04-17T07:29:31.272581+0000, chain.objs.size()=2 2024-04-17T05:46:03.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.024+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~2PYar7YZaqm8fURa7eZY0ZcoqypxJ5p.1 2024-04-17T05:46:03.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~2PYar7YZaqm8fURa7eZY0ZcoqypxJ5p.2 2024-04-17T05:46:03.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.028+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6456909908457376902', time=2024-04-17T07:29:48.755880+0000, chain.objs.size()=1 2024-04-17T05:46:03.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.028+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.518__shadow_.AY6h1zV9XUS4389SGM5NoNuIm4a6I_9_0 2024-04-17T05:46:03.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/47325 2024-04-17T05:46:03.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-04-17T05:46:03.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6924684407882246395', time=2024-04-17T07:30:46.513305+0000, chain.objs.size()=1 2024-04-17T05:46:03.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.540__shadow_.5ZZOwt6j3LiwBZJlkd0bQRJZ317LUVi_0 2024-04-17T05:46:03.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17723929439740412674', time=2024-04-17T07:32:32.356422+0000, chain.objs.size()=1 2024-04-17T05:46:03.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:-U7NcW6K1-nwYALh3qwLMeENh32ydGp_.rdhO427Gv1KkFd3t-dEaNWsM5U8a57b_0 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13868386188564749216', time=2024-04-17T07:32:34.403608+0000, chain.objs.size()=1 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:O3o7PXx0HRUZBnCocdOz9pJyNk64Wnl_.XigB0pdHUVU02dN4AptLIsddrpjYS2O_0 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12405979508910832285', time=2024-04-17T07:32:47.211665+0000, chain.objs.size()=1 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:YjvzGC13oH8AfPbKEE30w.kp4wjpJbc_.Fcg9GvddvQ1SYZQ9krTK3Awxsv_EtZb_0 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8458453281366365082', time=2024-04-17T07:33:28.842681+0000, chain.objs.size()=1 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:G6tok1c29AwHHD1lXAzk-vYZyhk6i8j_.WaMpCDaIud838nX3G3hrU2_2kTC7_KI_0 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10264625863506303785', time=2024-04-17T07:36:27.283761+0000, chain.objs.size()=1 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.614__shadow_.tlkl6dV9zL0SWdQpdWdtDtEmB2iPwXI_0 2024-04-17T05:46:03.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15311014938805425276', time=2024-04-17T07:36:36.029916+0000, chain.objs.size()=1 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.622__shadow_.zfvvVWpFCVDVrzdg3AgrbM25wzyJXPz_0 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6272932966959985025', time=2024-04-17T07:37:35.909443+0000, chain.objs.size()=1 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.648__shadow_.Oljld5TqnsKUUWVYacm6qF414PC1NvK_0 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.uqS6ilJ7LY2EpBPAG5cpUnIji_2RErQ_1_f1bfbbf581860c0c823d8d0918c5c3d5', time=2024-04-17T07:37:42.283920+0000, chain.objs.size()=1 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.651__shadow_.uqS6ilJ7LY2EpBPAG5cpUnIji_2RErQ_0 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11615830485716111615', time=2024-04-17T07:37:53.930407+0000, chain.objs.size()=1 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.664__shadow_.H7MzHQuHPJe13ZFbSKKZGrESs3eJHea_0 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1146655807496647847', time=2024-04-17T07:38:11.879580+0000, chain.objs.size()=1 2024-04-17T05:46:03.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.683__shadow:QrKJrsh6bOSjFS3jZ2VVDM8D1kFA3iM_.0ftBDGc4qkGH5XDm1ybgRQqFtp0-LO-_0 2024-04-17T05:46:03.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2875300281767825583', time=2024-04-17T07:38:21.196367+0000, chain.objs.size()=1 2024-04-17T05:46:03.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.032+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.690__shadow:6udmvN9nvGtGOjk-pWwOoNDA1V7m6MY_.jn8DTqyxV5CShwTtyNbtoTR3vEyLS0D_0 2024-04-17T05:46:03.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.040+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-17T05:46:03.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/40265' 2024-04-17T05:46:03.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_ZhQcogAWnJenZGq-lqqo0jCKuWceFem', time=2024-04-17T07:10:34.482139+0000, chain.objs.size()=2 2024-04-17T05:46:03.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.155__shadow_.VYGNrz3T6i3O4rzdFWYrUlrYrWYV38x_0 2024-04-17T05:46:03.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.044+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.155__shadow_.VYGNrz3T6i3O4rzdFWYrUlrYrWYV38x_1 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7157897100858596164', time=2024-04-17T07:14:38.636246+0000, chain.objs.size()=1 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.227__shadow_.rNeGG_mDZZFimqI0zQ2zzZxoBsLJRmD_0 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6419028015496640845', time=2024-04-17T07:14:40.964063+0000, chain.objs.size()=1 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.228__shadow_.uNJZosYOPU52l6fXxqUAJ0RojrZ0z-T_0 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13094622896624957166', time=2024-04-17T07:15:00.972961+0000, chain.objs.size()=1 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.240__shadow_.fuQwTfmmScgY_8UOQ9XWeWJfW66rbAB_0 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13730417475017756733', time=2024-04-17T07:15:09.154258+0000, chain.objs.size()=1 2024-04-17T05:46:03.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.245__shadow_.l0ExnP0D2tYB2pzHH3I8ijVRVn5FyaB_0 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6649434642645902856', time=2024-04-17T07:15:13.773110+0000, chain.objs.size()=1 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.248__shadow_.XCNdZ745i_Z_9vYUsfq1hGVOqlRHmZ7_0 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5427611210386072478', time=2024-04-17T07:17:02.559193+0000, chain.objs.size()=1 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.i08xuaKryJcqhrWoIQi_sSHn3eHBYdT_0 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16781074816149166972', time=2024-04-17T07:17:03.493898+0000, chain.objs.size()=1 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vsmNC3KwAlCqillYuKKz_Su8kdLjW9J_0 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11218671105516320174', time=2024-04-17T07:17:03.495814+0000, chain.objs.size()=1 2024-04-17T05:46:03.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8JJWDQAxi-UieZJ7HjjnVoKxk4XgTBA_0 2024-04-17T05:46:03.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2548148276017395935', time=2024-04-17T07:17:06.511689+0000, chain.objs.size()=1 2024-04-17T05:46:03.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._j9WtCVe77NZJFuUoG2LavcuqlM5w-m_0 2024-04-17T05:46:03.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3292680148274212134', time=2024-04-17T07:17:07.255658+0000, chain.objs.size()=1 2024-04-17T05:46:03.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.048+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7oR40LDl7MLkCSaQy2EUmMe7HG8HhhX_0 2024-04-17T05:46:03.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13491165559792106102', time=2024-04-17T07:17:07.754889+0000, chain.objs.size()=1 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yTmO6vdfDESfJ1-p4gBpipbNUHkOKQ0_0 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15486515854929338689', time=2024-04-17T07:17:09.849985+0000, chain.objs.size()=1 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nYmWAV1GkobMSY0dACyf-_ZrE7RjF7K_0 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7495180418889276153', time=2024-04-17T07:17:09.964793+0000, chain.objs.size()=1 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.70XqfGOiVfpkleipZJS9swM0TcC4ckX_0 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5960193579785669027', time=2024-04-17T07:17:10.439426+0000, chain.objs.size()=1 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.d1Yra1D3sGn3qZGJQGK0jpGpZfHHJjl_0 2024-04-17T05:46:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11083463209961887546', time=2024-04-17T07:17:10.884253+0000, chain.objs.size()=1 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fy1W5kA2Ep2HW69k4QGIq5vz1Tj7NYh_0 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17249885420117662599', time=2024-04-17T07:17:10.886619+0000, chain.objs.size()=1 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BvhVvZRmFZI3EwH7uMNeAq6X0fXt_Dv_0 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1128885090921965553', time=2024-04-17T07:17:11.731273+0000, chain.objs.size()=1 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.G5B_Xl3jlEzA-ZF-4Xvma9pPk32Ju9z_0 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17135008332263113403', time=2024-04-17T07:17:12.795429+0000, chain.objs.size()=1 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5grwiGfYJtJYeSJwY1gOBMkSFe-0eH4_0 2024-04-17T05:46:03.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9981720637114624406', time=2024-04-17T07:17:14.768549+0000, chain.objs.size()=1 2024-04-17T05:46:03.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NlBYnnVIFrp49rjCpk0xl3Sjl50e42q_0 2024-04-17T05:46:03.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2607558942625801880', time=2024-04-17T07:17:14.770282+0000, chain.objs.size()=1 2024-04-17T05:46:03.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kX83xtxlzgjYcrGOeXBAQeibElY_ZtS_0 2024-04-17T05:46:03.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10736701572261251797', time=2024-04-17T07:17:14.930490+0000, chain.objs.size()=1 2024-04-17T05:46:03.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.052+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YxiYudXdVLzt2FZC6Im65WDlwyTpTMQ_0 2024-04-17T05:46:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7468430990035043543', time=2024-04-17T07:17:15.044633+0000, chain.objs.size()=1 2024-04-17T05:46:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aqR13XRI_Dq0M8dDCJASHXQdg5m52IP_0 2024-04-17T05:46:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2052734683398075027', time=2024-04-17T07:17:15.045894+0000, chain.objs.size()=1 2024-04-17T05:46:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PONCpZfeu4TMs5A4n9tmoGjMbdoRotj_0 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9727827710197165410', time=2024-04-17T07:17:15.779904+0000, chain.objs.size()=1 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.m5TOVMyxHDhECuDYAD7mWWg6l-jXG_c_0 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5855734836974434505', time=2024-04-17T07:17:17.045993+0000, chain.objs.size()=1 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ygv5rTuszxkE1euQpVcqGr2KnlnK7Ay_0 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18259501043870751969', time=2024-04-17T07:17:17.977894+0000, chain.objs.size()=1 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.B3S5Yyi2vhG8D71drkWxpBjpZJKMJRv_0 2024-04-17T05:46:03.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14240333773857245060', time=2024-04-17T07:17:18.796868+0000, chain.objs.size()=1 2024-04-17T05:46:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8GHUfw9LsNJSXstbxy-ZUK-2LpyYoDq_0 2024-04-17T05:46:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1470420584404863906', time=2024-04-17T07:17:18.986879+0000, chain.objs.size()=1 2024-04-17T05:46:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2zXKJnNmwNLWIWPYqu6omtmeGaibVzW_0 2024-04-17T05:46:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7045756330163921739', time=2024-04-17T07:17:19.311859+0000, chain.objs.size()=1 2024-04-17T05:46:03.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.q8ztSbOqbW_OSYxVyDGuYHdPXgPCsMj_0 2024-04-17T05:46:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3679074392172518937', time=2024-04-17T07:17:19.865073+0000, chain.objs.size()=1 2024-04-17T05:46:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3xtUFVPIo4PEyZbiYDAivc-M2rAO8-A_0 2024-04-17T05:46:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6684716647003059585', time=2024-04-17T07:17:20.100973+0000, chain.objs.size()=1 2024-04-17T05:46:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mm5PR9tEdVmduRx9bLVsGXhn0hYnBB__0 2024-04-17T05:46:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16143667907301156887', time=2024-04-17T07:17:20.435545+0000, chain.objs.size()=1 2024-04-17T05:46:03.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.056+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wBQJcLJ8szOCVb8wGVwPK_ur4MUDazY_0 2024-04-17T05:46:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3900801827520429160', time=2024-04-17T07:17:22.060546+0000, chain.objs.size()=1 2024-04-17T05:46:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lR2LpBUKvFzRfnxBfM50AQ7mrXLQFTg_0 2024-04-17T05:46:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.716164469316199393', time=2024-04-17T07:17:22.351356+0000, chain.objs.size()=1 2024-04-17T05:46:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Y7Lb4BWdr2OAARAVTBm10JGur9AbOtE_0 2024-04-17T05:46:03.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10775768093044443921', time=2024-04-17T07:17:22.542038+0000, chain.objs.size()=1 2024-04-17T05:46:03.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rQ5VpMh-iXIKFwep_QL4PEu8jnvGuQp_0 2024-04-17T05:46:03.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2907610274347872626', time=2024-04-17T07:17:23.531881+0000, chain.objs.size()=1 2024-04-17T05:46:03.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.THaPQcE_cRFI6zM7HSSRxqUhdjiDjpo_0 2024-04-17T05:46:03.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13028770613427351319', time=2024-04-17T07:17:23.949835+0000, chain.objs.size()=1 2024-04-17T05:46:03.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.IZ_EHnn7gwCwmbD2n-huniYTSmxuI7P_0 2024-04-17T05:46:03.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18391481355766647028', time=2024-04-17T07:17:24.715633+0000, chain.objs.size()=1 2024-04-17T05:46:03.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.if_MVPBrYo28Tb6ccrJnPEuKah5-jwK_0 2024-04-17T05:46:03.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14524690023355091509', time=2024-04-17T07:17:24.850421+0000, chain.objs.size()=1 2024-04-17T05:46:03.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.060+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EPh2D0daLeNKnlyqUNuQ9cSaR_HXo65_0 2024-04-17T05:46:03.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14141157766195685390', time=2024-04-17T07:17:25.962918+0000, chain.objs.size()=1 2024-04-17T05:46:03.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._Db5KUQz7y2bDqfRL0bEoiAZzeODHqC_0 2024-04-17T05:46:03.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17468569724915961135', time=2024-04-17T07:17:27.460522+0000, chain.objs.size()=1 2024-04-17T05:46:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TYY-ECSPVgV_Gu8598A9BfaSdg2u6zq_0 2024-04-17T05:46:03.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17466044318070576400', time=2024-04-17T07:17:27.730943+0000, chain.objs.size()=1 2024-04-17T05:46:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dUkEwol_ciIgiQRRvlxktZqqLgUaoMe_0 2024-04-17T05:46:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7861081607133457662', time=2024-04-17T07:17:29.163540+0000, chain.objs.size()=1 2024-04-17T05:46:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VuWBArZWEBFWdqlvVpUwztdtiNS6L9l_0 2024-04-17T05:46:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.644072215526929438', time=2024-04-17T07:17:31.221884+0000, chain.objs.size()=1 2024-04-17T05:46:03.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.20TDfWPvsE89A8mCXIwDdfc3FiBt5vh_0 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7822665965820097561', time=2024-04-17T07:17:31.741976+0000, chain.objs.size()=1 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iJQbMBeF5IQNS_zDFvzuSCueOkcRe6m_0 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10041546913780717153', time=2024-04-17T07:18:03.351000+0000, chain.objs.size()=1 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.272__shadow_.A1Ur7i9_wFmTZkCJ0WLumDToDGzNNv7_0 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3667915584677958539', time=2024-04-17T07:18:06.540896+0000, chain.objs.size()=1 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.274__shadow_.kNa7gpyAtX60AWMlo0KphpiTAlKOfSv_0 2024-04-17T05:46:03.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6482378572336373994', time=2024-04-17T07:18:22.916316+0000, chain.objs.size()=1 2024-04-17T05:46:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.nXs90i7qc5fP3R5ZFFi5zZ-8ZSJvYDH_0 2024-04-17T05:46:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15828010810046202567', time=2024-04-17T07:18:23.235365+0000, chain.objs.size()=1 2024-04-17T05:46:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.eck_AV9QVCARv24DqkRJLac8fatmUqm_0 2024-04-17T05:46:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.752810587436173476', time=2024-04-17T07:18:23.237114+0000, chain.objs.size()=1 2024-04-17T05:46:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.064+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.VHwitpFV-rK2eOb_ZGhVokwl25_HEK0_0 2024-04-17T05:46:03.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14640013971693919563', time=2024-04-17T07:18:34.910062+0000, chain.objs.size()=1 2024-04-17T05:46:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.287__shadow_.JQtppYiVT0rXQLc78hg7X0YywSJWB1M_0 2024-04-17T05:46:03.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.197246389747311431', time=2024-04-17T07:18:40.067730+0000, chain.objs.size()=1 2024-04-17T05:46:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.290__shadow_.5ZEwslDAc04hQIHU-1vho6VSLBrMBoW_0 2024-04-17T05:46:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9143229035796888044', time=2024-04-17T07:18:43.305365+0000, chain.objs.size()=1 2024-04-17T05:46:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.292__shadow_.WUvllSPpPDu9JIecV2tB9sevKtxt7mO_0 2024-04-17T05:46:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13843785372568868902', time=2024-04-17T07:21:00.011692+0000, chain.objs.size()=1 2024-04-17T05:46:03.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.inEtxDITVBVAlPsbsfp5WNxktqfvLbT_0 2024-04-17T05:46:03.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4011521586427750384', time=2024-04-17T07:21:00.700087+0000, chain.objs.size()=1 2024-04-17T05:46:03.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.068+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kqzrOdGE0vI1vlof7rVCi7U8m3AiXUO_0 2024-04-17T05:46:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4018706083553706088', time=2024-04-17T07:21:02.603316+0000, chain.objs.size()=1 2024-04-17T05:46:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lyb6RzGuOXFoCefYJ3FWDx5LN8TmR7M_0 2024-04-17T05:46:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16957658289399022168', time=2024-04-17T07:21:03.875275+0000, chain.objs.size()=1 2024-04-17T05:46:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LcDMDXkzdsYplmhyXwGbQe_jsgBmP_M_0 2024-04-17T05:46:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8280824016198340269', time=2024-04-17T07:21:10.069416+0000, chain.objs.size()=1 2024-04-17T05:46:03.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zSBGhDoiD6nufdj1fs9Ao2utLaonccQ_0 2024-04-17T05:46:03.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16809833349789840445', time=2024-04-17T07:21:12.720021+0000, chain.objs.size()=1 2024-04-17T05:46:03.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PGZO--mBbH2fWheosP9aWx31owhohf3_0 2024-04-17T05:46:03.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7888730312539936705', time=2024-04-17T07:21:12.903188+0000, chain.objs.size()=1 2024-04-17T05:46:03.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nouK11utiyXCKrV4kpv2AgLHGAgkFFQ_0 2024-04-17T05:46:03.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10500483312813214217', time=2024-04-17T07:21:13.542566+0000, chain.objs.size()=1 2024-04-17T05:46:03.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.e3dCCyOCriMPdqx0MY0jPTvzfALHBA1_0 2024-04-17T05:46:03.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1708175758387203550', time=2024-04-17T07:21:13.996530+0000, chain.objs.size()=1 2024-04-17T05:46:03.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._EB0VMyvGSh9DBK7o-txglg1QrheIiG_0 2024-04-17T05:46:03.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2091495294889220948', time=2024-04-17T07:21:14.107034+0000, chain.objs.size()=1 2024-04-17T05:46:03.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.J38kc0Z557QxJwOj2P6pmn4eIHhDP9j_0 2024-04-17T05:46:03.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6590483906209549533', time=2024-04-17T07:21:14.211682+0000, chain.objs.size()=1 2024-04-17T05:46:03.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.072+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JZTXvB_P86O9LSzgpvqXPdbC6Ph1Zn4_0 2024-04-17T05:46:03.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17112498838108517057', time=2024-04-17T07:21:16.121417+0000, chain.objs.size()=1 2024-04-17T05:46:03.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.AYkmRIVyFZqUZNTdGcEaxxwZ0ZPVG0M_0 2024-04-17T05:46:03.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5079458980242000273', time=2024-04-17T07:21:16.885663+0000, chain.objs.size()=1 2024-04-17T05:46:03.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BDzpqcmIg01jG641k2sz3MVVK4ZmyX7_0 2024-04-17T05:46:03.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.141681819140464268', time=2024-04-17T07:21:17.146832+0000, chain.objs.size()=1 2024-04-17T05:46:03.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.AkaPQGOe6auNim_ef_2QHNQE3jolIUr_0 2024-04-17T05:46:03.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4807825678783742614', time=2024-04-17T07:21:17.302137+0000, chain.objs.size()=1 2024-04-17T05:46:03.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VwXXhMsgWTF84ncthGPYleELtxSV0Qu_0 2024-04-17T05:46:03.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.822330008980054644', time=2024-04-17T07:21:17.739600+0000, chain.objs.size()=1 2024-04-17T05:46:03.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3FLR1o2QXvJi1HymmT47esF4p1KgLgh_0 2024-04-17T05:46:03.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3437792388933501180', time=2024-04-17T07:21:18.134320+0000, chain.objs.size()=1 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8D47IimLQTBpXZWdFFYa0-2jqYMibr0_0 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3815881958666974842', time=2024-04-17T07:21:19.490029+0000, chain.objs.size()=1 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JwOt6tZrXahxZoB8hB3b2nJv3CmsowU_0 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11016368337904599162', time=2024-04-17T07:21:20.171324+0000, chain.objs.size()=1 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7khBa58Wf_MwaPTmav6graY4DbQz_FL_0 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11451082688481131046', time=2024-04-17T07:21:24.045382+0000, chain.objs.size()=1 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.076+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CfIjKZGC0HSXFOnXaSQ_CirdloekFKo_0 2024-04-17T05:46:03.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11035652492538631324', time=2024-04-17T07:21:24.049748+0000, chain.objs.size()=1 2024-04-17T05:46:03.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UxoEN-Cnu0vxRONqTdmY65fvgFrtApo_0 2024-04-17T05:46:03.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13807534669412304802', time=2024-04-17T07:21:24.604629+0000, chain.objs.size()=1 2024-04-17T05:46:03.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WjGkbVHGWB4F-iRzGoXTIS1WzgkOD98_0 2024-04-17T05:46:03.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8951106408183217234', time=2024-04-17T07:21:24.654600+0000, chain.objs.size()=1 2024-04-17T05:46:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DIWiXAwR69BaNkpC4wU3uIMfUKIdn7S_0 2024-04-17T05:46:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14480849641277084672', time=2024-04-17T07:21:24.656287+0000, chain.objs.size()=1 2024-04-17T05:46:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LdTk96WcS-1GUOJGc3Ztcb_PCk_DGbB_0 2024-04-17T05:46:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12792795266345571187', time=2024-04-17T07:21:27.500374+0000, chain.objs.size()=1 2024-04-17T05:46:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ACHHFbbjRsV3GSMIGnXOQPavKBSrB47_0 2024-04-17T05:46:03.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14358777127666774455', time=2024-04-17T07:21:27.502595+0000, chain.objs.size()=1 2024-04-17T05:46:03.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3ufzkVpFkHfuNDX4odNHGYc4MmARxey_0 2024-04-17T05:46:03.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8197340077817886143', time=2024-04-17T07:21:27.886791+0000, chain.objs.size()=1 2024-04-17T05:46:03.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.080+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DRyMZd5E5zg1ruQqlf6Pr4zmyaKWRjs_0 2024-04-17T05:46:03.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17826124212411444293', time=2024-04-17T07:21:28.187774+0000, chain.objs.size()=1 2024-04-17T05:46:03.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.si1JMr05rzhXkczxvOQ5wIsLoXREvuV_0 2024-04-17T05:46:03.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2042676255135699053', time=2024-04-17T07:21:28.640235+0000, chain.objs.size()=1 2024-04-17T05:46:03.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2tdLRmeOotTn_5PCJ1E7fZLRjt9fmkU_0 2024-04-17T05:46:03.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9367536083049133982', time=2024-04-17T07:21:28.798985+0000, chain.objs.size()=1 2024-04-17T05:46:03.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SxEWfJBCmi2sdgO9IsN7QdSxQOn5RCQ_0 2024-04-17T05:46:03.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12114770104426941214', time=2024-04-17T07:21:29.254658+0000, chain.objs.size()=1 2024-04-17T05:46:03.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.084+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IP4wjAGj0XXW6nWrHgzVlqTPTXNh12B_0 2024-04-17T05:46:03.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.180442374796598411', time=2024-04-17T07:23:12.633038+0000, chain.objs.size()=1 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.V_vAxN9B3nf3OuntghhW46x432-3fK5_0 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2042426711288042741', time=2024-04-17T07:23:15.971536+0000, chain.objs.size()=1 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pWIF9lGzz0E62wiQKzWP6Kj2mktBxtF_0 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14481033003455780430', time=2024-04-17T07:23:17.734541+0000, chain.objs.size()=1 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.REcZP2E_6z71cIVk4YEDADmwkhQKxQa_0 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9277221680172421632', time=2024-04-17T07:23:18.573460+0000, chain.objs.size()=1 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Oaun0y_Hut5AEP6TtIArVS5B-ETM3vv_0 2024-04-17T05:46:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5343753478076735560', time=2024-04-17T07:23:19.296205+0000, chain.objs.size()=1 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BSfS0pUS56wEehTXIa-PVxq8OwDS67f_0 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9393371303780470450', time=2024-04-17T07:23:19.298298+0000, chain.objs.size()=1 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BvOLTbsyfTRCzMRvoYyzgso4cY91lC2_0 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1504953686633994621', time=2024-04-17T07:23:19.449697+0000, chain.objs.size()=1 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VWtAP42RfY4vY11hmpNdRYN_9XlQymN_0 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5381971998176014338', time=2024-04-17T07:23:19.621394+0000, chain.objs.size()=1 2024-04-17T05:46:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EXtvaD2An3N5r4heuQKgq5aINBKusNK_0 2024-04-17T05:46:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11127193281384125132', time=2024-04-17T07:23:19.863421+0000, chain.objs.size()=1 2024-04-17T05:46:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ekSBiPWcSbpURJeqlGZbuzVZy1eGXzs_0 2024-04-17T05:46:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14890631309727714628', time=2024-04-17T07:23:21.778891+0000, chain.objs.size()=1 2024-04-17T05:46:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gxA1hZMaanttuIHqyBUEp3EESN1068__0 2024-04-17T05:46:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11662284577710613288', time=2024-04-17T07:23:22.331455+0000, chain.objs.size()=1 2024-04-17T05:46:03.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.088+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-ctK35yT4kxsby5mg8MQ6U02upsUK_J_0 2024-04-17T05:46:03.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2599033035111801132', time=2024-04-17T07:23:22.553798+0000, chain.objs.size()=1 2024-04-17T05:46:03.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Q5X6QrPJyXVa9ikVJ3UMWaChA1r__fT_0 2024-04-17T05:46:03.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11179172756752003386', time=2024-04-17T07:23:23.383143+0000, chain.objs.size()=1 2024-04-17T05:46:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9Ccer3BV5VMf4PrmWVUTiJxBRHkLqMT_0 2024-04-17T05:46:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13804564491600581978', time=2024-04-17T07:23:23.452212+0000, chain.objs.size()=1 2024-04-17T05:46:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rLcWzI3HKaSMw2jIN435N7C8e0glcVu_0 2024-04-17T05:46:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15668235099135350876', time=2024-04-17T07:23:23.454742+0000, chain.objs.size()=1 2024-04-17T05:46:03.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.092+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Un5jBz5Ate0t9kEcDRHgvpo16Y4e4qi_0 2024-04-17T05:46:03.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.096+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/40265 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=33, truncated=0, next_marker='' 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10854405745677322715', time=2024-04-17T07:23:25.364309+0000, chain.objs.size()=1 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8COUkFxfsxJiouxm1VDlhqdgxUsrzey_0 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4968857686049668341', time=2024-04-17T07:23:25.518547+0000, chain.objs.size()=1 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pgIEyBxCC905_jrFi3HS6oqTNiO4xCy_0 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13191478091956806670', time=2024-04-17T07:23:25.799658+0000, chain.objs.size()=1 2024-04-17T05:46:03.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RHdONJa7R6aFRgtxxmyZWIAGVelkV6E_0 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4556219034536176384', time=2024-04-17T07:23:26.783679+0000, chain.objs.size()=1 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UO6-C7wKSeHvGvUIR-qaxYtfmuTwJLq_0 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3042121731210346030', time=2024-04-17T07:23:26.987569+0000, chain.objs.size()=1 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vENloI4M92q9IosxT3mIOuRbwxTTrgs_0 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17491812826700340423', time=2024-04-17T07:23:27.477618+0000, chain.objs.size()=1 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mF64ogjjIxPvv7oYiZ_LDEefTGuz8mS_0 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14056336574988063228', time=2024-04-17T07:23:29.108426+0000, chain.objs.size()=1 2024-04-17T05:46:03.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.T6geEAgAOBmsZoHkVqIFjh6TbCZY0zb_0 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16986341304038748637', time=2024-04-17T07:23:29.276319+0000, chain.objs.size()=1 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.741O-ADedRiHdjAvffLOd7R5fnEJptF_0 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4126701405273675343', time=2024-04-17T07:23:29.736022+0000, chain.objs.size()=1 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.StZI6S48a5ijtIZypPD0QSL84HSAAkp_0 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3221127383106293898', time=2024-04-17T07:23:29.882521+0000, chain.objs.size()=1 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6IwwwKXqqnObEQFDfj0AkMFZS0XTUU8_0 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5184926006728492314', time=2024-04-17T07:23:30.297957+0000, chain.objs.size()=1 2024-04-17T05:46:03.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.koGsWYjdhheal6wDrmR4YtHDWMlugrr_0 2024-04-17T05:46:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5986353255622963806', time=2024-04-17T07:23:32.154908+0000, chain.objs.size()=1 2024-04-17T05:46:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.100+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Rp0K7QAIslxs7aNKis6wkfoRLiIUYh5_0 2024-04-17T05:46:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2078473994611663256', time=2024-04-17T07:23:32.276927+0000, chain.objs.size()=1 2024-04-17T05:46:03.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EqcQQy-2nhDPNKUEBaNtF_wS8ItMhaP_0 2024-04-17T05:46:03.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4025855935054853543', time=2024-04-17T07:23:32.279214+0000, chain.objs.size()=1 2024-04-17T05:46:03.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MyTbke3tKkonTpy0faj2MTbVgEE6DxI_0 2024-04-17T05:46:03.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6960300204677331079', time=2024-04-17T07:23:32.510539+0000, chain.objs.size()=1 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._ONpG9BQQqbHX48XTHXv45RBRxWeYJI_0 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1699125016794959161', time=2024-04-17T07:23:32.512579+0000, chain.objs.size()=1 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.j3s_gnc0qOsbCMAwDqF2bRWwainHhRF_0 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1514511148138161596', time=2024-04-17T07:23:32.539764+0000, chain.objs.size()=1 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4CG23YWSLaxFViLtkX5zWjYSpT4Az_l_0 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3949575499179364171', time=2024-04-17T07:23:34.864826+0000, chain.objs.size()=1 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.AWmTCeCCbb26KsptOS4EFXQnpZc7-Rd_0 2024-04-17T05:46:03.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17795557917184318684', time=2024-04-17T07:23:37.203418+0000, chain.objs.size()=1 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Z9nliYeMHf9Cw0U41uVtDscnYf7_tVp_0 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7173866083880614699', time=2024-04-17T07:23:37.432411+0000, chain.objs.size()=1 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jia1axjE4BwB-0HMTNxQ4mrdEVkizK6_0 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1999729939773839671', time=2024-04-17T07:23:38.630914+0000, chain.objs.size()=1 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.U1NHKJgPC7kVcrhQr4zliFgLUnki8y5_0 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15596534210123670159', time=2024-04-17T07:23:39.438592+0000, chain.objs.size()=1 2024-04-17T05:46:03.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.104+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Us-tQ2Hg3te_tMVef7Nt1fKPeud4IYj_0 2024-04-17T05:46:03.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.108+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.183477538119529468', time=2024-04-17T07:23:40.118306+0000, chain.objs.size()=1 2024-04-17T05:46:03.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.108+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gXva3lCZbeFesTrueHt1POK-G5JzdzS_0 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9744386761918688367', time=2024-04-17T07:26:18.781490+0000, chain.objs.size()=1 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.474__shadow_.wqyGskZKL6QkSpgRCES7-VY-ehm9L1Y_0 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8527891851665691253', time=2024-04-17T07:31:02.270276+0000, chain.objs.size()=1 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.545__shadow_.KYQIOoaMJNym4GytMvLzMpR-GHSUfIF_0 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3952876276095858145', time=2024-04-17T07:31:47.921180+0000, chain.objs.size()=1 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:tSMfrpzzsqfbiAlR-YSwn-zmbEV2yw8_.hRrHvuHLpzbRJgm5-bg1xPB_-kjMtaI_0 2024-04-17T05:46:03.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6637401385851833158', time=2024-04-17T07:31:59.577778+0000, chain.objs.size()=1 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.560__shadow_.NYhBr9HubvovwgsMgo6y_dhekjz6gS2_0 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17886291774144545512', time=2024-04-17T07:32:25.385884+0000, chain.objs.size()=1 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:ZiIA0vub8ovE0ejRC3wMHFod.PQQTjO_.n0ONtkGhGGiNlWNMXSB5ycdbPqj67bs_0 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3825739763142013844', time=2024-04-17T07:32:31.182638+0000, chain.objs.size()=1 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:4m22RjkPXjOg36ueO-z.ogvEqlpNovq_.QaukWOpZyMsHzaRg974Uvkz-EM8zYm4_0 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF', time=2024-04-17T07:36:25.444590+0000, chain.objs.size()=6 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612__multipart_multipart_enc.2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF.1 2024-04-17T05:46:03.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612__multipart_multipart_enc.2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF.2 2024-04-17T05:46:03.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612__multipart_multipart_enc.2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF.3 2024-04-17T05:46:03.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612__multipart_multipart_enc.2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF.4 2024-04-17T05:46:03.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.112+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612__multipart_multipart_enc.2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF.5 2024-04-17T05:46:03.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.612__multipart_multipart_enc.2~bJdZFxAxHWTsMAJdk8quQu_xehv2tjF.6 2024-04-17T05:46:03.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12083936048105833300', time=2024-04-17T07:36:33.513030+0000, chain.objs.size()=1 2024-04-17T05:46:03.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.620__shadow_.7FcHuZ4EQyaX7HZx5zmDcNDrPQRAsX__0 2024-04-17T05:46:03.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10008935019144469205', time=2024-04-17T07:37:04.790962+0000, chain.objs.size()=1 2024-04-17T05:46:03.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.639__shadow_.ieCUX0iJfOjBdPRbC11kpZdsd7N3OqX_0 2024-04-17T05:46:03.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5215259046898880810', time=2024-04-17T07:38:42.498919+0000, chain.objs.size()=1 2024-04-17T05:46:03.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.116+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.700__shadow:qfFFOUVvBXWrQjHevYYj0HgJZRa66Wo_.JvO9SptvW7cMLhTw1mwOsC2wQAbQ_fC_0 2024-04-17T05:46:03.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.120+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-17T05:46:03.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47857' 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1973766590575475411', time=2024-04-17T07:14:40.964119+0000, chain.objs.size()=1 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.228__shadow_.6CbemrW3vIEjh-zLo91JCybtNRL407__0 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14383058893815336659', time=2024-04-17T07:14:43.429899+0000, chain.objs.size()=1 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.230__shadow_.gU83AocAuEEJzaetIBZO2tHTOLvgQmS_0 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14634298480842924351', time=2024-04-17T07:15:00.955018+0000, chain.objs.size()=1 2024-04-17T05:46:03.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.240__shadow_._3jqiUAgtjlc5tq5vh2WMhYb8gL4-Af_0 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13371505694220878143', time=2024-04-17T07:15:02.658081+0000, chain.objs.size()=1 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.241__shadow_._-i2d8WZtJPrQ0cZYpUz3mTYa7duAEf_0 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7520219404435102034', time=2024-04-17T07:15:16.817876+0000, chain.objs.size()=1 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.250__shadow_.COWf7BtqKoOTc7HATIuDekQLEOfsqO9_0 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3726211522456136308', time=2024-04-17T07:17:03.274346+0000, chain.objs.size()=1 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UwqhIcLXxWWqIxu7nc4aVkhu9f2ppef_0 2024-04-17T05:46:03.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10346374413213089134', time=2024-04-17T07:17:03.629881+0000, chain.objs.size()=1 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ETt0Bem7BdQPbaBUUAZUnRDgacVo56__0 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3430450540154353260', time=2024-04-17T07:17:04.310184+0000, chain.objs.size()=1 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.unc3XITcjpbUaIxr7kTIupxQDHYY8RO_0 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6828323938672289472', time=2024-04-17T07:17:05.565250+0000, chain.objs.size()=1 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.95_sU7bVyQOAzfyc-_EYT7Mqjexz4mN_0 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11683415447441814553', time=2024-04-17T07:17:06.847952+0000, chain.objs.size()=1 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5Opyn3dbOsjzKYBZo92r5zdHPhFU4tf_0 2024-04-17T05:46:03.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9336933009729250295', time=2024-04-17T07:17:10.881309+0000, chain.objs.size()=1 2024-04-17T05:46:03.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.qhSxuCbApwjD2hz7E08PItbT79Q9XAC_0 2024-04-17T05:46:03.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2751519639854596971', time=2024-04-17T07:17:11.102127+0000, chain.objs.size()=1 2024-04-17T05:46:03.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.124+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.w_DqMnGuPL73eRGIeie2PVOCJQl1bPG_0 2024-04-17T05:46:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.85081579932582930', time=2024-04-17T07:17:11.731241+0000, chain.objs.size()=1 2024-04-17T05:46:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._D6EZzNf6Rte0K1QfqCbAKeQYxXl3_0_0 2024-04-17T05:46:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3512819936537135427', time=2024-04-17T07:17:12.366501+0000, chain.objs.size()=1 2024-04-17T05:46:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.67SaRm62t73tbQJwyUwgdTtD6jOyDF6_0 2024-04-17T05:46:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.128+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6435664813177080567', time=2024-04-17T07:17:12.927643+0000, chain.objs.size()=1 2024-04-17T05:46:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.128+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c-f_UbblGiz2R61FpdmoBNzzvaXsiE-_0 2024-04-17T05:46:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.500201554402110019', time=2024-04-17T07:17:15.692597+0000, chain.objs.size()=1 2024-04-17T05:46:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SRbpDVVS8R1pHZBv-ACVUJzL_mMtFQe_0 2024-04-17T05:46:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11681761126910246526', time=2024-04-17T07:17:16.656405+0000, chain.objs.size()=1 2024-04-17T05:46:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6uwX_yjPpFQVR2IrGd1FoGOlM1Qt_lc_0 2024-04-17T05:46:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15068015461341517668', time=2024-04-17T07:17:16.790414+0000, chain.objs.size()=1 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XwD7MwewBd9XOW0QMMdFciQAja64Vfe_0 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5735985847986906593', time=2024-04-17T07:17:17.046031+0000, chain.objs.size()=1 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6IAjxcvdGaWKKl4aLTrSo4qfoX44k0p_0 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17310827587411711789', time=2024-04-17T07:17:18.990496+0000, chain.objs.size()=1 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.p8EJ_UjIHOfJgUlxPqFndsdsulCa1ef_0 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2358919901336915328', time=2024-04-17T07:17:20.642949+0000, chain.objs.size()=1 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RKs_FNw9EMgKNMgRMQSbuKgwOOuAeO1_0 2024-04-17T05:46:03.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11729493174949407999', time=2024-04-17T07:17:20.704367+0000, chain.objs.size()=1 2024-04-17T05:46:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.50_NmQemZZhV_eBNuAbVqS4kQRr9va6_0 2024-04-17T05:46:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4097481186953095953', time=2024-04-17T07:17:22.545540+0000, chain.objs.size()=1 2024-04-17T05:46:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.132+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.h6tbn5Gd7YbkmWka44xxVDknFayGpe3_0 2024-04-17T05:46:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17029264462903223320', time=2024-04-17T07:17:24.098809+0000, chain.objs.size()=1 2024-04-17T05:46:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VIXJVvYwIp-_w7SnQk-0aQN3XKxzAea_0 2024-04-17T05:46:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7388081081603375688', time=2024-04-17T07:17:25.129329+0000, chain.objs.size()=1 2024-04-17T05:46:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dk5HoHHM9vwDXLkFv4Il6P8fSh4e5Vj_0 2024-04-17T05:46:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9679921663081283259', time=2024-04-17T07:17:26.095667+0000, chain.objs.size()=1 2024-04-17T05:46:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zyh9xkmYZ52dKGTN7g-J7cFhkrrcCeG_0 2024-04-17T05:46:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13130587421723620420', time=2024-04-17T07:17:26.097154+0000, chain.objs.size()=1 2024-04-17T05:46:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7ztsaShIUbxyufy7eCM6wTXKn5fJPOp_0 2024-04-17T05:46:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7774806171568972124', time=2024-04-17T07:17:26.597772+0000, chain.objs.size()=1 2024-04-17T05:46:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.oH0t5rQaMnLi63FfxplwzHXtDyenE5N_0 2024-04-17T05:46:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6134811688538310064', time=2024-04-17T07:17:27.309452+0000, chain.objs.size()=1 2024-04-17T05:46:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LmzkCoC4YeFXWwjbeFIS4cpu9a9ToCV_0 2024-04-17T05:46:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16561529808201758789', time=2024-04-17T07:17:27.966687+0000, chain.objs.size()=1 2024-04-17T05:46:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EB-8HAusG4hR4WCksFoQiIQV3HsSaoh_0 2024-04-17T05:46:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3611820001634064454', time=2024-04-17T07:17:32.263215+0000, chain.objs.size()=1 2024-04-17T05:46:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RcdaAxXaeLhfGENcxEmC7lRgPYDH8Ws_0 2024-04-17T05:46:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2845454455762745074', time=2024-04-17T07:17:47.388744+0000, chain.objs.size()=1 2024-04-17T05:46:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.262__shadow_.X2zbDLluAQTG3Op0zOuXH4nQh2xJFuT_0 2024-04-17T05:46:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16512612189587442382', time=2024-04-17T07:17:53.603803+0000, chain.objs.size()=1 2024-04-17T05:46:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.266__shadow_.ch2vb1CgiGQrkl4jjYBDHLjl3zPbzQ6_0 2024-04-17T05:46:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10548167922567368920', time=2024-04-17T07:18:11.620304+0000, chain.objs.size()=1 2024-04-17T05:46:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.136+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.277__shadow_.-3N9I8ogBpKStYAFCvbTVNFh1H-9HDU_0 2024-04-17T05:46:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9527810211635140155', time=2024-04-17T07:18:13.186371+0000, chain.objs.size()=1 2024-04-17T05:46:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.278__shadow_.7Y-CTUci4rmPYdQbuJmsbFR-eD9OOmJ_0 2024-04-17T05:46:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1225374934149213008', time=2024-04-17T07:18:27.979435+0000, chain.objs.size()=1 2024-04-17T05:46:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.ykZ0dG_gSwNu6Eq5tGrfToq8CN1YKNg_0 2024-04-17T05:46:03.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16968700236361414859', time=2024-04-17T07:18:27.998340+0000, chain.objs.size()=1 2024-04-17T05:46:03.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.NeO6VQtfd2-Zi49cShR6sfHiEno3bcy_0 2024-04-17T05:46:03.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16958865021443640964', time=2024-04-17T07:18:36.626629+0000, chain.objs.size()=1 2024-04-17T05:46:03.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.288__shadow_.wPvWzRGGj2AqyKGKtGKk7eFX0LbpH_t_0 2024-04-17T05:46:03.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4726402404528105428', time=2024-04-17T07:18:40.096737+0000, chain.objs.size()=1 2024-04-17T05:46:03.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.140+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.290__shadow_.TodGTYAxQZfrYNh6ApXOeBL3tnZ4V6Z_0 2024-04-17T05:46:03.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12508709531037304595', time=2024-04-17T07:19:06.255487+0000, chain.objs.size()=1 2024-04-17T05:46:03.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.305__shadow_.AhEmqAtswRdNIM83S8-9LG78wcaOHrD_0 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3386042816374136100', time=2024-04-17T07:19:11.275670+0000, chain.objs.size()=1 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.309__shadow:n3Uw7s29r3HdSU7MHHubXjpiZYd712R_.oOeeSXJ4mqj05S4OgEU7TiaYGhuMCWZ_0 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7801237501918604109', time=2024-04-17T07:19:15.450966+0000, chain.objs.size()=1 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.312__shadow_.LWTITJB_uPGTvgc-ZRM-AsN9lbgjMFl_0 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12030081416144769605', time=2024-04-17T07:21:00.501106+0000, chain.objs.size()=1 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lUSK_po1MVChYwtuQDnVkvX6qWnXesi_0 2024-04-17T05:46:03.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149940833150648917', time=2024-04-17T07:21:01.152291+0000, chain.objs.size()=1 2024-04-17T05:46:03.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.v7JFdX7dtrWKmsm-CA2xcs5aZD0FaW4_0 2024-04-17T05:46:03.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16298773953639173425', time=2024-04-17T07:21:02.088695+0000, chain.objs.size()=1 2024-04-17T05:46:03.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.a3LLYo8wXw6CbjRVyFzlg9uaXKE5loJ_0 2024-04-17T05:46:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15811393593274893299', time=2024-04-17T07:21:02.198318+0000, chain.objs.size()=1 2024-04-17T05:46:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.qb1_PFZS65SJF5hS-Mmelja_ZfXoTc4_0 2024-04-17T05:46:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9403606089075047690', time=2024-04-17T07:21:02.401088+0000, chain.objs.size()=1 2024-04-17T05:46:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iFhr9F52rYFgy_RJ9NaGj8fu6wWMUIo_0 2024-04-17T05:46:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13067400485090918600', time=2024-04-17T07:21:02.715913+0000, chain.objs.size()=1 2024-04-17T05:46:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZyY2kC4keOJHbGSyYl4di7AfrQlD2jS_0 2024-04-17T05:46:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14827010341778440172', time=2024-04-17T07:21:02.775245+0000, chain.objs.size()=1 2024-04-17T05:46:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.144+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.24LDItaI7X2xxR1Jd5vNvo1OmY4tf94_0 2024-04-17T05:46:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10163402166917681007', time=2024-04-17T07:21:04.524322+0000, chain.objs.size()=1 2024-04-17T05:46:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ait-0GV9jDS6_0c9ipCJ4Adx9M7FIoL_0 2024-04-17T05:46:03.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18184607081532813393', time=2024-04-17T07:21:04.595910+0000, chain.objs.size()=1 2024-04-17T05:46:03.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6Wyn_xWpRFi4yd27XHVruhKiZFX2H3H_0 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9308261196451886674', time=2024-04-17T07:21:05.947925+0000, chain.objs.size()=1 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7lyMXBxo3ulQ9fx5NnavafU68M7u0xv_0 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9041100008852241826', time=2024-04-17T07:21:06.765558+0000, chain.objs.size()=1 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.63yU6A8-y_91cs76GuatHfmkKboFT_M_0 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17578194065317144365', time=2024-04-17T07:21:08.786531+0000, chain.objs.size()=1 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0wjJneX-DVTFzNmEuh5l3BRzX7UAgZM_0 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.508889218092748892', time=2024-04-17T07:21:08.891065+0000, chain.objs.size()=1 2024-04-17T05:46:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.37FgEti8bDThK4kGUwCQ4j8ZbEw21R9_0 2024-04-17T05:46:03.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7467409097649415363', time=2024-04-17T07:21:09.768598+0000, chain.objs.size()=1 2024-04-17T05:46:03.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.148+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2A_px5YZn5-UgMczhIeI0CKxd3b4s14_0 2024-04-17T05:46:03.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6661646625165173479', time=2024-04-17T07:21:10.862037+0000, chain.objs.size()=1 2024-04-17T05:46:03.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2REf702AE07c3MInzr2cbcMWp7ReNPK_0 2024-04-17T05:46:03.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6654483671763762271', time=2024-04-17T07:21:12.235005+0000, chain.objs.size()=1 2024-04-17T05:46:03.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2OlH-JMyNVYYT46jD0tF_7c4zRC3fQQ_0 2024-04-17T05:46:03.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2076671088523752159', time=2024-04-17T07:21:14.584992+0000, chain.objs.size()=1 2024-04-17T05:46:03.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0aByrheZCmE6i77-upU4j9aFVTTUOXW_0 2024-04-17T05:46:03.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16135785399512910363', time=2024-04-17T07:21:14.830840+0000, chain.objs.size()=1 2024-04-17T05:46:03.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.O0k3xtleC51kviZv0D3PaqyFBXwr76e_0 2024-04-17T05:46:03.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15127920014399122265', time=2024-04-17T07:21:15.767179+0000, chain.objs.size()=1 2024-04-17T05:46:03.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vRuMJliQmXu9nfvkZtFpLbQgdSo4xeB_0 2024-04-17T05:46:03.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5334806138631319927', time=2024-04-17T07:21:16.552841+0000, chain.objs.size()=1 2024-04-17T05:46:03.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5jnlK_IghMcfKWiXiWFQlWcssZPGBMl_0 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9655153745780154109', time=2024-04-17T07:21:16.685933+0000, chain.objs.size()=1 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.hvUfO9mmCFmHBXDRAWYBRxhoDhY7-1t_0 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12384574557991798730', time=2024-04-17T07:21:16.926622+0000, chain.objs.size()=1 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.i5PNrxNyzUBmaKsxYXL9L0pNFXM5cCV_0 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.604419878095113026', time=2024-04-17T07:21:17.026302+0000, chain.objs.size()=1 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iAEHjnxWviynzGpRoYN5jzZM029hdoc_0 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8645192267079058670', time=2024-04-17T07:21:17.512514+0000, chain.objs.size()=1 2024-04-17T05:46:03.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mqQOoSIWUEhgANtDx83LKXpTjGGr5Ln_0 2024-04-17T05:46:03.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1179640858254144046', time=2024-04-17T07:21:21.894363+0000, chain.objs.size()=1 2024-04-17T05:46:03.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.T9ydifPkwR70JcqBeMWglr4rzkEG74v_0 2024-04-17T05:46:03.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6704456625788261427', time=2024-04-17T07:21:21.896745+0000, chain.objs.size()=1 2024-04-17T05:46:03.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.152+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1WsRhZ1DDDgPBtkqbY15jdUMKmLrz7A_0 2024-04-17T05:46:03.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3045934921062446687', time=2024-04-17T07:21:21.898332+0000, chain.objs.size()=1 2024-04-17T05:46:03.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nHQtrKhOoBnY3SIZuJWi_ABoS1JYFwQ_0 2024-04-17T05:46:03.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11358565817729689260', time=2024-04-17T07:21:22.033766+0000, chain.objs.size()=1 2024-04-17T05:46:03.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PncFc3cNJUtzNd3A2xrakD-JzNjWYHF_0 2024-04-17T05:46:03.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8099211616582382997', time=2024-04-17T07:21:23.725042+0000, chain.objs.size()=1 2024-04-17T05:46:03.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6Zc7vGha0KHdBbMglK-dWg7_B8szFAc_0 2024-04-17T05:46:03.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8130424092548332659', time=2024-04-17T07:21:23.995362+0000, chain.objs.size()=1 2024-04-17T05:46:03.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.156+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Hwa8qQTluYibK-_ge5q0xm-vZBO4iIa_0 2024-04-17T05:46:03.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4485155762031529962', time=2024-04-17T07:21:26.106139+0000, chain.objs.size()=1 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UFvG963YsEmKXILCwjRqz_vEXeqG3Pd_0 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1587693982115048936', time=2024-04-17T07:21:26.273236+0000, chain.objs.size()=1 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.p6ZUnLP_lmYuRjisnlt7Lb8atlRbqYg_0 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11509189160884342437', time=2024-04-17T07:21:26.958297+0000, chain.objs.size()=1 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UP7Dq8uPwhUShCR2iGuGWSSHbhuiq1g_0 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17864464246676338817', time=2024-04-17T07:21:27.032913+0000, chain.objs.size()=1 2024-04-17T05:46:03.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8Z4QDX30adEqADf1psi-wiFV8lXfY7m_0 2024-04-17T05:46:03.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7056729114726725112', time=2024-04-17T07:21:27.035062+0000, chain.objs.size()=1 2024-04-17T05:46:03.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PbXu2HISz4OMY6MzzfSLeQrQGYHn7uk_0 2024-04-17T05:46:03.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10774321253269159295', time=2024-04-17T07:21:27.037004+0000, chain.objs.size()=1 2024-04-17T05:46:03.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4Y9V4uwrkqckOqSh8hfvbhGbFUHqIoO_0 2024-04-17T05:46:03.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9512021850581222544', time=2024-04-17T07:23:12.742343+0000, chain.objs.size()=1 2024-04-17T05:46:03.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Mst1uXx2OpIwwmul6iAgP5XSfz33AaD_0 2024-04-17T05:46:03.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7702795634466956531', time=2024-04-17T07:23:13.846204+0000, chain.objs.size()=1 2024-04-17T05:46:03.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EIEaVQrS_8Flovu5SYwdOEynJ-ph_zq_0 2024-04-17T05:46:03.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7677720057660895177', time=2024-04-17T07:23:14.820222+0000, chain.objs.size()=1 2024-04-17T05:46:03.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.160+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.b1YblCYLAuSiPDXdttfvkgQuNj_FU4C_0 2024-04-17T05:46:03.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2004193807230746575', time=2024-04-17T07:23:15.181849+0000, chain.objs.size()=1 2024-04-17T05:46:03.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JU2HWHbAE1HkkRp7IMHwcjw7_-i0zj2_0 2024-04-17T05:46:03.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10897190674232621888', time=2024-04-17T07:23:15.791875+0000, chain.objs.size()=1 2024-04-17T05:46:03.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XDZpOjvbWbhM1obQueMPmzFcD8EhJRs_0 2024-04-17T05:46:03.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14480739491683849396', time=2024-04-17T07:23:17.076311+0000, chain.objs.size()=1 2024-04-17T05:46:03.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qig2-fw6V1cuHHy0kRsJHOy1pPj2XL5_0 2024-04-17T05:46:03.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14810260022929053964', time=2024-04-17T07:23:17.427040+0000, chain.objs.size()=1 2024-04-17T05:46:03.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wG5HK9IQ5BL9N2Kv5ezezwDW-2Go7fy_0 2024-04-17T05:46:03.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7570845187859861343', time=2024-04-17T07:23:17.502416+0000, chain.objs.size()=1 2024-04-17T05:46:03.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PL_YlFd7sFI57gX0s7jsNUDH0IriNt5_0 2024-04-17T05:46:03.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15940393496666907442', time=2024-04-17T07:23:18.928300+0000, chain.objs.size()=1 2024-04-17T05:46:03.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.q4ZBXo2hlqn80fsJKBTi7rnLyQ8QZE3_0 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12831816829206779756', time=2024-04-17T07:23:20.889732+0000, chain.objs.size()=1 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.P61t-W1-_7DFBumJZcZaZGmf22L08im_0 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17163151856134021312', time=2024-04-17T07:23:21.458813+0000, chain.objs.size()=1 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Zn8qRry5Urp4QwpOjz0fCoDyqC_G_BO_0 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6773525340674552414', time=2024-04-17T07:23:22.092297+0000, chain.objs.size()=1 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jgvPARX9CMnAzpwPYgXAZisq3QZts_1_0 2024-04-17T05:46:03.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16515550273004878315', time=2024-04-17T07:23:22.330911+0000, chain.objs.size()=1 2024-04-17T05:46:03.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.164+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.whDQ0_E-50sXgvjoB1iivAl30gpkmBn_0 2024-04-17T05:46:03.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.168+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16563115133838772832', time=2024-04-17T07:23:23.836196+0000, chain.objs.size()=1 2024-04-17T05:46:03.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.168+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gMiAmbhOXlktLmJneaDqT61hx9Ydli-_0 2024-04-17T05:46:03.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.168+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15685371059673619077', time=2024-04-17T07:23:25.747079+0000, chain.objs.size()=1 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.168+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dd2HsAuvaBbHP0hjCmpc0kxwzvyWIHv_0 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.168+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1841358496589309140', time=2024-04-17T07:23:26.185965+0000, chain.objs.size()=1 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.168+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GvE-7kcfvNB8NpE8LXu06jqdUCk7tCs_0 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18321804773697097155', time=2024-04-17T07:23:27.527966+0000, chain.objs.size()=1 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.O1_yLF_h_a1ki98AGSSVkMtF_yeRQhI_0 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3989351955694862624', time=2024-04-17T07:23:28.554433+0000, chain.objs.size()=1 2024-04-17T05:46:03.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J-0bDoj6B0d2-CXKUGLhVRh3ctf63YL_0 2024-04-17T05:46:03.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7853842741738516551', time=2024-04-17T07:23:29.020842+0000, chain.objs.size()=1 2024-04-17T05:46:03.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KIuQofmQMgiuVspyq8dW0nehKFG5ZrW_0 2024-04-17T05:46:03.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7162963378700094075', time=2024-04-17T07:23:30.046257+0000, chain.objs.size()=1 2024-04-17T05:46:03.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.JwOWFPPFflwppOuHJdHhbmTgRSFDJvR_0 2024-04-17T05:46:03.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10222922134856643293', time=2024-04-17T07:23:30.645620+0000, chain.objs.size()=1 2024-04-17T05:46:03.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Lt2JTA7jknpIhDQzw-gQ8GSw1c1ik_9_0 2024-04-17T05:46:03.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2646379684129631801', time=2024-04-17T07:23:33.007131+0000, chain.objs.size()=1 2024-04-17T05:46:03.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.172+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mibNFROa2SXCm_mMr7_ZhcvFjf9vjzc_0 2024-04-17T05:46:03.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.176+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/47857 2024-04-17T05:46:03.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.176+0000 7f039e29b640 0 -- 172.21.15.88:0/3940810918 >> [v2:172.21.15.154:6800/1245445760,v1:172.21.15.154:6801/1245445760] conn(0x7f038801d040 msgr2=0x7f026807f6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:46:03.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=24, truncated=0, next_marker='' 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17165184819650478199', time=2024-04-17T07:23:33.716875+0000, chain.objs.size()=1 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.o3hAVVarrx2xBiudkTciVfNIf-5xgCy_0 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15379857281373931015', time=2024-04-17T07:23:34.022789+0000, chain.objs.size()=1 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.l6YieGYz3zGLDu65BGQ8fvQQ9BQkBHH_0 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9675337999459123009', time=2024-04-17T07:23:34.138599+0000, chain.objs.size()=1 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eRFPVjXiVzW_OGeS8fqhGoNG6BwKdBO_0 2024-04-17T05:46:03.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13362878197050447777', time=2024-04-17T07:23:35.899676+0000, chain.objs.size()=1 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.H4VCTN-1jb5rx5iHKLL8euIpCXQMwvO_0 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2856237738443057562', time=2024-04-17T07:23:36.545704+0000, chain.objs.size()=1 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.GcIdgkJFxG0oSDMpCay47E_JVoT5j-b_0 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9971942303041365928', time=2024-04-17T07:23:36.561580+0000, chain.objs.size()=1 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.H_URJarUyY9VqEErxscu13Z9IVjaKUX_0 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15214169847463207137', time=2024-04-17T07:23:38.517105+0000, chain.objs.size()=1 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lcPuF2NxJ1a8UZaVAYJXKYrU2SEHRKr_0 2024-04-17T05:46:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3995058427043955236', time=2024-04-17T07:23:40.668399+0000, chain.objs.size()=1 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-JS9m7fBz1V0MkpvvgoeWY48He2mtma_0 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18403388743264148684', time=2024-04-17T07:23:40.837809+0000, chain.objs.size()=1 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nAQSUQHJ-bXlB_PFD8NJLr63Eadnq68_0 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6608105120245195326', time=2024-04-17T07:23:41.434628+0000, chain.objs.size()=1 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kG4yF91i22CVytakaK-B-g2O_N3iOJu_0 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9976220038043975898', time=2024-04-17T07:23:41.664139+0000, chain.objs.size()=1 2024-04-17T05:46:03.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IpqyriwR5fFFIzNqajISDtYAv4Dy58R_0 2024-04-17T05:46:03.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14165566580618547527', time=2024-04-17T07:23:48.997631+0000, chain.objs.size()=1 2024-04-17T05:46:03.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.322__shadow_.R3vveMg-68VKTsQe25S8kioLghKsNuO_0 2024-04-17T05:46:03.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15564602342980919609', time=2024-04-17T07:24:30.485437+0000, chain.objs.size()=1 2024-04-17T05:46:03.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.180+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.369__shadow_.un6YPGGii0QhcbQvaURsNH-Zwfm3ZnU_0 2024-04-17T05:46:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17807874972605770497', time=2024-04-17T07:25:47.411057+0000, chain.objs.size()=1 2024-04-17T05:46:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.447__shadow_.9l0XRQYJtIIJeNQI_G44VEXLPOcWulc_0 2024-04-17T05:46:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1319066988016135373', time=2024-04-17T07:25:54.076465+0000, chain.objs.size()=1 2024-04-17T05:46:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.451__shadow_.OjefBSbTrjZeGrHg3PjNZLYTRkN2ZeS_0 2024-04-17T05:46:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6796838962078824122', time=2024-04-17T07:27:02.449629+0000, chain.objs.size()=1 2024-04-17T05:46:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.503__multipart_mymultipart.2~6-PXrA-oz74eeK8HXWzYJ-_Vg91VOyn.1 2024-04-17T05:46:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4735365564609358312', time=2024-04-17T07:27:47.326122+0000, chain.objs.size()=3 2024-04-17T05:46:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~-9LUMyaFBaikZ5Y72xVwZb7MggwKjiD.1 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~-9LUMyaFBaikZ5Y72xVwZb7MggwKjiD.2 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.511__multipart_mymultipart.2~-9LUMyaFBaikZ5Y72xVwZb7MggwKjiD.3 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10137129491754049821', time=2024-04-17T07:29:45.461284+0000, chain.objs.size()=4 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.517__multipart_mymultipart.2~X0E6xHiwkZmwTBip0UWa_9Fx3c2V7S1.1 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.517__multipart_mymultipart.2~X0E6xHiwkZmwTBip0UWa_9Fx3c2V7S1.2 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.517__multipart_mymultipart.2~X0E6xHiwkZmwTBip0UWa_9Fx3c2V7S1.3 2024-04-17T05:46:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.517__multipart_mymultipart.2~X0E6xHiwkZmwTBip0UWa_9Fx3c2V7S1.4 2024-04-17T05:46:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12955183737977717488', time=2024-04-17T07:31:45.752536+0000, chain.objs.size()=1 2024-04-17T05:46:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.184+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:wCRVt8.n7ymAcIZ6754l0XOm7dnvIq9_.2sUEZSw7IYrp0SKjT1UHjdX9OKpkn0Z_0 2024-04-17T05:46:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11157075958496995927', time=2024-04-17T07:31:53.174770+0000, chain.objs.size()=1 2024-04-17T05:46:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:EdzAGosO1h8xFL9ptrt54X2.f1zU142_.n1Q8xtWa9vaqeybJ8OULaw8V7d_XfqU_0 2024-04-17T05:46:03.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.188+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10041791022304726922', time=2024-04-17T07:32:06.956429+0000, chain.objs.size()=1 2024-04-17T05:46:03.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.188+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:JN06iCR6Y86JFO3vSPW5vOFJUh3FVV1_.PI8TKO09-rQSVFU8mPlWxArwOE9acQW_0 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13769424301730856816', time=2024-04-17T07:33:23.506203+0000, chain.objs.size()=4 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~gHzhTyj3I6k-6qbxaP_i6ObBwizw42e.1 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~gHzhTyj3I6k-6qbxaP_i6ObBwizw42e.2 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~gHzhTyj3I6k-6qbxaP_i6ObBwizw42e.3 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~gHzhTyj3I6k-6qbxaP_i6ObBwizw42e.4 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8882931800815599115', time=2024-04-17T07:37:37.928860+0000, chain.objs.size()=1 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.649__shadow_.gjvUXLvrXwGLZtf9Fh4H5MisCFG_JnS_0 2024-04-17T05:46:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13094553609670341574', time=2024-04-17T07:37:37.930585+0000, chain.objs.size()=1 2024-04-17T05:46:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.649__shadow_.VVbc2J2Atz_Gj3CLZtJGRmDlaJVkr94_0 2024-04-17T05:46:03.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.192+0000 7f039da9a640 0 -- 172.21.15.88:0/3940810918 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f03880620b0 msgr2=0x7f038805c5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:46:03.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.196+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-17T05:46:03.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039d299640 0 -- 172.21.15.88:0/3940810918 >> [v2:172.21.15.88:6821/217804779,v1:172.21.15.88:6824/217804779] conn(0x559271df32a0 msgr2=0x7f02680841d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:46:03.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48000' 2024-04-17T05:46:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6053957862682827594', time=2024-04-17T07:10:03.188945+0000, chain.objs.size()=3 2024-04-17T05:46:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-LUKEWARM.2~GhrRxP0A8822bmX5lBSYhKhkVz9OgYk.1 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-LUKEWARM.2~GhrRxP0A8822bmX5lBSYhKhkVz9OgYk.2 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-LUKEWARM.2~GhrRxP0A8822bmX5lBSYhKhkVz9OgYk.3 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_0cLUwwS0yX5hZjXsntQYF69BSqQ5EgC', time=2024-04-17T07:10:29.973564+0000, chain.objs.size()=1 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.153__shadow_.JXWrZC5TOyMNCCHPb2thoiLszjR6FRE_0 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15689855682722387420', time=2024-04-17T07:10:41.487042+0000, chain.objs.size()=1 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.161__shadow_.-OG0OFOOdzzXnevfOuuotac-T4Eg3HN_0 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11672866916626038113', time=2024-04-17T07:14:28.132908+0000, chain.objs.size()=1 2024-04-17T05:46:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.221__shadow_.JxUrQXaXh_K0NcDgJwmWVf3JiXqduu3_0 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7873741755817299373', time=2024-04-17T07:14:33.055365+0000, chain.objs.size()=1 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.224__shadow_._3xdw5-Ae7Nafx0tj-KKKCfm2TTuLFp_0 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13117641341944446765', time=2024-04-17T07:14:38.651136+0000, chain.objs.size()=1 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.227__shadow_.AIcO_tZ7E2E6tltmrlCjVbFAWKeR3A3_0 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8597711590410439012', time=2024-04-17T07:14:40.971731+0000, chain.objs.size()=1 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.228__shadow_.s4eOpvsv6tFxz6HNUmDlMSB23n0pgpY_0 2024-04-17T05:46:03.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9062468909681170870', time=2024-04-17T07:14:48.879306+0000, chain.objs.size()=1 2024-04-17T05:46:03.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.233__shadow_.3xDKMO8EHMoOhaotMm59IG9oBlbb-PD_0 2024-04-17T05:46:03.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2290171668680761408', time=2024-04-17T07:14:51.256771+0000, chain.objs.size()=1 2024-04-17T05:46:03.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.234__shadow_.i4tRHxRS3B569BfBaTL9MTmgHd--uJo_0 2024-04-17T05:46:03.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4759342911831188695', time=2024-04-17T07:15:00.972535+0000, chain.objs.size()=1 2024-04-17T05:46:03.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.200+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.240__shadow_.-cwR_TnRg1myaeO0QdNwiy_NcHFWfgt_0 2024-04-17T05:46:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6469945627052976036', time=2024-04-17T07:15:18.457760+0000, chain.objs.size()=1 2024-04-17T05:46:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.251__shadow_.3n_YvqV3bP7ugXjt5fQc3NOzA3AD7k9_0 2024-04-17T05:46:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15618114238164651798', time=2024-04-17T07:17:01.274995+0000, chain.objs.size()=1 2024-04-17T05:46:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0Q9ZqR0EW286BUBk_uQA4zcRnc9FGGz_0 2024-04-17T05:46:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11677137650777171303', time=2024-04-17T07:17:01.276128+0000, chain.objs.size()=1 2024-04-17T05:46:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5ZiqSCft58YFJeNjnkPMFFxwwz_oxXM_0 2024-04-17T05:46:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12123604227648474687', time=2024-04-17T07:17:01.550925+0000, chain.objs.size()=1 2024-04-17T05:46:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2SJMar_mBQCm2_a96OChZdYRxHb-2ob_0 2024-04-17T05:46:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14770320171694071747', time=2024-04-17T07:17:02.889896+0000, chain.objs.size()=1 2024-04-17T05:46:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YcJRXyVRQAF-AII74d3fpymOmvnnz1T_0 2024-04-17T05:46:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14239680864999468146', time=2024-04-17T07:17:06.499790+0000, chain.objs.size()=1 2024-04-17T05:46:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.liCz7-7XfWNOecYXT3uIpMDTNW0OnAd_0 2024-04-17T05:46:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6025792745694864770', time=2024-04-17T07:17:07.903776+0000, chain.objs.size()=1 2024-04-17T05:46:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.oOxhQ2TUB0Lpgp7QmNJV_wc3jeQrqtU_0 2024-04-17T05:46:03.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6735108707215983027', time=2024-04-17T07:17:09.876402+0000, chain.objs.size()=1 2024-04-17T05:46:03.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CG5LlclLTNtQO2E7yvU4AnkxHU15rt5_0 2024-04-17T05:46:03.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18341864105127197786', time=2024-04-17T07:17:10.630619+0000, chain.objs.size()=1 2024-04-17T05:46:03.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8bnZHN88X5sMOwYi66p8c9H7SmtytMw_0 2024-04-17T05:46:03.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6930955472132969490', time=2024-04-17T07:17:11.340034+0000, chain.objs.size()=1 2024-04-17T05:46:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xv76tzP_FUzQb8bzlCNt4ZsGouzCF_4_0 2024-04-17T05:46:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14123472874476607241', time=2024-04-17T07:17:11.626210+0000, chain.objs.size()=1 2024-04-17T05:46:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OLbalLnWYWFUa6y2ta2SmUvApIUTqlf_0 2024-04-17T05:46:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5174939085936200334', time=2024-04-17T07:17:12.274344+0000, chain.objs.size()=1 2024-04-17T05:46:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.208+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.e9Q01g2-lktECnvZf2NDOYGYuyasgBe_0 2024-04-17T05:46:03.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17857552252325382448', time=2024-04-17T07:17:12.697752+0000, chain.objs.size()=1 2024-04-17T05:46:03.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PU9fPe8GtTQP9kY8FNwgla9ixTyCpGx_0 2024-04-17T05:46:03.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8146846539618539833', time=2024-04-17T07:17:14.064534+0000, chain.objs.size()=1 2024-04-17T05:46:03.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.B0CG-dne53KoZWWj4JXBSp-7ch8pGJ__0 2024-04-17T05:46:03.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14107157297766443612', time=2024-04-17T07:17:15.364704+0000, chain.objs.size()=1 2024-04-17T05:46:03.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HKDMPwWIXUKkJY5BsbWOq7nqieyfyxu_0 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18401746800925208555', time=2024-04-17T07:17:15.635847+0000, chain.objs.size()=1 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.drUij0RTJj_5YjFJqUaWwRhLcQrLX9X_0 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4618512351132841780', time=2024-04-17T07:17:17.907238+0000, chain.objs.size()=1 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CuHUZkfKAygZMs3OB0sOJF-PNjN9QY2_0 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.529279182987611115', time=2024-04-17T07:17:17.976578+0000, chain.objs.size()=1 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.N8hvicoy2Flr-_gTZGyPTvRPkjqc4LG_0 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3924005992974095105', time=2024-04-17T07:17:17.978029+0000, chain.objs.size()=1 2024-04-17T05:46:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Q8IhfClcf0ESP2iNk3kGA8QE7nQDTuW_0 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16687118067818909186', time=2024-04-17T07:17:20.244796+0000, chain.objs.size()=1 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RqYbUiJAE8DL0MlOfyg_iIGALPwOZjF_0 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2449410884585892862', time=2024-04-17T07:17:20.317186+0000, chain.objs.size()=1 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1Lejenjfy0hDUTO2-I-73nOhvv2xDTH_0 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5746025812040387673', time=2024-04-17T07:17:20.842493+0000, chain.objs.size()=1 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LWoZfeTWEdUXy0DL47zKFTv7XXEfmne_0 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4395115201970099619', time=2024-04-17T07:17:22.582595+0000, chain.objs.size()=1 2024-04-17T05:46:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.212+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.q7_BMMjXolA8EBbqOG6-HY1mhmBVP2H_0 2024-04-17T05:46:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15337000428733459966', time=2024-04-17T07:17:23.096686+0000, chain.objs.size()=1 2024-04-17T05:46:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pLL2xhFJZWaXUZg9yuDDBrJM-p49EEw_0 2024-04-17T05:46:03.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18072359240760553294', time=2024-04-17T07:17:23.942433+0000, chain.objs.size()=1 2024-04-17T05:46:03.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.eefK8no8FP0H9DfMcwdjXZVf8WYr2_N_0 2024-04-17T05:46:03.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12531244578299791188', time=2024-04-17T07:17:23.944986+0000, chain.objs.size()=1 2024-04-17T05:46:03.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xpeSyv5eXqKn2tJiEBhUZUD_CModbz__0 2024-04-17T05:46:03.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10038988884255967701', time=2024-04-17T07:17:24.994805+0000, chain.objs.size()=1 2024-04-17T05:46:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bKpRYHUgaARnBQCtHWmohXNZg5ojUAQ_0 2024-04-17T05:46:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4244048558886918658', time=2024-04-17T07:17:26.068786+0000, chain.objs.size()=1 2024-04-17T05:46:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.E8vhs5LBw3HskkZ3Jmrmda4diCEFs6d_0 2024-04-17T05:46:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14159370178516073883', time=2024-04-17T07:17:29.879397+0000, chain.objs.size()=1 2024-04-17T05:46:03.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.216+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.beJXyRitcY2ijZQwBnuBaDo81-_Tk31_0 2024-04-17T05:46:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14961486240328412802', time=2024-04-17T07:17:29.930777+0000, chain.objs.size()=1 2024-04-17T05:46:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DKNUmG9fBBcdsK9jQqVNqXNN22CKdUi_0 2024-04-17T05:46:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1339274183442426866', time=2024-04-17T07:17:30.310823+0000, chain.objs.size()=1 2024-04-17T05:46:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JvDdHGlGD0sKUOZbNcfPVpan3GkbdGU_0 2024-04-17T05:46:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15833862312838388534', time=2024-04-17T07:17:30.717683+0000, chain.objs.size()=1 2024-04-17T05:46:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xtv9m-uNLcyFNRtgMtEvh7J2JIF8Sed_0 2024-04-17T05:46:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6540663768317302663', time=2024-04-17T07:17:31.390570+0000, chain.objs.size()=1 2024-04-17T05:46:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GmxJvCRnlY--pkwLTUUy3Mu04hOMsG-_0 2024-04-17T05:46:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18237988150988948822', time=2024-04-17T07:17:31.851651+0000, chain.objs.size()=1 2024-04-17T05:46:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mf7Z_GTGcfh8oAuQLdWlnZ8cv6kdERs_0 2024-04-17T05:46:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6770364692457651208', time=2024-04-17T07:17:32.274327+0000, chain.objs.size()=1 2024-04-17T05:46:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.220+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.WCcChgOqHesPHGyCPI9dj0QQNY2pNj7_0 2024-04-17T05:46:03.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16268012794220942596', time=2024-04-17T07:17:32.276479+0000, chain.objs.size()=1 2024-04-17T05:46:03.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HldsedvevebUzdrIxQc-aoAWUhhtbZU_0 2024-04-17T05:46:03.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10006442554956050296', time=2024-04-17T07:17:33.913604+0000, chain.objs.size()=1 2024-04-17T05:46:03.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.253__shadow_.iREyhSQhvzPlKgGr7LUO_D18E48-ttS_0 2024-04-17T05:46:03.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12036937257653880930', time=2024-04-17T07:18:00.088691+0000, chain.objs.size()=1 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.270__shadow_.TkyM2CjoCo_Beglgh6s2FS1rn_St-ph_0 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17469357084482082667', time=2024-04-17T07:18:08.294521+0000, chain.objs.size()=1 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.275__shadow_.XmJiTmq3tv-uyvO_KwbRYp1Fd9_AIlG_0 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7609464225383643239', time=2024-04-17T07:18:23.025942+0000, chain.objs.size()=1 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.PsrqSn0Gpbkfq8E8MH7VLcskZs7liWw_0 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8403145407931166368', time=2024-04-17T07:18:28.283737+0000, chain.objs.size()=1 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.X2r7svrmhDDm_Qi_aCtT_5wHgtiRUCz_0 2024-04-17T05:46:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5398796921191137766', time=2024-04-17T07:18:43.326027+0000, chain.objs.size()=1 2024-04-17T05:46:03.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.292__shadow_.SPftC25daGquqS5qWCrInfRKRDj-R_T_0 2024-04-17T05:46:03.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9642293011963983580', time=2024-04-17T07:19:14.007586+0000, chain.objs.size()=1 2024-04-17T05:46:03.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.311__shadow_.y9AsksCh1ejULP5cmFwh9jtFhLU-8wX_0 2024-04-17T05:46:03.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13278572728076630371', time=2024-04-17T07:21:00.549050+0000, chain.objs.size()=1 2024-04-17T05:46:03.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.frtir9zOzzV-2G-bOBPNxMgc4u5aOcR_0 2024-04-17T05:46:03.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16021005361498849949', time=2024-04-17T07:21:00.551094+0000, chain.objs.size()=1 2024-04-17T05:46:03.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.G9-G1W46Od_gQ0ZlYs-BSm1zjMty5Od_0 2024-04-17T05:46:03.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10113576948039362339', time=2024-04-17T07:21:01.865623+0000, chain.objs.size()=1 2024-04-17T05:46:03.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.224+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.l-DS36GeGx3zLFLj-UgjPsi49iLaLYx_0 2024-04-17T05:46:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16475546473501423470', time=2024-04-17T07:21:02.095025+0000, chain.objs.size()=1 2024-04-17T05:46:03.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vXnL_aUzElAZiqaBl45q7434KHWfb9u_0 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.304222119054637669', time=2024-04-17T07:21:04.269265+0000, chain.objs.size()=1 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.K2Y-Zt1SmhyWnkmxu1QH9uLDpV0ENR9_0 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8974622609747738597', time=2024-04-17T07:21:04.694106+0000, chain.objs.size()=1 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.p352ndlI1GB1j-OQEECduv7nB7Yg3aM_0 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15601775483482098940', time=2024-04-17T07:21:06.039056+0000, chain.objs.size()=1 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.D3SFDdoddlL0nggyhGcXwShPNpdNsLH_0 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7294884363748474556', time=2024-04-17T07:21:06.473251+0000, chain.objs.size()=1 2024-04-17T05:46:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.64mXWwxJsm4coJ4SnzLyzOZ58cj9F6Y_0 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9694786691282879931', time=2024-04-17T07:21:06.475607+0000, chain.objs.size()=1 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t_uec0Obcxu2Y2U0oXzJ6U6FHGjBKQk_0 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14750768861561168220', time=2024-04-17T07:21:09.676986+0000, chain.objs.size()=1 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xc8VUuSIp8kcVj_JVlUQCFjqlrlVGlD_0 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1372452204295628739', time=2024-04-17T07:21:11.855310+0000, chain.objs.size()=1 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.fNSSUEAozZRjTE_ZUUa8cvzCPcXEE_t_0 2024-04-17T05:46:03.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3106568378723858962', time=2024-04-17T07:21:13.374884+0000, chain.objs.size()=1 2024-04-17T05:46:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YZVECVSYTlHOkP_RSQ98HprbL6tdVkZ_0 2024-04-17T05:46:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17826789048486720363', time=2024-04-17T07:21:13.825065+0000, chain.objs.size()=1 2024-04-17T05:46:03.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3r2ORMIRiW7uJAZyM7xJt4w3rgvPoYI_0 2024-04-17T05:46:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1195855096565414252', time=2024-04-17T07:21:14.333144+0000, chain.objs.size()=1 2024-04-17T05:46:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.232+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aaGbJ_u31E3FM1Wz6kzEDVGtAQkWdJq_0 2024-04-17T05:46:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1030749469919935299', time=2024-04-17T07:21:14.787672+0000, chain.objs.size()=1 2024-04-17T05:46:03.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HingsHMO-0FxpKPOF0SkuM6f5ufT7Ol_0 2024-04-17T05:46:03.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9131244600278133654', time=2024-04-17T07:21:14.834727+0000, chain.objs.size()=1 2024-04-17T05:46:03.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sDZYuRzfGAkHfySji5J7pDkcIMMhyJm_0 2024-04-17T05:46:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7492609547311817761', time=2024-04-17T07:21:16.182243+0000, chain.objs.size()=1 2024-04-17T05:46:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._oNEWIUyfv4CrAI0Sn6vBwdWsy6KWJV_0 2024-04-17T05:46:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12949837230654573542', time=2024-04-17T07:21:16.309813+0000, chain.objs.size()=1 2024-04-17T05:46:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Gq99eAmZdixa5rY98rhVqyKcT5Zzhco_0 2024-04-17T05:46:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1612774515352139008', time=2024-04-17T07:21:16.745592+0000, chain.objs.size()=1 2024-04-17T05:46:03.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.236+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.bPm4-e_y8jjs2uKtfPr7XOQzqplyBFH_0 2024-04-17T05:46:03.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.686125877227922633', time=2024-04-17T07:21:17.459646+0000, chain.objs.size()=1 2024-04-17T05:46:03.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.b0cM1XCbByxOPuEV79Ka40xlEX1ubAB_0 2024-04-17T05:46:03.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14833267655583745902', time=2024-04-17T07:21:21.125923+0000, chain.objs.size()=1 2024-04-17T05:46:03.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HsA-BwgwNLT87ccgxVx-pp1u-dEXMKe_0 2024-04-17T05:46:03.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9702687452824960149', time=2024-04-17T07:21:21.688815+0000, chain.objs.size()=1 2024-04-17T05:46:03.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aaJ3J7GaIybj-PMu4OsIHooN5YL-pp1_0 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5814807245053544973', time=2024-04-17T07:21:23.677189+0000, chain.objs.size()=1 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LtFcpchmay6P7Fy3Dk1rcVWhc7SBc-s_0 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12698543734923114191', time=2024-04-17T07:21:24.171429+0000, chain.objs.size()=1 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.cVxBVIdhCX7t6AqnEUl2LDfgYc4qwO2_0 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13087486467001640961', time=2024-04-17T07:21:25.765107+0000, chain.objs.size()=1 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.a3H0wA7QylnOvFIF0aaCXnXcZpsJ6Jd_0 2024-04-17T05:46:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8556904816865052066', time=2024-04-17T07:21:26.097008+0000, chain.objs.size()=1 2024-04-17T05:46:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gZYoW0aWn9RBPB86KRt9A-An3iT-FeE_0 2024-04-17T05:46:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10467924523715668506', time=2024-04-17T07:21:26.234812+0000, chain.objs.size()=1 2024-04-17T05:46:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rxqNqRujgcFHZZjYhDQ5-6TjHk50XIZ_0 2024-04-17T05:46:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5057165917625575432', time=2024-04-17T07:21:28.314987+0000, chain.objs.size()=1 2024-04-17T05:46:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t20C15prmjtzIHhAwbjeckdg0ckQk1Z_0 2024-04-17T05:46:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14410346018128025927', time=2024-04-17T07:23:12.637893+0000, chain.objs.size()=1 2024-04-17T05:46:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Rh06_nL2zNombbhnEnQ5GUwvgo4dUBu_0 2024-04-17T05:46:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4884157538705096330', time=2024-04-17T07:23:14.766199+0000, chain.objs.size()=1 2024-04-17T05:46:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.240+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.T0fjx_L6limwpWqHutw7jCh_YsjQtex_0 2024-04-17T05:46:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2228693927658703124', time=2024-04-17T07:23:14.838936+0000, chain.objs.size()=1 2024-04-17T05:46:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.68Y-HpHIEyNID0lMttF1xCyXyWqW7ZM_0 2024-04-17T05:46:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8387185158793490184', time=2024-04-17T07:23:17.102841+0000, chain.objs.size()=1 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.W-S99RBrDYBfF76ckuT8jkCZB95TsBf_0 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7434222119571824027', time=2024-04-17T07:23:17.679255+0000, chain.objs.size()=1 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qTGCZl8JlPo8JWOx0CLvLevUEaFUHIW_0 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5472535584120306896', time=2024-04-17T07:23:17.680935+0000, chain.objs.size()=1 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vf9w96Pz2Oxx8JPySlNfpv69LelJuiC_0 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12522100231457801435', time=2024-04-17T07:23:17.852407+0000, chain.objs.size()=1 2024-04-17T05:46:03.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NLNvctzWLgm0TfqEcKN_ooGq-ICfsOf_0 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15067004284983149958', time=2024-04-17T07:23:18.423305+0000, chain.objs.size()=1 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.beQXycgvq8tdLK75Y8rIlxrBqU1uMqQ_0 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14833720703016846659', time=2024-04-17T07:23:19.244597+0000, chain.objs.size()=1 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hLzBETE5MTvIlDY0TCxS93FhJgU6-v4_0 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13165269496059175745', time=2024-04-17T07:23:19.379181+0000, chain.objs.size()=1 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wMz6lNkTft4iYevhSr7lT4ga6Yw-GRL_0 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1826323575331874732', time=2024-04-17T07:23:19.585406+0000, chain.objs.size()=1 2024-04-17T05:46:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.244+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iaa8EAXPMU28mDWRwBrPExVkYOjUA4V_0 2024-04-17T05:46:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6278339106075707875', time=2024-04-17T07:23:19.889192+0000, chain.objs.size()=1 2024-04-17T05:46:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aVAHDiPJwHNLOwaQk3ZMlsOoYK6KRnY_0 2024-04-17T05:46:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16306899773228761655', time=2024-04-17T07:23:19.937881+0000, chain.objs.size()=1 2024-04-17T05:46:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TKyp84G_5DBmjd2i3aSNyHPQ7VhFdbF_0 2024-04-17T05:46:03.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3928625229941276106', time=2024-04-17T07:23:19.951764+0000, chain.objs.size()=1 2024-04-17T05:46:03.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nqcfEVbWI121O8G4KKm3a-iC7GYLfWD_0 2024-04-17T05:46:03.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7919857741045463014', time=2024-04-17T07:23:21.009960+0000, chain.objs.size()=1 2024-04-17T05:46:03.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NiHKZCbGhJmDSZOlDedd9cqWdbdgPjv_0 2024-04-17T05:46:03.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5203488644160275118', time=2024-04-17T07:23:21.336014+0000, chain.objs.size()=1 2024-04-17T05:46:03.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.248+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xea4LlrNi1rQsKnGlJdXMphEFHYDc7Y_0 2024-04-17T05:46:03.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7197254122618084216', time=2024-04-17T07:23:21.890857+0000, chain.objs.size()=1 2024-04-17T05:46:03.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.pUnz8Yj7UO447FBVfRLrkrm_n0k_fce_0 2024-04-17T05:46:03.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2068008690175942831', time=2024-04-17T07:23:22.136807+0000, chain.objs.size()=1 2024-04-17T05:46:03.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.hGCWrVWf5i5IOJWo5cCMFD7NsiDLKJo_0 2024-04-17T05:46:03.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.252+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5477284594539862687', time=2024-04-17T07:23:22.537003+0000, chain.objs.size()=1 2024-04-17T05:46:03.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.252+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2NF3Z5yV3LTBeVbxLxDeo03trFZo_n-_0 2024-04-17T05:46:03.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/48000 2024-04-17T05:46:03.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=34, truncated=0, next_marker='' 2024-04-17T05:46:03.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18280041194209811553', time=2024-04-17T07:23:23.020410+0000, chain.objs.size()=1 2024-04-17T05:46:03.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.83Ib3Alo0pZ0YM-D17jqzbccZKh-EpJ_0 2024-04-17T05:46:03.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6298193334032526576', time=2024-04-17T07:23:23.763803+0000, chain.objs.size()=1 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1mqTH8nN8zSBi3qXShlEd-HO42ItC2__0 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7673123416693497448', time=2024-04-17T07:23:23.857703+0000, chain.objs.size()=1 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aIuQgqONMJvBq4rURfCN8CKHi01v-YS_0 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4132583016022212370', time=2024-04-17T07:23:25.293990+0000, chain.objs.size()=1 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oHduy0_5HKjU6s-wWedRwb4CWKVWmfN_0 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1902533289722445908', time=2024-04-17T07:23:25.380289+0000, chain.objs.size()=1 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VBoLxo8jFGszy78bhg-C3ETbOmgr2ae_0 2024-04-17T05:46:03.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2902504022932925767', time=2024-04-17T07:23:26.581646+0000, chain.objs.size()=1 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qEJUymIfZWmYFBncNZM9PGTrJzl4kcv_0 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14818320328722305717', time=2024-04-17T07:23:27.518801+0000, chain.objs.size()=1 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-lX2fNiMRIYKGDObEFR2AEOAyJEOjDm_0 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1083811066471146571', time=2024-04-17T07:23:28.565338+0000, chain.objs.size()=1 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.06toL0M370dWyKtefGGM1CpJ4Too9Px_0 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4527525260919180385', time=2024-04-17T07:23:29.531339+0000, chain.objs.size()=1 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oZDKpqeXCWxSkpe721iMGYbiTPBovBR_0 2024-04-17T05:46:03.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1229642474967189393', time=2024-04-17T07:23:29.647014+0000, chain.objs.size()=1 2024-04-17T05:46:03.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qXeqheT8BhwJunqnELaQzn-zFXZXrSm_0 2024-04-17T05:46:03.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4751070291258117032', time=2024-04-17T07:23:31.162921+0000, chain.objs.size()=1 2024-04-17T05:46:03.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RKatc-vcIFqMEzs-Kcj11WO1jpKmdc-_0 2024-04-17T05:46:03.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5161420770301283969', time=2024-04-17T07:23:32.196734+0000, chain.objs.size()=1 2024-04-17T05:46:03.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.256+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.X8auw25N1xunwVLkGJta9Z977ArV0-x_0 2024-04-17T05:46:03.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.260+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11278144539267966403', time=2024-04-17T07:23:34.976352+0000, chain.objs.size()=1 2024-04-17T05:46:03.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.260+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SWVbhyXjpCeZQOIEd_Px7-Ja4SYdtc6_0 2024-04-17T05:46:03.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3692208434076358676', time=2024-04-17T07:23:35.792649+0000, chain.objs.size()=1 2024-04-17T05:46:03.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.n9B4A2LQz18189uxRFPW0tHh1iNaFON_0 2024-04-17T05:46:03.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17758397779798382268', time=2024-04-17T07:23:35.794857+0000, chain.objs.size()=1 2024-04-17T05:46:03.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.70KhE3SkwJkx4yw_D4CqErjfeeYQ3qB_0 2024-04-17T05:46:03.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9509806166988705346', time=2024-04-17T07:23:36.190594+0000, chain.objs.size()=1 2024-04-17T05:46:03.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.S_0tPGCQsW88Qwstm0cqWo6trHGbnBW_0 2024-04-17T05:46:03.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7005732790836286112', time=2024-04-17T07:23:36.572651+0000, chain.objs.size()=1 2024-04-17T05:46:03.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oXTdAm6IJNN4TEKgInTPu_9ryOnXnEl_0 2024-04-17T05:46:03.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8495941470921518584', time=2024-04-17T07:23:37.430433+0000, chain.objs.size()=1 2024-04-17T05:46:03.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2MSd_0ez1V8h-R184MEplzY0fUBavdk_0 2024-04-17T05:46:03.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3621532980826036783', time=2024-04-17T07:23:37.784063+0000, chain.objs.size()=1 2024-04-17T05:46:03.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.B6IlMeZodcWyCYrTz2Gd6HYS79Gdba6_0 2024-04-17T05:46:03.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8041182398360467828', time=2024-04-17T07:23:41.275767+0000, chain.objs.size()=1 2024-04-17T05:46:03.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qfQdoBpf88ydTZ_IzsTAH2U_m_gfT1K_0 2024-04-17T05:46:03.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5135705803152814142', time=2024-04-17T07:23:43.505245+0000, chain.objs.size()=1 2024-04-17T05:46:03.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.316__shadow_.h9vyDYQkasZYkV6ovwxn-GLb1-TMQde_0 2024-04-17T05:46:03.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15618668321199065299', time=2024-04-17T07:23:56.482951+0000, chain.objs.size()=1 2024-04-17T05:46:03.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.330__shadow_.SSaqLcYiSegqSSdRrXBgGoQmogcMmzI_0 2024-04-17T05:46:03.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4432564616257090559', time=2024-04-17T07:25:18.698126+0000, chain.objs.size()=1 2024-04-17T05:46:03.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.422__shadow_.AXr6OUbj3xefGfIqt1OF-zGoWhCfKNL_0 2024-04-17T05:46:03.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16828232802957200685', time=2024-04-17T07:25:57.496119+0000, chain.objs.size()=1 2024-04-17T05:46:03.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.264+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.453__shadow_.iTY3GDufPOTryD7OZkbZBgNEODS0kdA_0 2024-04-17T05:46:03.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10876332981591562200', time=2024-04-17T07:26:02.359165+0000, chain.objs.size()=1 2024-04-17T05:46:03.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.456__shadow_.9UAdlsDkMCf8dwtbTHNHkMlUxuL3zdS_0 2024-04-17T05:46:03.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2_87f3975f7b3d0ccd522953c70014dcff', time=2024-04-17T07:26:56.574007+0000, chain.objs.size()=6 2024-04-17T05:46:03.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.1 2024-04-17T05:46:03.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2 2024-04-17T05:46:03.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.3 2024-04-17T05:46:03.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.4 2024-04-17T05:46:03.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.5 2024-04-17T05:46:03.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.6 2024-04-17T05:46:03.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY', time=2024-04-17T07:27:55.725003+0000, chain.objs.size()=1 2024-04-17T05:46:03.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY.1 2024-04-17T05:46:03.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15611656390767011316', time=2024-04-17T07:31:15.229916+0000, chain.objs.size()=2 2024-04-17T05:46:03.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.547__shadow_.4kHmeauvUr9YU6CBc6EWhYsdRetbFmH_0 2024-04-17T05:46:03.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.268+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.547__shadow_.4kHmeauvUr9YU6CBc6EWhYsdRetbFmH_1 2024-04-17T05:46:03.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9047473062194478797', time=2024-04-17T07:31:18.562812+0000, chain.objs.size()=1 2024-04-17T05:46:03.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.549__shadow_.IlKG34TaWUV-Rlm46HkFZnRZHw2U9ou_0 2024-04-17T05:46:03.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7305230112538975565', time=2024-04-17T07:31:37.275504+0000, chain.objs.size()=1 2024-04-17T05:46:03.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:Yy0hHJq16ITIiG7bmfYcZFpJsjFD6AL_.UHUBd4wWvHF3oBGgAdCBB9N8GZC3GYm_0 2024-04-17T05:46:03.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7837703940823176135', time=2024-04-17T07:31:39.918643+0000, chain.objs.size()=1 2024-04-17T05:46:03.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:PwIXx2DC6z9gHTN7NrI.g33erWrDgyy_.I_iKbK2dq6VVkxhnODvDIJDlsuVclrX_0 2024-04-17T05:46:03.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14657104035852489442', time=2024-04-17T07:33:43.007589+0000, chain.objs.size()=1 2024-04-17T05:46:03.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.575__shadow:Azt7llQrQUgMUrup2HAE57dAHu7DsHL_.Yi9rVezmfNNi-SAZOpehAiH-gH83Dux_0 2024-04-17T05:46:03.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4635765530835836533', time=2024-04-17T07:37:34.135262+0000, chain.objs.size()=6 2024-04-17T05:46:03.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.272+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647__multipart_foo.2~RRhiXpAqI0M42YAgzneiGwqs0R1q91z.1 2024-04-17T05:46:03.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647__multipart_foo.2~RRhiXpAqI0M42YAgzneiGwqs0R1q91z.2 2024-04-17T05:46:03.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647__multipart_foo.2~RRhiXpAqI0M42YAgzneiGwqs0R1q91z.3 2024-04-17T05:46:03.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647__multipart_foo.2~RRhiXpAqI0M42YAgzneiGwqs0R1q91z.4 2024-04-17T05:46:03.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647__multipart_foo.2~RRhiXpAqI0M42YAgzneiGwqs0R1q91z.5 2024-04-17T05:46:03.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647__multipart_foo.2~RRhiXpAqI0M42YAgzneiGwqs0R1q91z.6 2024-04-17T05:46:03.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4652018106325193695', time=2024-04-17T07:38:55.435407+0000, chain.objs.size()=1 2024-04-17T05:46:03.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.276+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.704__shadow_.NcrAv0bkypiqbnf2Tn4bStwPPWVUk9m_0 2024-04-17T05:46:03.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.280+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.284+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-17T05:46:03.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.284+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.284+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/51171' 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14405181655350820888', time=2024-04-17T07:10:03.117060+0000, chain.objs.size()=3 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-FROZEN.2~H4R5yLddg9rljXa6Z377IUq1RBCFzgH.1 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-FROZEN.2~H4R5yLddg9rljXa6Z377IUq1RBCFzgH.2 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.149__multipart_mymultipart-FROZEN.2~H4R5yLddg9rljXa6Z377IUq1RBCFzgH.3 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7445449329250813381', time=2024-04-17T07:14:54.497270+0000, chain.objs.size()=1 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.236__shadow_.d1Vscqjiea9EmFzrz8tbS_CB-czZ3q9_0 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17805541764903462421', time=2024-04-17T07:15:02.621722+0000, chain.objs.size()=1 2024-04-17T05:46:03.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.241__shadow_.beArQGrjUQRJ16PXAX_761U5xHBJEHm_0 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15640149527711684431', time=2024-04-17T07:17:01.791508+0000, chain.objs.size()=1 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.S6_lrrGNTTXBL1SFsaqYXfWuu4YiE0Y_0 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1718344235310293509', time=2024-04-17T07:17:02.002741+0000, chain.objs.size()=1 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.spkV6feC3WtkKswrP-KlRNJ20S-rI_I_0 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17715019926367976183', time=2024-04-17T07:17:02.092738+0000, chain.objs.size()=1 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6q_q5lOeAc-_qyR6L4uZVdwmxoILIWL_0 2024-04-17T05:46:03.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12404869092261530539', time=2024-04-17T07:17:02.258873+0000, chain.objs.size()=1 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.g7gRmxRwHrJNZJFGlSXuAZUXJ3EHUEG_0 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6386936574037712418', time=2024-04-17T07:17:02.421235+0000, chain.objs.size()=1 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LV69Et2FDgauCAQr8JMkSy0NaZDlKJI_0 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9654257516531050483', time=2024-04-17T07:17:06.075597+0000, chain.objs.size()=1 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.H24U9Mg4GW6Au_XjPiq-w6BTa6p8mPU_0 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6780072734546082090', time=2024-04-17T07:17:08.037603+0000, chain.objs.size()=1 2024-04-17T05:46:03.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.288+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LN0gNUwfOt9pfeyKRGYOcmfQtkbkHaw_0 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3042981143711683424', time=2024-04-17T07:17:08.039402+0000, chain.objs.size()=1 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tV2TNj0nL58xObYQ9iIK6sO4pmx7s2z_0 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15158006444207490053', time=2024-04-17T07:17:10.331993+0000, chain.objs.size()=1 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2sShDBiLtM-CQX41tzpJdpopo5oCiHD_0 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7937092044219475897', time=2024-04-17T07:17:10.802958+0000, chain.objs.size()=1 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ec_ALzAQ7Fw7UTZhjOY05yX7tXdiaAg_0 2024-04-17T05:46:03.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9945599140470349117', time=2024-04-17T07:17:12.931288+0000, chain.objs.size()=1 2024-04-17T05:46:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.c6czkFkg6fgmF5AuWqEPHppVlQ4cvHV_0 2024-04-17T05:46:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11011600876161151195', time=2024-04-17T07:17:12.933607+0000, chain.objs.size()=1 2024-04-17T05:46:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zpqDEx9TdEK-jDrMkNqXXLyjEZNTqjw_0 2024-04-17T05:46:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8183289577210675518', time=2024-04-17T07:17:13.946273+0000, chain.objs.size()=1 2024-04-17T05:46:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.44-JOMXf6NEQXASyeZb5nKZL-rleGxD_0 2024-04-17T05:46:03.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11701840992739678743', time=2024-04-17T07:17:15.200235+0000, chain.objs.size()=1 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jbQL5kxMX41bzfW92J0K_bhYHSGFfGY_0 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5606170160712758204', time=2024-04-17T07:17:16.882727+0000, chain.objs.size()=1 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.q26_FoyBXiY8urIXE4kbLzw1RBNwdFZ_0 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13367276170603438023', time=2024-04-17T07:17:19.861760+0000, chain.objs.size()=1 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hGP4CVaOlHLpM73dG0Z4bvbnm1AJTPF_0 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10303724717056333083', time=2024-04-17T07:17:23.095590+0000, chain.objs.size()=1 2024-04-17T05:46:03.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.7ehHVjyIGOo9vYHNlpbIiZhAxH9sVNi_0 2024-04-17T05:46:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16308699591211038144', time=2024-04-17T07:17:24.883192+0000, chain.objs.size()=1 2024-04-17T05:46:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.292+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.H9jbbAXPYFRDOg46wVdEdroE_fWeWty_0 2024-04-17T05:46:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14437411084451373636', time=2024-04-17T07:17:26.155810+0000, chain.objs.size()=1 2024-04-17T05:46:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GiGzV54jQ-ylThc9cELeSwB0MNJ5I6U_0 2024-04-17T05:46:03.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1565189985561612270', time=2024-04-17T07:17:29.218321+0000, chain.objs.size()=1 2024-04-17T05:46:03.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.9HGiqV649ID7OQjczI848LMi4dV7vsG_0 2024-04-17T05:46:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4810795188722824555', time=2024-04-17T07:17:30.960037+0000, chain.objs.size()=1 2024-04-17T05:46:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZTty1iTJLm3bI2sDyunf5st8CnuCw76_0 2024-04-17T05:46:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2036090000095971889', time=2024-04-17T07:17:35.429556+0000, chain.objs.size()=1 2024-04-17T05:46:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.254__shadow_.kLmsKKGg4D4TSrKHfV1uap8QscV6Vzy_0 2024-04-17T05:46:03.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.117227797685566028', time=2024-04-17T07:17:42.925403+0000, chain.objs.size()=1 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.259__shadow_.aUoEdlY_ZtYnUp52Llr23qq2oQGoxl5_0 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17078207376925375614', time=2024-04-17T07:17:47.382063+0000, chain.objs.size()=1 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.262__shadow_.1h9Qq8rFaErq8pG0-agCwIPXNXjFlZa_0 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8831187914837416773', time=2024-04-17T07:17:58.464816+0000, chain.objs.size()=1 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.269__shadow_.3gJhF7Wq5lVoYcwEFAQi8HTpbN56K28_0 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12197798313146245592', time=2024-04-17T07:18:08.256634+0000, chain.objs.size()=1 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.275__shadow_.qZz0Yz8sP970WobB9nvPmM3A7hSxSFN_0 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13086154350995556501', time=2024-04-17T07:18:28.279562+0000, chain.objs.size()=1 2024-04-17T05:46:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.gkX_tZzer-XTwveCPl-q-9lycAPuHu3_0 2024-04-17T05:46:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14920668568218210075', time=2024-04-17T07:21:00.230540+0000, chain.objs.size()=1 2024-04-17T05:46:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._Nzu2ch8yJ8PoJlKQ1Vx3cs1aA6w2he_0 2024-04-17T05:46:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8974085556391917386', time=2024-04-17T07:21:00.560529+0000, chain.objs.size()=1 2024-04-17T05:46:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.296+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.k1z03fnCjXmSHJmArdVv-ryAiY_BLsG_0 2024-04-17T05:46:03.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6044300823730155462', time=2024-04-17T07:21:00.997529+0000, chain.objs.size()=1 2024-04-17T05:46:03.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9fYayhWenGU-_n7ZcGnpKo7v9933UUP_0 2024-04-17T05:46:03.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3442367191968809158', time=2024-04-17T07:21:04.764114+0000, chain.objs.size()=1 2024-04-17T05:46:03.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Xc-3Bk_jMw9vcm9-cyccN4QIw7rLwza_0 2024-04-17T05:46:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17425484135559722891', time=2024-04-17T07:21:04.918758+0000, chain.objs.size()=1 2024-04-17T05:46:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.CaAdjyy5R27G8lVMWqR8rf8aWhrzHN5_0 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12847371735486434931', time=2024-04-17T07:21:06.028011+0000, chain.objs.size()=1 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eMSVuSyu4cCwpxJ11okjd4Q_KMivEnF_0 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7491727991933309484', time=2024-04-17T07:21:06.386828+0000, chain.objs.size()=1 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.hUpH18mHMbc7mfKG1dRca0GSjWbXuDM_0 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11380450631919980183', time=2024-04-17T07:21:06.766899+0000, chain.objs.size()=1 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2FRRnBJ4oazMzijQAnIqYpYmnOEgjSb_0 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1641580532375328764', time=2024-04-17T07:21:08.007642+0000, chain.objs.size()=1 2024-04-17T05:46:03.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.300+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.tyiNxzU9-bn4NfuOTHrdw0sT2RQXz9P_0 2024-04-17T05:46:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3069918090158005153', time=2024-04-17T07:21:09.575543+0000, chain.objs.size()=1 2024-04-17T05:46:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ndTj1h-XfGQCqOpqYt6jT8kNl77lX5T_0 2024-04-17T05:46:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8709423861749690923', time=2024-04-17T07:21:10.225070+0000, chain.objs.size()=1 2024-04-17T05:46:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oDmrcS9hz1GEh8jjULa1GOpK_P4loF-_0 2024-04-17T05:46:03.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1284248006231683586', time=2024-04-17T07:21:10.804594+0000, chain.objs.size()=1 2024-04-17T05:46:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.6wVxH1gyKW0mi3wKBm_31IyMF8mqO_B_0 2024-04-17T05:46:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4782848822573522476', time=2024-04-17T07:21:10.860396+0000, chain.objs.size()=1 2024-04-17T05:46:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HZVdqVglOWaNaVoa3do19Vus_vM3QJy_0 2024-04-17T05:46:03.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9112310065160368057', time=2024-04-17T07:21:12.973446+0000, chain.objs.size()=1 2024-04-17T05:46:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._p-lm6mCZB4yBSSkKbP_wZGitH5xVes_0 2024-04-17T05:46:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8380650548992157217', time=2024-04-17T07:21:13.786988+0000, chain.objs.size()=1 2024-04-17T05:46:03.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.304+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MSDWADEGkbBSrHMnaTJMu6CMHtcXmJC_0 2024-04-17T05:46:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1298871015427194452', time=2024-04-17T07:21:14.410490+0000, chain.objs.size()=1 2024-04-17T05:46:03.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PFEIYuqTGMqRtgETmZPRx1XbZL9ZXKu_0 2024-04-17T05:46:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2641351526568675093', time=2024-04-17T07:21:15.950806+0000, chain.objs.size()=1 2024-04-17T05:46:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mr8D7skpOXeiftUxU1Wy0SCoN3KNDbQ_0 2024-04-17T05:46:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1606950646595782245', time=2024-04-17T07:21:16.277197+0000, chain.objs.size()=1 2024-04-17T05:46:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.OXO7Si-hRtuTnGT0hr61a5vv62ylL91_0 2024-04-17T05:46:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6336287979863158149', time=2024-04-17T07:21:16.602112+0000, chain.objs.size()=1 2024-04-17T05:46:03.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.P9S370s6iyywaZV5w9vc0DmAJvXGG7y_0 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3274974567535769783', time=2024-04-17T07:21:16.682269+0000, chain.objs.size()=1 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yK8_41itzcmKng37UzbkVCsWygRasks_0 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2970304254399071943', time=2024-04-17T07:21:16.926860+0000, chain.objs.size()=1 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZeZ09vov1f8CSM_iPp_JoiVdJknaFU8_0 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5786872990806797814', time=2024-04-17T07:21:17.389065+0000, chain.objs.size()=1 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RutLXmePzQTGbiJAJS9aCK7zB3rUyNL_0 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6408906285363411790', time=2024-04-17T07:21:20.293280+0000, chain.objs.size()=1 2024-04-17T05:46:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.OWfB8I-ArkwY7wdVtU8Bmfa_x0OFq8R_0 2024-04-17T05:46:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7143878518550618478', time=2024-04-17T07:21:20.810575+0000, chain.objs.size()=1 2024-04-17T05:46:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.308+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uhJ898A_UioXxsPr8qGErbBGk0EWDx1_0 2024-04-17T05:46:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13645133791670041520', time=2024-04-17T07:21:21.226092+0000, chain.objs.size()=1 2024-04-17T05:46:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.63xuim3usofyZDwJogULfzlG6Mmgwdx_0 2024-04-17T05:46:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3631747362718810070', time=2024-04-17T07:21:21.304717+0000, chain.objs.size()=1 2024-04-17T05:46:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ERfua5lBef-Xq3YkEmX3SX8uyvozxFx_0 2024-04-17T05:46:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18287234244830071556', time=2024-04-17T07:21:21.454980+0000, chain.objs.size()=1 2024-04-17T05:46:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2ahNrIUM486J2uyWe8wZb_pSRUfYUpn_0 2024-04-17T05:46:03.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2712120310188684366', time=2024-04-17T07:21:22.417283+0000, chain.objs.size()=1 2024-04-17T05:46:03.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.312+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.B2HAdCooM5KmAmX5bLqHTtmjiRPjvx4_0 2024-04-17T05:46:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1350491164576758581', time=2024-04-17T07:21:22.472522+0000, chain.objs.size()=1 2024-04-17T05:46:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IoqTMw_dswmJkNWDnEwd0sHBEC7ukwc_0 2024-04-17T05:46:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17517757664444816629', time=2024-04-17T07:21:24.836296+0000, chain.objs.size()=1 2024-04-17T05:46:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DYgBgfa0pIx9CTHeCgt2a8pgG-8Uug__0 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10852154656244820100', time=2024-04-17T07:21:25.350709+0000, chain.objs.size()=1 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.W-kFbQ5X25byLZPCDqNeQnYuopLF8Ik_0 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8157898193282980084', time=2024-04-17T07:21:27.096488+0000, chain.objs.size()=1 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zhh9SUIrG9YJiHKYz-wtJkQq_AssDVH_0 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17207810442327156120', time=2024-04-17T07:21:27.500899+0000, chain.objs.size()=1 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ffWduFa8uqJC1aua8zH49W42hwaPEC1_0 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11327579044061349898', time=2024-04-17T07:21:27.786902+0000, chain.objs.size()=1 2024-04-17T05:46:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4Uaq6-paXDkf0faqr6emg_sdOX300L2_0 2024-04-17T05:46:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9957327319055031117', time=2024-04-17T07:21:27.798705+0000, chain.objs.size()=1 2024-04-17T05:46:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.iFgcBMUjL9p6sVWStiZ_LMUsdjJcRP__0 2024-04-17T05:46:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17741758800257482298', time=2024-04-17T07:21:28.317748+0000, chain.objs.size()=1 2024-04-17T05:46:03.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mW4o0Z9N_MyVF3fwsGOatiXbk_IH3oY_0 2024-04-17T05:46:03.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7882578722329432618', time=2024-04-17T07:23:12.633042+0000, chain.objs.size()=1 2024-04-17T05:46:03.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.jLjQGKJ0rSONrx0j1JSJpP37vljYJ-__0 2024-04-17T05:46:03.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13524305554854786704', time=2024-04-17T07:23:13.842083+0000, chain.objs.size()=1 2024-04-17T05:46:03.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.316+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.41VswuvSWzIfmf8qmF1rq1iP4lfbgpF_0 2024-04-17T05:46:03.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10141379019338209636', time=2024-04-17T07:23:13.966320+0000, chain.objs.size()=1 2024-04-17T05:46:03.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kuSXpeAfysCHbA1ptRqZKRioIPCD3sK_0 2024-04-17T05:46:03.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4494632634286197206', time=2024-04-17T07:23:19.553140+0000, chain.objs.size()=1 2024-04-17T05:46:03.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ojzA8df-vZ7Dsozy5vKGvSiovkmqhGl_0 2024-04-17T05:46:03.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15516009222422570573', time=2024-04-17T07:23:19.810685+0000, chain.objs.size()=1 2024-04-17T05:46:03.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bp6aYo1-ot6yYJ0JVhi8DgoaWwhpzvI_0 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13023140447064371509', time=2024-04-17T07:23:22.202641+0000, chain.objs.size()=1 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Thcu5CGzv3F_RUlFChTjqfWXtqY7soi_0 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15526328748085023193', time=2024-04-17T07:23:22.331524+0000, chain.objs.size()=1 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.AuzMCYYSlJM6sfIhcI2Mu9sitv96SiF_0 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6764437501011982450', time=2024-04-17T07:23:22.577088+0000, chain.objs.size()=1 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5gkRisR4BwyI1BnjFr379DEcvrz0jSa_0 2024-04-17T05:46:03.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5057716921144339331', time=2024-04-17T07:23:23.383255+0000, chain.objs.size()=1 2024-04-17T05:46:03.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NINtua5ieexkljcEZPcZe-PVrxZX-JM_0 2024-04-17T05:46:03.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12940546621851730865', time=2024-04-17T07:23:23.449560+0000, chain.objs.size()=1 2024-04-17T05:46:03.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.320+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oSim25-Bo9tWlAjn2edfljapXFzxYRP_0 2024-04-17T05:46:03.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6425641356959581698', time=2024-04-17T07:23:24.933266+0000, chain.objs.size()=1 2024-04-17T05:46:03.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IlIrv_ThNcDEBbWFxYjI5bniJt73r26_0 2024-04-17T05:46:03.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8225724168984556824', time=2024-04-17T07:23:25.516803+0000, chain.objs.size()=1 2024-04-17T05:46:03.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.66mfF_-LPC0C0P1M6b01Y0matcL_js7_0 2024-04-17T05:46:03.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15815509763228422267', time=2024-04-17T07:23:26.444583+0000, chain.objs.size()=1 2024-04-17T05:46:03.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QGXzKPPF3U9Mr_p6YdkDDENEnARI8fX_0 2024-04-17T05:46:03.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8500524097300976841', time=2024-04-17T07:23:27.001158+0000, chain.objs.size()=1 2024-04-17T05:46:03.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1W2z31W8TA4I1Eqvf02q9RGjfBDzD9A_0 2024-04-17T05:46:03.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10088002710890595314', time=2024-04-17T07:23:27.559740+0000, chain.objs.size()=1 2024-04-17T05:46:03.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.324+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.i22CvT-XCAAJ0hnSklh9yrfWE_06DO8_0 2024-04-17T05:46:03.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2430500330605685916', time=2024-04-17T07:23:28.554487+0000, chain.objs.size()=1 2024-04-17T05:46:03.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eAeyezmOOCoJ_JiFTDeMqzdoupx4_d6_0 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10015302415970529780', time=2024-04-17T07:23:28.656687+0000, chain.objs.size()=1 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lzrM7b70NkzKrYNPioMtZPwxR2rrXI8_0 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7817556684282011519', time=2024-04-17T07:23:28.933512+0000, chain.objs.size()=1 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kDI-dnn64Q8mtxd6FByHPLTy8gVpOuL_0 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14826464524473270268', time=2024-04-17T07:23:30.295654+0000, chain.objs.size()=1 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.luhiYl3mM9z-G5zMDALORYXyVPvJ2Pz_0 2024-04-17T05:46:03.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12543561165566144247', time=2024-04-17T07:23:31.053527+0000, chain.objs.size()=1 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kKswOhuoC32gO-5pff6EI1Y4ye1Whed_0 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4131476477758728742', time=2024-04-17T07:23:32.729989+0000, chain.objs.size()=1 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.juFi3tz_BkPmxV6sJ0KHFVNOL3BMQ7E_0 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3666085302254255792', time=2024-04-17T07:23:33.716848+0000, chain.objs.size()=1 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IWUn0gh8hZu9-jFPDb4WFC8GdZqGGbs_0 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13236864635706049383', time=2024-04-17T07:23:33.891708+0000, chain.objs.size()=1 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VCRbF4O94Tf4FRufIf_sGtPmqRMQZZU_0 2024-04-17T05:46:03.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8914982549288443645', time=2024-04-17T07:23:34.573661+0000, chain.objs.size()=1 2024-04-17T05:46:03.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ibwYl1vK3i2AzXvEI6cpI7As6IRK4gL_0 2024-04-17T05:46:03.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17601731018178543067', time=2024-04-17T07:23:34.808299+0000, chain.objs.size()=1 2024-04-17T05:46:03.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9XUgZ-wfxpsmoZms1BbncDK8kN6D-Vp_0 2024-04-17T05:46:03.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10411512089864920183', time=2024-04-17T07:23:37.316363+0000, chain.objs.size()=1 2024-04-17T05:46:03.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.328+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sshzIDlmXeRTlyFSoBoECwgpMYA_Kxh_0 2024-04-17T05:46:03.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.332+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13734374094277629613', time=2024-04-17T07:23:38.668605+0000, chain.objs.size()=1 2024-04-17T05:46:03.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.332+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qanEU9hUgEaktA0Sf8aO_IVzAp4z7lu_0 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14814073421088681756', time=2024-04-17T07:23:52.326338+0000, chain.objs.size()=1 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.325__shadow_.ubcHhEww2i-kKOluiAIRHsq_vUWScAI_0 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15649550326910287797', time=2024-04-17T07:24:57.777519+0000, chain.objs.size()=1 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.397__shadow_.Pm1IeR6SmoJqFAp_9nFPb-SDgJt3Sr5_0 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1094601702193026937', time=2024-04-17T07:25:15.725756+0000, chain.objs.size()=1 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.419__shadow_.3ApXsT88Jl0YMCJJot253SX3FSVopUX_0 2024-04-17T05:46:03.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14527433085813889153', time=2024-04-17T07:25:24.217371+0000, chain.objs.size()=1 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.427__shadow_.UiJu1FU9G4rj-jUiFy3e0DgWjp2CUFn_0 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17697650060080350416', time=2024-04-17T07:25:52.405952+0000, chain.objs.size()=1 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.450__shadow_.SArpzlKieeKEcaBnZMf0gbWLwNf2jVY_0 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.478__shadow_.aTjwmnxXceHlLJ5q0ieACeDqPlahZvP_1_4a4ca73aacda97bdde8ef468192f2527', time=2024-04-17T07:26:26.940374+0000, chain.objs.size()=1 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.478__shadow_.aTjwmnxXceHlLJ5q0ieACeDqPlahZvP_0 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.485__shadow_.gR0kZO6gxFVbi-o29wZANI9uu3sD_wh_1_347f6dc07de5a8cc73badbab667bfefc', time=2024-04-17T07:26:33.135364+0000, chain.objs.size()=1 2024-04-17T05:46:03.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.336+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.485__shadow_.gR0kZO6gxFVbi-o29wZANI9uu3sD_wh_0 2024-04-17T05:46:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.340+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/51171 2024-04-17T05:46:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=23, truncated=0, next_marker='' 2024-04-17T05:46:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.487__shadow_.gfcCpZ5c8UYRyumKrZBGQ2gfrms4cZS_1_85bd67c07df9fa935acad82e10eca093', time=2024-04-17T07:26:36.344445+0000, chain.objs.size()=1 2024-04-17T05:46:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.487__shadow_.gfcCpZ5c8UYRyumKrZBGQ2gfrms4cZS_0 2024-04-17T05:46:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_1_d5aa9f11c520d23986ec747d241fbd52', time=2024-04-17T07:26:43.471147+0000, chain.objs.size()=1 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_0 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6696739894297501044', time=2024-04-17T07:27:15.681157+0000, chain.objs.size()=1 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.507__shadow_.lBs3C-QYf_H5R0nzkiu-J-bSQgKFzLV_0 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4378815089410714849', time=2024-04-17T07:27:29.135612+0000, chain.objs.size()=6 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.509__multipart_mymultipart.2~Teuxq3rPNrFPxzVl4eqdombQU-nme-C.1 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.509__multipart_mymultipart.2~Teuxq3rPNrFPxzVl4eqdombQU-nme-C.2 2024-04-17T05:46:03.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.509__multipart_mymultipart.2~Teuxq3rPNrFPxzVl4eqdombQU-nme-C.3 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.509__multipart_mymultipart.2~Teuxq3rPNrFPxzVl4eqdombQU-nme-C.4 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.509__multipart_mymultipart.2~Teuxq3rPNrFPxzVl4eqdombQU-nme-C.5 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.509__multipart_mymultipart.2~Teuxq3rPNrFPxzVl4eqdombQU-nme-C.6 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4126769974963139633', time=2024-04-17T07:29:02.352752+0000, chain.objs.size()=6 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A.1 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.V6b5QOsRiusExhqsVhv98di-IXv1syM8.2 2024-04-17T05:46:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.344+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.EZ6eKWdGS4mUgOzCxJ6IEHvUO5aytglr.3 2024-04-17T05:46:03.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A.4 2024-04-17T05:46:03.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A.5 2024-04-17T05:46:03.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~QaE2ohY5qvnDcOl7rftx84lcBlA-__A.6 2024-04-17T05:46:03.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3597727869583055165', time=2024-04-17T07:29:14.282644+0000, chain.objs.size()=1 2024-04-17T05:46:03.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~jTYQ6kWHNJ5ioTORtY8Q-Qbvl861N2S.1 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4402950179130819894', time=2024-04-17T07:29:17.811189+0000, chain.objs.size()=2 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~DveI9zVRqkV_qnFaPm7SXEldGOPQXx9.1 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~DveI9zVRqkV_qnFaPm7SXEldGOPQXx9.2 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1125344603562406360', time=2024-04-17T07:31:57.482207+0000, chain.objs.size()=1 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.559__shadow:v8yxi3VFzRhJGS6XJ0.51sde3lbNCni_.rSWPd9Zg4gV5frFE1A_RiYVxOLUjoCJ_0 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18103359709039281437', time=2024-04-17T07:31:58.166613+0000, chain.objs.size()=1 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.559__shadow:0usQcNakV-XkIhMz.N71RLZWcXL0WtK_.E8iX2OQloR69QdPGLsjD6kyYuQazfOZ_0 2024-04-17T05:46:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17123469241093318053', time=2024-04-17T07:32:24.801164+0000, chain.objs.size()=1 2024-04-17T05:46:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.565__shadow:DxFVDQ-1w0fp7Itlg.bvq3Y.Ogt4PZA_.idJMMXR-gV4zAnqCYuwvLxbhZlCjtIA_0 2024-04-17T05:46:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15798723613333977754', time=2024-04-17T07:32:41.420614+0000, chain.objs.size()=1 2024-04-17T05:46:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:KjSS7JeVyo33K9OUxvU7mwrlKrPQQ3S_.oeUbya87GPw1EWObi5_za0XWrR9SfBD_0 2024-04-17T05:46:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13432057310640026588', time=2024-04-17T07:32:48.973543+0000, chain.objs.size()=1 2024-04-17T05:46:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.348+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:mpiDpvSNNfXNPnzaDsPhdrNE9-p8YSC_.YfRXcMSQjAKq_gBqhgqzbK7TWDnlwek_0 2024-04-17T05:46:03.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.76057181728395492', time=2024-04-17T07:33:04.140804+0000, chain.objs.size()=1 2024-04-17T05:46:03.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:yJDHK50Dxu7cw9o2nIW6SxG47ofeM3N_.UFYCPxi-XOMYR5omplv_ySJLkBJlbDe_0 2024-04-17T05:46:03.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17959994530866140482', time=2024-04-17T07:33:28.877140+0000, chain.objs.size()=1 2024-04-17T05:46:03.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:mKM9KH9Ii0refwCT35LK8zCQ2b2hGzQ_.IZhRFUY9K1Yigc2bx6DlfhUYd40XRV6_0 2024-04-17T05:46:03.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18136382918732310187', time=2024-04-17T07:33:29.136980+0000, chain.objs.size()=1 2024-04-17T05:46:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:sHiS.ua1ZANrRLw9fWxdf.XKPTU7ni9_.9OJxmi19bRFSBI_IDuKSxbVCi5QYE9e_0 2024-04-17T05:46:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4024711332345840490', time=2024-04-17T07:33:29.441659+0000, chain.objs.size()=1 2024-04-17T05:46:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:8vYTT5z1sG3t3jyXoT3TnFyYEJRkVSZ_.k0F845vHNAIdQ-rkJo4RXmzKb2l__nA_0 2024-04-17T05:46:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6894345034733022002', time=2024-04-17T07:33:33.584595+0000, chain.objs.size()=1 2024-04-17T05:46:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:aYkwN-EcPPlwfAR2VHy7nBHocJML3oL_.TDjeRwHRu8PYUf7zcW4M64h1eexc6Ie_0 2024-04-17T05:46:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9600794642340158925', time=2024-04-17T07:33:41.118653+0000, chain.objs.size()=1 2024-04-17T05:46:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.574__shadow:yIxlmqXGrbHV7B5E8QCAAtslmI3ZJpS_.kg8vThoJZ-m_7lzzeEFNNogQfMWwhxJ_0 2024-04-17T05:46:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16728138135404305804', time=2024-04-17T07:33:46.071153+0000, chain.objs.size()=1 2024-04-17T05:46:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:cHhex68LVQ.nmH4MXf9hcHFX7Y4ScTZ_.3W2bYRdFbn1R4Tl1RNFVyGGBp8r2ohS_0 2024-04-17T05:46:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16884857764190296959', time=2024-04-17T07:36:26.381258+0000, chain.objs.size()=1 2024-04-17T05:46:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.613__shadow_.BsTgqG-ESdyI0j93e-jOZ-g3lyP0Q9d_0 2024-04-17T05:46:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4742076628080429736', time=2024-04-17T07:36:32.537884+0000, chain.objs.size()=1 2024-04-17T05:46:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.619__shadow_.F6WBesddHqN6n2_NT1s4sSu2tjJXmMO_0 2024-04-17T05:46:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4648140596687025235', time=2024-04-17T07:37:09.399708+0000, chain.objs.size()=1 2024-04-17T05:46:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.641__shadow_.pfx9VWfAcdbFQY6GFJwXXdQD5M0D8oz_0 2024-04-17T05:46:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.653__shadow_.fIyq-YVRCiWKiFyX6otswI2odm6RgTN_1_9c3bd26e5b70779bd58de3414c080166', time=2024-04-17T07:37:44.414052+0000, chain.objs.size()=1 2024-04-17T05:46:03.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.352+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.653__shadow_.fIyq-YVRCiWKiFyX6otswI2odm6RgTN_0 2024-04-17T05:46:03.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.360+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.360+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-17T05:46:03.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45484' 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11852457485367886850', time=2024-04-17T07:09:50.690251+0000, chain.objs.size()=1 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.147__shadow:NB9gsg2zWjsa4OiNjcRTOhoIDiMqvjG_.wlRWV3FZKkoKmvsaCTuuXf2th2Tg5p6_0 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_vcoXZLTOHPcNpH2arno2QJiSoQzcufM', time=2024-04-17T07:10:25.385252+0000, chain.objs.size()=2 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.dLP92h1yMsUrF5ECiRcEma59CMQC7lW_0 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.151__shadow_.dLP92h1yMsUrF5ECiRcEma59CMQC7lW_1 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_6_IDpjYgnfFRsHQ2Vg2KWlKGFAQztUy', time=2024-04-17T07:10:34.407172+0000, chain.objs.size()=2 2024-04-17T05:46:03.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.155__shadow_.VtyN2OuIwScbH2fG13WkfMJbIDPfVW7_0 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.155__shadow_.VtyN2OuIwScbH2fG13WkfMJbIDPfVW7_1 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4484367418910184150', time=2024-04-17T07:10:56.214351+0000, chain.objs.size()=1 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.167__shadow_.83iciZaaTWuo5QYtpKfMT6SVz-9foC6_0 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3823468169375202791', time=2024-04-17T07:14:36.332995+0000, chain.objs.size()=1 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.226__shadow_.0ModVofxQVXQAd4pnpwABqbPbpa9vMd_0 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3094759832082207084', time=2024-04-17T07:14:42.197367+0000, chain.objs.size()=1 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.229__shadow_.CQsf6XP6YRjYHcQXXFPNCwwAbLPKhKV_0 2024-04-17T05:46:03.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13380486837470941255', time=2024-04-17T07:15:02.633915+0000, chain.objs.size()=1 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.241__shadow_.FRYFx_RDSE7lXDho215e_Swq7quWNPm_0 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13946327520618193701', time=2024-04-17T07:17:01.707979+0000, chain.objs.size()=1 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bSO17gzdMGsziQcvlRnceNdMxkcZFav_0 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3865553055732782606', time=2024-04-17T07:17:02.457881+0000, chain.objs.size()=1 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.37ILdEwnBGf-OlsGbUOFeYWzvyHTTUD_0 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8670585590199663163', time=2024-04-17T07:17:03.370474+0000, chain.objs.size()=1 2024-04-17T05:46:03.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.364+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.CD3AVb3oGAwrZdgQ9PsBKbRoroJt5PC_0 2024-04-17T05:46:03.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7860823185518928648', time=2024-04-17T07:17:03.493265+0000, chain.objs.size()=1 2024-04-17T05:46:03.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mBdpuUlqhAPhNuD65QHJL8rGhrWNEic_0 2024-04-17T05:46:03.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3493970335062285980', time=2024-04-17T07:17:03.495272+0000, chain.objs.size()=1 2024-04-17T05:46:03.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.jBdAb7s6tr5kApcGGl_v0-vgJZTOZ5P_0 2024-04-17T05:46:03.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11895489552570334975', time=2024-04-17T07:17:03.735982+0000, chain.objs.size()=1 2024-04-17T05:46:03.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.urpMCylaoE6slXmEOweyFSKo9CytnR7_0 2024-04-17T05:46:03.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5396034797518319294', time=2024-04-17T07:17:06.736076+0000, chain.objs.size()=1 2024-04-17T05:46:03.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.J1BbqIpL7CELo8YvAT6t8-OJdgC5YxT_0 2024-04-17T05:46:03.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.854169885793400486', time=2024-04-17T07:17:10.237525+0000, chain.objs.size()=1 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YEN9ZlJ1qUK-l-psnxJgCQAKjeukjud_0 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11638003896049779849', time=2024-04-17T07:17:10.374284+0000, chain.objs.size()=1 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sCLB-32u5xJWEo2kIRUHIahYmqHUQRn_0 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16734074745786961420', time=2024-04-17T07:17:10.631713+0000, chain.objs.size()=1 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.RmCO59Pv34desZaM1dLJrkJtjJZhXTO_0 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1840107796620415622', time=2024-04-17T07:17:12.222421+0000, chain.objs.size()=1 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.l-Scu09PniA5LlhHT0VPq01Ydp916b8_0 2024-04-17T05:46:03.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4873603898985725397', time=2024-04-17T07:17:12.848004+0000, chain.objs.size()=1 2024-04-17T05:46:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mQwK6Ukrg5VCsFFw-3uW8OXZD8BzuXG_0 2024-04-17T05:46:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13688962392560563199', time=2024-04-17T07:17:12.978682+0000, chain.objs.size()=1 2024-04-17T05:46:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.vSNpfCOlsV9kSAyT2z2FJC2nBdT59mp_0 2024-04-17T05:46:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12260150335962777842', time=2024-04-17T07:17:14.110322+0000, chain.objs.size()=1 2024-04-17T05:46:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.368+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wxbnGEqpZu43Ojwft1zS9sm3zFGG19j_0 2024-04-17T05:46:03.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10076432397211696991', time=2024-04-17T07:17:14.626750+0000, chain.objs.size()=1 2024-04-17T05:46:03.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.R5uqPPQYZM6Me7oujFJGnFPcGgdS6Jn_0 2024-04-17T05:46:03.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5642106684553014791', time=2024-04-17T07:17:14.628651+0000, chain.objs.size()=1 2024-04-17T05:46:03.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4dtvViI-Ipx-j_5m8CgFUxu3ka90O5p_0 2024-04-17T05:46:03.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6513507085329629068', time=2024-04-17T07:17:15.861362+0000, chain.objs.size()=1 2024-04-17T05:46:03.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2km4LJ4Awz80u6LeKEjQwAIEb95z_yn_0 2024-04-17T05:46:03.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5552888211287548249', time=2024-04-17T07:17:15.924283+0000, chain.objs.size()=1 2024-04-17T05:46:03.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tZMOB1ndWjlBp0Nmf40H6Ho_J1Uz7zu_0 2024-04-17T05:46:03.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.678719088035059187', time=2024-04-17T07:17:18.387455+0000, chain.objs.size()=1 2024-04-17T05:46:03.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.372+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zLT85MGu7gYyJTDRG2ZQiKKShtO7LXv_0 2024-04-17T05:46:03.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6633473826568107080', time=2024-04-17T07:17:18.512265+0000, chain.objs.size()=1 2024-04-17T05:46:03.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YHmmFO8oFyX7nlSDvdncqzIhawigw3E_0 2024-04-17T05:46:03.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13308737964709121900', time=2024-04-17T07:17:19.307978+0000, chain.objs.size()=1 2024-04-17T05:46:03.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.whWVtAiZWb26HJFEu7tGWvaYN87ZdDK_0 2024-04-17T05:46:03.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.384554804532672464', time=2024-04-17T07:17:22.022441+0000, chain.objs.size()=1 2024-04-17T05:46:03.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EuxmZmVXfBmyLJusoO0wUWXcpMDx48f_0 2024-04-17T05:46:03.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4939705728178488009', time=2024-04-17T07:17:23.635227+0000, chain.objs.size()=1 2024-04-17T05:46:03.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UelWLUM_h4etHvGKUyMSZv5fyP_FUS__0 2024-04-17T05:46:03.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17092087054609052354', time=2024-04-17T07:17:24.323345+0000, chain.objs.size()=1 2024-04-17T05:46:03.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NUkv9GU4ne4Z4IYC6zCHvU1CaLlLxrD_0 2024-04-17T05:46:03.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15353098246750172136', time=2024-04-17T07:17:26.166339+0000, chain.objs.size()=1 2024-04-17T05:46:03.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JJ98WodgeakKJq3w1DQ2DcvUNQqYb4h_0 2024-04-17T05:46:03.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7368488717704295354', time=2024-04-17T07:17:29.158687+0000, chain.objs.size()=1 2024-04-17T05:46:03.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Dj_dSXBI3R4FD5xALF-9EOANlPfoBVX_0 2024-04-17T05:46:03.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2534695731878853601', time=2024-04-17T07:17:30.668662+0000, chain.objs.size()=1 2024-04-17T05:46:03.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PiDhcrdE4NjjiFlghKCxYbcVkQGVCjj_0 2024-04-17T05:46:03.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16606468252358961263', time=2024-04-17T07:17:31.138222+0000, chain.objs.size()=1 2024-04-17T05:46:03.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.08yEqJzsu-8HMFBRQDR6FasqWb9aMrX_0 2024-04-17T05:46:03.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11590394317813969319', time=2024-04-17T07:17:48.857549+0000, chain.objs.size()=1 2024-04-17T05:46:03.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.376+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.263__shadow_.Gy29kjLeTJKpkPUZ5alo3pcTsUmSOed_0 2024-04-17T05:46:03.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10623499238405080923', time=2024-04-17T07:18:00.092046+0000, chain.objs.size()=1 2024-04-17T05:46:03.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.270__shadow_.yCSXr8eTwSvpoEZRsjmeBZSiJoif0F1_0 2024-04-17T05:46:03.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039da9a640 0 -- 172.21.15.88:0/3940810918 >> [v2:172.21.15.88:6800/1670897847,v1:172.21.15.88:6801/1670897847] conn(0x7f038801d040 msgr2=0x7f026807f6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-17T05:46:03.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1850666575308353778', time=2024-04-17T07:18:08.285492+0000, chain.objs.size()=1 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.275__shadow_.OjjZ41CTwYTpRQUtioGy--213V4ljGK_0 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4550380354469195280', time=2024-04-17T07:18:10.010126+0000, chain.objs.size()=1 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.276__shadow_.q76-vNmO1Jg7jMaEzOy9GYETWAH4uA-_0 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13922661479357064716', time=2024-04-17T07:18:13.224380+0000, chain.objs.size()=1 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.278__shadow_.YDLJw2ZNgK5FEwqM57aeOnzrkv5Uki0_0 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16894568578701829470', time=2024-04-17T07:18:14.887845+0000, chain.objs.size()=1 2024-04-17T05:46:03.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.279__shadow_.kQlrCut621Uy0JeOIzbDwYBTW0Jwr5H_0 2024-04-17T05:46:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7915128466155070293', time=2024-04-17T07:18:22.934668+0000, chain.objs.size()=1 2024-04-17T05:46:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.380+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.tEtjNOjug5snz2Fid2r7lSFrO-IB4P3_0 2024-04-17T05:46:03.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15459706953197543891', time=2024-04-17T07:18:31.505135+0000, chain.objs.size()=1 2024-04-17T05:46:03.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.285__shadow_.Wf1cD0ZtuUXthAk9Gw01W7VkgEGu_W3_0 2024-04-17T05:46:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3044676977574525200', time=2024-04-17T07:19:13.983417+0000, chain.objs.size()=1 2024-04-17T05:46:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.311__shadow_.fEYOzZx95rwjGeVGzo1WDULdpVo83kT_0 2024-04-17T05:46:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2734495707972477227', time=2024-04-17T07:21:00.918137+0000, chain.objs.size()=1 2024-04-17T05:46:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._b7KCjIkJVGqmIm50wMBtV9LcrpKBs8_0 2024-04-17T05:46:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8319485600001097001', time=2024-04-17T07:21:02.311762+0000, chain.objs.size()=1 2024-04-17T05:46:03.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.X3WwPAPHQc-IXFZlom_Mp6wXGIewmss_0 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8941416306313570714', time=2024-04-17T07:21:02.314823+0000, chain.objs.size()=1 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u5qc2RU5eEVdszSNqq8618B-R0QP6cN_0 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15603774587358913376', time=2024-04-17T07:21:02.833059+0000, chain.objs.size()=1 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VbXBXV21u03MwGa9UkTMJzRMZv1Z0RF_0 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12273817503444110255', time=2024-04-17T07:21:05.005060+0000, chain.objs.size()=1 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LOMOY9IW05Zg36lvdziQzIDpDvTiKlK_0 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16894745721719196534', time=2024-04-17T07:21:05.102778+0000, chain.objs.size()=1 2024-04-17T05:46:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UloCnW5kHbpUFS0MiZYcXPZkblKpZow_0 2024-04-17T05:46:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4671102446779576472', time=2024-04-17T07:21:05.311628+0000, chain.objs.size()=1 2024-04-17T05:46:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uoYHlZlGjqTLGBsIwm42ceSLcRmFMFr_0 2024-04-17T05:46:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14422669730800882040', time=2024-04-17T07:21:05.780522+0000, chain.objs.size()=1 2024-04-17T05:46:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Wtbxzi1GXTp600ILJzwwaPRjboopYi__0 2024-04-17T05:46:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7221125024684943942', time=2024-04-17T07:21:05.957632+0000, chain.objs.size()=1 2024-04-17T05:46:03.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.384+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u9GY2fioxlNtd6_Tl8fmiNWkDw1Fv76_0 2024-04-17T05:46:03.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7037014321826819248', time=2024-04-17T07:21:06.847203+0000, chain.objs.size()=1 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.0ImuLlv55QGLISaymLk0mXFacRxaHyK_0 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4864084849822161233', time=2024-04-17T07:21:10.253470+0000, chain.objs.size()=1 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xJmC-ISeGUOFw95PPBQhu4h_jypjTKp_0 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17184857747029052828', time=2024-04-17T07:21:16.599058+0000, chain.objs.size()=1 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.49GV0_Tlr9rfrZiJowJY8-r0htmvsHJ_0 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6251059135087010744', time=2024-04-17T07:21:17.758849+0000, chain.objs.size()=1 2024-04-17T05:46:03.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Pt4luIcMXxdmkemMQP05MF4FogcBtkd_0 2024-04-17T05:46:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13412900515667860044', time=2024-04-17T07:21:17.856642+0000, chain.objs.size()=1 2024-04-17T05:46:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zy-i0EIrTCiuLM25UjWtRWWr11ruj46_0 2024-04-17T05:46:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9536084685501085951', time=2024-04-17T07:21:18.517526+0000, chain.objs.size()=1 2024-04-17T05:46:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.7OZcojzTe42aSnOpVlpyeesedkSiS0L_0 2024-04-17T05:46:03.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15081689423514283373', time=2024-04-17T07:21:20.998297+0000, chain.objs.size()=1 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-g1AVQbCg0f3Uq30s1sZ8a2cCnI5ztm_0 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15016806534906214862', time=2024-04-17T07:21:21.788796+0000, chain.objs.size()=1 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Q7p9aEu0lZ0irshVQgTgLixRp94SYI8_0 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.917618645681365678', time=2024-04-17T07:21:22.053828+0000, chain.objs.size()=1 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.388+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.A_AabC0cQ3KrUmQY15sX12iLJD8zYG3_0 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.647398403816266955', time=2024-04-17T07:21:22.419703+0000, chain.objs.size()=1 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nno9URkA3W-o9_W1UMp3nmTDqx3xz_3_0 2024-04-17T05:46:03.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.392+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4828171347186760806', time=2024-04-17T07:21:23.724371+0000, chain.objs.size()=1 2024-04-17T05:46:03.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.392+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.-EAg9OWvhV4wXr0mxoqfp6ZMxPapRcX_0 2024-04-17T05:46:03.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15340719777110765429', time=2024-04-17T07:21:24.416653+0000, chain.objs.size()=1 2024-04-17T05:46:03.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Cl5YA4L7LrFRHGPKp03234-J7Z3kkJ6_0 2024-04-17T05:46:03.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1997450627890400536', time=2024-04-17T07:21:27.121633+0000, chain.objs.size()=1 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Nt2wzjW5-BlP9r2EQntzI3dfLyK1N3w_0 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14154683941682259735', time=2024-04-17T07:21:27.385671+0000, chain.objs.size()=1 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.C66UxM7Kvmrms9nP1Tx1COq99xo9xDz_0 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13642571748466945411', time=2024-04-17T07:21:27.824189+0000, chain.objs.size()=1 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9bDBbX-x-1ipC3g_p8aX1CJ7HOR3vc-_0 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17066599952291165804', time=2024-04-17T07:21:28.211442+0000, chain.objs.size()=1 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VBMcyrXpk2hPW6NAqNK8DIyv9TRzvrj_0 2024-04-17T05:46:03.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15602376824411481159', time=2024-04-17T07:21:29.254115+0000, chain.objs.size()=1 2024-04-17T05:46:03.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4_8d4PR4uuBdrWy1-awK6KhukXiSu_l_0 2024-04-17T05:46:03.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10558684075518815479', time=2024-04-17T07:21:29.258792+0000, chain.objs.size()=1 2024-04-17T05:46:03.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.WtqaoexsqXc_9rEBJLMul_CQSmkxP4J_0 2024-04-17T05:46:03.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5599258428356857101', time=2024-04-17T07:23:13.050204+0000, chain.objs.size()=1 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VySjr45drBBcosLgHVu-YTgQjV1mlPO_0 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11316211432950243857', time=2024-04-17T07:23:13.692496+0000, chain.objs.size()=1 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sIWBPwHFrWqjj-Q-aqfrD3M9R1_tsZH_0 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14540476790963656188', time=2024-04-17T07:23:13.856791+0000, chain.objs.size()=1 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TOYXwQEw_SIw3XkY9m4pQ3UvMjRdCfO_0 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1454925196100280432', time=2024-04-17T07:23:14.237076+0000, chain.objs.size()=1 2024-04-17T05:46:03.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.396+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7pNdm0wttkhlopFfGQKtVeHoi2L7ucZ_0 2024-04-17T05:46:03.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17327660445441009052', time=2024-04-17T07:23:14.643492+0000, chain.objs.size()=1 2024-04-17T05:46:03.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mlaeyKZe_Piu5ElGWM3ftJ_OtpqJJ57_0 2024-04-17T05:46:03.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5933337164606351604', time=2024-04-17T07:23:17.732976+0000, chain.objs.size()=1 2024-04-17T05:46:03.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dKLBxnvQppSk_Jsj6XoKXFS6TWtoPD4_0 2024-04-17T05:46:03.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17416437944520938409', time=2024-04-17T07:23:18.154107+0000, chain.objs.size()=1 2024-04-17T05:46:03.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3trO4tx4jvs_JlUmduKBbj-z6qfOSwP_0 2024-04-17T05:46:03.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15019136772615413245', time=2024-04-17T07:23:18.160146+0000, chain.objs.size()=1 2024-04-17T05:46:03.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0kO5JP1kPTF5oH-b2fm80a1ZM01YlwT_0 2024-04-17T05:46:03.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1207140847106752386', time=2024-04-17T07:23:18.567910+0000, chain.objs.size()=1 2024-04-17T05:46:03.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.LRxnSmQTc6nUq-oeyJpuZk7URdOJIgT_0 2024-04-17T05:46:03.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13555635621503058093', time=2024-04-17T07:23:21.472735+0000, chain.objs.size()=1 2024-04-17T05:46:03.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rCJHuvMPyXGmC5tO5C_HmPPMnmpU1Pl_0 2024-04-17T05:46:03.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17467362488208719078', time=2024-04-17T07:23:21.474636+0000, chain.objs.size()=1 2024-04-17T05:46:03.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.400+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ErCXrMZj6IwWvQ6OcvwoXfTJO3-iSpV_0 2024-04-17T05:46:03.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7461718386806147318', time=2024-04-17T07:23:21.905207+0000, chain.objs.size()=1 2024-04-17T05:46:03.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RftpKaQkBgzRuBS5S0zHasmdSETvq5n_0 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9817891832633566069', time=2024-04-17T07:23:23.423212+0000, chain.objs.size()=1 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wOy_HIa4wwujqjxOegsGsbOuim7tQqu_0 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1893397071169670760', time=2024-04-17T07:23:24.726451+0000, chain.objs.size()=1 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8yKjsVQ_fQhmUgDdKqNimh6xl1J_0P-_0 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14937156164009659910', time=2024-04-17T07:23:26.328952+0000, chain.objs.size()=1 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XmIrfBTSXkfU1mILd_13jB2kLxedA-g_0 2024-04-17T05:46:03.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8485409061391642815', time=2024-04-17T07:23:26.487863+0000, chain.objs.size()=1 2024-04-17T05:46:03.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.404+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Jm2if9_460uwabwqCwt47Z5lVcwUAYk_0 2024-04-17T05:46:03.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17841316165344503420', time=2024-04-17T07:23:27.001023+0000, chain.objs.size()=1 2024-04-17T05:46:03.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1TMOoZlXBLe3uG1V3xUmRb3HAF7c23w_0 2024-04-17T05:46:03.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8061537230147918971', time=2024-04-17T07:23:27.274502+0000, chain.objs.size()=1 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5fy7a0V2X6Q1WlzlnmM7rSkq7oBCUDd_0 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17815732895808831077', time=2024-04-17T07:23:29.401787+0000, chain.objs.size()=1 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yU_XXtbYUiQx9r14UL-iKxmJvXEZG2w_0 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8531222633458497737', time=2024-04-17T07:23:29.910097+0000, chain.objs.size()=1 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8tjHWqAY5nKBOimKmHxm6PI9g3Z5saQ_0 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10508795537589623300', time=2024-04-17T07:23:29.972080+0000, chain.objs.size()=1 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4XBZAqNOtStiFXNNcmbzGk1V7UShWH8_0 2024-04-17T05:46:03.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12586088645324619615', time=2024-04-17T07:23:30.682481+0000, chain.objs.size()=1 2024-04-17T05:46:03.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.408+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6a5PMFUVxrvkxxEPLBinrDFM0MbVDhg_0 2024-04-17T05:46:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6260900158993510004', time=2024-04-17T07:23:32.502481+0000, chain.objs.size()=1 2024-04-17T05:46:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DLW0px8V_Imq36VonlaJU4zhPp3Q4Be_0 2024-04-17T05:46:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6188762773334719801', time=2024-04-17T07:23:33.749815+0000, chain.objs.size()=1 2024-04-17T05:46:03.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BMsr5U5QVefTUEBB_2wV1nl4Qcexil__0 2024-04-17T05:46:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4735211524401864622', time=2024-04-17T07:23:33.851813+0000, chain.objs.size()=1 2024-04-17T05:46:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CcjWmbrd3KqwMjH-PbrWlG-2zp01Etl_0 2024-04-17T05:46:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8675977018937282963', time=2024-04-17T07:23:34.577642+0000, chain.objs.size()=1 2024-04-17T05:46:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8JB6_Jrbq9H50YhGLedmr2m52CrXfQR_0 2024-04-17T05:46:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6063795158528540446', time=2024-04-17T07:23:35.853474+0000, chain.objs.size()=1 2024-04-17T05:46:03.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TfFFB8CmmWL63uExDiQL25NKRhDkZIC_0 2024-04-17T05:46:03.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5453445877062292072', time=2024-04-17T07:23:36.636988+0000, chain.objs.size()=1 2024-04-17T05:46:03.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Jge0VO0YNirHwgdKI2Vh3L0lEMSnQH8_0 2024-04-17T05:46:03.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5347286167862149910', time=2024-04-17T07:23:36.638838+0000, chain.objs.size()=1 2024-04-17T05:46:03.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.412+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wf1IaS22xcSnLSne2t405uXkPjJspR9_0 2024-04-17T05:46:03.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45484 2024-04-17T05:46:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=28, truncated=0, next_marker='' 2024-04-17T05:46:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10735404746130111486', time=2024-04-17T07:23:37.786595+0000, chain.objs.size()=1 2024-04-17T05:46:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.aHGEra3MkZ2c7MGGsb4GG3RHrrrX2B4_0 2024-04-17T05:46:03.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14373964996053988782', time=2024-04-17T07:23:40.116843+0000, chain.objs.size()=1 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9Tdrc8-5rX4_vC_FvOFbyxIYHR9viF9_0 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1592361728969962754', time=2024-04-17T07:23:46.253634+0000, chain.objs.size()=1 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.319__shadow_.TTEvZuZBeoZbyhkkDW_zgvifXxL64q6_0 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18213652774639077926', time=2024-04-17T07:24:19.231357+0000, chain.objs.size()=1 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.359__shadow_.a5b6xD9Rs6vPq9bI1L-sW1znbmPtz9o_0 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15936037968460370741', time=2024-04-17T07:24:28.028097+0000, chain.objs.size()=1 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.367__shadow_.Gt4k_RRViRV03hRXo_gaKsyAQHoMLNw_0 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2120844597314546521', time=2024-04-17T07:25:54.072598+0000, chain.objs.size()=1 2024-04-17T05:46:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.451__shadow_.g6i41jsKLBLpgtmgtvvOgrd0nboft5G_0 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3547805600864247715', time=2024-04-17T07:26:17.168593+0000, chain.objs.size()=1 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.biOg-g3OcpB9jlZ-jxknS5JCOuXiNOV_0 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5525094581812964889', time=2024-04-17T07:28:08.737130+0000, chain.objs.size()=6 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.x9ak8OeBroW3Lx0EdNP-jb7mLL1UP_HJ.1 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY.2 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY.3 2024-04-17T05:46:03.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY.4 2024-04-17T05:46:03.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.420+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY.5 2024-04-17T05:46:03.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.512__multipart_mymultipart.2~veegGW9g7DGmOvUc0zLQHPkUYSXexeY.6 2024-04-17T05:46:03.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.424+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1200421717623979494', time=2024-04-17T07:29:35.924935+0000, chain.objs.size()=3 2024-04-17T05:46:03.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.424+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~rJbQTBLMEgTOL-CXuz2hcNqF6tqj0ZS.1 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~rJbQTBLMEgTOL-CXuz2hcNqF6tqj0ZS.2 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515__multipart_mymultipart.2~rJbQTBLMEgTOL-CXuz2hcNqF6tqj0ZS.3 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5392269724561368555', time=2024-04-17T07:31:01.899473+0000, chain.objs.size()=1 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.545__shadow_.ZKzyyBz1xOqNw7kRTZkAldUe807D4d9_0 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10668694046871378437', time=2024-04-17T07:31:13.788893+0000, chain.objs.size()=1 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.547__shadow_._BOy4l3wTYZtd2Ap0Pf2RuqKdm-kBEx_0 2024-04-17T05:46:03.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4331028183472481491', time=2024-04-17T07:31:43.605594+0000, chain.objs.size()=1 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:kQVktMsysYpsjJ5QVq3sCkDPk.4TTmV_.giUJ2-nhr0Ea3iI7kQpwq75HFs0OU_5_0 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18126547242101955703', time=2024-04-17T07:31:44.188996+0000, chain.objs.size()=1 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:5NWu0xH1jBtijMh5-wIYU31N8IrET58_.YAa4k0bLPCENHfVOtloZ8KmaHT8sakj_0 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5868202745712724562', time=2024-04-17T07:32:02.918467+0000, chain.objs.size()=1 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.562__shadow_.Ujrsa3F3WKe8EOpFISczN6XHvH6nlVM_0 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8657375672809818028', time=2024-04-17T07:32:06.072198+0000, chain.objs.size()=1 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow_.ewUwHLYxKr4OJyTMh4bjeVDPA0CFxqI_0 2024-04-17T05:46:03.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3170314296839524382', time=2024-04-17T07:32:11.669837+0000, chain.objs.size()=1 2024-04-17T05:46:03.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:1uVyGwKzUJNVsLUAxfz2RfaRn9Ym7qn_.jQsEoyDLBjzewKrW_Mmjux0N3vyWeOz_0 2024-04-17T05:46:03.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2031283275601724917', time=2024-04-17T07:32:12.778577+0000, chain.objs.size()=1 2024-04-17T05:46:03.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:N52ShmUazUkqovWnLg8HXNPd2zI4l2s_.nERwV_mM5kerC1UPI4aBSc56VFt6-t__0 2024-04-17T05:46:03.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2258474804660083101', time=2024-04-17T07:33:29.107102+0000, chain.objs.size()=1 2024-04-17T05:46:03.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.428+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.568__shadow:VgX609fobi52cQcWuUMx.06M84ki0Ff_.4icexeEDkLiP_F9oPHFpxtWQqDLCkQp_0 2024-04-17T05:46:03.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:emuOQEr.v5f7US0CQbgNiL6oyTvPHBn_.8B0PANt9SXHPvpiPHZO8Id_om9c4vas_1_07dcff9491202f62bb86122b0eb9bb97', time=2024-04-17T07:33:33.419982+0000, chain.objs.size()=1 2024-04-17T05:46:03.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:emuOQEr.v5f7US0CQbgNiL6oyTvPHBn_.8B0PANt9SXHPvpiPHZO8Id_om9c4vas_0 2024-04-17T05:46:03.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9173168413364236204', time=2024-04-17T07:36:34.805167+0000, chain.objs.size()=1 2024-04-17T05:46:03.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.621__shadow_.QRZjjO5i_ftJOMwUu5x6shl4evFPvXM_0 2024-04-17T05:46:03.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14003151752713341557', time=2024-04-17T07:36:37.874440+0000, chain.objs.size()=1 2024-04-17T05:46:03.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.624__shadow_.fRhb_jE8xkGpvwsrwPMa5j53Rmdz9v2_0 2024-04-17T05:46:03.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9466393959993552620', time=2024-04-17T07:36:42.312095+0000, chain.objs.size()=1 2024-04-17T05:46:03.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.627__shadow_.lU6SII0JQkmtwGHSN_gZNojW4t_TYVK_0 2024-04-17T05:46:03.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9032846754438272123', time=2024-04-17T07:36:48.526566+0000, chain.objs.size()=1 2024-04-17T05:46:03.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.630__shadow_.QAXe9UrR65U-MzjvG4x_dOpvetuxnFz_0 2024-04-17T05:46:03.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2126290538203665273', time=2024-04-17T07:37:37.914092+0000, chain.objs.size()=1 2024-04-17T05:46:03.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.649__shadow_.OPlWpnEFfU1TUqXUygr5bbsBexPwnSg_0 2024-04-17T05:46:03.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4601686777425064286', time=2024-04-17T07:37:39.732318+0000, chain.objs.size()=1 2024-04-17T05:46:03.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.650__shadow_.Q1bCQV__2gj9HMYGeOOBIftSHY6Y-N__0 2024-04-17T05:46:03.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8391572248814184626', time=2024-04-17T07:38:09.634751+0000, chain.objs.size()=1 2024-04-17T05:46:03.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.681__shadow:kf79eAgE413-mYNd-orh4pl5yjv8-Ir_.RYBHDw_zESPAGyapNEXYPX-11-dldm7_0 2024-04-17T05:46:03.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4953971917288713401', time=2024-04-17T07:38:13.049840+0000, chain.objs.size()=1 2024-04-17T05:46:03.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.684__shadow:5pKtDLFJfq9jp0Po80TE0tICMvNXFX8_.cINRuJSx6gmAffC9z-po-mJT0PZGkAa_0 2024-04-17T05:46:03.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9214805412732303821', time=2024-04-17T07:38:14.301604+0000, chain.objs.size()=1 2024-04-17T05:46:03.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.432+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.685__shadow:O3RqZ-U1Hi6faXZWLRtIChqYdVwhecQ_.WSMW4fu05EWiGvmWZFP47IxMAPpojNP_0 2024-04-17T05:46:03.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.440+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.440+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-17T05:46:03.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44057' 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4454045635937170280', time=2024-04-17T07:10:54.917719+0000, chain.objs.size()=1 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.166__shadow_._JgAsvqP7TZYZEpmiErL9TFUoocN376_0 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6532614436251348934', time=2024-04-17T07:14:28.135492+0000, chain.objs.size()=1 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.221__shadow_.CPBJa5MU0_GFgzmV3kpOBdQOJIhI3tG_0 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2576858879595458901', time=2024-04-17T07:14:45.014620+0000, chain.objs.size()=1 2024-04-17T05:46:03.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.231__shadow_.qhz6De7XAJ-ZVwfJDnwd4pk6RiFi1lc_0 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11174519677325183558', time=2024-04-17T07:15:18.481051+0000, chain.objs.size()=1 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.251__shadow_.aU6-j5yQZe3oBBwK3JPlF0Eei38nV5O_0 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6140959610818057233', time=2024-04-17T07:17:02.760688+0000, chain.objs.size()=1 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iGOOJxz8Zqlonz4LznJBZnuMkyevcqs_0 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11404754268213138933', time=2024-04-17T07:17:03.987147+0000, chain.objs.size()=1 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sQ5rVBmLcjizt1yQspregnH0y1BEPMQ_0 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7437735078123076213', time=2024-04-17T07:17:04.224990+0000, chain.objs.size()=1 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.1emEb9tk5ezF85K9cZ9JySj6-f7JWA4_0 2024-04-17T05:46:03.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15723541800035302252', time=2024-04-17T07:17:06.076622+0000, chain.objs.size()=1 2024-04-17T05:46:03.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.59kWtsuHzJqB7c-O9iY7hbFS1lhltSt_0 2024-04-17T05:46:03.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17571000398364254351', time=2024-04-17T07:17:06.576583+0000, chain.objs.size()=1 2024-04-17T05:46:03.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tQm4rqiN3t2whPZ17Urx94eemSV2PoJ_0 2024-04-17T05:46:03.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8600987123459091165', time=2024-04-17T07:17:08.609171+0000, chain.objs.size()=1 2024-04-17T05:46:03.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.23v8mYGvxBQRoB_-V6ZkbHvbGnjJpla_0 2024-04-17T05:46:03.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12193174083240511366', time=2024-04-17T07:17:10.003225+0000, chain.objs.size()=1 2024-04-17T05:46:03.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._ro26o8vTbt-RdrqQP2dDI7YND4eXxl_0 2024-04-17T05:46:03.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6783468229117539276', time=2024-04-17T07:17:10.298429+0000, chain.objs.size()=1 2024-04-17T05:46:03.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.444+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.PC7xIeUhoVjn_Ypgo6wLjqIidx0EE4D_0 2024-04-17T05:46:03.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6817702751473669877', time=2024-04-17T07:17:10.507773+0000, chain.objs.size()=1 2024-04-17T05:46:03.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YKjqaEyuQMS7i4A14mc3A6RH1g3hCZr_0 2024-04-17T05:46:03.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2588507504025859516', time=2024-04-17T07:17:11.414793+0000, chain.objs.size()=1 2024-04-17T05:46:03.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zFIDRFE_HNL3xrhwp36VoOXlCYWRW4x_0 2024-04-17T05:46:03.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10326098444084773510', time=2024-04-17T07:17:14.556121+0000, chain.objs.size()=1 2024-04-17T05:46:03.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NmgC_voj8X24huZzv0C65b6jYbw10_q_0 2024-04-17T05:46:03.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16601446659706872570', time=2024-04-17T07:17:15.165859+0000, chain.objs.size()=1 2024-04-17T05:46:03.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._97DdMTqFj0HsSrvpByMZml2DciuiS9_0 2024-04-17T05:46:03.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14009776201048625326', time=2024-04-17T07:17:15.167387+0000, chain.objs.size()=1 2024-04-17T05:46:03.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sqLDOhKyXJLxrOkHS4fLgmKPCHPrwVX_0 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7235982104987364061', time=2024-04-17T07:17:15.417469+0000, chain.objs.size()=1 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MXNDd3wTEXAVBKoThvKHPV2yHGgWmX2_0 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8930445274111136989', time=2024-04-17T07:17:15.917664+0000, chain.objs.size()=1 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fCSNk6EauffGreDyCzxt2cXGs9spHVb_0 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12808929177402396928', time=2024-04-17T07:17:16.964115+0000, chain.objs.size()=1 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.p094ORn58ggjDoCWBc1gCwG4OdPWBk8_0 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.954558301343276231', time=2024-04-17T07:17:17.972972+0000, chain.objs.size()=1 2024-04-17T05:46:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.40K3rW5ZBGDSD44yf5AElPJcIiUPTBn_0 2024-04-17T05:46:03.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11382974307879982994', time=2024-04-17T07:17:20.240324+0000, chain.objs.size()=1 2024-04-17T05:46:03.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.448+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Tv_q4kLo11NAqWqLYpTQ-4zYnrEOeNF_0 2024-04-17T05:46:03.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.452+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6519397167967453180', time=2024-04-17T07:17:20.527529+0000, chain.objs.size()=1 2024-04-17T05:46:03.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.452+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8jkvILqhgqA3WQAAhBohfcIVa9OKTDa_0 2024-04-17T05:46:03.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2986798956269520755', time=2024-04-17T07:17:21.092028+0000, chain.objs.size()=1 2024-04-17T05:46:03.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kPvslVY12KxhBpNg2Emm4bmSJrcUUAb_0 2024-04-17T05:46:03.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3766002616334722409', time=2024-04-17T07:17:22.080804+0000, chain.objs.size()=1 2024-04-17T05:46:03.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.NX4qfIPZ-TF-Vd9WisevSwubQStWTBD_0 2024-04-17T05:46:03.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16541937517317710663', time=2024-04-17T07:17:22.660844+0000, chain.objs.size()=1 2024-04-17T05:46:03.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zuRm-fcVOUUbS0vtPNHn0xW7ZssTW0E_0 2024-04-17T05:46:03.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11573508225395925833', time=2024-04-17T07:17:22.909552+0000, chain.objs.size()=1 2024-04-17T05:46:03.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.15usm9OwP4bau-jfq4cTrkxLphDObns_0 2024-04-17T05:46:03.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12912095094875835078', time=2024-04-17T07:17:23.251280+0000, chain.objs.size()=1 2024-04-17T05:46:03.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gM7BM5gpZr572Yb6csM8uFgvfieiA5W_0 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14070671333787752910', time=2024-04-17T07:17:24.476713+0000, chain.objs.size()=1 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QQ2Qw72zsWOcXanQz5xQdesff3qjJ7x_0 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9516422117664713400', time=2024-04-17T07:17:25.105881+0000, chain.objs.size()=1 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lRopPhfNKSZNa14R8HXmZbS8WhJkeJy_0 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12490738549307844174', time=2024-04-17T07:17:26.032315+0000, chain.objs.size()=1 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Hjv-X3EdXrGamoclDFdR9D_hiGyixUb_0 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15995286292624514892', time=2024-04-17T07:17:30.364534+0000, chain.objs.size()=1 2024-04-17T05:46:03.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZcD209Lfw9yJZGHovRrzNz4inJd0A4S_0 2024-04-17T05:46:03.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4924854025217219008', time=2024-04-17T07:17:31.425423+0000, chain.objs.size()=1 2024-04-17T05:46:03.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SwJ7TN2julpIw_Wej24MWNdziav9n5Z_0 2024-04-17T05:46:03.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13746756796640429183', time=2024-04-17T07:17:32.065681+0000, chain.objs.size()=1 2024-04-17T05:46:03.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.456+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._oQkQn8GU3zUKecerJM8DbBn0w412KE_0 2024-04-17T05:46:03.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17841590751194942165', time=2024-04-17T07:17:51.932344+0000, chain.objs.size()=1 2024-04-17T05:46:03.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.265__shadow_.SKuY9EzrUs1M2tW1SMam-s_NC0NMmqt_0 2024-04-17T05:46:03.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9222760256925809197', time=2024-04-17T07:17:53.594600+0000, chain.objs.size()=1 2024-04-17T05:46:03.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.266__shadow_.lgNJ11ib4OCbiYokluxZ52FsSfCLSPH_0 2024-04-17T05:46:03.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7080226134817200279', time=2024-04-17T07:17:56.867489+0000, chain.objs.size()=1 2024-04-17T05:46:03.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.268__shadow_.2Bw9hSfFckG9y2YVsVmpEElb65QOAiw_0 2024-04-17T05:46:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6558389559182508197', time=2024-04-17T07:17:58.464771+0000, chain.objs.size()=1 2024-04-17T05:46:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.269__shadow_.zCnr-paUxxdcmDuxPkGKpJgBZ93_ZUb_0 2024-04-17T05:46:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9701090419015493156', time=2024-04-17T07:18:04.980660+0000, chain.objs.size()=1 2024-04-17T05:46:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.273__shadow_.Hb3DlVThQ0Vi6h2sEaYkxjMf6p5Pklw_0 2024-04-17T05:46:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12969997546145992851', time=2024-04-17T07:18:18.418928+0000, chain.objs.size()=1 2024-04-17T05:46:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.460+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.281__shadow_.OEfP-znYv6CC-CZD0MxYeZ6tZ_3CFo-_0 2024-04-17T05:46:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16414143959420233776', time=2024-04-17T07:18:22.988326+0000, chain.objs.size()=1 2024-04-17T05:46:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.dzmgz6ZCBaIzjrX2-H80O7QRM8nVFMb_0 2024-04-17T05:46:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9133041802604841583', time=2024-04-17T07:18:22.990574+0000, chain.objs.size()=1 2024-04-17T05:46:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.7pJ2RbIAFY-aLhlVkXHPBXC8nu2x845_0 2024-04-17T05:46:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6653352677765119879', time=2024-04-17T07:18:34.911419+0000, chain.objs.size()=1 2024-04-17T05:46:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.287__shadow_.Suvy4wQ4bchYOGoWd85avZTqu5vHY5g_0 2024-04-17T05:46:03.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17312478035999963057', time=2024-04-17T07:18:41.690072+0000, chain.objs.size()=1 2024-04-17T05:46:03.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.291__shadow_.dYch1kdFaAaLp5wD0o4jLh0BAL0kNKY_0 2024-04-17T05:46:03.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10649014816578974253', time=2024-04-17T07:18:52.952896+0000, chain.objs.size()=1 2024-04-17T05:46:03.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.464+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.297__shadow:Gw.KX5yTqGnSPt.F8.OhSsB-D3rojAL_.sypPzMi-DWq6W6CFS3gQa9e9Il5z5VN_0 2024-04-17T05:46:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8053142509141607893', time=2024-04-17T07:19:11.005999+0000, chain.objs.size()=1 2024-04-17T05:46:03.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.309__shadow:yl2R895poLfL9KLQs0r41iTp0AuMXgz_.gOvM4Z62sXIXPAitMPenHoaKv2PpTG-_0 2024-04-17T05:46:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16357565276821899652', time=2024-04-17T07:21:02.015105+0000, chain.objs.size()=1 2024-04-17T05:46:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kWr1fICkZG-zbRy__V-8Pw1zh1DHo1k_0 2024-04-17T05:46:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8744915195861021086', time=2024-04-17T07:21:04.629370+0000, chain.objs.size()=1 2024-04-17T05:46:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.OfATTq44H1Mvd0b92A8RGclSXhrHOBQ_0 2024-04-17T05:46:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16355422676964798950', time=2024-04-17T07:21:06.010244+0000, chain.objs.size()=1 2024-04-17T05:46:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sSIt1T1PxEGLX-UTg7e8aUqnk2Oe3Cw_0 2024-04-17T05:46:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4556028534065991660', time=2024-04-17T07:21:06.379218+0000, chain.objs.size()=1 2024-04-17T05:46:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DDtMcVfdtpy6cqAGn5oXT8A1K41vwiA_0 2024-04-17T05:46:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7687474202289067869', time=2024-04-17T07:21:06.381490+0000, chain.objs.size()=1 2024-04-17T05:46:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UeP9n_QDOuqisYRqUg_0ji52nlLWDIY_0 2024-04-17T05:46:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11094824374276650487', time=2024-04-17T07:21:09.035195+0000, chain.objs.size()=1 2024-04-17T05:46:03.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4sSEryncg8hSZzt0AuyhZFQqaTR9Byl_0 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6090919671959411295', time=2024-04-17T07:21:09.571269+0000, chain.objs.size()=1 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ah098953aB8WrxnIpDIj5l-FGzkrwTy_0 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16883295202058101522', time=2024-04-17T07:21:10.250259+0000, chain.objs.size()=1 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EInSTi1MKTSpnG3bYn_Ezvsz-SX0zR5_0 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9117935330927314121', time=2024-04-17T07:21:11.834652+0000, chain.objs.size()=1 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Zmvlfv5lWAfse1jM6lZ-N0rM1sNLlPd_0 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15698633584382061353', time=2024-04-17T07:21:12.160383+0000, chain.objs.size()=1 2024-04-17T05:46:03.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.468+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UQvGJzFEQ7DpiD12OBNELiE6OMWsjKP_0 2024-04-17T05:46:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12902075789776711858', time=2024-04-17T07:21:13.914018+0000, chain.objs.size()=1 2024-04-17T05:46:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1SDea352em1JkTBPrZ4Fx-aWUMkonPk_0 2024-04-17T05:46:03.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9082625853596878969', time=2024-04-17T07:21:16.091448+0000, chain.objs.size()=1 2024-04-17T05:46:03.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.f3sWDfr9Lheo1JcqXRRM_PAU-UKwYqY_0 2024-04-17T05:46:03.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7703538970273808369', time=2024-04-17T07:21:17.153756+0000, chain.objs.size()=1 2024-04-17T05:46:03.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._qRNNxQSRmmlQVjoLqWZalyg2zZz__a_0 2024-04-17T05:46:03.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.574123822177526096', time=2024-04-17T07:21:17.742358+0000, chain.objs.size()=1 2024-04-17T05:46:03.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YsNVy32OqAG3cQ_ODXb0TnWY7-lLuAJ_0 2024-04-17T05:46:03.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10531539324043658752', time=2024-04-17T07:21:18.600933+0000, chain.objs.size()=1 2024-04-17T05:46:03.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pLfTsI-5zDY9ypX1_LNiUNXwpP6aXYc_0 2024-04-17T05:46:03.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14701197771443956213', time=2024-04-17T07:21:19.696270+0000, chain.objs.size()=1 2024-04-17T05:46:03.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.S1Dftt_q2aKdGjJGa8_9kwmn4E9H6wp_0 2024-04-17T05:46:03.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12873117265861351854', time=2024-04-17T07:21:20.350665+0000, chain.objs.size()=1 2024-04-17T05:46:03.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HCXAyNwr26aY-fIZmHg740E64UxeqEk_0 2024-04-17T05:46:03.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6023425248852806837', time=2024-04-17T07:21:21.218716+0000, chain.objs.size()=1 2024-04-17T05:46:03.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JuTFW8dj5kIAjvJDB9eDJ9Hk7y-8p5A_0 2024-04-17T05:46:03.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17168015908152844313', time=2024-04-17T07:21:21.220973+0000, chain.objs.size()=1 2024-04-17T05:46:03.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.472+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.V95wdaEgJBF7WaALm9WA9U0B6w-xGUP_0 2024-04-17T05:46:03.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1212088647348258287', time=2024-04-17T07:21:22.509013+0000, chain.objs.size()=1 2024-04-17T05:46:03.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uI63gGTbfBkmNPRPjDo6fn6nCUK_c5g_0 2024-04-17T05:46:03.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11606300976077840888', time=2024-04-17T07:21:23.739947+0000, chain.objs.size()=1 2024-04-17T05:46:03.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yipqt5cFyICVANuwPvtPQeCDmxAKNjX_0 2024-04-17T05:46:03.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9807058624862722237', time=2024-04-17T07:21:25.083107+0000, chain.objs.size()=1 2024-04-17T05:46:03.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wnSmkAzQT9nBROxiFep-nYIVkaHCc_i_0 2024-04-17T05:46:03.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4562969938435651864', time=2024-04-17T07:21:25.322119+0000, chain.objs.size()=1 2024-04-17T05:46:03.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ozbr5BLog1aapz_JWVJ51eJwChLPZ4M_0 2024-04-17T05:46:03.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8687567800442982023', time=2024-04-17T07:21:25.668356+0000, chain.objs.size()=1 2024-04-17T05:46:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.E0EPuXybyLcO7zuJjgo8lubtSRrLQbV_0 2024-04-17T05:46:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7215651728935054876', time=2024-04-17T07:21:27.099712+0000, chain.objs.size()=1 2024-04-17T05:46:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.476+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lg2ycLa8y3Wb4lC0SEMoSdHVvwEPZpe_0 2024-04-17T05:46:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10012127227028044505', time=2024-04-17T07:21:28.408126+0000, chain.objs.size()=1 2024-04-17T05:46:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TvnkduTHopZ95PZCHNUx5TfnRcHQ8JX_0 2024-04-17T05:46:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14844230739887003074', time=2024-04-17T07:21:28.719474+0000, chain.objs.size()=1 2024-04-17T05:46:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vjoj30KiL_R5fhcc8vLlZF5iS_KiGgf_0 2024-04-17T05:46:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12360266253032655158', time=2024-04-17T07:21:29.319495+0000, chain.objs.size()=1 2024-04-17T05:46:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.dtz1u-QMm5i2cLEV1nuF2MGHxaRCYyA_0 2024-04-17T05:46:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4313186567170326596', time=2024-04-17T07:23:12.740408+0000, chain.objs.size()=1 2024-04-17T05:46:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PkbyzdRssJGBEdCbuse9M5tCkaHkKgx_0 2024-04-17T05:46:03.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15431126582636969463', time=2024-04-17T07:23:12.742016+0000, chain.objs.size()=1 2024-04-17T05:46:03.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.w8231xY_rBUYi6K77mFx9dPrpRSAujK_0 2024-04-17T05:46:03.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2844813396812467814', time=2024-04-17T07:23:13.961322+0000, chain.objs.size()=1 2024-04-17T05:46:03.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.V4FRw4vmxwjgijmblblSYgjtRi4ONby_0 2024-04-17T05:46:03.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17854312058544184863', time=2024-04-17T07:23:13.962970+0000, chain.objs.size()=1 2024-04-17T05:46:03.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.480+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._I7L2w5ll6M3_Xp2XSJiUn3bj0OsF9i_0 2024-04-17T05:46:03.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16944700666804896832', time=2024-04-17T07:23:14.230531+0000, chain.objs.size()=1 2024-04-17T05:46:03.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.izyQb-JzEw2QGnrA8XNBi3K8mOO-PJx_0 2024-04-17T05:46:03.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7188837114852071904', time=2024-04-17T07:23:14.451010+0000, chain.objs.size()=1 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.akcsXZxEwzPf1V77By_MZH9cYUyi7Uu_0 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3001276255411778312', time=2024-04-17T07:23:14.455051+0000, chain.objs.size()=1 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PxP_fM7J4jbQ5JXTG40MbqhwgyXSBkG_0 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7064968442600257110', time=2024-04-17T07:23:15.902726+0000, chain.objs.size()=1 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YuCEo9L_kRtjmvxUKdw1kI8LxEcZsjy_0 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9878711762054330046', time=2024-04-17T07:23:17.391399+0000, chain.objs.size()=1 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QRexWPmRM6a0iCUNpXBF5Cj-Y0x6Cx8_0 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4386872987855958789', time=2024-04-17T07:23:18.336421+0000, chain.objs.size()=1 2024-04-17T05:46:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iCCEHzHnVhOqC2_PP9eP-LEjUohcF2C_0 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.953797714665302250', time=2024-04-17T07:23:18.469005+0000, chain.objs.size()=1 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RODOTkOXRqnlqGq8Sz7gJ1DktIwA69R_0 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10272287857358017651', time=2024-04-17T07:23:19.137727+0000, chain.objs.size()=1 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3wbhpPlOFswzNLCapwNzq3uVaqh_mas_0 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7695288254205357306', time=2024-04-17T07:23:20.858029+0000, chain.objs.size()=1 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.9dB1pJUHaGjMdFeqNNQmdfbIaXeU3Y9_0 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17003592701316367400', time=2024-04-17T07:23:21.784647+0000, chain.objs.size()=1 2024-04-17T05:46:03.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.484+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vgcPiStTeZSxY_1-GlqyTX3h__GY3gw_0 2024-04-17T05:46:03.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.488+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6305454368074627200', time=2024-04-17T07:23:21.893848+0000, chain.objs.size()=1 2024-04-17T05:46:03.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.488+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.srSD10gHTQ8cDydU7ba4qSFlUgpOm8N_0 2024-04-17T05:46:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11538102851432745160', time=2024-04-17T07:23:22.143575+0000, chain.objs.size()=1 2024-04-17T05:46:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BW8fCnIez8giRpwA4nykmFRwgB5zLrg_0 2024-04-17T05:46:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8089472706322423887', time=2024-04-17T07:23:22.726964+0000, chain.objs.size()=1 2024-04-17T05:46:03.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bK43uEEzVl3rkTkIwLZuPCioqE_TsKT_0 2024-04-17T05:46:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5603616039638833212', time=2024-04-17T07:23:23.012182+0000, chain.objs.size()=1 2024-04-17T05:46:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kV9JZBzzHU_b9SrPu_miKGZhaQ-XyL0_0 2024-04-17T05:46:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17391688375463662885', time=2024-04-17T07:23:24.941157+0000, chain.objs.size()=1 2024-04-17T05:46:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cQll6SxtpNVcaSb-au9eHpwbbeIfklZ_0 2024-04-17T05:46:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10767433046780720807', time=2024-04-17T07:23:25.804838+0000, chain.objs.size()=1 2024-04-17T05:46:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.492+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.f9layoLab354OPkkzqvCrrl1tzb0tnU_0 2024-04-17T05:46:03.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16717764797675485104', time=2024-04-17T07:23:26.746746+0000, chain.objs.size()=1 2024-04-17T05:46:03.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.D4HtAYyr9iCIAfczEPCcc95rRPCFMLR_0 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9382480087727373376', time=2024-04-17T07:23:27.117020+0000, chain.objs.size()=1 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UUdJYCIMb69tWgiUa1t2Xa2hf-FeQAW_0 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12797088433286453221', time=2024-04-17T07:23:29.402446+0000, chain.objs.size()=1 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.exEL3NwXo6hH67cM_Ou_05ql7cuPvRp_0 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2011764199450620920', time=2024-04-17T07:23:29.988357+0000, chain.objs.size()=1 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lUrwPrWU4mx8LDbfAaIpvwN1rlwKTkG_0 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7197264033332674811', time=2024-04-17T07:23:30.314725+0000, chain.objs.size()=1 2024-04-17T05:46:03.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qE3koQptqkDT5KJ9qzMm8oShX5sLkY-_0 2024-04-17T05:46:03.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2008941065289932718', time=2024-04-17T07:23:30.839263+0000, chain.objs.size()=1 2024-04-17T05:46:03.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.496+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iexaeHKzrKgP4og46aUN2oJ7DDG8EL9_0 2024-04-17T05:46:03.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.500+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/44057 2024-04-17T05:46:03.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=41, truncated=0, next_marker='' 2024-04-17T05:46:03.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7052474731644041064', time=2024-04-17T07:23:31.198287+0000, chain.objs.size()=1 2024-04-17T05:46:03.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.30Ov1rINFYlrZXWhagC1aWdOJBkG36i_0 2024-04-17T05:46:03.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9395442890092591348', time=2024-04-17T07:23:32.460472+0000, chain.objs.size()=1 2024-04-17T05:46:03.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IXWa4ACpWBkj2hzSVIzHWKqZijiore9_0 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11922558035566903407', time=2024-04-17T07:23:32.982879+0000, chain.objs.size()=1 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.KNT8ZoklkvYImLfgrTleJvhyxh3G6Ls_0 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10539612476184620351', time=2024-04-17T07:23:34.225214+0000, chain.objs.size()=1 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.xZySsdph_HZ1dwWuwRbwPGJBh34bTaV_0 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14562958863684968177', time=2024-04-17T07:23:34.398829+0000, chain.objs.size()=1 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yR8C5H9Go8tx2niAmyIoZJ44-_El2SQ_0 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4490710441606103175', time=2024-04-17T07:23:36.098553+0000, chain.objs.size()=1 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.x824AKdlP5FP9bbbDpXc6Wk9kImDVDi_0 2024-04-17T05:46:03.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5282606080606894519', time=2024-04-17T07:23:36.559779+0000, chain.objs.size()=1 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Yp-IKu1_WlKfuESaxO9f4HLbVA0jaEH_0 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10238039807896533603', time=2024-04-17T07:23:36.626010+0000, chain.objs.size()=1 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7VxkHbduHax0azDPu3NXFVPjYO7-3x0_0 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12646757478760994159', time=2024-04-17T07:23:36.916395+0000, chain.objs.size()=1 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.zYkLv5ym-_3q1WyscLOUVyftRNxGM4A_0 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18398191334749989687', time=2024-04-17T07:23:37.252265+0000, chain.objs.size()=1 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NYZHb3EV1q-rR4HTTNmpt5PaYfKCSwN_0 2024-04-17T05:46:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11284201976616109611', time=2024-04-17T07:23:37.903452+0000, chain.objs.size()=1 2024-04-17T05:46:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.nvl4RCCoQliBMdxQ5U2VtwsXj8f5vji_0 2024-04-17T05:46:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4452702511843763742', time=2024-04-17T07:23:39.361874+0000, chain.objs.size()=1 2024-04-17T05:46:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.504+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.PBCvafjFvjgzaoPEuVOvSSK9ot7KaMO_0 2024-04-17T05:46:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9255216567301754555', time=2024-04-17T07:23:40.846670+0000, chain.objs.size()=1 2024-04-17T05:46:03.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kKVzx5gUtSxHpFKqP65UTDF_nX-MoKw_0 2024-04-17T05:46:03.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15642365816899329375', time=2024-04-17T07:23:41.244062+0000, chain.objs.size()=1 2024-04-17T05:46:03.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7Yk2KmzN4htIHUqmobhzM3RXWvqJK8p_0 2024-04-17T05:46:03.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18230893780162004622', time=2024-04-17T07:23:41.301963+0000, chain.objs.size()=1 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TutbJab19pN_zvCQoOgSSJDoiGIqxL2_0 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.434316231910034767', time=2024-04-17T07:24:59.513260+0000, chain.objs.size()=1 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.399__shadow_.neqSn5PQZV9UMbMyWGg2e14fQ0lFLad_0 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18339055090781988059', time=2024-04-17T07:25:35.740166+0000, chain.objs.size()=1 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.437__shadow_.BemFSsgMeRQV8Az1t5ycQ2TgfM7h5JM_0 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13035252802742626556', time=2024-04-17T07:25:38.542916+0000, chain.objs.size()=1 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.440__shadow_.YtFoLUIA3PlZa4hC4I_8XGRou9fx8OH_0 2024-04-17T05:46:03.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15367723132861078924', time=2024-04-17T07:26:00.292199+0000, chain.objs.size()=1 2024-04-17T05:46:03.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.455__shadow_.8c9k3iUE4MHHo8Cb37cpAx0YCvtbWF4_0 2024-04-17T05:46:03.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5193200648099473444', time=2024-04-17T07:26:17.122587+0000, chain.objs.size()=1 2024-04-17T05:46:03.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.508+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.KBMSWNjkHGTJ8MLqWDP4J9NkUMIs8_r_0 2024-04-17T05:46:03.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1924150806678835568', time=2024-04-17T07:26:17.163299+0000, chain.objs.size()=1 2024-04-17T05:46:03.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.19W-YUPDxog7y1162OrjgRgNytVWg21_0 2024-04-17T05:46:03.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18421370041585785317', time=2024-04-17T07:26:26.953124+0000, chain.objs.size()=1 2024-04-17T05:46:03.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.478__shadow_.aTjwmnxXceHlLJ5q0ieACeDqPlahZvP_0 2024-04-17T05:46:03.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3166956568211016848', time=2024-04-17T07:26:33.127861+0000, chain.objs.size()=1 2024-04-17T05:46:03.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.485__shadow_.gR0kZO6gxFVbi-o29wZANI9uu3sD_wh_0 2024-04-17T05:46:03.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_1_e587e76990dff9b6da0fe69f96d36bec', time=2024-04-17T07:26:43.464248+0000, chain.objs.size()=1 2024-04-17T05:46:03.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_0 2024-04-17T05:46:03.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_1_9d4428067c6b846838270a1a15b03d3b', time=2024-04-17T07:26:43.466043+0000, chain.objs.size()=1 2024-04-17T05:46:03.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_0 2024-04-17T05:46:03.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2_d0ee02d650da97b9dedeca62af2832cb', time=2024-04-17T07:26:56.978764+0000, chain.objs.size()=6 2024-04-17T05:46:03.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.1 2024-04-17T05:46:03.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.512+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2 2024-04-17T05:46:03.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.3 2024-04-17T05:46:03.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.4 2024-04-17T05:46:03.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.5 2024-04-17T05:46:03.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.6 2024-04-17T05:46:03.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3686584199109613246', time=2024-04-17T07:27:14.640423+0000, chain.objs.size()=1 2024-04-17T05:46:03.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.508__multipart_mymultipart.2~E9kElZ0Kuu6r8WJ_KpbA82PVo6AjUzm.1 2024-04-17T05:46:03.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3891546560132524698', time=2024-04-17T07:29:15.697237+0000, chain.objs.size()=2 2024-04-17T05:46:03.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~xmHP2-abJyrZslG80XPHTeFYMk5narN.1 2024-04-17T05:46:03.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.513__multipart_mymultipart.2~xmHP2-abJyrZslG80XPHTeFYMk5narN.2 2024-04-17T05:46:03.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.848432625730330019', time=2024-04-17T07:31:36.788952+0000, chain.objs.size()=1 2024-04-17T05:46:03.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.516+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:1Jx.BeLsB3vWUs3LC6ws.HZ-tCuCcfL_.4S8ULQ1fBAit4qQKEl1rZQvlpTLktxf_0 2024-04-17T05:46:03.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9532056721981163505', time=2024-04-17T07:32:09.186992+0000, chain.objs.size()=1 2024-04-17T05:46:03.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:a7wKRjBEc69BeRb6CqrS-WsdViivK9O_.Tzx6LfwjRpSGVI0Zr24BDAl_oVyaq1J_0 2024-04-17T05:46:03.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1672415353206614752', time=2024-04-17T07:32:12.253320+0000, chain.objs.size()=1 2024-04-17T05:46:03.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:lZlCe8bcXiCmXED63.Rt8GpYssbWOPY_.INPsRkSM2rHHhZXPHj_QPzG5bcoMpBA_0 2024-04-17T05:46:03.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9713087316479376305', time=2024-04-17T07:32:28.493323+0000, chain.objs.size()=1 2024-04-17T05:46:03.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:zdeTEgomkWQCUBJ8Ch.GOuZNPA4uiws_.rrbXa9PU_uT-etRYTwQ9ne0IJILdwYJ_0 2024-04-17T05:46:03.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17044222155749350136', time=2024-04-17T07:32:45.437333+0000, chain.objs.size()=1 2024-04-17T05:46:03.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:at-Ie1DlPAVSFXEjufqGb6DPLpV5DRq_.f0uY57_IW8mUSQEU4PgdtltyHO6BK-g_0 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9297309195398467435', time=2024-04-17T07:32:57.405266+0000, chain.objs.size()=1 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:ke9EYyjDKkTqVABZSLwkFcChG.0HFpM_.CL8us7Q49xgSi4TGAaxTNrrs9VgUaGq_0 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9762326944196651862', time=2024-04-17T07:33:05.078144+0000, chain.objs.size()=1 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:k..HPZJofZXjLZ12huxh5uiBzM2HjkO_.sIsvReZ04s2eyIW9Z6FLsX5T8_bVk7B_0 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1997266082438955175', time=2024-04-17T07:33:46.350817+0000, chain.objs.size()=1 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.576__shadow:JglXAcjpP4pXUJ2uiHLRV21DqaQOMX-_.DT1_YP5g2HGryN_wQU6LSa82_dg0Tyc_0 2024-04-17T05:46:03.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5954195306075005969', time=2024-04-17T07:35:25.696920+0000, chain.objs.size()=6 2024-04-17T05:46:03.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.603__multipart_multipart_enc.2~VTwzFFs6fzjPkL5YoyCT0wOiLTej9qu.1 2024-04-17T05:46:03.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.603__multipart_multipart_enc.2~VTwzFFs6fzjPkL5YoyCT0wOiLTej9qu.2 2024-04-17T05:46:03.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.603__multipart_multipart_enc.2~VTwzFFs6fzjPkL5YoyCT0wOiLTej9qu.3 2024-04-17T05:46:03.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.520+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.603__multipart_multipart_enc.2~VTwzFFs6fzjPkL5YoyCT0wOiLTej9qu.4 2024-04-17T05:46:03.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.524+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.603__multipart_multipart_enc.2~VTwzFFs6fzjPkL5YoyCT0wOiLTej9qu.5 2024-04-17T05:46:03.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.524+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.603__multipart_multipart_enc.2~VTwzFFs6fzjPkL5YoyCT0wOiLTej9qu.6 2024-04-17T05:46:03.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.524+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx', time=2024-04-17T07:36:16.715096+0000, chain.objs.size()=6 2024-04-17T05:46:03.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.524+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.611__multipart_multipart_enc.2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx.1 2024-04-17T05:46:03.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.611__multipart_multipart_enc.2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx.2 2024-04-17T05:46:03.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.611__multipart_multipart_enc.2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx.3 2024-04-17T05:46:03.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.611__multipart_multipart_enc.2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx.4 2024-04-17T05:46:03.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.611__multipart_multipart_enc.2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx.5 2024-04-17T05:46:03.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.611__multipart_multipart_enc.2~6sPgdt1ZO8eRhUS6zeN68ivCcbO2NFx.6 2024-04-17T05:46:03.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12037247736826891117', time=2024-04-17T07:36:39.756054+0000, chain.objs.size()=1 2024-04-17T05:46:03.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.626__shadow_.frSzdcAAkekvW2tu_WK_TS8-_osi9cI_0 2024-04-17T05:46:03.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.827431018346395730', time=2024-04-17T07:36:59.302091+0000, chain.objs.size()=1 2024-04-17T05:46:03.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.635__shadow_.YFc_xSPZeHkkJIEZgTZebZMr1-yhPyP_0 2024-04-17T05:46:03.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6428865068674802508', time=2024-04-17T07:38:25.604539+0000, chain.objs.size()=1 2024-04-17T05:46:03.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.528+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.694__shadow_.30YyAyBqyyRJx-Iq6y95V1C-phFXVcA_0 2024-04-17T05:46:03.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.536+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-17T05:46:03.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.540+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42749' 2024-04-17T05:46:03.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17530515863704341414', time=2024-04-17T07:09:56.600485+0000, chain.objs.size()=1 2024-04-17T05:46:03.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.148__shadow_.cd_DkAfooVwHocWdUkBGQ7cXQ90-hy__0 2024-04-17T05:46:03.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13308606159721318572', time=2024-04-17T07:10:08.613837+0000, chain.objs.size()=2 2024-04-17T05:46:03.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.B9JEc5aU4EuDDXQKx6vjVPpw_EERRJp_0 2024-04-17T05:46:03.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.B9JEc5aU4EuDDXQKx6vjVPpw_EERRJp_1 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9474906589512252469', time=2024-04-17T07:10:29.631672+0000, chain.objs.size()=1 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.152__shadow_.hA5GIO7tohpTgs7mhs8bbZW4JCvqT-R_0 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9270857354272025653', time=2024-04-17T07:11:37.990417+0000, chain.objs.size()=1 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.190__shadow_.Ldbv2TQyXAmTWaqWYSHn7XgSydmBS10_0 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12490445344628477919', time=2024-04-17T07:14:38.613540+0000, chain.objs.size()=1 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.227__shadow_.V4AMmDQ9IfYkLJXN1C4NLwg9rbtoeII_0 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10530537117794683223', time=2024-04-17T07:17:01.309340+0000, chain.objs.size()=1 2024-04-17T05:46:03.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Snc6DcaezFTmafbQAkE-6Ws8-FCIcWO_0 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9076987118707074883', time=2024-04-17T07:17:02.096654+0000, chain.objs.size()=1 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.2As3B29x2p-EE3b-e_L2v8nOFxBm7Ut_0 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10421477632290185742', time=2024-04-17T07:17:02.544387+0000, chain.objs.size()=1 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.flVsUYZ8kpXtq5vrsos_2HrVslnG6s__0 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17068049727699569681', time=2024-04-17T07:17:03.794466+0000, chain.objs.size()=1 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VjrxFUQMxsWEkLmafmRDJWt8YWG3oFc_0 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11195448069976173718', time=2024-04-17T07:17:05.723802+0000, chain.objs.size()=1 2024-04-17T05:46:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YvvszjpwRBx7p_ex_YVO0kR6wAXVXrd_0 2024-04-17T05:46:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4824476931289731479', time=2024-04-17T07:17:06.032705+0000, chain.objs.size()=1 2024-04-17T05:46:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.u80viPCh9QJlFPvB-c-7_Odvec_-MfD_0 2024-04-17T05:46:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13785027717898244808', time=2024-04-17T07:17:06.426161+0000, chain.objs.size()=1 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.544+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EHW41eM1aiX8xFgzycMvCiLV_bd7Qc8_0 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3598518935734975381', time=2024-04-17T07:17:12.680854+0000, chain.objs.size()=1 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pEN98tLxCou1DjwsJFIyrWKZoAev6lV_0 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6118553115080187673', time=2024-04-17T07:17:12.954389+0000, chain.objs.size()=1 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zk2lg3WuWQ6ZWh00If8_gW3PrvCSP-o_0 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15454745084680570387', time=2024-04-17T07:17:14.041455+0000, chain.objs.size()=1 2024-04-17T05:46:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QHGXK-PyqDAEOq8txK8imlxLQ0ZeZD__0 2024-04-17T05:46:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10815320837995235199', time=2024-04-17T07:17:14.648344+0000, chain.objs.size()=1 2024-04-17T05:46:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OxbRqlHow6cxNX2ZzPEmgr-owpk-KGV_0 2024-04-17T05:46:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2416196807200085119', time=2024-04-17T07:17:14.674660+0000, chain.objs.size()=1 2024-04-17T05:46:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.GG3KXTu6nmno3bMSIL0WIuFCuDKs21g_0 2024-04-17T05:46:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9810705920304811838', time=2024-04-17T07:17:14.989499+0000, chain.objs.size()=1 2024-04-17T05:46:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wrCZXB1QEL8kNudoHdydnYYmVOwMHW__0 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8535138198952193810', time=2024-04-17T07:17:15.436490+0000, chain.objs.size()=1 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JdYwmizmftrPgd5RuclcVMq_VwwLRr1_0 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.861003797027885775', time=2024-04-17T07:17:16.216608+0000, chain.objs.size()=1 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dcKPpyQ-Iouty9-jVK48jUn_dWszBcH_0 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16113199286837151883', time=2024-04-17T07:17:16.798306+0000, chain.objs.size()=1 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.updMlWi7F5nGNB0GE5WawRtFXBh84l5_0 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2189568930438159004', time=2024-04-17T07:17:21.919207+0000, chain.objs.size()=1 2024-04-17T05:46:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.E6k5zpejoGZQabiDQXgiEC8wRQHF6IY_0 2024-04-17T05:46:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7059401000430508524', time=2024-04-17T07:17:22.494731+0000, chain.objs.size()=1 2024-04-17T05:46:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.548+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.OkOyS6XVWLEApxzhZeQsLyrT_EwQh5q_0 2024-04-17T05:46:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15871924469134666670', time=2024-04-17T07:17:23.209934+0000, chain.objs.size()=1 2024-04-17T05:46:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DWz07GUPBkDK-ThHzCJzAqkkRMttjMG_0 2024-04-17T05:46:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1072350990433035709', time=2024-04-17T07:17:23.521158+0000, chain.objs.size()=1 2024-04-17T05:46:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kDhhkEymJJWjP6sXr2WDwyXpwtcfs1p_0 2024-04-17T05:46:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13353666515331737403', time=2024-04-17T07:17:25.125879+0000, chain.objs.size()=1 2024-04-17T05:46:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._GWXtw0mfV5od7Lc92GhPEld2kulEVo_0 2024-04-17T05:46:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4314652358637108947', time=2024-04-17T07:17:26.465691+0000, chain.objs.size()=1 2024-04-17T05:46:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MzCZayhNTBIq6tZn7HjvC6_o3JyYqvP_0 2024-04-17T05:46:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15871642888875252504', time=2024-04-17T07:17:27.171585+0000, chain.objs.size()=1 2024-04-17T05:46:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.552+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MdzxKjs1AyIS7mVjHBCwloXmKV94b8V_0 2024-04-17T05:46:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9759217176707966779', time=2024-04-17T07:17:27.458231+0000, chain.objs.size()=1 2024-04-17T05:46:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.F4T49uNjOH-2ZVapPvU5JkQYt__FAma_0 2024-04-17T05:46:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.875745550113461629', time=2024-04-17T07:17:28.678803+0000, chain.objs.size()=1 2024-04-17T05:46:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Aqkuc4_F0B-A9QD-8n3QWzBCz5xt3GM_0 2024-04-17T05:46:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16865964403098716731', time=2024-04-17T07:17:29.005801+0000, chain.objs.size()=1 2024-04-17T05:46:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.I6zFcEqh6sHbdT_FDx_yrxkpddaXxMM_0 2024-04-17T05:46:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4789284290868942245', time=2024-04-17T07:17:30.645292+0000, chain.objs.size()=1 2024-04-17T05:46:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.hQ8o4SNKQ_ENYQ6L0dIdMjj_RIuwG0h_0 2024-04-17T05:46:03.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17799010279715288569', time=2024-04-17T07:17:31.196579+0000, chain.objs.size()=1 2024-04-17T05:46:03.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EeIuN6cFAQeqY39XI7KWgRKs88wZ1Df_0 2024-04-17T05:46:03.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7036770915535474416', time=2024-04-17T07:17:31.830084+0000, chain.objs.size()=1 2024-04-17T05:46:03.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BXZjwNreX8T-_j7d2F_3fS1rYlf3-kv_0 2024-04-17T05:46:03.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11128149011428280029', time=2024-04-17T07:17:35.442195+0000, chain.objs.size()=1 2024-04-17T05:46:03.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.556+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.254__shadow_.sfnGJywswlvRyGRvIc9cuLkXuOGdcz3_0 2024-04-17T05:46:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12073833349121618731', time=2024-04-17T07:18:04.958301+0000, chain.objs.size()=1 2024-04-17T05:46:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.273__shadow_.dlZZGZbNx4tJdXINEk2jqjbGDB4HQSx_0 2024-04-17T05:46:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11856659119396943859', time=2024-04-17T07:18:22.982000+0000, chain.objs.size()=1 2024-04-17T05:46:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.vDpZAOXcZb13f7H44Ow-Hal8NMS4oEY_0 2024-04-17T05:46:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11270134527935323910', time=2024-04-17T07:18:22.985844+0000, chain.objs.size()=1 2024-04-17T05:46:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.UWhoLk4pHQhsRUYbU5WYVzMGWkspTWU_0 2024-04-17T05:46:03.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10112743498818501715', time=2024-04-17T07:18:22.990060+0000, chain.objs.size()=1 2024-04-17T05:46:03.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.5q1xdnVVk7kOV8QNgtb6W4MMZz2LEf3_0 2024-04-17T05:46:03.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3689818545247077312', time=2024-04-17T07:18:28.213347+0000, chain.objs.size()=1 2024-04-17T05:46:03.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.283__shadow_.2Sr32B4S0Oq-rwyFGtFWZtd3iKMWLu5_0 2024-04-17T05:46:03.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3514208903322712156', time=2024-04-17T07:18:33.116733+0000, chain.objs.size()=1 2024-04-17T05:46:03.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.286__shadow_.GhX5hPtWLLtmaqrA_uAgeSXhFTw-C8h_0 2024-04-17T05:46:03.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10468167105366856192', time=2024-04-17T07:21:00.302508+0000, chain.objs.size()=1 2024-04-17T05:46:03.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.yauOUJe7WDmuN3qChwTMjjnz-KWbz9J_0 2024-04-17T05:46:03.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10665249819072124516', time=2024-04-17T07:21:00.865527+0000, chain.objs.size()=1 2024-04-17T05:46:03.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.560+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HmTjMViVkrqL0elO1WXIxDflm56AR6a_0 2024-04-17T05:46:03.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16025215467417194043', time=2024-04-17T07:21:02.496439+0000, chain.objs.size()=1 2024-04-17T05:46:03.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PzPUM_9PV80Ian6VUJkCvMbIGdoDz5N_0 2024-04-17T05:46:03.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17492447939221863326', time=2024-04-17T07:21:02.517169+0000, chain.objs.size()=1 2024-04-17T05:46:03.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TjOpJv3peM2Yee0_8eEXWez-9JYeXSU_0 2024-04-17T05:46:03.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6667886907144403272', time=2024-04-17T07:21:04.006209+0000, chain.objs.size()=1 2024-04-17T05:46:03.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sCJ4ajcDrVvIxFQXnrjMah8h4MklinJ_0 2024-04-17T05:46:03.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17746419859795985474', time=2024-04-17T07:21:04.584794+0000, chain.objs.size()=1 2024-04-17T05:46:03.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vu3AfIHPD7FVDbCHgLCtjLvnJZBoLVX_0 2024-04-17T05:46:03.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3151925568289409618', time=2024-04-17T07:21:04.826500+0000, chain.objs.size()=1 2024-04-17T05:46:03.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HUAqPLYkPxM5ahC1Q-KoQd57aTb5DZb_0 2024-04-17T05:46:03.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14389551184972726002', time=2024-04-17T07:21:05.072832+0000, chain.objs.size()=1 2024-04-17T05:46:03.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.564+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kKTE9YxlrkR4x62_FK0ohmTOT6GijKu_0 2024-04-17T05:46:03.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13853854982866103647', time=2024-04-17T07:21:05.075830+0000, chain.objs.size()=1 2024-04-17T05:46:03.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DMSVkAuMlDyr1f01lYfUEH1dYOgnG6v_0 2024-04-17T05:46:03.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16167512341424933388', time=2024-04-17T07:21:05.263311+0000, chain.objs.size()=1 2024-04-17T05:46:03.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SCJ5plkuotGz2UvkoyzBC7h53YWvhCE_0 2024-04-17T05:46:03.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9873619624012360303', time=2024-04-17T07:21:06.001431+0000, chain.objs.size()=1 2024-04-17T05:46:03.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3YYZwE9kUkdpk8R8KHdeUPUCrupqp37_0 2024-04-17T05:46:03.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5352336259992750366', time=2024-04-17T07:21:08.420509+0000, chain.objs.size()=1 2024-04-17T05:46:03.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kZ9ZuaPRw84Jvw0CjlQd0OyXc9iq-Qx_0 2024-04-17T05:46:03.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15833178351124078046', time=2024-04-17T07:21:09.972791+0000, chain.objs.size()=1 2024-04-17T05:46:03.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PvJ0sNSNgvt7QkwRO8mCQ-eTPh5YPc7_0 2024-04-17T05:46:03.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1956495620978728006', time=2024-04-17T07:21:10.419071+0000, chain.objs.size()=1 2024-04-17T05:46:03.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LhZt9cf9VoX2VV7P2EUTc2fSkPy6NA9_0 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12039285978899478284', time=2024-04-17T07:21:12.236283+0000, chain.objs.size()=1 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oDtCpsKvWTONgQUWKm4pXmwNcZgt8rw_0 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14209626362080351915', time=2024-04-17T07:21:13.093458+0000, chain.objs.size()=1 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._Bf5Y1e4c6SFGDk5Fl1lcyNb_AqpPXX_0 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7753451266000495281', time=2024-04-17T07:21:13.387587+0000, chain.objs.size()=1 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.568+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zpc1sKq7OyMIWMc34C6XB3gUXYqdxGo_0 2024-04-17T05:46:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18195705497019634525', time=2024-04-17T07:21:16.116922+0000, chain.objs.size()=1 2024-04-17T05:46:03.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vysuq_gUh9fY4Cu2nyXwjZgIQ8nNw5Q_0 2024-04-17T05:46:03.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14031048666222211952', time=2024-04-17T07:21:17.044618+0000, chain.objs.size()=1 2024-04-17T05:46:03.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.jQ33q7wipDDNwpLnTF7ksyO-zG8IiAR_0 2024-04-17T05:46:03.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13936965488097767891', time=2024-04-17T07:21:17.154447+0000, chain.objs.size()=1 2024-04-17T05:46:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KitESHLQVcgov5v2O6nHIG8BuzZK0bY_0 2024-04-17T05:46:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4639412381745349169', time=2024-04-17T07:21:18.533810+0000, chain.objs.size()=1 2024-04-17T05:46:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.o8PwRf9O9McfWEOh9rjiUc7blpt6L1W_0 2024-04-17T05:46:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.188210938358292402', time=2024-04-17T07:21:19.577117+0000, chain.objs.size()=1 2024-04-17T05:46:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.572+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9NdgTd8OEkS7zNr9ibUeSYhQUcnW1WK_0 2024-04-17T05:46:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5314235629078356282', time=2024-04-17T07:21:19.988656+0000, chain.objs.size()=1 2024-04-17T05:46:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.kLTgfxhJbM5IKgwXhlssTL6R_Q-IhRh_0 2024-04-17T05:46:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17472318436066943661', time=2024-04-17T07:21:20.629092+0000, chain.objs.size()=1 2024-04-17T05:46:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.5Dt0pM2cL9BaaayRL_2K6BA9F2guarO_0 2024-04-17T05:46:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15320225181698815334', time=2024-04-17T07:21:21.121813+0000, chain.objs.size()=1 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.H0K1rpKSASoc5oFs7Bv2tL3hwYWsBLa_0 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17354594576514220092', time=2024-04-17T07:21:22.564940+0000, chain.objs.size()=1 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XTpW_Sz1jIWOa4afPX_LfwI0wzojYmm_0 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6551344140628390132', time=2024-04-17T07:21:24.200051+0000, chain.objs.size()=1 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3-shradXOyRCa9kEMtzSsZ27MlD0Hcf_0 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11486794216551409564', time=2024-04-17T07:21:24.417669+0000, chain.objs.size()=1 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8jhlM13z_FLGl_CnalxRh86aUmpp603_0 2024-04-17T05:46:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11914380682408944536', time=2024-04-17T07:21:24.719612+0000, chain.objs.size()=1 2024-04-17T05:46:03.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.576+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LAzT1XMRrL6fZgj31PHFTFR_57SPG7u_0 2024-04-17T05:46:03.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7633399427909066691', time=2024-04-17T07:21:25.255655+0000, chain.objs.size()=1 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.242g4LLCG5FL5_4tOE2gCYBNUrt-p_r_0 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13164761590727019909', time=2024-04-17T07:21:25.912615+0000, chain.objs.size()=1 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IzFMnWO0w7I_TZdLHTiJFyazZoT1CwD_0 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8098820357032315956', time=2024-04-17T07:21:27.506735+0000, chain.objs.size()=1 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.oxH2XZCB8uYYWJEWSeKwOWmMKFOAUts_0 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5425576930451837902', time=2024-04-17T07:21:28.680853+0000, chain.objs.size()=1 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xYuaQcQQ8DdOY17QRe--RKgkzlk1xpR_0 2024-04-17T05:46:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4412990119561779185', time=2024-04-17T07:21:29.377751+0000, chain.objs.size()=1 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lXoIwWqiI1oShIoeV81zg3LbSMvUz9O_0 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14322799337457813020', time=2024-04-17T07:23:12.749458+0000, chain.objs.size()=1 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rKjqxEZJsMwPyvy5PyPXx0wgwY5AAJw_0 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9273330332041334040', time=2024-04-17T07:23:13.085906+0000, chain.objs.size()=1 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QrXKHs7CFemM5aur5B4p4Q0PPYePzv2_0 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9581736456293810041', time=2024-04-17T07:23:13.588998+0000, chain.objs.size()=1 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.r-JBpop3r-nWUPs_c4diF9MKDFBtC60_0 2024-04-17T05:46:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16565895223500381102', time=2024-04-17T07:23:13.944104+0000, chain.objs.size()=1 2024-04-17T05:46:03.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.bzcPHVSLvz7-rkpjRTYsnSkoM1I8i4V_0 2024-04-17T05:46:03.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1479069642479859198', time=2024-04-17T07:23:19.617374+0000, chain.objs.size()=1 2024-04-17T05:46:03.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.iG4GTwj2VAOS4TtBaefJrFNWtAVy0L2_0 2024-04-17T05:46:03.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13617593614493955738', time=2024-04-17T07:23:20.907273+0000, chain.objs.size()=1 2024-04-17T05:46:03.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.580+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._3CIgueXHQXRe3-uPn1M8NLdwqh95aF_0 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10728443131633688099', time=2024-04-17T07:23:21.393780+0000, chain.objs.size()=1 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UNo-ybTM2da1cIdhQIMoX2URa48GvEo_0 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2795062683314865528', time=2024-04-17T07:23:23.014506+0000, chain.objs.size()=1 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XItKuFBNJ1XSWPgF48z4CeOb0L3clqq_0 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10235843129683025455', time=2024-04-17T07:23:23.094967+0000, chain.objs.size()=1 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.QQ2KxPN8fYmg44N76fOxUrf_g-NmPYu_0 2024-04-17T05:46:03.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14466753529907752780', time=2024-04-17T07:23:23.526742+0000, chain.objs.size()=1 2024-04-17T05:46:03.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._cUWqLmUAPA5MIw25wWP5CH-UofZX0o_0 2024-04-17T05:46:03.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10138348786676318667', time=2024-04-17T07:23:23.818168+0000, chain.objs.size()=1 2024-04-17T05:46:03.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.584+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TBS6_iMwdSTlBwqYFu1-W50uhslz8dC_0 2024-04-17T05:46:03.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4666445327904404551', time=2024-04-17T07:23:26.051917+0000, chain.objs.size()=1 2024-04-17T05:46:03.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kX1xeqBs5cXOMtKWhMN8RY5eqnK-RTx_0 2024-04-17T05:46:03.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2773313056425608214', time=2024-04-17T07:23:27.387167+0000, chain.objs.size()=1 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0-XlEynYdfunWACUvL7h1otqZDAz_dp_0 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1666468950221893456', time=2024-04-17T07:23:28.543706+0000, chain.objs.size()=1 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OcomlPSxKxHEACFCK0-gg7ox-8kYD5V_0 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14496737586222008799', time=2024-04-17T07:23:28.878437+0000, chain.objs.size()=1 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.U7CICu2rWPK9UqJvfPagIuzsAN3Vxkz_0 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2144306928830187573', time=2024-04-17T07:23:31.331030+0000, chain.objs.size()=1 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MhQClGOnhJoYY1BIXmzaBKe7zKjCWe4_0 2024-04-17T05:46:03.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9398717126050969889', time=2024-04-17T07:23:32.164659+0000, chain.objs.size()=1 2024-04-17T05:46:03.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.588+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.upIQeku_hc-5WICYbHIiIh8U-xrtyPx_0 2024-04-17T05:46:03.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.347311180919108414', time=2024-04-17T07:23:32.970749+0000, chain.objs.size()=1 2024-04-17T05:46:03.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.sghMQr2ldY1V95qIcpaWQ65f9DQWE53_0 2024-04-17T05:46:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12384144529616070486', time=2024-04-17T07:23:33.402644+0000, chain.objs.size()=1 2024-04-17T05:46:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Nm60Ms3KpRXyVxOChLvzQQaG7IiiRBE_0 2024-04-17T05:46:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.275122897352081658', time=2024-04-17T07:23:33.866825+0000, chain.objs.size()=1 2024-04-17T05:46:03.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.E_LkTF4Ix0ebZciIsGwAFSFABVo_vs6_0 2024-04-17T05:46:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.400705509043040665', time=2024-04-17T07:23:35.006477+0000, chain.objs.size()=1 2024-04-17T05:46:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.HCiKkQAeP-mTQZjjd7csukBpymyNUGV_0 2024-04-17T05:46:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.72426301779751817', time=2024-04-17T07:23:35.862958+0000, chain.objs.size()=1 2024-04-17T05:46:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3z_eDs4HUWJZuaf8a0u_FFfGds0nysZ_0 2024-04-17T05:46:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7622871500212636333', time=2024-04-17T07:23:37.013757+0000, chain.objs.size()=1 2024-04-17T05:46:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Dpds-MmE9HY4KidJl1uZ-s4gad014uT_0 2024-04-17T05:46:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8436844023696717660', time=2024-04-17T07:23:37.306801+0000, chain.objs.size()=1 2024-04-17T05:46:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.FBi4-KKmeWSPG57sfoJtP_CMFDVYYQa_0 2024-04-17T05:46:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15201979217540923887', time=2024-04-17T07:23:38.220516+0000, chain.objs.size()=1 2024-04-17T05:46:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.592+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uzn3Y5z5mRwsVY3oD05JZFzSV_61yXR_0 2024-04-17T05:46:03.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.596+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/42749 2024-04-17T05:46:03.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=20, truncated=0, next_marker='' 2024-04-17T05:46:03.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6669896286810902119', time=2024-04-17T07:23:38.684730+0000, chain.objs.size()=1 2024-04-17T05:46:03.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NjY5MK4ZABNc-XQoTubCMgCXenGWhOy_0 2024-04-17T05:46:03.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4712952871185198200', time=2024-04-17T07:23:39.442905+0000, chain.objs.size()=1 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.z3Mlna8Gf5IhN-oWyVl2MBllTsPX7l4_0 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8538877282858662343', time=2024-04-17T07:23:40.404456+0000, chain.objs.size()=1 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZvTEWlCsl24C-0YlywYdXB4BgqVucas_0 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2906119380748295158', time=2024-04-17T07:23:40.592846+0000, chain.objs.size()=1 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oBZNeg9IxByKJedFDtkeJ7PBVSAfzbP_0 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13946090223965620848', time=2024-04-17T07:26:12.396734+0000, chain.objs.size()=1 2024-04-17T05:46:03.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.471__shadow_.GtbnEk4kqnc9NmQ-sYaRHbFQMa4bCM9_0 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1839998117329762747', time=2024-04-17T07:26:18.774881+0000, chain.objs.size()=1 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.474__shadow_.QmWpj7EzUXpbEjleFkWf2Hl9VIZuYyv_0 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.481__shadow_.eL2eHW3eUU-sEozEsPNPA5YYY7CNvn6_1_4f9bdbddf4079fd422b5203a2408f2b7', time=2024-04-17T07:26:30.640255+0000, chain.objs.size()=1 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.481__shadow_.eL2eHW3eUU-sEozEsPNPA5YYY7CNvn6_0 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9270919243087580660', time=2024-04-17T07:26:36.345225+0000, chain.objs.size()=1 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.487__shadow_.gfcCpZ5c8UYRyumKrZBGQ2gfrms4cZS_0 2024-04-17T05:46:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8496677363491800324', time=2024-04-17T07:31:05.771892+0000, chain.objs.size()=1 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.546__shadow_.mmalu73KhSfrFrfCMqEAHd55tF7EFWg_0 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~kOfQ8xq5KXiDOghZ0Mt8qvjn8IIqpeI', time=2024-04-17T07:31:19.808855+0000, chain.objs.size()=1 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.550__multipart_mymultipart.2~kOfQ8xq5KXiDOghZ0Mt8qvjn8IIqpeI.1 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18017243542845909450', time=2024-04-17T07:31:51.583583+0000, chain.objs.size()=1 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:N1n0hHTXQxmcveqpTzP5z1PwxhpPUVl_.wj3mFRjY9w_vCMd0YBliXDx-3oOWHYB_0 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12318567130771466351', time=2024-04-17T07:32:08.140412+0000, chain.objs.size()=1 2024-04-17T05:46:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.600+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:UIY9a0oKbLKQGF8tswaAxPlW2T.zaY7_.C86bBKrSWrDpeXJdlhWljJNyAOpmVmI_0 2024-04-17T05:46:03.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16509196021029107359', time=2024-04-17T07:32:35.222477+0000, chain.objs.size()=1 2024-04-17T05:46:03.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:m1IkM4v0xRBy4e4kyXQNUYRa9mlyufU_.QwhvMGsybDF3J6MwpIvpekR_0qnla75_0 2024-04-17T05:46:03.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9081367925141443847', time=2024-04-17T07:32:44.622015+0000, chain.objs.size()=1 2024-04-17T05:46:03.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:zGAECd2ze09aLUSV1rKMxjcPcpxdA5B_.kek0NOfyIRln8a4_JP3aqD87AuOKbC9_0 2024-04-17T05:46:03.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15432632714899778225', time=2024-04-17T07:33:05.897651+0000, chain.objs.size()=1 2024-04-17T05:46:03.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:i.dNy8F3rO5lXUgvSloK9U-tQSQYoSL_.EFKtHOYGIcmJ_OBhGl1wnDyBXBxA5WV_0 2024-04-17T05:46:03.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10042863176938457567', time=2024-04-17T07:33:07.192112+0000, chain.objs.size()=1 2024-04-17T05:46:03.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:IYd36UX2BZlXok3XIq0T0PxUK74.a.7_.VhSqqLDMD7AsEaSmGvpnAPjq0LFU4KY_0 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16993036042211376355', time=2024-04-17T07:33:22.135500+0000, chain.objs.size()=4 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~3jWehLLzzqgWCc4nFIN4S6xWIUL4Zxs.1 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~3jWehLLzzqgWCc4nFIN4S6xWIUL4Zxs.2 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~3jWehLLzzqgWCc4nFIN4S6xWIUL4Zxs.3 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.567__multipart_testobj.2~3jWehLLzzqgWCc4nFIN4S6xWIUL4Zxs.4 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7640169749507638197', time=2024-04-17T07:36:29.584480+0000, chain.objs.size()=1 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.616__shadow_.NTZ_nBbm6CrzXfG_rJBptfrKh8RkkO1_0 2024-04-17T05:46:03.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14783130288924565066', time=2024-04-17T07:37:53.946817+0000, chain.objs.size()=1 2024-04-17T05:46:03.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.664__shadow_.pcGdEETs19_bXTFcbrQhXNPsPSiy6BO_0 2024-04-17T05:46:03.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2990383235458317524', time=2024-04-17T07:38:10.555172+0000, chain.objs.size()=1 2024-04-17T05:46:03.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.604+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.682__shadow:Q1Rj0dIybF1IYiIS2frLGThifMZUZRh_.xg_l8M5lfg5IWI-IgEOluopRNsCLWaK_0 2024-04-17T05:46:03.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.612+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.612+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-17T05:46:03.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43909' 2024-04-17T05:46:03.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6271382954820420114', time=2024-04-17T07:14:45.013374+0000, chain.objs.size()=1 2024-04-17T05:46:03.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.231__shadow_.3PlseF3AM0dwjL0e5GyVXNVGEgm9Rry_0 2024-04-17T05:46:03.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10268515534439537584', time=2024-04-17T07:15:04.252057+0000, chain.objs.size()=1 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.242__shadow_.RKSkWQHKrfqZ5C6Bue9owf15vHRLrVq_0 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9379715432151520817', time=2024-04-17T07:15:16.834025+0000, chain.objs.size()=1 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.250__shadow_.NNRveeNWOWwKMkCm3AT_iq1jrmSQkIA_0 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9445726398452616546', time=2024-04-17T07:17:03.051811+0000, chain.objs.size()=1 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.XWHNiXEiSM6i2tIWm39qd_uavHbVO-g_0 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12246045286696543758', time=2024-04-17T07:17:05.822099+0000, chain.objs.size()=1 2024-04-17T05:46:03.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.p78FFapkMoonVBFJX79vCAAtMkAyYpQ_0 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.192614068059870301', time=2024-04-17T07:17:06.619329+0000, chain.objs.size()=1 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wY9mFdymYJHTs4v9IfFzu-b9W2jpqrT_0 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2569821690263076732', time=2024-04-17T07:17:07.882668+0000, chain.objs.size()=1 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gx84uaxKCOx0T8m0nHgXU0lvzc-cp4K_0 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8671580035382001550', time=2024-04-17T07:17:08.788845+0000, chain.objs.size()=1 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.i89R8Iv_IL4btxMt8Pe7VYqYGMAGU_m_0 2024-04-17T05:46:03.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5133005909748122704', time=2024-04-17T07:17:09.849889+0000, chain.objs.size()=1 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yyOoG-ckijO-xl3vbMHWs6H70oGtJAD_0 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5936441372216347064', time=2024-04-17T07:17:09.964965+0000, chain.objs.size()=1 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nIiCBPaajzPa1ASNsvUoaPaTF86mh61_0 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9612437537239350936', time=2024-04-17T07:17:10.333706+0000, chain.objs.size()=1 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kYD8-dhMjmambjE7FfwU6q1-TsolRc4_0 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17260303404380901169', time=2024-04-17T07:17:10.438027+0000, chain.objs.size()=1 2024-04-17T05:46:03.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.616+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xcDvS8B9dbhRYzYqHW7Ui4V42isxZhi_0 2024-04-17T05:46:03.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3617486248152155117', time=2024-04-17T07:17:11.106316+0000, chain.objs.size()=1 2024-04-17T05:46:03.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nUalJsiHlm5VvE-Qe2II2IRpSmKiV01_0 2024-04-17T05:46:03.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13429649973369564599', time=2024-04-17T07:17:11.436749+0000, chain.objs.size()=1 2024-04-17T05:46:03.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MAvUm3rYOGlMHJkCIerL_2pRrWI5TlU_0 2024-04-17T05:46:03.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12633794294140712359', time=2024-04-17T07:17:12.157924+0000, chain.objs.size()=1 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.MPltyZCFH8zVK4G4ae9LLi0gyiCRWuM_0 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9224606119465315266', time=2024-04-17T07:17:12.364706+0000, chain.objs.size()=1 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.h4AlUHjRZrYcevqTPPsyuopc2p9nQ0f_0 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7519034233295031358', time=2024-04-17T07:17:14.435039+0000, chain.objs.size()=1 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nMwo49uPpoK0VxznnTLTb0hPLJaXubS_0 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13112266683037864442', time=2024-04-17T07:17:15.200835+0000, chain.objs.size()=1 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.HqYZQdDfNqxi5Se8CGp7abGbOHw5iDB_0 2024-04-17T05:46:03.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12476469300009473257', time=2024-04-17T07:17:17.049378+0000, chain.objs.size()=1 2024-04-17T05:46:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EuvWLOZktnsi1MVTSCdPhpUiwCDKeyF_0 2024-04-17T05:46:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14831330378244973336', time=2024-04-17T07:17:18.992522+0000, chain.objs.size()=1 2024-04-17T05:46:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.P32g2V2-MddlpjBD_SUV0ZhqLER6I6Y_0 2024-04-17T05:46:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15926541644146617029', time=2024-04-17T07:17:20.101838+0000, chain.objs.size()=1 2024-04-17T05:46:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Mlv6jjgJ9sK2-zitFbwk_JDsNxUuY3Y_0 2024-04-17T05:46:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16570217154062330618', time=2024-04-17T07:17:20.505820+0000, chain.objs.size()=1 2024-04-17T05:46:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VMWpS-c5MQeGyyjQT-2WhleEcxe1gTj_0 2024-04-17T05:46:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12915320490231411040', time=2024-04-17T07:17:22.061492+0000, chain.objs.size()=1 2024-04-17T05:46:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.620+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.U4HLm5iqzAwVgiwJmTXImjW2C8XNxib_0 2024-04-17T05:46:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16605114904015897992', time=2024-04-17T07:17:22.359936+0000, chain.objs.size()=1 2024-04-17T05:46:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UzJoPMQCrydezugqpd-xOAXEWTyVbTZ_0 2024-04-17T05:46:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5674092866277222617', time=2024-04-17T07:17:22.661341+0000, chain.objs.size()=1 2024-04-17T05:46:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.n7FEGa-JIfEFy1WiMBbLUvQk6VKc8bn_0 2024-04-17T05:46:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.624+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9817189705212923654', time=2024-04-17T07:17:22.799905+0000, chain.objs.size()=1 2024-04-17T05:46:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.624+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.IUo1jiotvlJ5QzU-C6n1tr4OlCqmjJJ_0 2024-04-17T05:46:03.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3733555351230718700', time=2024-04-17T07:17:24.461924+0000, chain.objs.size()=1 2024-04-17T05:46:03.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tFwlhnnnlRkOmUt3fyLPf5r15wHLs3N_0 2024-04-17T05:46:03.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15753194724664691583', time=2024-04-17T07:17:24.599499+0000, chain.objs.size()=1 2024-04-17T05:46:03.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.TyzJDFETyUBCmYTQe7JQv5pCmzq2suv_0 2024-04-17T05:46:03.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1971164603863936696', time=2024-04-17T07:17:27.567312+0000, chain.objs.size()=1 2024-04-17T05:46:03.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SOvkpQqiKrJrDpTCgYl58F4j4w976hp_0 2024-04-17T05:46:03.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14481396886670055821', time=2024-04-17T07:17:28.626229+0000, chain.objs.size()=1 2024-04-17T05:46:03.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.x8Z6u-Up2z5DAlHgw6XsROSeWUlvQMy_0 2024-04-17T05:46:03.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16145178061832314118', time=2024-04-17T07:17:29.004710+0000, chain.objs.size()=1 2024-04-17T05:46:03.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pESb6-WTx2rwRWyA-QUjt5GQm2xJ1dk_0 2024-04-17T05:46:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17796386330516617023', time=2024-04-17T07:17:29.006274+0000, chain.objs.size()=1 2024-04-17T05:46:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QwjstA_Xtyb93tgUxH_1UolEHZMqNnk_0 2024-04-17T05:46:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2638092718253550904', time=2024-04-17T07:17:30.043760+0000, chain.objs.size()=1 2024-04-17T05:46:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dLz-tC48-rp3Dhiq-xmQjku1qfvdpxI_0 2024-04-17T05:46:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1956607271674779929', time=2024-04-17T07:17:30.774211+0000, chain.objs.size()=1 2024-04-17T05:46:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.U6gAkPs7sx4YmJr5yJ_3QUQ6DNKE43F_0 2024-04-17T05:46:03.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9880597138602154508', time=2024-04-17T07:17:31.886529+0000, chain.objs.size()=1 2024-04-17T05:46:03.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zFtHvP7ggJp2cPnX7ikn4VRN-KwD_h7_0 2024-04-17T05:46:03.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3035762728963573038', time=2024-04-17T07:17:33.902497+0000, chain.objs.size()=1 2024-04-17T05:46:03.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.628+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.253__shadow_.S8em-vnhW5ueTMJEyHyo9DFEZOxLzTl_0 2024-04-17T05:46:03.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2649968900421970017', time=2024-04-17T07:17:45.909581+0000, chain.objs.size()=1 2024-04-17T05:46:03.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.261__shadow_.GQRGArdMrtHCzU-hLTa464E8MZN5xFS_0 2024-04-17T05:46:03.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8670774593862048952', time=2024-04-17T07:18:34.910034+0000, chain.objs.size()=1 2024-04-17T05:46:03.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.287__shadow_.8AOkqDnA7ktTyCXZi3uE7gmQDLS6ROA_0 2024-04-17T05:46:03.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11284945567975515356', time=2024-04-17T07:18:36.613571+0000, chain.objs.size()=1 2024-04-17T05:46:03.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.288__shadow_.D1hpBNkaDn6Lv-POU8lyrX1B5cG6a0X_0 2024-04-17T05:46:03.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13817606405734988017', time=2024-04-17T07:21:00.104054+0000, chain.objs.size()=1 2024-04-17T05:46:03.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.632+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JUjspkaHwwARWgjDkbVGrEv-StlY8tc_0 2024-04-17T05:46:03.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1387142922032939048', time=2024-04-17T07:21:00.877838+0000, chain.objs.size()=1 2024-04-17T05:46:03.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.NZrUMafdJlx1JNoROA8d9zRGDVhgKY3_0 2024-04-17T05:46:03.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8980976649120964706', time=2024-04-17T07:21:01.388705+0000, chain.objs.size()=1 2024-04-17T05:46:03.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.bz-I8SE7JpvTExRXqoP2upVg1i0L9bo_0 2024-04-17T05:46:03.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3560787915418581645', time=2024-04-17T07:21:01.517956+0000, chain.objs.size()=1 2024-04-17T05:46:03.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wBpvHtDAxo3URBqWPnXrTdUN5YBM1PT_0 2024-04-17T05:46:03.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15266820861255275274', time=2024-04-17T07:21:04.780845+0000, chain.objs.size()=1 2024-04-17T05:46:03.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.s5yc6fNKqtslfsVbk1vOWUynDAErXTW_0 2024-04-17T05:46:03.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1823780232562723347', time=2024-04-17T07:21:09.146863+0000, chain.objs.size()=1 2024-04-17T05:46:03.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.po4IGvCKr69sBPRuWuEpMFkn0Wi1jMd_0 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3282004805409185591', time=2024-04-17T07:21:11.831989+0000, chain.objs.size()=1 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.mng0vAvmUQaM5jNB27MqVji1eI_vJIc_0 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1842414175599564369', time=2024-04-17T07:21:12.292657+0000, chain.objs.size()=1 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eFpDUfwknrS0o-akIFoU1XHyDXP6Mnp_0 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2991053526184744214', time=2024-04-17T07:21:13.495246+0000, chain.objs.size()=1 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.pcpTbjPomT1awkcnL2GnTzDiKF3lp5a_0 2024-04-17T05:46:03.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14528780835623271453', time=2024-04-17T07:21:14.211613+0000, chain.objs.size()=1 2024-04-17T05:46:03.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.E8Y7uKnHVmOvjJQNl5hiEX0H09lFxqA_0 2024-04-17T05:46:03.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13646884607986981455', time=2024-04-17T07:21:15.673200+0000, chain.objs.size()=1 2024-04-17T05:46:03.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.H6o1ZW5L8meZQdrASg7ki8Y26o1teT6_0 2024-04-17T05:46:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11513119756733709057', time=2024-04-17T07:21:18.137730+0000, chain.objs.size()=1 2024-04-17T05:46:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.636+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.bRmCXSCYHpBoyten6rCuzC1I4Mg1_sv_0 2024-04-17T05:46:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16221848891815171166', time=2024-04-17T07:21:18.139681+0000, chain.objs.size()=1 2024-04-17T05:46:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.czXMWT7-ah-RIDdGWpY7pXo03LXrOdT_0 2024-04-17T05:46:03.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.140921309772489789', time=2024-04-17T07:21:19.883064+0000, chain.objs.size()=1 2024-04-17T05:46:03.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Q57BI2tuf-szS2HPRTHbaeRWo_eJiJt_0 2024-04-17T05:46:03.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12402782555051272661', time=2024-04-17T07:21:20.451168+0000, chain.objs.size()=1 2024-04-17T05:46:03.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.c-lqvEY3gbCsaJn7lx5Xp-DC5oRMgxf_0 2024-04-17T05:46:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17797612385770328719', time=2024-04-17T07:21:20.473048+0000, chain.objs.size()=1 2024-04-17T05:46:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ad-iJJ_4GkXUTbMCJxxkFdG_DqSN2zW_0 2024-04-17T05:46:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.254405715665678566', time=2024-04-17T07:21:22.418443+0000, chain.objs.size()=1 2024-04-17T05:46:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gmijDNnwaRjIy_jJC-pkZhyPgPvDCmW_0 2024-04-17T05:46:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7551823206523649586', time=2024-04-17T07:21:23.376678+0000, chain.objs.size()=1 2024-04-17T05:46:03.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.A_bGUp7-FstGqtJo5Z3UKUbI5PtkQuT_0 2024-04-17T05:46:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6639494017668393456', time=2024-04-17T07:21:24.655090+0000, chain.objs.size()=1 2024-04-17T05:46:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.KuLkwEXR83jpAFvO_38dKZssZt0PgoX_0 2024-04-17T05:46:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1000828934521578567', time=2024-04-17T07:21:25.014907+0000, chain.objs.size()=1 2024-04-17T05:46:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZU1bAveaRDa4IgzjPCUwFXE4Ee4Pi3F_0 2024-04-17T05:46:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9817770936948063981', time=2024-04-17T07:21:25.322047+0000, chain.objs.size()=1 2024-04-17T05:46:03.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t-oxSTe-Ql4EEDUmovYD-ENSLkmnGdP_0 2024-04-17T05:46:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8605821881424251599', time=2024-04-17T07:21:25.350592+0000, chain.objs.size()=1 2024-04-17T05:46:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.UFh1-z2orB1ZFpUZ9rPwT0X4uU6BHgC_0 2024-04-17T05:46:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11698542713366228414', time=2024-04-17T07:21:27.926583+0000, chain.objs.size()=1 2024-04-17T05:46:03.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.640+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Q90BI78bpsQSDf0YPxOTrnCcpNJavGg_0 2024-04-17T05:46:03.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17077720536425146449', time=2024-04-17T07:21:28.424167+0000, chain.objs.size()=1 2024-04-17T05:46:03.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.d5N4QKCFDD-UcxLEkXYNYxKJL9UWd2a_0 2024-04-17T05:46:03.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15995451128316802619', time=2024-04-17T07:23:18.934086+0000, chain.objs.size()=1 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vL--zXFnOW5ULBvog0jdRETRxxht_Kb_0 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2397142598324575191', time=2024-04-17T07:23:19.299586+0000, chain.objs.size()=1 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.lG8T9gAcMxcWnEqtfI7ggnYJisfKuEt_0 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18208211040703346601', time=2024-04-17T07:23:19.863558+0000, chain.objs.size()=1 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.K2rM4x6-FM9k8PXdxUeKyKujCGY_jrt_0 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8636533420697015410', time=2024-04-17T07:23:21.778875+0000, chain.objs.size()=1 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.CaPesxaxiKpDOU3Ee0RFT-cYPmPzn8A_0 2024-04-17T05:46:03.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13254027551670003640', time=2024-04-17T07:23:22.024839+0000, chain.objs.size()=1 2024-04-17T05:46:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.644+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.b-Ycsg2BqliMlX8M8tEsBez9D521v9n_0 2024-04-17T05:46:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13185791850061981318', time=2024-04-17T07:23:22.597305+0000, chain.objs.size()=1 2024-04-17T05:46:03.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fksnzuPF4BJhGTghw-q535dqb1zKg-R_0 2024-04-17T05:46:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.767641463938049283', time=2024-04-17T07:23:23.093866+0000, chain.objs.size()=1 2024-04-17T05:46:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BG5iBtPPBocW1eY-AXlX3Fv0Gl_cULu_0 2024-04-17T05:46:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1550409246752812654', time=2024-04-17T07:23:24.768165+0000, chain.objs.size()=1 2024-04-17T05:46:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.-abXbFvkJ9zaPfDI6-rorir7lYNMmAT_0 2024-04-17T05:46:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10461123475680615476', time=2024-04-17T07:23:25.294034+0000, chain.objs.size()=1 2024-04-17T05:46:03.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XXgV8PW4GKmN3rAaZT2QcxxAQ-KsG52_0 2024-04-17T05:46:03.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9944696973382476140', time=2024-04-17T07:23:25.994083+0000, chain.objs.size()=1 2024-04-17T05:46:03.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0i_uzS_lrvX_0eJa9N1x6SVGkOtgiCk_0 2024-04-17T05:46:03.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6922842350713559849', time=2024-04-17T07:23:28.831270+0000, chain.objs.size()=1 2024-04-17T05:46:03.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WweFY4QJbGP684LgapdGgkawBSgfm3u_0 2024-04-17T05:46:03.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13136151679150197207', time=2024-04-17T07:23:31.053827+0000, chain.objs.size()=1 2024-04-17T05:46:03.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.648+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wRbwSw5mv8YebGJlAVEUBePakK_zdYa_0 2024-04-17T05:46:03.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18221495156381360145', time=2024-04-17T07:23:31.225547+0000, chain.objs.size()=1 2024-04-17T05:46:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SdrAb0raSL2gxudGCbgHxA7LUrXROiM_0 2024-04-17T05:46:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7426531145201496926', time=2024-04-17T07:23:32.277647+0000, chain.objs.size()=1 2024-04-17T05:46:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.668-aa9vovl-MDr_WoxRUWAFNVD0oTA_0 2024-04-17T05:46:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16961787088802715229', time=2024-04-17T07:23:32.635318+0000, chain.objs.size()=1 2024-04-17T05:46:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.B2KV_jzE6KXU2ZYeQwmq03mL5Aw4fI8_0 2024-04-17T05:46:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3494603274313333641', time=2024-04-17T07:23:34.974555+0000, chain.objs.size()=1 2024-04-17T05:46:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.K8Am33tlIKvGEzUymynRKFA4jWaGtgi_0 2024-04-17T05:46:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17594897207876579069', time=2024-04-17T07:23:35.783514+0000, chain.objs.size()=1 2024-04-17T05:46:03.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.652+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.1Quer9vHeO5B8iEqwNwjxQdPXcRbu_0_0 2024-04-17T05:46:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6382097692871699030', time=2024-04-17T07:23:36.271924+0000, chain.objs.size()=1 2024-04-17T05:46:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dWiXuRE_9Fh8b94reRtDpWrLfsEInoH_0 2024-04-17T05:46:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17671326313060945984', time=2024-04-17T07:23:36.816948+0000, chain.objs.size()=1 2024-04-17T05:46:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YPrSKyVQGzCt1ndKozakYHn7u8Eupb7_0 2024-04-17T05:46:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15654020160123453135', time=2024-04-17T07:23:37.602602+0000, chain.objs.size()=1 2024-04-17T05:46:03.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.f_19JGkfI0DGhhOWinqsJPMIsRh9sk4_0 2024-04-17T05:46:03.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10487629491801001895', time=2024-04-17T07:23:38.294370+0000, chain.objs.size()=1 2024-04-17T05:46:03.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6IxGbfjmaaGiIT5F-puEl64z5SiTeVw_0 2024-04-17T05:46:03.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7494726583323525951', time=2024-04-17T07:23:38.296470+0000, chain.objs.size()=1 2024-04-17T05:46:03.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wTiFhPy2gLxUYkQd4VILSfcGGVir6jB_0 2024-04-17T05:46:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14296463025413241279', time=2024-04-17T07:23:38.345126+0000, chain.objs.size()=1 2024-04-17T05:46:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qkmBAsPua6SDzSHxNbrx46So6pxRr-X_0 2024-04-17T05:46:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7006593910600500447', time=2024-04-17T07:23:39.375720+0000, chain.objs.size()=1 2024-04-17T05:46:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.WwDcY0Xj_fQRSs7mh8cFoTBuVdEN9Vs_0 2024-04-17T05:46:03.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8428886462915139239', time=2024-04-17T07:23:39.477447+0000, chain.objs.size()=1 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3tTxvWfWg6ajkqOxRJwwsWun80CPq73_0 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13745849254358922347', time=2024-04-17T07:23:39.794607+0000, chain.objs.size()=1 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.d0UIklLXchc8J7ouIIp6vR4Y4dl-eyj_0 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14574324821220742214', time=2024-04-17T07:23:45.300198+0000, chain.objs.size()=1 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.318__shadow_.mlBeSgPv_YX55X17KuitIUlgWUSYt46_0 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10615516573343206932', time=2024-04-17T07:24:21.085758+0000, chain.objs.size()=1 2024-04-17T05:46:03.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.656+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.361__shadow_.BGCEBtp-TI5S5UuGXhY-srxfaxKvBS4_0 2024-04-17T05:46:03.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11743335683695227409', time=2024-04-17T07:25:59.277595+0000, chain.objs.size()=1 2024-04-17T05:46:03.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.454__shadow_.JQr525i8ur93mvGJIMlNn_sKH1Mf6WC_0 2024-04-17T05:46:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6378026012685054863', time=2024-04-17T07:26:02.131212+0000, chain.objs.size()=1 2024-04-17T05:46:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.456__shadow_.XkkC1c0TCZW41F3c2DfH0yVyP4vNkwY_0 2024-04-17T05:46:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11512196778218962863', time=2024-04-17T07:26:17.168452+0000, chain.objs.size()=1 2024-04-17T05:46:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.dXG3mV6FpVvsJuzML06fVD7rlrFbsTB_0 2024-04-17T05:46:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5865613418516356282', time=2024-04-17T07:26:17.208586+0000, chain.objs.size()=1 2024-04-17T05:46:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.660+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.gR4PZp13ZUMtc6cnf9BYgrsu-_FQWx8_0 2024-04-17T05:46:03.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3859247051649689201', time=2024-04-17T07:26:27.911897+0000, chain.objs.size()=1 2024-04-17T05:46:03.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.479__shadow_.6zYR7W3aF2B-ECd04umBtP3d-Pr0QdM_0 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_1_209583041d9b0090dec5c9217e18263f', time=2024-04-17T07:26:44.293323+0000, chain.objs.size()=1 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.493__shadow:DI6VUD8moxAH1yjDvtjHy.JYWvrckYN_.epBQueOpaojyPNSkyOe4RdfT0MkZ6dj_0 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2140938058969346681', time=2024-04-17T07:31:19.343017+0000, chain.objs.size()=1 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.550__multipart_mymultipart.2~yaRMuC8QI6yYIgH1-Gdtk5TNvoOhYVG.1 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15231151653976424027', time=2024-04-17T07:31:30.944417+0000, chain.objs.size()=1 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.555__shadow_.vZEr-sT3rxtODMfMkF1aI8l_6o2DUrM_0 2024-04-17T05:46:03.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.201315763518469076', time=2024-04-17T07:32:10.939164+0000, chain.objs.size()=1 2024-04-17T05:46:03.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.664+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.563__shadow:1l909rCGoTRAnXYitIFWJj4rmRI-fM8_.ZdRgcpPN4q1xN-KCc0ksXRqlMOKDFK2_0 2024-04-17T05:46:03.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.668+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/43909 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4948369247521533803', time=2024-04-17T07:33:01.876413+0000, chain.objs.size()=1 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:dSddlf0Q1WyMZZNAh3OzoOnRC7IBe1J_.3NnSJ5XLCk76ZpYjGafUxlateHGFrrM_0 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10925739223131483762', time=2024-04-17T07:33:35.166192+0000, chain.objs.size()=1 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.571__shadow:UhZcGn4nyp7YD22mFUv.ouctF-2n4NV_.h-N-MtTvw9WZk_U5A7VpltEVqm9BaU-_0 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5091235885611977649', time=2024-04-17T07:33:43.072044+0000, chain.objs.size()=1 2024-04-17T05:46:03.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.575__shadow:KyE5YEXhIxBJoErxl12UPUowB9rsNQM_.CpnGHCxO-X7X0V8jbvrlVOg9I8IdOwH_0 2024-04-17T05:46:03.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7684449329788307588', time=2024-04-17T07:38:23.269975+0000, chain.objs.size()=1 2024-04-17T05:46:03.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.692__shadow:xknldNRt52VV.nYBheUcT78QcZT.q6j_.1QzY-vBK7Sb1inNWfVJyukFK2FmZaKI_0 2024-04-17T05:46:03.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.672+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-17T05:46:03.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.676+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45299' 2024-04-17T05:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8191350836494128403', time=2024-04-17T07:09:41.922101+0000, chain.objs.size()=1 2024-04-17T05:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.141__shadow_.p2PbipVDw_ByK8lKXc1Hkzh3llAI2N9_0 2024-04-17T05:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7846603281485966765', time=2024-04-17T07:10:40.303214+0000, chain.objs.size()=1 2024-04-17T05:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.160__shadow_.YSZgbVUQh7l1E2FBXEFr-eDJCA-Qv5-_0 2024-04-17T05:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17643085189589450131', time=2024-04-17T07:14:48.918503+0000, chain.objs.size()=1 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.233__shadow_.bHVgBP6HXkATiLLUNMny4XfvjQ_sD1z_0 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16715369114889998352', time=2024-04-17T07:15:15.223365+0000, chain.objs.size()=1 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.249__shadow_.rRW7OCv9P50okHmKHUo2vCo2xQcPIMo_0 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15791400224468707016', time=2024-04-17T07:17:01.544965+0000, chain.objs.size()=1 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KdlPyF9zZnALbGzRItutLcVM4u6QURS_0 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8385167734590311205', time=2024-04-17T07:17:02.102255+0000, chain.objs.size()=1 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xsZhTmA1sa2_zGS73e4Jr__zwLF8KKA_0 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15998518957269124259', time=2024-04-17T07:17:03.493234+0000, chain.objs.size()=1 2024-04-17T05:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dXKvjYcNY3_f9SOrS2wBxc1QNyj_vuj_0 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.665308533972874362', time=2024-04-17T07:17:03.495254+0000, chain.objs.size()=1 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.z2WXuR27LGRbNntnYSwtCJRJGvWZDem_0 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4637411808978208075', time=2024-04-17T07:17:04.353186+0000, chain.objs.size()=1 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.cx2h5ArA0VAEPuG7KIrP-xQiWXg7hMY_0 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13263969909680378611', time=2024-04-17T07:17:05.491049+0000, chain.objs.size()=1 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gga05sMnYVtNScKyI1-oS7_OnJQQ4dT_0 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9224093079459559501', time=2024-04-17T07:17:07.023402+0000, chain.objs.size()=1 2024-04-17T05:46:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Gi2m0WzrWYo5neAXNmk425IOvI-E3KU_0 2024-04-17T05:46:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16393004450294710092', time=2024-04-17T07:17:07.147390+0000, chain.objs.size()=1 2024-04-17T05:46:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.680+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.T1PNORF0NpilX-KzrUDH7lmiJ7U-Fu-_0 2024-04-17T05:46:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15105149068506857722', time=2024-04-17T07:17:08.672639+0000, chain.objs.size()=1 2024-04-17T05:46:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Dn6iUbH_M5F1Y_13PKIRGLzGgD82QCE_0 2024-04-17T05:46:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3834114959074223052', time=2024-04-17T07:17:09.803090+0000, chain.objs.size()=1 2024-04-17T05:46:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.cgCCJI6_J3godCX_EDZipoyuLn7GGgv_0 2024-04-17T05:46:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9249778791209358109', time=2024-04-17T07:17:09.875751+0000, chain.objs.size()=1 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ISTwNDnrhGDsKEPy4Z7W73Vju7Gfmc8_0 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12193966092145130449', time=2024-04-17T07:17:11.682248+0000, chain.objs.size()=1 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lcBGZy4BBQvJBDFnfA8X9YiyfS_VLhC_0 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6630565193971866783', time=2024-04-17T07:17:11.886538+0000, chain.objs.size()=1 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.DA8jkN5Z_FpgsxwNZyiCDMJn3wphK0M_0 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11090882108909822265', time=2024-04-17T07:17:12.027445+0000, chain.objs.size()=1 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.cp-3xVqsbN-JrsJfc73B-bRrfqaBCdn_0 2024-04-17T05:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12750350252751991472', time=2024-04-17T07:17:12.510817+0000, chain.objs.size()=1 2024-04-17T05:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.U2imiHJtdEeDY9gNcFcSooy_N0GyStv_0 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4157094815976815381', time=2024-04-17T07:17:15.362478+0000, chain.objs.size()=1 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0D_S2IAORPHj1uwd66JpcNJhumrhpM6_0 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10210715712216045487', time=2024-04-17T07:17:16.144531+0000, chain.objs.size()=1 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8R01RKnFY9Z0737ibx7W5nRwjJxLyul_0 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1838288187273964285', time=2024-04-17T07:17:17.983702+0000, chain.objs.size()=1 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ZbpubIdvJUVA54WCtLO38ss2C2-yLpb_0 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2751973889887951408', time=2024-04-17T07:17:18.287254+0000, chain.objs.size()=1 2024-04-17T05:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.684+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.U_a7xBFeZrPTpWezxIfnyUlpSpWoXYy_0 2024-04-17T05:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.643180480996273941', time=2024-04-17T07:17:19.188315+0000, chain.objs.size()=1 2024-04-17T05:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.SpMiIao_himA-7vAS-b2_R8AyrCwJzI_0 2024-04-17T05:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3597065881093379773', time=2024-04-17T07:17:19.288183+0000, chain.objs.size()=1 2024-04-17T05:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.3ofIwSNhAe9FpcF6qO7cEkefzfPi5jL_0 2024-04-17T05:46:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14765027974657400977', time=2024-04-17T07:17:19.705892+0000, chain.objs.size()=1 2024-04-17T05:46:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.8fvT9Z90pQjaIjG9cHJnBOgsuQox-OM_0 2024-04-17T05:46:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2233393790382891557', time=2024-04-17T07:17:20.242589+0000, chain.objs.size()=1 2024-04-17T05:46:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Y-9S787NQAjNo7bCPsnxao5vfmCtae1_0 2024-04-17T05:46:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14252930042992595027', time=2024-04-17T07:17:20.438741+0000, chain.objs.size()=1 2024-04-17T05:46:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_._mieu6doS8-6LbLZm_pPb8rwAEj_AZl_0 2024-04-17T05:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2092073706956266486', time=2024-04-17T07:17:22.021336+0000, chain.objs.size()=1 2024-04-17T05:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xpQmRDj9M0CSHwHPvAyysoNxjgFjIWz_0 2024-04-17T05:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13608325599543362268', time=2024-04-17T07:17:22.766495+0000, chain.objs.size()=1 2024-04-17T05:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.688+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.B46fKVLcyOQOCFIjWX7L3dGcsyoGbIf_0 2024-04-17T05:46:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12966651048944871313', time=2024-04-17T07:17:23.083338+0000, chain.objs.size()=1 2024-04-17T05:46:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Uz51Uf859U2xm0nEe8Fe9bvKgEETfrx_0 2024-04-17T05:46:03.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5295171823023305091', time=2024-04-17T07:17:23.084977+0000, chain.objs.size()=1 2024-04-17T05:46:03.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.fkZUkS7reJSOF1yBGuv7OByju2aUepg_0 2024-04-17T05:46:03.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7863171060649330714', time=2024-04-17T07:17:24.801700+0000, chain.objs.size()=1 2024-04-17T05:46:03.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JKlIU9UFZRavZcUjfISwuj87eoKb63Z_0 2024-04-17T05:46:03.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8952050597559954515', time=2024-04-17T07:17:25.004812+0000, chain.objs.size()=1 2024-04-17T05:46:03.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YKJdV8lN5_MlZQckcMlLR4W8JW771m2_0 2024-04-17T05:46:03.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4141167481578582328', time=2024-04-17T07:17:25.024951+0000, chain.objs.size()=1 2024-04-17T05:46:03.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.D-99CXhcnbrbhGhpDWtPBbY7LPpEeip_0 2024-04-17T05:46:03.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6025077490603953166', time=2024-04-17T07:17:25.043624+0000, chain.objs.size()=1 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.ysxe5Yqh27EekGNveDsVyhkp3jZJG05_0 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15486353362160061991', time=2024-04-17T07:17:26.569193+0000, chain.objs.size()=1 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.cejVFPNr8rJnqlmbw_Q-XjHHKJTR0z8_0 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12220131553348645610', time=2024-04-17T07:17:28.426985+0000, chain.objs.size()=1 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.N0c6qW0GCUljbeTpeXC_CJgLxEuhnmU_0 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9804520307244614066', time=2024-04-17T07:17:30.009766+0000, chain.objs.size()=1 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.04Mc_wQjVTfgA9Lk0aXZKzmHOmk5hUf_0 2024-04-17T05:46:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7258523216791129926', time=2024-04-17T07:17:30.245556+0000, chain.objs.size()=1 2024-04-17T05:46:03.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.692+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.R8NHMHVWDkPFksGbjbD5Y-LeROqJVZN_0 2024-04-17T05:46:03.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.540214197312447124', time=2024-04-17T07:17:30.816443+0000, chain.objs.size()=1 2024-04-17T05:46:03.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5IljC0v7l8c-j5wutkSkrKa3Wmr7N-6_0 2024-04-17T05:46:03.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7628924243124789283', time=2024-04-17T07:18:00.070933+0000, chain.objs.size()=1 2024-04-17T05:46:03.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.270__shadow_.bcCPWiM7oeYoyYDlZvwD989fPRKhpjS_0 2024-04-17T05:46:03.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17034374805306516338', time=2024-04-17T07:18:14.901827+0000, chain.objs.size()=1 2024-04-17T05:46:03.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.279__shadow_.BT-iCWm2hPjt5LLclvbWZhG1mW3hSEM_0 2024-04-17T05:46:03.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12694438019001790427', time=2024-04-17T07:18:22.965825+0000, chain.objs.size()=1 2024-04-17T05:46:03.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.f1l0CbZVmNWmkkymW1i_yEEYiz6T6ug_0 2024-04-17T05:46:03.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12147138133206730582', time=2024-04-17T07:18:29.892005+0000, chain.objs.size()=1 2024-04-17T05:46:03.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.284__shadow_.4srMAY8l6sLv3vTl3nCcKi3RxI9qcfp_0 2024-04-17T05:46:03.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6931576971649104020', time=2024-04-17T07:18:44.968115+0000, chain.objs.size()=1 2024-04-17T05:46:03.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.293__shadow_.rp8zBnPOIvtRYnqD8ib5jC6fJTuOS_H_0 2024-04-17T05:46:03.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10803145253461142473', time=2024-04-17T07:20:59.908125+0000, chain.objs.size()=1 2024-04-17T05:46:03.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VycyDdGpmUuUGoqtxdwlfn-h5481wtt_0 2024-04-17T05:46:03.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4760919132066265212', time=2024-04-17T07:21:00.033020+0000, chain.objs.size()=1 2024-04-17T05:46:03.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GUIoa7oHB3C3XTc1WaJ8QslV-lVyyjE_0 2024-04-17T05:46:03.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13589272661988120516', time=2024-04-17T07:21:00.162843+0000, chain.objs.size()=1 2024-04-17T05:46:03.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.uqdXtoR68ce_zMskQ0mHM7D5zB1z3wp_0 2024-04-17T05:46:03.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12819356960710202537', time=2024-04-17T07:21:00.297525+0000, chain.objs.size()=1 2024-04-17T05:46:03.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.696+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ceGGDeKa95uZeQVlGB63XfwDz0hCLjd_0 2024-04-17T05:46:03.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6141529677698342413', time=2024-04-17T07:21:00.812661+0000, chain.objs.size()=1 2024-04-17T05:46:03.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.YOWkyLBYCsqBmpVrG3h7qycxBtScw_5_0 2024-04-17T05:46:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9394940559671594694', time=2024-04-17T07:21:01.387634+0000, chain.objs.size()=1 2024-04-17T05:46:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.cYKMD2rRLYoIRvKF-RnLLP66eWT379K_0 2024-04-17T05:46:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.700+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15051672244153023476', time=2024-04-17T07:21:01.902297+0000, chain.objs.size()=1 2024-04-17T05:46:03.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.700+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FwgrwxFvg2DtOFD6o4KBNS2h71p8p7j_0 2024-04-17T05:46:03.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1976118137931652107', time=2024-04-17T07:21:02.667390+0000, chain.objs.size()=1 2024-04-17T05:46:03.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ee45Ft5S_MEk1QOEpiAZthJJuT-8ziH_0 2024-04-17T05:46:03.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.262299377571261166', time=2024-04-17T07:21:06.294040+0000, chain.objs.size()=1 2024-04-17T05:46:03.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wBtcg-ah2JPvQAJRoUL8M2kvmwBTXfF_0 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7111355790280679543', time=2024-04-17T07:21:07.972913+0000, chain.objs.size()=1 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SkzirYi1MSaLkKXBYMcEJMmHN9da_VE_0 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3797070337885153959', time=2024-04-17T07:21:08.383134+0000, chain.objs.size()=1 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.c1re48s1G-LzEHgMedLkrXvru7a2tcE_0 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2793220619703677623', time=2024-04-17T07:21:08.706014+0000, chain.objs.size()=1 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.FYbRRgW-lskTBjHMlMBtfAaVDVX1ykp_0 2024-04-17T05:46:03.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16228776775521979282', time=2024-04-17T07:21:09.286377+0000, chain.objs.size()=1 2024-04-17T05:46:03.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RU3q3UwP2jnBfkBengNOYbRTK6zyQrj_0 2024-04-17T05:46:03.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7351421599588065780', time=2024-04-17T07:21:09.628626+0000, chain.objs.size()=1 2024-04-17T05:46:03.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xRdOsFpinBR7LXu-ynxR3Q15Q6NNuab_0 2024-04-17T05:46:03.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5542284365155086629', time=2024-04-17T07:21:09.746522+0000, chain.objs.size()=1 2024-04-17T05:46:03.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sEoCRK0ZBy2k1R7cW5ko1Mpb0fR4enK_0 2024-04-17T05:46:03.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13239229265635002561', time=2024-04-17T07:21:10.008399+0000, chain.objs.size()=1 2024-04-17T05:46:03.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._hpJOSsZjbKzYD4zWUO3wZ7rvjsibiG_0 2024-04-17T05:46:03.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18064015915882583865', time=2024-04-17T07:21:10.482051+0000, chain.objs.size()=1 2024-04-17T05:46:03.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sp1FwHdLoFSgKUpwUH0tuxAmxOXw1XS_0 2024-04-17T05:46:03.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13484485836942662566', time=2024-04-17T07:21:10.485084+0000, chain.objs.size()=1 2024-04-17T05:46:03.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.704+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2Ngb7rHUYyzwY6OUwQ_PJY6xpq4VkGZ_0 2024-04-17T05:46:03.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16653947863867356233', time=2024-04-17T07:21:10.881681+0000, chain.objs.size()=1 2024-04-17T05:46:03.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QVwd44Jqt6bmyR-o1Y9JbrqBZzhCoxs_0 2024-04-17T05:46:03.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15313171405072934193', time=2024-04-17T07:21:13.554282+0000, chain.objs.size()=1 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.arThf_rpKQCoCawePtmbpIfOkkdg78E_0 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.356510256810159678', time=2024-04-17T07:21:14.470519+0000, chain.objs.size()=1 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.t8jPUVC00k7oLDnsHGfZyeE5MIz-rcx_0 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.337710805913389140', time=2024-04-17T07:21:17.320605+0000, chain.objs.size()=1 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.aCf87_1keSqr36ILnkyX7HXU4zC8sp5_0 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16988075296837122964', time=2024-04-17T07:21:17.398961+0000, chain.objs.size()=1 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.XiEcqRpZ9lonP9yqL6dFxaZhA5PvBg__0 2024-04-17T05:46:03.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5829609596306903581', time=2024-04-17T07:21:17.955899+0000, chain.objs.size()=1 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._vXpFh53S_1LIcCfcEwvNuw8-y0Er1c_0 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3474640904676203219', time=2024-04-17T07:21:18.168189+0000, chain.objs.size()=1 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._CUqzN6s77slI4dLsiqKb0sf-Wah02p_0 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13018290229243449928', time=2024-04-17T07:21:18.516935+0000, chain.objs.size()=1 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.P9sknnYAc0hZabnnvIKnUJqZUtwunk9_0 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2298723676915267918', time=2024-04-17T07:21:19.939062+0000, chain.objs.size()=1 2024-04-17T05:46:03.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.708+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PdUmkFxs6PG1BFFsYNnIRrY-u6UP9GF_0 2024-04-17T05:46:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8278954047966282888', time=2024-04-17T07:21:20.625372+0000, chain.objs.size()=1 2024-04-17T05:46:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.GLHp4eSsrDaDv7wEasU6kZ2mX5Xn-db_0 2024-04-17T05:46:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4867995501834837648', time=2024-04-17T07:21:23.242927+0000, chain.objs.size()=1 2024-04-17T05:46:03.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LsNhoeFA13hKPn6L_Ebyt7TIixIyryE_0 2024-04-17T05:46:03.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.551051697936713309', time=2024-04-17T07:21:24.283088+0000, chain.objs.size()=1 2024-04-17T05:46:03.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.L1NmsKjYZ8Dn2hK-2uLtCmdHL-LS7iT_0 2024-04-17T05:46:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4132213373211722033', time=2024-04-17T07:21:24.490668+0000, chain.objs.size()=1 2024-04-17T05:46:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nI5XrJCNvGis7LS2qNGBhnF6IDEuIZ5_0 2024-04-17T05:46:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12019277500280104693', time=2024-04-17T07:21:24.728211+0000, chain.objs.size()=1 2024-04-17T05:46:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.E3gU3svGldI4Q_8Z0wWe1rAhrBWPNrn_0 2024-04-17T05:46:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15488635472236591385', time=2024-04-17T07:21:25.134054+0000, chain.objs.size()=1 2024-04-17T05:46:03.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.712+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.eCArxciSPf0_PT-wsJV2oEJhkh3OF65_0 2024-04-17T05:46:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3317666264506402414', time=2024-04-17T07:21:25.547291+0000, chain.objs.size()=1 2024-04-17T05:46:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.8jNzOQpQiCU92kNf-qzsUcZbyjolQWr_0 2024-04-17T05:46:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17120787723850002936', time=2024-04-17T07:21:27.121634+0000, chain.objs.size()=1 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.LPnRkVNNkdSmLGBBENyhhoCYSaizsV1_0 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3138108567553970942', time=2024-04-17T07:21:27.385523+0000, chain.objs.size()=1 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.wLS_UgB1MNcLiQNOLCsUmb8o5eMMv9-_0 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3985348737822676903', time=2024-04-17T07:21:29.254211+0000, chain.objs.size()=1 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.rP7wk9NUBpRa_6oIDP5ZoaXEBZ7E51Z_0 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9751684355008517669', time=2024-04-17T07:23:14.237455+0000, chain.objs.size()=1 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NS5OI9xkRNfA4bf5lriytpUrnQp6ek7_0 2024-04-17T05:46:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12826693021975646428', time=2024-04-17T07:23:14.300504+0000, chain.objs.size()=1 2024-04-17T05:46:03.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.XgQIAzwIXo8JuuOgWbJtCE3eZQ6Y76j_0 2024-04-17T05:46:03.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6338920783206180608', time=2024-04-17T07:23:14.737917+0000, chain.objs.size()=1 2024-04-17T05:46:03.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.716+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N5nkPTvyp9KmbsFhZbFf38gP9Frje2M_0 2024-04-17T05:46:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13206629545178822670', time=2024-04-17T07:23:15.333955+0000, chain.objs.size()=1 2024-04-17T05:46:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.R5EcBNdp0BCRs8eiDximMJoFNutYxk3_0 2024-04-17T05:46:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3164516984073752611', time=2024-04-17T07:23:15.892528+0000, chain.objs.size()=1 2024-04-17T05:46:03.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.cF-5juRg8IANjjWoy5RJxK4ghUrl4x7_0 2024-04-17T05:46:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1690255921528548583', time=2024-04-17T07:23:15.970941+0000, chain.objs.size()=1 2024-04-17T05:46:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7FHeazgyJ7Y_CYQQ4ocBZQu0UUgzHl9_0 2024-04-17T05:46:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1929267361800955921', time=2024-04-17T07:23:18.154162+0000, chain.objs.size()=1 2024-04-17T05:46:03.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5ebV66dEuG39MGFms0dV_4zoJGnt3BB_0 2024-04-17T05:46:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9272989085629316916', time=2024-04-17T07:23:21.037246+0000, chain.objs.size()=1 2024-04-17T05:46:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5vsheaV3zFZOo6U63bC9PJzjZZxUY5m_0 2024-04-17T05:46:03.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.715717181479313582', time=2024-04-17T07:23:21.171890+0000, chain.objs.size()=1 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yMXLyYrSLY63X7iJtclMESXc1eUXje3_0 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4999813621611502623', time=2024-04-17T07:23:21.565835+0000, chain.objs.size()=1 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SAXKaytbcln5mmRJX17e82fVKVbdvFT_0 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8647148893396912766', time=2024-04-17T07:23:22.317045+0000, chain.objs.size()=1 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.q-0Lv5S1irMfFgLeAPcJmlQ7YGclNmN_0 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15275062008061707769', time=2024-04-17T07:23:22.645764+0000, chain.objs.size()=1 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.0pFQP9deZj-v5xsVaPFvwNLkYlqHffo_0 2024-04-17T05:46:03.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.378602635509234863', time=2024-04-17T07:23:22.922475+0000, chain.objs.size()=1 2024-04-17T05:46:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MAKiWXu7evV5MoDySzlgD1udvkB-Ytq_0 2024-04-17T05:46:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15879539531511463868', time=2024-04-17T07:23:25.005306+0000, chain.objs.size()=1 2024-04-17T05:46:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.720+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.a8i2eIh7dSaBDJ4Qb4c6rmn3roLOvyh_0 2024-04-17T05:46:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.292305286850143719', time=2024-04-17T07:23:25.314027+0000, chain.objs.size()=1 2024-04-17T05:46:03.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.7mP4ELimzCUAuazwDh4kq5p2Q-z-Mco_0 2024-04-17T05:46:03.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.809771497197669323', time=2024-04-17T07:23:25.565562+0000, chain.objs.size()=1 2024-04-17T05:46:03.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kG06eeYQrFp3gxP99YJqFBArdOLT7W5_0 2024-04-17T05:46:03.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.724+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2364142081268816486', time=2024-04-17T07:23:26.551468+0000, chain.objs.size()=1 2024-04-17T05:46:03.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.724+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.4kClCQxKtyojEOQxazI3ETZxE3jaTnC_0 2024-04-17T05:46:03.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.728+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45299 2024-04-17T05:46:03.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=34, truncated=0, next_marker='' 2024-04-17T05:46:03.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16544073419120881652', time=2024-04-17T07:23:27.192828+0000, chain.objs.size()=1 2024-04-17T05:46:03.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.LKmteIogvMJXtSRT_5OSKlSWK904biK_0 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.888025739394636791', time=2024-04-17T07:23:27.393773+0000, chain.objs.size()=1 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.SeUeFDKZlxedttz7kqR6kDK5w--2Qe3_0 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8906017009776672031', time=2024-04-17T07:23:28.675503+0000, chain.objs.size()=1 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uCMr3YSFmJARAu7glBtXxEBWlp_sGHZ_0 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15532585984580329421', time=2024-04-17T07:23:29.588220+0000, chain.objs.size()=1 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.kgGApKP3wNnH2ONe32_2f_SwpqcRuPJ_0 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9003729966182010572', time=2024-04-17T07:23:31.077162+0000, chain.objs.size()=1 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.NEElT81kg0RYUaPUGH8gOg7yKRTGXh6_0 2024-04-17T05:46:03.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16677247084133590123', time=2024-04-17T07:23:31.276143+0000, chain.objs.size()=1 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.88JtvBzSzKoCSxdUUivT_YRn6WPK1Yp_0 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8799456289226922561', time=2024-04-17T07:23:33.008875+0000, chain.objs.size()=1 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.75WnEIrIEJFW36v8y6IFnA63yvoYRBt_0 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13870597328046307866', time=2024-04-17T07:23:33.294063+0000, chain.objs.size()=1 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Mlb6QrMg06xPsjCGjBgm_Y36yGTNR7m_0 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17757598209544668172', time=2024-04-17T07:23:34.295899+0000, chain.objs.size()=1 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.yaCEU-Hs_KxIyql5iDLtTk_GWmHef5L_0 2024-04-17T05:46:03.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7245526110253675732', time=2024-04-17T07:23:35.853652+0000, chain.objs.size()=1 2024-04-17T05:46:03.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.rXi9E9Sv_3bAqImWggsmTdSIWNWip27_0 2024-04-17T05:46:03.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17707669191231936943', time=2024-04-17T07:23:36.221450+0000, chain.objs.size()=1 2024-04-17T05:46:03.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EcNhE5vGTO-hX4JXz4zQHyr1098Dvj5_0 2024-04-17T05:46:03.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1288710018248182292', time=2024-04-17T07:23:36.223529+0000, chain.objs.size()=1 2024-04-17T05:46:03.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.732+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UAFsqiSBTUbmiznLJclXLun-s9GMJes_0 2024-04-17T05:46:03.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1443403278491607723', time=2024-04-17T07:23:38.511843+0000, chain.objs.size()=1 2024-04-17T05:46:03.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.L9PpzOS7ApK0FtTJcW304DqJzgZiibB_0 2024-04-17T05:46:03.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4930987862041478144', time=2024-04-17T07:23:39.791277+0000, chain.objs.size()=1 2024-04-17T05:46:03.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.LYGr2i5rRwOzd8FDBZZiHHjO3Tzl4NL_0 2024-04-17T05:46:03.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12641607173521754458', time=2024-04-17T07:23:40.116885+0000, chain.objs.size()=1 2024-04-17T05:46:03.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OD-BuIw8-kEkoyxIFaQiXwcbn8IhJSs_0 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18204460091856892049', time=2024-04-17T07:23:40.195841+0000, chain.objs.size()=1 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Vt7q54ReQri9KzQMdkhLijn5MMrbvdz_0 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7937378112754836833', time=2024-04-17T07:23:58.729250+0000, chain.objs.size()=1 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.332__shadow_.PDWCOfMTRRcLOljGfJ1p8IHmjjt26d-_0 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9872497943723182572', time=2024-04-17T07:24:02.253440+0000, chain.objs.size()=1 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.336__shadow_.Y2xnXiq5UVBx9QbN2GukzHlSzOkxl2D_0 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8089181022073378356', time=2024-04-17T07:25:28.427083+0000, chain.objs.size()=1 2024-04-17T05:46:03.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.431__shadow_.UCYirn3c4XcfVRr4X2n40vLd3RCzbna_0 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2822735829424016970', time=2024-04-17T07:26:17.187381+0000, chain.objs.size()=1 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.473__shadow_.Kg63cnB-Jvu8frDxBbggA0kSC0o6MKo_0 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4928685374333150233', time=2024-04-17T07:26:19.374954+0000, chain.objs.size()=1 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.475__shadow_.8mjDazlS6WlIK7q_LDeDyenvPJErB1U_0 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12263942175365374773', time=2024-04-17T07:26:28.957569+0000, chain.objs.size()=1 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.480__shadow_.4ffkC7moil15YUDgLXwL3wajGOzsTBh_0 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6499932045617730355', time=2024-04-17T07:26:30.278746+0000, chain.objs.size()=1 2024-04-17T05:46:03.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.736+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.481__shadow_.eL2eHW3eUU-sEozEsPNPA5YYY7CNvn6_0 2024-04-17T05:46:03.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.740+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2_ce922d0ded238127f15329761f1a58eb', time=2024-04-17T07:26:56.120062+0000, chain.objs.size()=6 2024-04-17T05:46:03.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.740+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.1 2024-04-17T05:46:03.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.740+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2 2024-04-17T05:46:03.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.740+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.3 2024-04-17T05:46:03.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.740+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.4 2024-04-17T05:46:03.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.740+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.5 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.6 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4405890143758652655', time=2024-04-17T07:27:15.672644+0000, chain.objs.size()=1 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.507__shadow_.rIkKo0ZGflRD9lTjBlCeWNwhdgGe3XQ_0 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~byucKcSDSOV9JhqxWCcILpinMnNgPn5', time=2024-04-17T07:30:05.188688+0000, chain.objs.size()=1 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.525__multipart_mymultipart.2~byucKcSDSOV9JhqxWCcILpinMnNgPn5.3 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5365597549888432167', time=2024-04-17T07:31:47.288174+0000, chain.objs.size()=1 2024-04-17T05:46:03.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:10jraTPAXtUDdCvSgCA7ukAkyNxgBKG_._qU9ydf1qmt7EKzVcXhA4r1Q6DAqlZh_0 2024-04-17T05:46:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='5a3ned8a0eget3srxya7j526leyep2og', time=2024-04-17T07:32:14.698250+0000, chain.objs.size()=1 2024-04-17T05:46:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.564__shadow_.FbeO3aGn-FJYjgdx3g5_uaYP3F6bQc5_0 2024-04-17T05:46:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7996359438187642688', time=2024-04-17T07:32:46.720182+0000, chain.objs.size()=1 2024-04-17T05:46:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:v7dE7UENGX0EPD22wQMbYzWuyNB-Y6b_.DX9gg8tBYXpQPdZEHXPcsaqTYcoabbt_0 2024-04-17T05:46:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:aYkwN-EcPPlwfAR2VHy7nBHocJML3oL_.TDjeRwHRu8PYUf7zcW4M64h1eexc6Ie_1_90a547a7ee94c8f97293c037fa636baf', time=2024-04-17T07:33:33.441453+0000, chain.objs.size()=1 2024-04-17T05:46:03.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.744+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.569__shadow:aYkwN-EcPPlwfAR2VHy7nBHocJML3oL_.TDjeRwHRu8PYUf7zcW4M64h1eexc6Ie_0 2024-04-17T05:46:03.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13322721171242137317', time=2024-04-17T07:33:43.119789+0000, chain.objs.size()=1 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.575__shadow:XRzCIzV30hSD-f6XvzvYGtwe2FMAPWi_.9-vO_cJD7hO-BrvmaEqNJN5reL9nXLW_0 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12289042495671638787', time=2024-04-17T07:37:00.388269+0000, chain.objs.size()=1 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.636__shadow_.o7pA43PRiHuVigj5adk1wnmxYB-BZXe_0 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14488880360956595444', time=2024-04-17T07:37:01.306717+0000, chain.objs.size()=1 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.637__shadow_.5J0CeALIjqPBXdjNWKD2jv9dkRUuDgs_0 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.888526218670242959', time=2024-04-17T07:38:30.452793+0000, chain.objs.size()=1 2024-04-17T05:46:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.748+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.698__shadow:SdRVemdJVLEiCboiQFcPcDJNS-e-kNV_.cH-4Zs8Q4bx545gXFyAdPy_uwIh2bzd_0 2024-04-17T05:46:03.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.752+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.752+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-17T05:46:03.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45806' 2024-04-17T05:46:03.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4506112063598407310', time=2024-04-17T07:10:34.127752+0000, chain.objs.size()=2 2024-04-17T05:46:03.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.154__multipart_foo.2~6LtU9Ds62KiUq9paRQkp_rQ_dsvotst.1 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.154__multipart_foo.2~6LtU9Ds62KiUq9paRQkp_rQ_dsvotst.2 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12274828164402352439', time=2024-04-17T07:14:56.129558+0000, chain.objs.size()=1 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.237__shadow_.nQDjS7Gz3eEDouWH11Ztf0xqfX1_fJl_0 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16569443403206562941', time=2024-04-17T07:15:07.505149+0000, chain.objs.size()=1 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.244__shadow_.09jpAEg7Ut0VC9Fo_aZFlxRYr8Wynu7_0 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4041511181145691496', time=2024-04-17T07:15:10.752252+0000, chain.objs.size()=1 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.246__shadow_.jCKKBFcUDplnxTn9z_dY6-VAdou8Zjl_0 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16455369027398890580', time=2024-04-17T07:17:01.309364+0000, chain.objs.size()=1 2024-04-17T05:46:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.aw174NsIqhHbttRntnSWnZqJhdni30E_0 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15173071984526383335', time=2024-04-17T07:17:02.217668+0000, chain.objs.size()=1 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4M99dI8bA-dttpwUYFS9Qd6W1raG_8A_0 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7183646004956724750', time=2024-04-17T07:17:03.240852+0000, chain.objs.size()=1 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pnkIZdKhAFdASd__IdwzBEBg79r_sv9_0 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10155725086875284816', time=2024-04-17T07:17:03.638039+0000, chain.objs.size()=1 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4VWhX1T7crkrXNZEyLsnNiI9dTB3oj__0 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11110552388420321539', time=2024-04-17T07:17:03.794718+0000, chain.objs.size()=1 2024-04-17T05:46:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.abDYkXlRVRfNIvdAHhn2EwcFx7obLic_0 2024-04-17T05:46:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16301653375934252016', time=2024-04-17T07:17:04.137440+0000, chain.objs.size()=1 2024-04-17T05:46:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sGvDEz7dXfLsPELUoIrkGhBl7u84_dI_0 2024-04-17T05:46:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18106565257195670773', time=2024-04-17T07:17:04.211341+0000, chain.objs.size()=1 2024-04-17T05:46:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.756+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.z6RbcjKpsd2-XUf4Q8Bg0R1X1B4rAWC_0 2024-04-17T05:46:03.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.760+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18117386582846508156', time=2024-04-17T07:17:05.494205+0000, chain.objs.size()=1 2024-04-17T05:46:03.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.760+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Jzkk-St_ddnI6H2jPc57sWvqax7QvNu_0 2024-04-17T05:46:03.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10908234079347822166', time=2024-04-17T07:17:06.425642+0000, chain.objs.size()=1 2024-04-17T05:46:03.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.99LPmjG4g9y17tWyVyojoiFplZ_wHjR_0 2024-04-17T05:46:03.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11349066858432862911', time=2024-04-17T07:17:06.775517+0000, chain.objs.size()=1 2024-04-17T05:46:03.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.e3CoRfspZav3hINXWnjVGbXY8Rt5CSA_0 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4895402254807761734', time=2024-04-17T07:17:07.074969+0000, chain.objs.size()=1 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dTQwopCxSxNghqBju3clOtLSoxIcWyq_0 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6513469806873635', time=2024-04-17T07:17:07.537188+0000, chain.objs.size()=1 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.AiiqJWlvwACFGuTWa5xL9Fw15ueCUoY_0 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15155651059267987040', time=2024-04-17T07:17:08.620457+0000, chain.objs.size()=1 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.z7yQvq9U06jPgu4x4ZRyCp-ufqTVqh4_0 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.732408635193782607', time=2024-04-17T07:17:08.676442+0000, chain.objs.size()=1 2024-04-17T05:46:03.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JbqLhCQnui6LBtdfCVIVFJ5FRupRgWJ_0 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4828754258267272185', time=2024-04-17T07:17:09.930903+0000, chain.objs.size()=1 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.UyXCaVI5uvTLLduH_dATCFE0jD3EIk__0 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3965776201795295924', time=2024-04-17T07:17:10.370305+0000, chain.objs.size()=1 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EfF593vuqY9DhDk_Ga4s9cDqTndNleb_0 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14075263529429229735', time=2024-04-17T07:17:10.882440+0000, chain.objs.size()=1 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.lhoG92CetnsmLDEclPsebWmHdvO4S5F_0 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5597812331616712407', time=2024-04-17T07:17:11.803961+0000, chain.objs.size()=1 2024-04-17T05:46:03.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.YOuniJ5bfNcwhjB7BoG27roVx0HCXR9_0 2024-04-17T05:46:03.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15176768028228756036', time=2024-04-17T07:17:12.117668+0000, chain.objs.size()=1 2024-04-17T05:46:03.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.764+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.BoxuVVPNbCFQZJoXuw6K8dYFuwK4tCO_0 2024-04-17T05:46:03.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16914421054298278523', time=2024-04-17T07:17:14.223907+0000, chain.objs.size()=1 2024-04-17T05:46:03.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.D85rhklIJ6g7CPDwtV6lgSl341wjkfi_0 2024-04-17T05:46:03.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17018987129182153985', time=2024-04-17T07:17:14.625190+0000, chain.objs.size()=1 2024-04-17T05:46:03.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.pXl23_d8OpaYc6xdNMP5V0pscg2sGp3_0 2024-04-17T05:46:03.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14221809737746088125', time=2024-04-17T07:17:16.216546+0000, chain.objs.size()=1 2024-04-17T05:46:03.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.kGXALcFHYN3hxPduNOOrusJspuZqUGT_0 2024-04-17T05:46:03.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1108670828071666431', time=2024-04-17T07:17:16.781803+0000, chain.objs.size()=1 2024-04-17T05:46:03.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.skHZ0x91NxbfYfv7nFvqWkby6HWo2NC_0 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15819680477545956982', time=2024-04-17T07:17:18.866106+0000, chain.objs.size()=1 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4Z5WCLqrfKiKPylFK_OtdrXLZqHsk3m_0 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7478383883682640932', time=2024-04-17T07:17:19.661731+0000, chain.objs.size()=1 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.bH9wgUeEkZwiMMzXA2OThWjmRccrFC-_0 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15114020392878755320', time=2024-04-17T07:17:19.670668+0000, chain.objs.size()=1 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.wspBO4iG2ty94SdjM7ZI_A1yK0ek5D3_0 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14924370002146719806', time=2024-04-17T07:17:19.674151+0000, chain.objs.size()=1 2024-04-17T05:46:03.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mgsDTyDFAe4wqPm0_uXC4VRhIvBmmaw_0 2024-04-17T05:46:03.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5722500971081543891', time=2024-04-17T07:17:19.768644+0000, chain.objs.size()=1 2024-04-17T05:46:03.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.JrStiFkyp4urp05WHtToYA0n91OzVxg_0 2024-04-17T05:46:03.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11923089810366594338', time=2024-04-17T07:17:22.441017+0000, chain.objs.size()=1 2024-04-17T05:46:03.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.x-uEgB_yi_IfWj-ruKSa1dRkJwVjFCm_0 2024-04-17T05:46:03.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.598316479942274712', time=2024-04-17T07:17:22.443667+0000, chain.objs.size()=1 2024-04-17T05:46:03.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.768+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.EHs7sejVyf9k95nNEvhymXxtm3viEA7_0 2024-04-17T05:46:03.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13502605822991054392', time=2024-04-17T07:17:23.814627+0000, chain.objs.size()=1 2024-04-17T05:46:03.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.nTfo-IBEq5Okk4NLqa20OI6zIR6uXNs_0 2024-04-17T05:46:03.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13817595404090167298', time=2024-04-17T07:17:24.689415+0000, chain.objs.size()=1 2024-04-17T05:46:03.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.QT7eccQyCwJfl80JY5Y9UdQoevdZEXq_0 2024-04-17T05:46:03.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.772+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17073552676135713771', time=2024-04-17T07:17:27.399334+0000, chain.objs.size()=1 2024-04-17T05:46:03.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.772+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.yKMLs26joJahRdwt1HVeQRbB3Jcp1fJ_0 2024-04-17T05:46:03.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11767524319191644362', time=2024-04-17T07:17:27.530590+0000, chain.objs.size()=1 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.tB0Ual3t9Eq_mFNpRwXSZIRCByyRYeh_0 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3085706758304470737', time=2024-04-17T07:17:29.169283+0000, chain.objs.size()=1 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.rcccp3GTWE9SYaLZZ9lCgvj4dktUmaF_0 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3991390354534982860', time=2024-04-17T07:17:30.851885+0000, chain.objs.size()=1 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Bq5wxw264yjWCXq-b4t6h3xrut0oRM5_0 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1730840241944493812', time=2024-04-17T07:17:31.879965+0000, chain.objs.size()=1 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.sLWONR71ILm8okmye2gFCl3x2j6Rckq_0 2024-04-17T05:46:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13796905920864590194', time=2024-04-17T07:17:32.223753+0000, chain.objs.size()=1 2024-04-17T05:46:03.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.iHcoEAA_9JYwO5RHWYoR9hVlnHl28eV_0 2024-04-17T05:46:03.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10137547116167161537', time=2024-04-17T07:17:42.938851+0000, chain.objs.size()=1 2024-04-17T05:46:03.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.259__shadow_.R3law0OAna9yjD6GwsN7K8hpmmD3p1P_0 2024-04-17T05:46:03.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10669599931282518751', time=2024-04-17T07:17:50.365628+0000, chain.objs.size()=1 2024-04-17T05:46:03.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.264__shadow_.scSOoknV0_OgnZHlIvzU6rA4e6kMYpt_0 2024-04-17T05:46:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4161155432241125692', time=2024-04-17T07:17:55.244197+0000, chain.objs.size()=1 2024-04-17T05:46:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.267__shadow_.az5BybkV1A82ER5vHhYPl0bFJIpxqGh_0 2024-04-17T05:46:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6545985381815178627', time=2024-04-17T07:18:10.033390+0000, chain.objs.size()=1 2024-04-17T05:46:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.276__shadow_.ep1fxRg-nIcMRzXjTg4IN6ASF_fapoh_0 2024-04-17T05:46:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.502988668537716858', time=2024-04-17T07:18:23.239017+0000, chain.objs.size()=1 2024-04-17T05:46:03.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.776+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.282__shadow_.3ct9ufFnjwYQAOjA3RQONQqsIYMFrrF_0 2024-04-17T05:46:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16312038733223108157', time=2024-04-17T07:18:44.943532+0000, chain.objs.size()=1 2024-04-17T05:46:03.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.293__shadow_.-3koBCVs6tEvDMDU3DDkra5eYao21_n_0 2024-04-17T05:46:03.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11448690068031290272', time=2024-04-17T07:21:00.305178+0000, chain.objs.size()=1 2024-04-17T05:46:03.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vNTtAMSYelJWY5m_ekKUU5xNh2hCXzV_0 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14283113380906404893', time=2024-04-17T07:21:00.865496+0000, chain.objs.size()=1 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.NSZIavG3WIOoXhNPBzdmzJvIF2UnDAJ_0 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.384772674896147479', time=2024-04-17T07:21:01.363640+0000, chain.objs.size()=1 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vpRr_9WxjPtmXpz7r1K8NzQei0DGD2N_0 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2770877462344185651', time=2024-04-17T07:21:01.689834+0000, chain.objs.size()=1 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.3cGPW6sYwSkaMYOK7Ib5yyVyf8nUJWM_0 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12355430279582238326', time=2024-04-17T07:21:02.029154+0000, chain.objs.size()=1 2024-04-17T05:46:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.gTL9FAdIkZm0vgBUmCzus2cb4-77bwq_0 2024-04-17T05:46:03.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10259909331883599166', time=2024-04-17T07:21:02.496432+0000, chain.objs.size()=1 2024-04-17T05:46:03.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IDZBJSAN_KpP1s0A0GCv-ZMHxBKLmVE_0 2024-04-17T05:46:03.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3413547984997837076', time=2024-04-17T07:21:02.508921+0000, chain.objs.size()=1 2024-04-17T05:46:03.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.780+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.xgrT7qmIc8i3wgnA3rG7X7H2QqCEwO4_0 2024-04-17T05:46:03.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11916040824814240646', time=2024-04-17T07:21:04.082723+0000, chain.objs.size()=1 2024-04-17T05:46:03.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ab0Tg0JQYidFpkyTTHGKL2dxgjH0pEj_0 2024-04-17T05:46:03.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17667575939744596030', time=2024-04-17T07:21:04.642229+0000, chain.objs.size()=1 2024-04-17T05:46:03.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.nA53Gk5AL-XKVkJ9vURA8vaPr93Oq8p_0 2024-04-17T05:46:03.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7324152647424209798', time=2024-04-17T07:21:06.531106+0000, chain.objs.size()=1 2024-04-17T05:46:03.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.O272Rhmh36q94VJF8brZgAuKrIG-bH3_0 2024-04-17T05:46:03.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17168779110486386854', time=2024-04-17T07:21:08.421648+0000, chain.objs.size()=1 2024-04-17T05:46:03.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.9iA-YBLPoKwvcMoQeB75QK9R8FzdB3O_0 2024-04-17T05:46:03.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4975663458585717971', time=2024-04-17T07:21:08.519044+0000, chain.objs.size()=1 2024-04-17T05:46:03.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Ck8O0c1EQ2zRJl5Qd6QzeFVK27Cd8UN_0 2024-04-17T05:46:03.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8429474637519244115', time=2024-04-17T07:21:08.520462+0000, chain.objs.size()=1 2024-04-17T05:46:03.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZFdZ5gPXquk9pjgM7DBq1elXhHDT5l6_0 2024-04-17T05:46:03.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18285454402897025679', time=2024-04-17T07:21:09.248756+0000, chain.objs.size()=1 2024-04-17T05:46:03.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4rtrrm3o2YFte-oXSl8vRS7oJq03ZjX_0 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.648036093338316652', time=2024-04-17T07:21:09.950304+0000, chain.objs.size()=1 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.2B141fZuI-1ocGCiNyIVVu7P5uLfA20_0 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5558614136843244588', time=2024-04-17T07:21:09.974242+0000, chain.objs.size()=1 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._xp64TfnrQQz59IoeNSoORNXq11mkrI_0 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13986495612496321664', time=2024-04-17T07:21:10.417499+0000, chain.objs.size()=1 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.PpkQwuS7jBijq_JT3SN5EmzcrHSpDMD_0 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5503579844228272615', time=2024-04-17T07:21:12.986894+0000, chain.objs.size()=1 2024-04-17T05:46:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.784+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ZNBL4i31B_F8ckU6Mlo8DA-Zkmr_8Bo_0 2024-04-17T05:46:03.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.788+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11970956255426622716', time=2024-04-17T07:21:13.720799+0000, chain.objs.size()=1 2024-04-17T05:46:03.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.788+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.l4Yo5Bz7vgNIlRqBIOoUvaTsJt7dw9N_0 2024-04-17T05:46:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9085197878382080622', time=2024-04-17T07:21:17.420333+0000, chain.objs.size()=1 2024-04-17T05:46:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.Z1eM72d7bKRVEAKmn8wblqtT5BDsCYz_0 2024-04-17T05:46:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11719581599210526330', time=2024-04-17T07:21:18.598417+0000, chain.objs.size()=1 2024-04-17T05:46:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.69pb7OK_ZRKm7MEFJ7Nv4lnLXOCNeVw_0 2024-04-17T05:46:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10416440665682370828', time=2024-04-17T07:21:18.726314+0000, chain.objs.size()=1 2024-04-17T05:46:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.ywjKSCBXtKy--wkjzQfNzt4ADvwMoVK_0 2024-04-17T05:46:03.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16241877934898882685', time=2024-04-17T07:21:20.868242+0000, chain.objs.size()=1 2024-04-17T05:46:03.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.28MkAvAFhqgqWaEhRtNK1b1ncOayAvH_0 2024-04-17T05:46:03.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.766939211962380594', time=2024-04-17T07:21:21.041647+0000, chain.objs.size()=1 2024-04-17T05:46:03.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BrjMWeGSabeKQUxviFR0AbLdfDIMG8E_0 2024-04-17T05:46:03.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12172466472963610489', time=2024-04-17T07:21:21.043077+0000, chain.objs.size()=1 2024-04-17T05:46:03.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4knKDXlnMzMzVdCofE5z5zKQ3DlWfEz_0 2024-04-17T05:46:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13118173726634283154', time=2024-04-17T07:21:21.218890+0000, chain.objs.size()=1 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QoyV5QUJwQTXmLWIcoYYLyNMntXXEQa_0 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4271434773165510252', time=2024-04-17T07:21:22.280033+0000, chain.objs.size()=1 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.BvKUjgRym3Cj7d2z75vIrbfobdPiGPM_0 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11856331417270300612', time=2024-04-17T07:21:23.293516+0000, chain.objs.size()=1 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.SfvzKHQ-BsRh2b4F7EePsiHsQShy9Ud_0 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13001718999046841202', time=2024-04-17T07:21:23.753224+0000, chain.objs.size()=1 2024-04-17T05:46:03.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.792+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.y096Ak3ks2zj7QDyaWWRXL7uudfi3TO_0 2024-04-17T05:46:03.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11491096602711736327', time=2024-04-17T07:21:24.195961+0000, chain.objs.size()=1 2024-04-17T05:46:03.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.RcCbCQnCQ_ZdUMq36pRegLf7N-IbIWW_0 2024-04-17T05:46:03.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4742370831347506636', time=2024-04-17T07:21:24.607735+0000, chain.objs.size()=1 2024-04-17T05:46:03.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.HlnVE4AJkdRn2BADqWFoZz0tMTE10Rl_0 2024-04-17T05:46:03.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8113310420917282951', time=2024-04-17T07:21:25.429952+0000, chain.objs.size()=1 2024-04-17T05:46:03.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.K_krloBoNT83oqDtpTTYN5B6CQQr7Hc_0 2024-04-17T05:46:03.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4415286259258784834', time=2024-04-17T07:23:15.265894+0000, chain.objs.size()=1 2024-04-17T05:46:03.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ZNin6r0yOFM-Z5rosvYJ_lE01J1Sx6D_0 2024-04-17T05:46:03.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10864047986204373224', time=2024-04-17T07:23:15.329837+0000, chain.objs.size()=1 2024-04-17T05:46:03.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.796+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UILuK16BlED7BeyfG3UEb_D25DX45XZ_0 2024-04-17T05:46:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16269415487829414901', time=2024-04-17T07:23:17.136238+0000, chain.objs.size()=1 2024-04-17T05:46:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.wMiJOZEOReu2AeM4EFccumzlkPawrce_0 2024-04-17T05:46:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2521619277347161510', time=2024-04-17T07:23:19.017440+0000, chain.objs.size()=1 2024-04-17T05:46:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gglKzrIXZWi2B3vdjoEjP_DQeF3KRKy_0 2024-04-17T05:46:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7248889317155838869', time=2024-04-17T07:23:19.883265+0000, chain.objs.size()=1 2024-04-17T05:46:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.a5KCuUKQ-aqmUX2GeY4ug_O3_6woU47_0 2024-04-17T05:46:03.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12525673201909212706', time=2024-04-17T07:23:20.836584+0000, chain.objs.size()=1 2024-04-17T05:46:03.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.P-aHvG8xFx0WHqIP-ckmYxaoSsf-trd_0 2024-04-17T05:46:03.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17625790468637692546', time=2024-04-17T07:23:23.004715+0000, chain.objs.size()=1 2024-04-17T05:46:03.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.dltnSGnCA8121nbL1sa6E1fAxcoQxBS_0 2024-04-17T05:46:03.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16015026101757197753', time=2024-04-17T07:23:23.009337+0000, chain.objs.size()=1 2024-04-17T05:46:03.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.MjN9aFTkGH_5FSxK_NSO_ptgyw1u5fT_0 2024-04-17T05:46:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5066246219561555366', time=2024-04-17T07:23:24.769610+0000, chain.objs.size()=1 2024-04-17T05:46:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.6e-wWuQVgLAhGpEJ_xvehXAPzpVjcKh_0 2024-04-17T05:46:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1090423216071336009', time=2024-04-17T07:23:24.953685+0000, chain.objs.size()=1 2024-04-17T05:46:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.800+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tIGO1xXjwFV_L1Y7M2JQcjZCPBqAQrg_0 2024-04-17T05:46:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8586174537218573559', time=2024-04-17T07:23:26.387986+0000, chain.objs.size()=1 2024-04-17T05:46:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.VumJHhd_z1JPj25j-9IPi7wVEtikLBO_0 2024-04-17T05:46:03.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1844685195983754471', time=2024-04-17T07:23:26.981178+0000, chain.objs.size()=1 2024-04-17T05:46:03.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.V11Xvda1rcaeGFBI0gViRDF_lWn06Nk_0 2024-04-17T05:46:03.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15063674033622955365', time=2024-04-17T07:23:28.878444+0000, chain.objs.size()=1 2024-04-17T05:46:03.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.gEVF07R7jNeDwPBMaGutdQbM6VBC0qi_0 2024-04-17T05:46:03.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2821038876713455987', time=2024-04-17T07:23:29.346815+0000, chain.objs.size()=1 2024-04-17T05:46:03.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.BT9G-U3J7IWDxy2ZiR-e24eQm8iBwvz_0 2024-04-17T05:46:03.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7811072375383789171', time=2024-04-17T07:23:29.595071+0000, chain.objs.size()=1 2024-04-17T05:46:03.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.l_ly8Xvr57eTNFbq-t4NZfk0nnIeGej_0 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8920111312950622497', time=2024-04-17T07:23:29.810777+0000, chain.objs.size()=1 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.ybnksk7s9mLKzxU7WYZeh_cEnZaj2Dx_0 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6725042507149467646', time=2024-04-17T07:23:29.969865+0000, chain.objs.size()=1 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.tMrYAV_lQgfGw5tmureLjJQ-08S-uir_0 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5926860956079854778', time=2024-04-17T07:23:30.292654+0000, chain.objs.size()=1 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8zTwrPUuI1hwGYEDM3Fdw0BXVY1SiBO_0 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11199920007739766190', time=2024-04-17T07:23:30.331014+0000, chain.objs.size()=1 2024-04-17T05:46:03.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.804+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.f9DobbZuvvoLj9HJvu4ftWuUld9yjyy_0 2024-04-17T05:46:03.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.808+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5543201371471073030', time=2024-04-17T07:23:30.939302+0000, chain.objs.size()=1 2024-04-17T05:46:03.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.808+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.5WpiSac5pKDzXnK-Nruye5fusNBJlv7_0 2024-04-17T05:46:03.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/45806 2024-04-17T05:46:03.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=18, truncated=0, next_marker='' 2024-04-17T05:46:03.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14792166799031307424', time=2024-04-17T07:23:32.839664+0000, chain.objs.size()=1 2024-04-17T05:46:03.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.d6DQ2iP-m3P90tAVMZ72LyyFJC56-XM_0 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3532482038365909677', time=2024-04-17T07:23:33.601968+0000, chain.objs.size()=1 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.FHct7Ntw17fKYqDgzW7e2ynmb9gaw5d_0 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11421374265997534836', time=2024-04-17T07:23:33.795358+0000, chain.objs.size()=1 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DXTNxs1RFENAEqQ23akVjiweClAE6y3_0 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14466380373786504529', time=2024-04-17T07:23:34.079797+0000, chain.objs.size()=1 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.oPvx3g9cC7fqjTFjVx_YvHcB6d9kRJZ_0 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2106601596914212623', time=2024-04-17T07:23:34.758390+0000, chain.objs.size()=1 2024-04-17T05:46:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2okPNl_N7ZG9kcXCYI1JH4dfynga6QF_0 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9791678671149988872', time=2024-04-17T07:23:36.468161+0000, chain.objs.size()=1 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.RGAGwAdavIxXe7vVLJOFUCDY0S0oUoP_0 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3182396231026362231', time=2024-04-17T07:23:37.056396+0000, chain.objs.size()=1 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_._lVn4PZRjAvAvXLElQOg54rsd517YH9_0 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11456485400186130908', time=2024-04-17T07:23:38.659585+0000, chain.objs.size()=1 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.vQMExed1NnVunL0Ib3U_XKGX2sqvpEf_0 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9189339454703877509', time=2024-04-17T07:23:39.354307+0000, chain.objs.size()=1 2024-04-17T05:46:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.IzfUFpiEan6XXMioBkUcVLnrb2cG0Dl_0 2024-04-17T05:46:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13655871830810810204', time=2024-04-17T07:23:40.474000+0000, chain.objs.size()=1 2024-04-17T05:46:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.EQYqeo9iMOmEtosu39bQLcFzlbf7Ahn_0 2024-04-17T05:46:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5009238513617688196', time=2024-04-17T07:23:40.739302+0000, chain.objs.size()=1 2024-04-17T05:46:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.uOQaIb9L1ZJZm0-rvFzrImZv9yyaJ23_0 2024-04-17T05:46:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12765710481673064032', time=2024-04-17T07:25:57.493674+0000, chain.objs.size()=1 2024-04-17T05:46:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.812+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.453__shadow_.7BFyzCtbaBPgWMv4MmhAbWaxhxoH39h_0 2024-04-17T05:46:03.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.816+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~vzerRUMT19bnr_hMvKYnTY6fYGTp_q_', time=2024-04-17T07:29:57.173626+0000, chain.objs.size()=2 2024-04-17T05:46:03.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.816+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.521__multipart_mymultipart.2~vzerRUMT19bnr_hMvKYnTY6fYGTp_q_.1 2024-04-17T05:46:03.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.521__multipart_mymultipart.2~vzerRUMT19bnr_hMvKYnTY6fYGTp_q_.2 2024-04-17T05:46:03.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='2~9QQhUpCG3yDC-gO6pqMVNqPD6JZAfMJ', time=2024-04-17T07:29:57.268818+0000, chain.objs.size()=1 2024-04-17T05:46:03.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.521__multipart_mymultipart2.2~9QQhUpCG3yDC-gO6pqMVNqPD6JZAfMJ.1 2024-04-17T05:46:03.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14045884666660168617', time=2024-04-17T07:31:44.662284+0000, chain.objs.size()=1 2024-04-17T05:46:03.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:x.KQdC-HyRXIHpKWZE36MV.5lO4b3ba_.V7B8PbFUF5Q3hCsxVwCW0cY0FtzkF5C_0 2024-04-17T05:46:03.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1038584837925738020', time=2024-04-17T07:32:35.928629+0000, chain.objs.size()=1 2024-04-17T05:46:03.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:rvYJDOaI8FizY0bmmSYanGPXmz53bjF_.PDZN10atCZSg-Lyr-CtJe9rst3YEeUO_0 2024-04-17T05:46:03.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9545762645403171209', time=2024-04-17T07:37:02.453774+0000, chain.objs.size()=1 2024-04-17T05:46:03.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.638__shadow_.9CUN8ACzYvJ1Ur-DTtfa6R6Cd_t4ANB_0 2024-04-17T05:46:03.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14194982509245427425', time=2024-04-17T07:39:20.876142+0000, chain.objs.size()=1 2024-04-17T05:46:03.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.820+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.722__shadow_.5JV1WOHIHDt6TokaD7AU3Sv0ZqpDasq_0 2024-04-17T05:46:03.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.824+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-17T05:46:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-17T05:46:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-17T05:46:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46276' 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7925047723503072343', time=2024-04-17T07:09:39.522157+0000, chain.objs.size()=1 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.137__shadow_.QQrNPnIFcuX744w8sORdcnC_Wuqcm6S_0 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='_rkFkW8frM4BeS7AhepIobLeIhmh98Xt', time=2024-04-17T07:10:12.132063+0000, chain.objs.size()=2 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.HU1A779zjCfu87w1N396HbXBijnW9mq_0 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.150__shadow_.HU1A779zjCfu87w1N396HbXBijnW9mq_1 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17767872079347490012', time=2024-04-17T07:10:57.454310+0000, chain.objs.size()=1 2024-04-17T05:46:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.168__shadow_.8OjrO7aEfiy-2thmoAc3q5FWsoCZTZ1_0 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2790416392136977633', time=2024-04-17T07:14:57.792718+0000, chain.objs.size()=1 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.238__shadow_.z8pg4jD6o3oEvLXxXF6AbK0hwCS2ssD_0 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9624997698302100967', time=2024-04-17T07:15:04.275084+0000, chain.objs.size()=1 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.242__shadow_.ZtKAw1X3ElxJ_gemRvgmcBIn7I4nic8_0 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12037083959852343525', time=2024-04-17T07:15:07.490232+0000, chain.objs.size()=1 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.244__shadow_.mCZ4IkvFEhY7yyE7ow7MH8xMRiywloC_0 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15765524858363902166', time=2024-04-17T07:15:12.235139+0000, chain.objs.size()=1 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.247__shadow_.1hS3G8mUAMnPaEQxEFOGWa1fCMal9OB_0 2024-04-17T05:46:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9797452028548190622', time=2024-04-17T07:15:13.780782+0000, chain.objs.size()=1 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.248__shadow_.hxZG8psTF7azrz7C1REJ-31232zwIKt_0 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4527612045707487021', time=2024-04-17T07:17:01.354462+0000, chain.objs.size()=1 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FjK7gXwF4OQKEyJ9Hh8OlAcmDJvrlvM_0 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4780127896549394356', time=2024-04-17T07:17:01.785772+0000, chain.objs.size()=1 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.l4vCprbxJhchbRFVMFQbH2rKCOwaPJ1_0 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16962441795118354498', time=2024-04-17T07:17:02.343472+0000, chain.objs.size()=1 2024-04-17T05:46:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.828+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Yw_5E-G3_EDd4dsQygNTiFSCgrwAnx__0 2024-04-17T05:46:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11365794686510004850', time=2024-04-17T07:17:03.047187+0000, chain.objs.size()=1 2024-04-17T05:46:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.dBGzf6pN_OMDUhBsCoLAbRl3qqC6Q1E_0 2024-04-17T05:46:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13703732215624560216', time=2024-04-17T07:17:06.193178+0000, chain.objs.size()=1 2024-04-17T05:46:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Z2_3fI2LG7fD02rljxl6UjPQIwlgDCg_0 2024-04-17T05:46:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.832+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17531428989487026050', time=2024-04-17T07:17:06.372359+0000, chain.objs.size()=1 2024-04-17T05:46:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.832+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.mzfahoWxoe4nerh4A4YV-CejDwINqO8_0 2024-04-17T05:46:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6132082713682484100', time=2024-04-17T07:17:06.574556+0000, chain.objs.size()=1 2024-04-17T05:46:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.l3OIIg-SkY2JQgkaKxK46rzW6Axc42V_0 2024-04-17T05:46:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7493011099831539225', time=2024-04-17T07:17:06.916097+0000, chain.objs.size()=1 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5g1RtlEDrh5F7GcJuJc2tfQUbZC2Jv8_0 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.937976844749527125', time=2024-04-17T07:17:07.110905+0000, chain.objs.size()=1 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FtXNUxTBAjy0gRNyi9bBthY1GfJYVKf_0 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17121307528482185651', time=2024-04-17T07:17:08.724022+0000, chain.objs.size()=1 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.6hKvmz1iJd1Ujcw74RFQUOeFIIQNVBs_0 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11708947707399226787', time=2024-04-17T07:17:11.004004+0000, chain.objs.size()=1 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gC2TgrocrMWo3Wkyj14lWWLCXKd5LKd_0 2024-04-17T05:46:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4893890285638738524', time=2024-04-17T07:17:15.164462+0000, chain.objs.size()=1 2024-04-17T05:46:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.FmgZczAg4IJ5KUGLooWCzbWOvc6VoWH_0 2024-04-17T05:46:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8320187038074110809', time=2024-04-17T07:17:15.627125+0000, chain.objs.size()=1 2024-04-17T05:46:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.VZcPUGLuaSgYvFQVpsruEJ7CXlFyykX_0 2024-04-17T05:46:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3103620380296814722', time=2024-04-17T07:17:16.363968+0000, chain.objs.size()=1 2024-04-17T05:46:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.B-klix1Gj_SZfBFNdmcJuW74v72Eoyz_0 2024-04-17T05:46:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13608120925570170938', time=2024-04-17T07:17:16.676610+0000, chain.objs.size()=1 2024-04-17T05:46:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.KkTdNYWHM2M2KuCXWrnI0laGDlRAusv_0 2024-04-17T05:46:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12941516271993822108', time=2024-04-17T07:17:16.962719+0000, chain.objs.size()=1 2024-04-17T05:46:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Ei8Vm_Gq9Z7sxAM4sRHvUq9rJGUvn2f_0 2024-04-17T05:46:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11610622350437173077', time=2024-04-17T07:17:18.985078+0000, chain.objs.size()=1 2024-04-17T05:46:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.836+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.43DVRLRsQNSsO8K6N5rJVterjkNVdk1_0 2024-04-17T05:46:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5764205522223877882', time=2024-04-17T07:17:20.429906+0000, chain.objs.size()=1 2024-04-17T05:46:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.cWf2l5W75a31igTuh1X24uhk0VyeWp-_0 2024-04-17T05:46:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15578307248487712591', time=2024-04-17T07:17:20.861287+0000, chain.objs.size()=1 2024-04-17T05:46:03.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.0n2vh93cWz-tRk3yUGGlS22R8Vg-tId_0 2024-04-17T05:46:03.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4008339699431369008', time=2024-04-17T07:17:20.996244+0000, chain.objs.size()=1 2024-04-17T05:46:03.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.5eRLe5TuukSrrjlnF8mhVse_SUBnmvD_0 2024-04-17T05:46:03.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12769600038788790628', time=2024-04-17T07:17:24.824317+0000, chain.objs.size()=1 2024-04-17T05:46:03.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Gl8WP9xJK6oI9gzhCYyrPGJ5KmRo3D-_0 2024-04-17T05:46:03.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5468315838072412479', time=2024-04-17T07:17:26.098417+0000, chain.objs.size()=1 2024-04-17T05:46:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.byv1V0Q4OhJCP8WrT4AZetLh7Bkpjob_0 2024-04-17T05:46:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3332597440016935586', time=2024-04-17T07:17:26.601496+0000, chain.objs.size()=1 2024-04-17T05:46:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.LrTlBx27pVZN0WG7Fbg4IK1piht4vhp_0 2024-04-17T05:46:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6448320149312407105', time=2024-04-17T07:17:28.647349+0000, chain.objs.size()=1 2024-04-17T05:46:03.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.WbevNASBnmVfJeyJdTdoGNPGboVl7KT_0 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8080681464389935918', time=2024-04-17T07:17:28.649237+0000, chain.objs.size()=1 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.4ZJUOj3R4DKv9yT1bv_WSS0I4a2XujF_0 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7268777089069733430', time=2024-04-17T07:17:28.695844+0000, chain.objs.size()=1 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.xALEjkrXCzzmm8FbRDHmvyXLY7VXGFv_0 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17917865107496649094', time=2024-04-17T07:17:30.045759+0000, chain.objs.size()=1 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.u0fl4GdqtrZrp5yVHnMjrFeWaVEfYkY_0 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7349954244740953645', time=2024-04-17T07:17:30.262305+0000, chain.objs.size()=1 2024-04-17T05:46:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.87sa3xPLLfz__uk4wKtXjhuRmNQ_epY_0 2024-04-17T05:46:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2964058324493650760', time=2024-04-17T07:17:30.961167+0000, chain.objs.size()=1 2024-04-17T05:46:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.840+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.amXNaVApm1HhgNGj6PBFtE4FnnLOnYf_0 2024-04-17T05:46:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12888730313568271551', time=2024-04-17T07:17:31.580349+0000, chain.objs.size()=1 2024-04-17T05:46:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.Cn9D2-ovl6XO1WRhkzz_0d7mryk7niu_0 2024-04-17T05:46:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16435799807112209033', time=2024-04-17T07:17:31.582407+0000, chain.objs.size()=1 2024-04-17T05:46:03.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.gMlSFaYo1XyYUMrlAUjP3XYGHGwNBK0_0 2024-04-17T05:46:03.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.844+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7493307036131481696', time=2024-04-17T07:17:32.300850+0000, chain.objs.size()=1 2024-04-17T05:46:03.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.844+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.252__shadow_.zwMQSO3RNiW4naRTy92qYBjCGFDZ994_0 2024-04-17T05:46:03.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.804214579927624084', time=2024-04-17T07:17:56.864423+0000, chain.objs.size()=1 2024-04-17T05:46:03.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.268__shadow_.E-nks64zcbj4fPClBXZhNfsFINYlCVH_0 2024-04-17T05:46:03.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2649218439208615125', time=2024-04-17T07:18:48.167946+0000, chain.objs.size()=1 2024-04-17T05:46:03.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.295__shadow_.oZzZk83jWzUkLH63L1xsJ3_Vws2y8sn_0 2024-04-17T05:46:03.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13329207176665717535', time=2024-04-17T07:20:59.784155+0000, chain.objs.size()=1 2024-04-17T05:46:03.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.sTYTUTY3P7Ih5MTXFq03WkbcsqYkD8N_0 2024-04-17T05:46:03.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4824654707819748984', time=2024-04-17T07:21:00.879329+0000, chain.objs.size()=1 2024-04-17T05:46:03.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.VEir7L9FQkCsu8nQi48ulRCHWpIWkCU_0 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11636997268097388399', time=2024-04-17T07:21:02.492805+0000, chain.objs.size()=1 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.diMWgeJidSZwgcqVEjmsVSkPTPeNr-O_0 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2068906905862982288', time=2024-04-17T07:21:02.735989+0000, chain.objs.size()=1 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.JTVZeVi7iFUlPyFNRSdy0rB15X8PLzd_0 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4647751267739882247', time=2024-04-17T07:21:03.859831+0000, chain.objs.size()=1 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._EaHwaSzivovsYbQVvXBBLLtCEEli9F_0 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18402036535243842762', time=2024-04-17T07:21:12.251599+0000, chain.objs.size()=1 2024-04-17T05:46:03.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.lwTtvQWDMrbCZy9BciF_8IQUK5Bcwu3_0 2024-04-17T05:46:03.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12275756376375719400', time=2024-04-17T07:21:12.315964+0000, chain.objs.size()=1 2024-04-17T05:46:03.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.j6kvTd7lG0zi8gEggwQPL3JHrmFxuca_0 2024-04-17T05:46:03.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16530395492662279709', time=2024-04-17T07:21:13.078719+0000, chain.objs.size()=1 2024-04-17T05:46:03.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.848+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.vJcUtSkEXNLbAE8I0w2beTPvp8-KKtF_0 2024-04-17T05:46:03.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7952344621090323214', time=2024-04-17T07:21:13.080831+0000, chain.objs.size()=1 2024-04-17T05:46:03.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.isSBJkK9x4A4K1CN89vhWyBYU0T5CVa_0 2024-04-17T05:46:03.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6548381341628240622', time=2024-04-17T07:21:13.624690+0000, chain.objs.size()=1 2024-04-17T05:46:03.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.EZ17q9-_98sL0cHeTi6KmIQmwU-bM4z_0 2024-04-17T05:46:03.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7931115277725849634', time=2024-04-17T07:21:14.827246+0000, chain.objs.size()=1 2024-04-17T05:46:03.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.MiFfTz23bUeLIkr6ifx2Wn8S55tIaB8_0 2024-04-17T05:46:03.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11917061789479656324', time=2024-04-17T07:21:15.872377+0000, chain.objs.size()=1 2024-04-17T05:46:03.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.QcJCRdKBsZcIOwctIWd6GQie0tw8Zc8_0 2024-04-17T05:46:03.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14207715503292152155', time=2024-04-17T07:21:16.261064+0000, chain.objs.size()=1 2024-04-17T05:46:03.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.852+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.v21jJKgBeastROs36mtDYijnXNVHktf_0 2024-04-17T05:46:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14232922973133518604', time=2024-04-17T07:21:16.554909+0000, chain.objs.size()=1 2024-04-17T05:46:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.1hWPh-CxYAW8tbnvIciS4O3g7xJxKia_0 2024-04-17T05:46:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9134875014219162222', time=2024-04-17T07:21:17.025029+0000, chain.objs.size()=1 2024-04-17T05:46:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.DMsoPj36AzYvDV-Ep79fllD56-cQC3g_0 2024-04-17T05:46:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2302737838449778933', time=2024-04-17T07:21:17.244146+0000, chain.objs.size()=1 2024-04-17T05:46:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.IpysGdRderPxeNlPz5UGs4sAmqtt9r2_0 2024-04-17T05:46:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17179549308446439213', time=2024-04-17T07:21:18.143403+0000, chain.objs.size()=1 2024-04-17T05:46:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.TlzwNey-a0afDUOZT-qVLxdsm3NowbM_0 2024-04-17T05:46:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14882373292157740171', time=2024-04-17T07:21:19.575468+0000, chain.objs.size()=1 2024-04-17T05:46:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.szELEg25iUg9TH65wKsRQ1H87XMe7Dx_0 2024-04-17T05:46:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11972420000325473358', time=2024-04-17T07:21:19.577559+0000, chain.objs.size()=1 2024-04-17T05:46:03.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_._-e3pmSxeFROXua8JkOvP42w8u1PzCV_0 2024-04-17T05:46:03.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12159183577065871055', time=2024-04-17T07:21:19.951029+0000, chain.objs.size()=1 2024-04-17T05:46:03.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.u9I2mzzOc_HU8IdS9pJSnFeEcGCXx_2_0 2024-04-17T05:46:03.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4321193711589864334', time=2024-04-17T07:21:20.157040+0000, chain.objs.size()=1 2024-04-17T05:46:03.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.N0sfjUMfHJNhTEWx8eg2wtV3aQUSear_0 2024-04-17T05:46:03.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12401811131073790434', time=2024-04-17T07:21:22.078904+0000, chain.objs.size()=1 2024-04-17T05:46:03.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.zxcutuMfH-DZNPm3zdAXsKIllP_MqZU_0 2024-04-17T05:46:03.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11762417992614181648', time=2024-04-17T07:21:27.454364+0000, chain.objs.size()=1 2024-04-17T05:46:03.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.77ZXGH6AuN9kQCdv_vPbIbIyHl4qOxF_0 2024-04-17T05:46:03.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14056645376963219622', time=2024-04-17T07:21:28.561511+0000, chain.objs.size()=1 2024-04-17T05:46:03.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.856+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.313__shadow_.4LL7nefJV-hfRG0yF33phHT1yK_jOMZ_0 2024-04-17T05:46:03.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.1366290939595583887', time=2024-04-17T07:23:13.083023+0000, chain.objs.size()=1 2024-04-17T05:46:03.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UiHoYFBsy7KhRSG8SYGRpJ8hdnBudNA_0 2024-04-17T05:46:03.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13259242013485276716', time=2024-04-17T07:23:13.569117+0000, chain.objs.size()=1 2024-04-17T05:46:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.3kh98CSA72nAIYzZ_oGvFYJ4FE1OZma_0 2024-04-17T05:46:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.5584119419899949430', time=2024-04-17T07:23:13.570584+0000, chain.objs.size()=1 2024-04-17T05:46:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.qAi3Rx7_jps73Eev3iIRfJSf8vgI0mF_0 2024-04-17T05:46:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14347469603618343354', time=2024-04-17T07:23:14.307713+0000, chain.objs.size()=1 2024-04-17T05:46:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.fN9ohV7z0c0z7AnSozxnMHU8zMufE8-_0 2024-04-17T05:46:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11955511870257797992', time=2024-04-17T07:23:14.453210+0000, chain.objs.size()=1 2024-04-17T05:46:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.YlocYQWpTMZzfjSrAF_g0WpIdHdE4h5_0 2024-04-17T05:46:03.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14621014350132424442', time=2024-04-17T07:23:19.135543+0000, chain.objs.size()=1 2024-04-17T05:46:03.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.b1bxNZWsxkIMVLFGYtUzde4AKfOSdSH_0 2024-04-17T05:46:03.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.6920505614668007494', time=2024-04-17T07:23:19.767657+0000, chain.objs.size()=1 2024-04-17T05:46:03.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.860+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Yh5-lXZkdQ7asFVcw0pdRXNsjXfBz07_0 2024-04-17T05:46:03.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.12580856541793125144', time=2024-04-17T07:23:21.782625+0000, chain.objs.size()=1 2024-04-17T05:46:03.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.2FFa0wkAjmxbk2bef-j5kXkQQlj8fMs_0 2024-04-17T05:46:03.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9491986966296278865', time=2024-04-17T07:23:22.550515+0000, chain.objs.size()=1 2024-04-17T05:46:03.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.Bw_QMOqN2VPd9u48an04FTncE75uIkN_0 2024-04-17T05:46:03.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.18168820556591799903', time=2024-04-17T07:23:24.769389+0000, chain.objs.size()=1 2024-04-17T05:46:03.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.N0IOfINlekI-JaXrcdzQSmSzpxgRw8w_0 2024-04-17T05:46:03.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.273085344627156710', time=2024-04-17T07:23:25.012717+0000, chain.objs.size()=1 2024-04-17T05:46:03.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.AwBIgSrVJas7TBv6G6IZus4FHUL5qwm_0 2024-04-17T05:46:03.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13814049593076985118', time=2024-04-17T07:23:26.415559+0000, chain.objs.size()=1 2024-04-17T05:46:03.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.faHIQy8Gg00BFk3wJ0zD8bgC4W3c6SQ_0 2024-04-17T05:46:03.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.11129264719518697967', time=2024-04-17T07:23:27.386847+0000, chain.objs.size()=1 2024-04-17T05:46:03.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.J5sEZeURFU7zKVs0BEvN8mBirq5xvLE_0 2024-04-17T05:46:03.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17430715528268567419', time=2024-04-17T07:23:27.523967+0000, chain.objs.size()=1 2024-04-17T05:46:03.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.DrEOoogx1PQQtEn59-nwtgZDMGjFKV7_0 2024-04-17T05:46:03.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.14860870997190672579', time=2024-04-17T07:23:28.546684+0000, chain.objs.size()=1 2024-04-17T05:46:03.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.flbH6dQgxVvMbz0ulv5y_HVFzTDNCFa_0 2024-04-17T05:46:03.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3822786591386378327', time=2024-04-17T07:23:30.152846+0000, chain.objs.size()=1 2024-04-17T05:46:03.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.8ZDvOStS3fZwE8HJAxqAnyu3_gLwaxM_0 2024-04-17T05:46:03.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2292924336909085321', time=2024-04-17T07:23:30.558128+0000, chain.objs.size()=1 2024-04-17T05:46:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.TaVyZ8md8BwYkN115xfW91pjJLMBl0u_0 2024-04-17T05:46:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13278967260417387784', time=2024-04-17T07:23:31.359525+0000, chain.objs.size()=1 2024-04-17T05:46:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.eoxvkSV_09H8ICN5Cthq8XFNjsmg5oh_0 2024-04-17T05:46:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9068634729229693341', time=2024-04-17T07:23:36.100849+0000, chain.objs.size()=1 2024-04-17T05:46:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.864+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.mBtodLIuPOnc00E0KIzpPb510IaHY0P_0 2024-04-17T05:46:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.8483004803636830428', time=2024-04-17T07:23:37.308009+0000, chain.objs.size()=1 2024-04-17T05:46:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.O3gshTrm9CctWem9lkwahYRJkaFm39M_0 2024-04-17T05:46:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.15589959319656340271', time=2024-04-17T07:23:40.401994+0000, chain.objs.size()=1 2024-04-17T05:46:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.OI-eCt8HB9Fz-F53CO7ufEu2R6ANGCR_0 2024-04-17T05:46:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.515695407243886404', time=2024-04-17T07:23:40.990214+0000, chain.objs.size()=1 2024-04-17T05:46:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.314__shadow_.UcK6w8CRKo8rEmtp84RW51MD9q7o0SE_0 2024-04-17T05:46:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.3467965871967732229', time=2024-04-17T07:26:24.463281+0000, chain.objs.size()=3 2024-04-17T05:46:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.868+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_0 2024-04-17T05:46:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_1 2024-04-17T05:46:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.476__shadow_.obeDca3C2mEdzyhhJIjNRL9pGoCzU8-_2 2024-04-17T05:46:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2_cc2a683d109de6382bb2db9b07ba5ac0', time=2024-04-17T07:26:56.122507+0000, chain.objs.size()=6 2024-04-17T05:46:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.1 2024-04-17T05:46:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.2 2024-04-17T05:46:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.3 2024-04-17T05:46:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.4 2024-04-17T05:46:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.5 2024-04-17T05:46:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.872+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.497__multipart_srcmultipart.2~SbErxs6X_rEftJtsHCg3UtBgvM9aQwK.6 2024-04-17T05:46:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7545137189350361501', time=2024-04-17T07:27:15.661303+0000, chain.objs.size()=2 2024-04-17T05:46:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.507__shadow_.pBI-ePrYW10EjECRXJ7LpXkeaFUqCvl_0 2024-04-17T05:46:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.507__shadow_.pBI-ePrYW10EjECRXJ7LpXkeaFUqCvl_1 2024-04-17T05:46:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.7467073771896163966', time=2024-04-17T07:31:11.084518+0000, chain.objs.size()=1 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.547__shadow_.3jvjx_j9EBBx3bZGE6mVejJJyeRlDRB_0 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2317520939509076849', time=2024-04-17T07:31:17.306592+0000, chain.objs.size()=1 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.548__shadow_.JzIeWnowdMiMJjt7xhMF0_pI-dRReWZ_0 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.2451117305625951122', time=2024-04-17T07:31:38.409235+0000, chain.objs.size()=1 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:eZ8crvrzE9bA8kpgxJbQyQ-eWkhxzp3_.bVEkU7SPtM3H59UxzDGEB505dD_TGJj_0 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.173290023604923664', time=2024-04-17T07:31:46.593570+0000, chain.objs.size()=1 2024-04-17T05:46:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.558__shadow:ZgrtcwMupiLsXvEzjnXfplvajM09Si-_.XRZeyICrM8FhgbIF9eOmmVLaNSIE_ZE_0 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.10288746715298405440', time=2024-04-17T07:32:33.450730+0000, chain.objs.size()=1 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:Ngjiub4oFgRz92zEUW9UhxCbU4qkvKk_.gvlJFXVGbIle-yesilcUj5s-B-o2pM8_0 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.16089705782204918277', time=2024-04-17T07:32:50.360123+0000, chain.objs.size()=1 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:ycO98VnN7CsaeKQWr7aNsesjpSbXKR8_.xtDGxC-1T_yGwFIcLu42Zo7tnp1sVR__0 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.13912938578712982721', time=2024-04-17T07:32:51.628981+0000, chain.objs.size()=1 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.566__shadow:GdR428sCb5f773x3kC4fCWg1-wMRIDW_.qDLPiP3a318VMCPD4OdHHsFcSGfR9K3_0 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.4266004487554710676', time=2024-04-17T07:37:39.740433+0000, chain.objs.size()=1 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.650__shadow_.lWGEefizPmRnc7GbP9sGRtMdBIWndkx_0 2024-04-17T05:46:03.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.17958933822409824327', time=2024-04-17T07:38:08.236540+0000, chain.objs.size()=1 2024-04-17T05:46:03.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.876+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.680__shadow_.uUkF6e9iTOUpxeWNff_Xdw3c0mxK9mL_0 2024-04-17T05:46:03.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 0/46276 2024-04-17T05:46:03.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-17T05:46:03.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.884+0000 7f039ff7f980 20 garbage collection: RGWGC::process iterating over entry tag='52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.9114902759637429835', time=2024-04-17T07:38:15.431581+0000, chain.objs.size()=1 2024-04-17T05:46:03.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:52c3b34b-e732-40a8-b5aa-dfbef5c9e391.4426.686__shadow:ll3VQC6Ad4tZ8ctFP6quxFNIYrl.zWh_.mDJnIAoVejx7dT4u7q9Rgv7r7Advrnx_0 2024-04-17T05:46:03.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.884+0000 7f039ff7f980 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-17T05:46:03.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.888+0000 7f024f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-17T05:46:03.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.892+0000 7f039ff7f980 20 remove_watcher() i=0 2024-04-17T05:46:03.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.892+0000 7f039ff7f980 2 removed watcher, disabling cache 2024-04-17T05:46:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.892+0000 7f039ff7f980 20 remove_watcher() i=1 2024-04-17T05:46:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.892+0000 7f039ff7f980 20 remove_watcher() i=2 2024-04-17T05:46:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.892+0000 7f039ff7f980 20 remove_watcher() i=3 2024-04-17T05:46:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.896+0000 7f039ff7f980 20 remove_watcher() i=4 2024-04-17T05:46:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.896+0000 7f039ff7f980 20 remove_watcher() i=5 2024-04-17T05:46:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.896+0000 7f039ff7f980 20 remove_watcher() i=6 2024-04-17T05:46:03.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-17T05:46:03.896+0000 7f039ff7f980 20 remove_watcher() i=7 2024-04-17T05:46:03.911 DEBUG:teuthology.run_tasks:Unwinding manager openssl_keys 2024-04-17T05:46:03.923 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-17T05:46:03.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T05:46:03.969 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:46:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:04.320 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:46:04.334 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2353,"stamp":"2024-04-17T05:46:03.280712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":852009202,"num_objects":4832,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":156306,"num_read_kb":2419174,"num_write":142134,"num_write_kb":2463295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13386,"ondisk_log_size":13386,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2940688,"kb_used_data":1339408,"kb_used_omap":12,"kb_used_meta":1601203,"kb_avail":752034032,"statfs":{"total":773094113280,"available":770082848768,"internally_reserved":0,"allocated":1371553792,"data_stored":1355316852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1639632433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008357"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151550+0000","last_change":"2024-04-17T05:07:45.742645+0000","last_active":"2024-04-17T05:42:45.151550+0000","last_peered":"2024-04-17T05:42:45.151550+0000","last_clean":"2024-04-17T05:42:45.151550+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:42:45.151550+0000","last_undegraded":"2024-04-17T05:42:45.151550+0000","last_fullsized":"2024-04-17T05:42:45.151550+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:06.072659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138115+0000","last_change":"2024-04-17T04:51:43.682048+0000","last_active":"2024-04-17T05:41:45.138115+0000","last_peered":"2024-04-17T05:41:45.138115+0000","last_clean":"2024-04-17T05:41:45.138115+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:41:45.138115+0000","last_undegraded":"2024-04-17T05:41:45.138115+0000","last_fullsized":"2024-04-17T05:41:45.138115+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:06.493069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164913+0000","last_change":"2024-04-17T04:51:39.638879+0000","last_active":"2024-04-17T05:41:40.164913+0000","last_peered":"2024-04-17T05:41:40.164913+0000","last_clean":"2024-04-17T05:41:40.164913+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:41:40.164913+0000","last_undegraded":"2024-04-17T05:41:40.164913+0000","last_fullsized":"2024-04-17T05:41:40.164913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:57:53.046398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":148,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139601+0000","last_change":"2024-04-17T04:47:56.968120+0000","last_active":"2024-04-17T05:42:10.139601+0000","last_peered":"2024-04-17T05:42:10.139601+0000","last_clean":"2024-04-17T05:42:10.139601+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:42:10.139601+0000","last_undegraded":"2024-04-17T05:42:10.139601+0000","last_fullsized":"2024-04-17T05:42:10.139601+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:39.089673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149910+0000","last_change":"2024-04-17T05:07:45.746586+0000","last_active":"2024-04-17T05:42:45.149910+0000","last_peered":"2024-04-17T05:42:45.149910+0000","last_clean":"2024-04-17T05:42:45.149910+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:42:45.149910+0000","last_undegraded":"2024-04-17T05:42:45.149910+0000","last_fullsized":"2024-04-17T05:42:45.149910+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:02.041624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'2","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124114+0000","last_change":"2024-04-17T04:51:43.661871+0000","last_active":"2024-04-17T05:41:45.124114+0000","last_peered":"2024-04-17T05:41:45.124114+0000","last_clean":"2024-04-17T05:41:45.124114+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:41:45.124114+0000","last_undegraded":"2024-04-17T05:41:45.124114+0000","last_fullsized":"2024-04-17T05:41:45.124114+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:46.906315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.137511+0000","last_change":"2024-04-17T04:51:39.640822+0000","last_active":"2024-04-17T05:41:40.137511+0000","last_peered":"2024-04-17T05:41:40.137511+0000","last_clean":"2024-04-17T05:41:40.137511+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:41:40.137511+0000","last_undegraded":"2024-04-17T05:41:40.137511+0000","last_fullsized":"2024-04-17T05:41:40.137511+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:51.213156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'96","reported_seq":350,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.143271+0000","last_change":"2024-04-17T04:47:56.937898+0000","last_active":"2024-04-17T05:42:15.143271+0000","last_peered":"2024-04-17T05:42:15.143271+0000","last_clean":"2024-04-17T05:42:15.143271+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:42:15.143271+0000","last_undegraded":"2024-04-17T05:42:15.143271+0000","last_fullsized":"2024-04-17T05:42:15.143271+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:02:44.307495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130858+0000","last_change":"2024-04-17T05:07:45.739412+0000","last_active":"2024-04-17T05:41:35.130858+0000","last_peered":"2024-04-17T05:41:35.130858+0000","last_clean":"2024-04-17T05:41:35.130858+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:41:35.130858+0000","last_undegraded":"2024-04-17T05:41:35.130858+0000","last_fullsized":"2024-04-17T05:41:35.130858+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:26.784407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162336+0000","last_change":"2024-04-17T04:51:43.682620+0000","last_active":"2024-04-17T05:43:50.162336+0000","last_peered":"2024-04-17T05:43:50.162336+0000","last_clean":"2024-04-17T05:43:50.162336+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:43:50.162336+0000","last_undegraded":"2024-04-17T05:43:50.162336+0000","last_fullsized":"2024-04-17T05:43:50.162336+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:29.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'2","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.164674+0000","last_change":"2024-04-17T04:51:39.605291+0000","last_active":"2024-04-17T05:43:50.164674+0000","last_peered":"2024-04-17T05:43:50.164674+0000","last_clean":"2024-04-17T05:43:50.164674+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:43:50.164674+0000","last_undegraded":"2024-04-17T05:43:50.164674+0000","last_fullsized":"2024-04-17T05:43:50.164674+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:40.774979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":121,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.150706+0000","last_change":"2024-04-17T04:47:56.953281+0000","last_active":"2024-04-17T05:43:00.150706+0000","last_peered":"2024-04-17T05:43:00.150706+0000","last_clean":"2024-04-17T05:43:00.150706+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:43:00.150706+0000","last_undegraded":"2024-04-17T05:43:00.150706+0000","last_fullsized":"2024-04-17T05:43:00.150706+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:02.682710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164774+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:41:40.164774+0000","last_peered":"2024-04-17T05:41:40.164774+0000","last_clean":"2024-04-17T05:41:40.164774+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:41:40.164774+0000","last_undegraded":"2024-04-17T05:41:40.164774+0000","last_fullsized":"2024-04-17T05:41:40.164774+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:05.623921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.195511+0000","last_change":"2024-04-17T04:51:43.676613+0000","last_active":"2024-04-17T05:43:55.195511+0000","last_peered":"2024-04-17T05:43:55.195511+0000","last_clean":"2024-04-17T05:43:55.195511+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:43:55.195511+0000","last_undegraded":"2024-04-17T05:43:55.195511+0000","last_fullsized":"2024-04-17T05:43:55.195511+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:26.269622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946142+0000","last_change":"2024-04-17T04:51:39.615233+0000","last_active":"2024-04-17T05:43:44.946142+0000","last_peered":"2024-04-17T05:43:44.946142+0000","last_clean":"2024-04-17T05:43:44.946142+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:43:44.946142+0000","last_undegraded":"2024-04-17T05:43:44.946142+0000","last_fullsized":"2024-04-17T05:43:44.946142+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:02.139603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":160,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183795+0000","last_change":"2024-04-17T04:47:56.969273+0000","last_active":"2024-04-17T05:43:00.183795+0000","last_peered":"2024-04-17T05:43:00.183795+0000","last_clean":"2024-04-17T05:43:00.183795+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:43:00.183795+0000","last_undegraded":"2024-04-17T05:43:00.183795+0000","last_fullsized":"2024-04-17T05:43:00.183795+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:24.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.194209+0000","last_change":"2024-04-17T05:07:45.745164+0000","last_active":"2024-04-17T05:43:50.194209+0000","last_peered":"2024-04-17T05:43:50.194209+0000","last_clean":"2024-04-17T05:43:50.194209+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:43:50.194209+0000","last_undegraded":"2024-04-17T05:43:50.194209+0000","last_fullsized":"2024-04-17T05:43:50.194209+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:44.184077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'1","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150275+0000","last_change":"2024-04-17T04:51:43.666389+0000","last_active":"2024-04-17T05:42:40.150275+0000","last_peered":"2024-04-17T05:42:40.150275+0000","last_clean":"2024-04-17T05:42:40.150275+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:42:40.150275+0000","last_undegraded":"2024-04-17T05:42:40.150275+0000","last_fullsized":"2024-04-17T05:42:40.150275+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:10.966058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.142876+0000","last_change":"2024-04-17T04:51:39.625539+0000","last_active":"2024-04-17T05:42:25.142876+0000","last_peered":"2024-04-17T05:42:25.142876+0000","last_clean":"2024-04-17T05:42:25.142876+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:42:25.142876+0000","last_undegraded":"2024-04-17T05:42:25.142876+0000","last_fullsized":"2024-04-17T05:42:25.142876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:55.423515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":200,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183684+0000","last_change":"2024-04-17T04:47:56.967026+0000","last_active":"2024-04-17T05:43:00.183684+0000","last_peered":"2024-04-17T05:43:00.183684+0000","last_clean":"2024-04-17T05:43:00.183684+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:43:00.183684+0000","last_undegraded":"2024-04-17T05:43:00.183684+0000","last_fullsized":"2024-04-17T05:43:00.183684+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:26.424709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158284+0000","last_change":"2024-04-17T05:07:45.754178+0000","last_active":"2024-04-17T05:43:10.158284+0000","last_peered":"2024-04-17T05:43:10.158284+0000","last_clean":"2024-04-17T05:43:10.158284+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:43:10.158284+0000","last_undegraded":"2024-04-17T05:43:10.158284+0000","last_fullsized":"2024-04-17T05:43:10.158284+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:49.281995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'4","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142841+0000","last_change":"2024-04-17T04:51:43.664767+0000","last_active":"2024-04-17T05:42:30.142841+0000","last_peered":"2024-04-17T05:42:30.142841+0000","last_clean":"2024-04-17T05:42:30.142841+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:42:30.142841+0000","last_undegraded":"2024-04-17T05:42:30.142841+0000","last_fullsized":"2024-04-17T05:42:30.142841+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:36.530590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":22528,"num_write":9,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"261'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.134896+0000","last_change":"2024-04-17T04:51:39.611306+0000","last_active":"2024-04-17T05:42:35.134896+0000","last_peered":"2024-04-17T05:42:35.134896+0000","last_clean":"2024-04-17T05:42:35.134896+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:42:35.134896+0000","last_undegraded":"2024-04-17T05:42:35.134896+0000","last_fullsized":"2024-04-17T05:42:35.134896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:51.035427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'74","reported_seq":283,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149482+0000","last_change":"2024-04-17T04:47:56.952081+0000","last_active":"2024-04-17T05:43:00.149482+0000","last_peered":"2024-04-17T05:43:00.149482+0000","last_clean":"2024-04-17T05:43:00.149482+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:43:00.149482+0000","last_undegraded":"2024-04-17T05:43:00.149482+0000","last_fullsized":"2024-04-17T05:43:00.149482+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:08:06.903329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145686+0000","last_change":"2024-04-17T05:07:45.754885+0000","last_active":"2024-04-17T05:42:45.145686+0000","last_peered":"2024-04-17T05:42:45.145686+0000","last_clean":"2024-04-17T05:42:45.145686+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:42:45.145686+0000","last_undegraded":"2024-04-17T05:42:45.145686+0000","last_fullsized":"2024-04-17T05:42:45.145686+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:02.913191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133462+0000","last_change":"2024-04-17T04:51:43.664884+0000","last_active":"2024-04-17T05:41:45.133462+0000","last_peered":"2024-04-17T05:41:45.133462+0000","last_clean":"2024-04-17T05:41:45.133462+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:41:45.133462+0000","last_undegraded":"2024-04-17T05:41:45.133462+0000","last_fullsized":"2024-04-17T05:41:45.133462+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:04.163089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'8","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122767+0000","last_change":"2024-04-17T04:51:39.621614+0000","last_active":"2024-04-17T05:41:40.122767+0000","last_peered":"2024-04-17T05:41:40.122767+0000","last_clean":"2024-04-17T05:41:40.122767+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:41:40.122767+0000","last_undegraded":"2024-04-17T05:41:40.122767+0000","last_fullsized":"2024-04-17T05:41:40.122767+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:01.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":23,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":74,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:20.127064+0000","last_change":"2024-04-17T04:47:56.955568+0000","last_active":"2024-04-17T05:41:20.127064+0000","last_peered":"2024-04-17T05:41:20.127064+0000","last_clean":"2024-04-17T05:41:20.127064+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:41:20.127064+0000","last_undegraded":"2024-04-17T05:41:20.127064+0000","last_fullsized":"2024-04-17T05:41:20.127064+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:09:24.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145754+0000","last_change":"2024-04-17T05:07:45.757519+0000","last_active":"2024-04-17T05:42:45.145754+0000","last_peered":"2024-04-17T05:42:45.145754+0000","last_clean":"2024-04-17T05:42:45.145754+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:42:45.145754+0000","last_undegraded":"2024-04-17T05:42:45.145754+0000","last_fullsized":"2024-04-17T05:42:45.145754+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:27.671557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165700+0000","last_change":"2024-04-17T04:51:43.686621+0000","last_active":"2024-04-17T05:41:45.165700+0000","last_peered":"2024-04-17T05:41:45.165700+0000","last_clean":"2024-04-17T05:41:45.165700+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:41:45.165700+0000","last_undegraded":"2024-04-17T05:41:45.165700+0000","last_fullsized":"2024-04-17T05:41:45.165700+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:36.829860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132003+0000","last_change":"2024-04-17T04:51:39.643524+0000","last_active":"2024-04-17T05:41:40.132003+0000","last_peered":"2024-04-17T05:41:40.132003+0000","last_clean":"2024-04-17T05:41:40.132003+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:41:40.132003+0000","last_undegraded":"2024-04-17T05:41:40.132003+0000","last_fullsized":"2024-04-17T05:41:40.132003+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:01.434777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":90,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163169+0000","last_change":"2024-04-17T04:47:56.932780+0000","last_active":"2024-04-17T05:43:35.163169+0000","last_peered":"2024-04-17T05:43:35.163169+0000","last_clean":"2024-04-17T05:43:35.163169+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:43:35.163169+0000","last_undegraded":"2024-04-17T05:43:35.163169+0000","last_fullsized":"2024-04-17T05:43:35.163169+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:07.644614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151106+0000","last_change":"2024-04-17T05:07:45.743105+0000","last_active":"2024-04-17T05:42:45.151106+0000","last_peered":"2024-04-17T05:42:45.151106+0000","last_clean":"2024-04-17T05:42:45.151106+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:42:45.151106+0000","last_undegraded":"2024-04-17T05:42:45.151106+0000","last_fullsized":"2024-04-17T05:42:45.151106+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:30.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176828+0000","last_change":"2024-04-17T04:51:43.664814+0000","last_active":"2024-04-17T05:42:30.176828+0000","last_peered":"2024-04-17T05:42:30.176828+0000","last_clean":"2024-04-17T05:42:30.176828+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:42:30.176828+0000","last_undegraded":"2024-04-17T05:42:30.176828+0000","last_fullsized":"2024-04-17T05:42:30.176828+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:39.897527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157053+0000","last_change":"2024-04-17T04:51:39.626132+0000","last_active":"2024-04-17T05:43:10.157053+0000","last_peered":"2024-04-17T05:43:10.157053+0000","last_clean":"2024-04-17T05:43:10.157053+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:43:10.157053+0000","last_undegraded":"2024-04-17T05:43:10.157053+0000","last_fullsized":"2024-04-17T05:43:10.157053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:23.450299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":80,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158447+0000","last_change":"2024-04-17T04:47:56.956719+0000","last_active":"2024-04-17T05:43:35.158447+0000","last_peered":"2024-04-17T05:43:35.158447+0000","last_clean":"2024-04-17T05:43:35.158447+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:43:35.158447+0000","last_undegraded":"2024-04-17T05:43:35.158447+0000","last_fullsized":"2024-04-17T05:43:35.158447+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:10.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149849+0000","last_change":"2024-04-17T05:07:45.738001+0000","last_active":"2024-04-17T05:42:45.149849+0000","last_peered":"2024-04-17T05:42:45.149849+0000","last_clean":"2024-04-17T05:42:45.149849+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:42:45.149849+0000","last_undegraded":"2024-04-17T05:42:45.149849+0000","last_fullsized":"2024-04-17T05:42:45.149849+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.048242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187309+0000","last_change":"2024-04-17T04:51:43.682839+0000","last_active":"2024-04-17T05:43:15.187309+0000","last_peered":"2024-04-17T05:43:15.187309+0000","last_clean":"2024-04-17T05:43:15.187309+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:43:15.187309+0000","last_undegraded":"2024-04-17T05:43:15.187309+0000","last_fullsized":"2024-04-17T05:43:15.187309+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:32.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'1","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141354+0000","last_change":"2024-04-17T04:51:39.639685+0000","last_active":"2024-04-17T05:42:25.141354+0000","last_peered":"2024-04-17T05:42:25.141354+0000","last_clean":"2024-04-17T05:42:25.141354+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:42:25.141354+0000","last_undegraded":"2024-04-17T05:42:25.141354+0000","last_fullsized":"2024-04-17T05:42:25.141354+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:41.954023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191081+0000","last_change":"2024-04-17T04:47:56.966803+0000","last_active":"2024-04-17T05:43:35.191081+0000","last_peered":"2024-04-17T05:43:35.191081+0000","last_clean":"2024-04-17T05:43:35.191081+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:43:35.191081+0000","last_undegraded":"2024-04-17T05:43:35.191081+0000","last_fullsized":"2024-04-17T05:43:35.191081+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:27.144075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145538+0000","last_change":"2024-04-17T05:07:45.747162+0000","last_active":"2024-04-17T05:42:45.145538+0000","last_peered":"2024-04-17T05:42:45.145538+0000","last_clean":"2024-04-17T05:42:45.145538+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:42:45.145538+0000","last_undegraded":"2024-04-17T05:42:45.145538+0000","last_fullsized":"2024-04-17T05:42:45.145538+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:25.853530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'3","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176921+0000","last_change":"2024-04-17T04:51:43.671647+0000","last_active":"2024-04-17T05:42:30.176921+0000","last_peered":"2024-04-17T05:42:30.176921+0000","last_clean":"2024-04-17T05:42:30.176921+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:42:30.176921+0000","last_undegraded":"2024-04-17T05:42:30.176921+0000","last_fullsized":"2024-04-17T05:42:30.176921+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:09:37.852227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5291313,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":8,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'10","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145913+0000","last_change":"2024-04-17T04:51:39.612849+0000","last_active":"2024-04-17T05:42:25.145913+0000","last_peered":"2024-04-17T05:42:25.145913+0000","last_clean":"2024-04-17T05:42:25.145913+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:42:25.145913+0000","last_undegraded":"2024-04-17T05:42:25.145913+0000","last_fullsized":"2024-04-17T05:42:25.145913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:41.680996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":46082,"num_write":14,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.161351+0000","last_change":"2024-04-17T04:47:56.937214+0000","last_active":"2024-04-17T05:43:35.161351+0000","last_peered":"2024-04-17T05:43:35.161351+0000","last_clean":"2024-04-17T05:43:35.161351+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:43:35.161351+0000","last_undegraded":"2024-04-17T05:43:35.161351+0000","last_fullsized":"2024-04-17T05:43:35.161351+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:02.809448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151449+0000","last_change":"2024-04-17T05:07:45.744163+0000","last_active":"2024-04-17T05:42:45.151449+0000","last_peered":"2024-04-17T05:42:45.151449+0000","last_clean":"2024-04-17T05:42:45.151449+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:42:45.151449+0000","last_undegraded":"2024-04-17T05:42:45.151449+0000","last_fullsized":"2024-04-17T05:42:45.151449+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:42.169974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'10","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.133958+0000","last_change":"2024-04-17T04:51:43.667351+0000","last_active":"2024-04-17T05:42:30.133958+0000","last_peered":"2024-04-17T05:42:30.133958+0000","last_clean":"2024-04-17T05:42:30.133958+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:42:30.133958+0000","last_undegraded":"2024-04-17T05:42:30.133958+0000","last_fullsized":"2024-04-17T05:42:30.133958+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:27.804721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":37890,"num_write":19,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143121+0000","last_change":"2024-04-17T04:51:39.639400+0000","last_active":"2024-04-17T05:42:25.143121+0000","last_peered":"2024-04-17T05:42:25.143121+0000","last_clean":"2024-04-17T05:42:25.143121+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:42:25.143121+0000","last_undegraded":"2024-04-17T05:42:25.143121+0000","last_fullsized":"2024-04-17T05:42:25.143121+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:39.092327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":211,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133386+0000","last_change":"2024-04-17T04:47:56.964248+0000","last_active":"2024-04-17T05:41:50.133386+0000","last_peered":"2024-04-17T05:41:50.133386+0000","last_clean":"2024-04-17T05:41:50.133386+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:41:50.133386+0000","last_undegraded":"2024-04-17T05:41:50.133386+0000","last_fullsized":"2024-04-17T05:41:50.133386+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:19.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163617+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:41:35.163617+0000","last_peered":"2024-04-17T05:41:35.163617+0000","last_clean":"2024-04-17T05:41:35.163617+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:41:35.163617+0000","last_undegraded":"2024-04-17T05:41:35.163617+0000","last_fullsized":"2024-04-17T05:41:35.163617+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:45.333124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'5","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157158+0000","last_change":"2024-04-17T04:51:43.670407+0000","last_active":"2024-04-17T05:43:05.157158+0000","last_peered":"2024-04-17T05:43:05.157158+0000","last_clean":"2024-04-17T05:43:05.157158+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:43:05.157158+0000","last_undegraded":"2024-04-17T05:43:05.157158+0000","last_fullsized":"2024-04-17T05:43:05.157158+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:31.623818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:59.936150+0000","last_change":"2024-04-17T04:51:39.613038+0000","last_active":"2024-04-17T05:42:59.936150+0000","last_peered":"2024-04-17T05:42:59.936150+0000","last_clean":"2024-04-17T05:42:59.936150+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:42:59.936150+0000","last_undegraded":"2024-04-17T05:42:59.936150+0000","last_fullsized":"2024-04-17T05:42:59.936150+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:51.929471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.130770+0000","last_change":"2024-04-17T04:47:56.962330+0000","last_active":"2024-04-17T05:41:40.130770+0000","last_peered":"2024-04-17T05:41:40.130770+0000","last_clean":"2024-04-17T05:41:40.130770+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:41:40.130770+0000","last_undegraded":"2024-04-17T05:41:40.130770+0000","last_fullsized":"2024-04-17T05:41:40.130770+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:38.049482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130958+0000","last_change":"2024-04-17T05:07:45.749757+0000","last_active":"2024-04-17T05:41:35.130958+0000","last_peered":"2024-04-17T05:41:35.130958+0000","last_clean":"2024-04-17T05:41:35.130958+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:41:35.130958+0000","last_undegraded":"2024-04-17T05:41:35.130958+0000","last_fullsized":"2024-04-17T05:41:35.130958+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:50:18.794745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'1","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157104+0000","last_change":"2024-04-17T04:51:43.650369+0000","last_active":"2024-04-17T05:43:05.157104+0000","last_peered":"2024-04-17T05:43:05.157104+0000","last_clean":"2024-04-17T05:43:05.157104+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:43:05.157104+0000","last_undegraded":"2024-04-17T05:43:05.157104+0000","last_fullsized":"2024-04-17T05:43:05.157104+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:16:08.661428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183604+0000","last_change":"2024-04-17T04:51:39.631783+0000","last_active":"2024-04-17T05:43:00.183604+0000","last_peered":"2024-04-17T05:43:00.183604+0000","last_clean":"2024-04-17T05:43:00.183604+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:43:00.183604+0000","last_undegraded":"2024-04-17T05:43:00.183604+0000","last_fullsized":"2024-04-17T05:43:00.183604+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:32.873589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":132,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.131963+0000","last_change":"2024-04-17T04:47:56.967539+0000","last_active":"2024-04-17T05:41:40.131963+0000","last_peered":"2024-04-17T05:41:40.131963+0000","last_clean":"2024-04-17T05:41:40.131963+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:41:40.131963+0000","last_undegraded":"2024-04-17T05:41:40.131963+0000","last_fullsized":"2024-04-17T05:41:40.131963+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:50.899642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"263'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.149607+0000","last_change":"2024-04-17T05:07:45.748765+0000","last_active":"2024-04-17T05:42:55.149607+0000","last_peered":"2024-04-17T05:42:55.149607+0000","last_clean":"2024-04-17T05:42:55.149607+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:42:55.149607+0000","last_undegraded":"2024-04-17T05:42:55.149607+0000","last_fullsized":"2024-04-17T05:42:55.149607+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:21.732472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'2","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.184978+0000","last_change":"2024-04-17T04:51:43.684076+0000","last_active":"2024-04-17T05:43:05.184978+0000","last_peered":"2024-04-17T05:43:05.184978+0000","last_clean":"2024-04-17T05:43:05.184978+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:43:05.184978+0000","last_undegraded":"2024-04-17T05:43:05.184978+0000","last_fullsized":"2024-04-17T05:43:05.184978+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.261686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":10240,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149546+0000","last_change":"2024-04-17T04:51:39.626930+0000","last_active":"2024-04-17T05:43:00.149546+0000","last_peered":"2024-04-17T05:43:00.149546+0000","last_clean":"2024-04-17T05:43:00.149546+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:43:00.149546+0000","last_undegraded":"2024-04-17T05:43:00.149546+0000","last_fullsized":"2024-04-17T05:43:00.149546+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:24:11.581902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122667+0000","last_change":"2024-04-17T04:47:56.946016+0000","last_active":"2024-04-17T05:41:40.122667+0000","last_peered":"2024-04-17T05:41:40.122667+0000","last_clean":"2024-04-17T05:41:40.122667+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:41:40.122667+0000","last_undegraded":"2024-04-17T05:41:40.122667+0000","last_fullsized":"2024-04-17T05:41:40.122667+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:39.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163578+0000","last_change":"2024-04-17T05:07:45.741199+0000","last_active":"2024-04-17T05:41:35.163578+0000","last_peered":"2024-04-17T05:41:35.163578+0000","last_clean":"2024-04-17T05:41:35.163578+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:41:35.163578+0000","last_undegraded":"2024-04-17T05:41:35.163578+0000","last_fullsized":"2024-04-17T05:41:35.163578+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:14:05.831006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150592+0000","last_change":"2024-04-17T04:51:43.673701+0000","last_active":"2024-04-17T05:43:05.150592+0000","last_peered":"2024-04-17T05:43:05.150592+0000","last_clean":"2024-04-17T05:43:05.150592+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:43:05.150592+0000","last_undegraded":"2024-04-17T05:43:05.150592+0000","last_fullsized":"2024-04-17T05:43:05.150592+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:00.395599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154845+0000","last_change":"2024-04-17T04:51:39.625916+0000","last_active":"2024-04-17T05:43:00.154845+0000","last_peered":"2024-04-17T05:43:00.154845+0000","last_clean":"2024-04-17T05:43:00.154845+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:43:00.154845+0000","last_undegraded":"2024-04-17T05:43:00.154845+0000","last_fullsized":"2024-04-17T05:43:00.154845+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:35:16.145280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":195,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164555+0000","last_change":"2024-04-17T04:47:56.960822+0000","last_active":"2024-04-17T05:41:40.164555+0000","last_peered":"2024-04-17T05:41:40.164555+0000","last_clean":"2024-04-17T05:41:40.164555+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:41:40.164555+0000","last_undegraded":"2024-04-17T05:41:40.164555+0000","last_fullsized":"2024-04-17T05:41:40.164555+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:26.089711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.147879+0000","last_change":"2024-04-17T05:07:45.745113+0000","last_active":"2024-04-17T05:42:35.147879+0000","last_peered":"2024-04-17T05:42:35.147879+0000","last_clean":"2024-04-17T05:42:35.147879+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:42:35.147879+0000","last_undegraded":"2024-04-17T05:42:35.147879+0000","last_fullsized":"2024-04-17T05:42:35.147879+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:03.124310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'4","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.190008+0000","last_change":"2024-04-17T04:51:43.682734+0000","last_active":"2024-04-17T05:43:30.190008+0000","last_peered":"2024-04-17T05:43:30.190008+0000","last_clean":"2024-04-17T05:43:30.190008+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:43:30.190008+0000","last_undegraded":"2024-04-17T05:43:30.190008+0000","last_fullsized":"2024-04-17T05:43:30.190008+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:32.531568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":9,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154462+0000","last_change":"2024-04-17T04:51:39.645602+0000","last_active":"2024-04-17T05:43:25.154462+0000","last_peered":"2024-04-17T05:43:25.154462+0000","last_clean":"2024-04-17T05:43:25.154462+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:43:25.154462+0000","last_undegraded":"2024-04-17T05:43:25.154462+0000","last_fullsized":"2024-04-17T05:43:25.154462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:02.515138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.173510+0000","last_change":"2024-04-17T04:47:56.962048+0000","last_active":"2024-04-17T05:42:15.173510+0000","last_peered":"2024-04-17T05:42:15.173510+0000","last_clean":"2024-04-17T05:42:15.173510+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:42:15.173510+0000","last_undegraded":"2024-04-17T05:42:15.173510+0000","last_fullsized":"2024-04-17T05:42:15.173510+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:15.157938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159170+0000","last_change":"2024-04-17T05:07:45.751549+0000","last_active":"2024-04-17T05:43:45.159170+0000","last_peered":"2024-04-17T05:43:45.159170+0000","last_clean":"2024-04-17T05:43:45.159170+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:43:45.159170+0000","last_undegraded":"2024-04-17T05:43:45.159170+0000","last_fullsized":"2024-04-17T05:43:45.159170+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:47.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.189964+0000","last_change":"2024-04-17T04:51:43.688891+0000","last_active":"2024-04-17T05:43:30.189964+0000","last_peered":"2024-04-17T05:43:30.189964+0000","last_clean":"2024-04-17T05:43:30.189964+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:43:30.189964+0000","last_undegraded":"2024-04-17T05:43:30.189964+0000","last_fullsized":"2024-04-17T05:43:30.189964+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:35.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":9216,"num_write":3,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'4","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.159519+0000","last_change":"2024-04-17T04:51:39.634707+0000","last_active":"2024-04-17T05:43:25.159519+0000","last_peered":"2024-04-17T05:43:25.159519+0000","last_clean":"2024-04-17T05:43:25.159519+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:43:25.159519+0000","last_undegraded":"2024-04-17T05:43:25.159519+0000","last_fullsized":"2024-04-17T05:43:25.159519+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:09.043206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":11264,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141762+0000","last_change":"2024-04-17T04:47:56.954332+0000","last_active":"2024-04-17T05:42:00.141762+0000","last_peered":"2024-04-17T05:42:00.141762+0000","last_clean":"2024-04-17T05:42:00.141762+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:42:00.141762+0000","last_undegraded":"2024-04-17T05:42:00.141762+0000","last_fullsized":"2024-04-17T05:42:00.141762+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:07.662187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141842+0000","last_change":"2024-04-17T05:07:45.741143+0000","last_active":"2024-04-17T05:42:25.141842+0000","last_peered":"2024-04-17T05:42:25.141842+0000","last_clean":"2024-04-17T05:42:25.141842+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:42:25.141842+0000","last_undegraded":"2024-04-17T05:42:25.141842+0000","last_fullsized":"2024-04-17T05:42:25.141842+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:50.193462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164848+0000","last_change":"2024-04-17T04:51:43.657675+0000","last_active":"2024-04-17T05:43:40.164848+0000","last_peered":"2024-04-17T05:43:40.164848+0000","last_clean":"2024-04-17T05:43:40.164848+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:43:40.164848+0000","last_undegraded":"2024-04-17T05:43:40.164848+0000","last_fullsized":"2024-04-17T05:43:40.164848+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:59:28.152355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'2","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.156097+0000","last_change":"2024-04-17T04:51:39.634454+0000","last_active":"2024-04-17T05:43:25.156097+0000","last_peered":"2024-04-17T05:43:25.156097+0000","last_clean":"2024-04-17T05:43:25.156097+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:43:25.156097+0000","last_undegraded":"2024-04-17T05:43:25.156097+0000","last_fullsized":"2024-04-17T05:43:25.156097+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:48.747740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":123,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136769+0000","last_change":"2024-04-17T04:47:56.963915+0000","last_active":"2024-04-17T05:42:00.136769+0000","last_peered":"2024-04-17T05:42:00.136769+0000","last_clean":"2024-04-17T05:42:00.136769+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:42:00.136769+0000","last_undegraded":"2024-04-17T05:42:00.136769+0000","last_fullsized":"2024-04-17T05:42:00.136769+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:58.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141220+0000","last_change":"2024-04-17T05:07:45.757127+0000","last_active":"2024-04-17T05:42:25.141220+0000","last_peered":"2024-04-17T05:42:25.141220+0000","last_clean":"2024-04-17T05:42:25.141220+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:42:25.141220+0000","last_undegraded":"2024-04-17T05:42:25.141220+0000","last_fullsized":"2024-04-17T05:42:25.141220+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:42.585132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130813+0000","last_change":"2024-04-17T04:52:04.726201+0000","last_active":"2024-04-17T05:41:35.130813+0000","last_peered":"2024-04-17T05:41:35.130813+0000","last_clean":"2024-04-17T05:41:35.130813+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:41:35.130813+0000","last_undegraded":"2024-04-17T05:41:35.130813+0000","last_fullsized":"2024-04-17T05:41:35.130813+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:14:03.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163394+0000","last_change":"2024-04-17T04:51:39.618531+0000","last_active":"2024-04-17T05:43:35.163394+0000","last_peered":"2024-04-17T05:43:35.163394+0000","last_clean":"2024-04-17T05:43:35.163394+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:43:35.163394+0000","last_undegraded":"2024-04-17T05:43:35.163394+0000","last_fullsized":"2024-04-17T05:43:35.163394+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:52.911539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":128,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139900+0000","last_change":"2024-04-17T04:47:56.962022+0000","last_active":"2024-04-17T05:42:00.139900+0000","last_peered":"2024-04-17T05:42:00.139900+0000","last_clean":"2024-04-17T05:42:00.139900+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:42:00.139900+0000","last_undegraded":"2024-04-17T05:42:00.139900+0000","last_fullsized":"2024-04-17T05:42:00.139900+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:04.015930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193091+0000","last_change":"2024-04-17T05:07:45.744264+0000","last_active":"2024-04-17T05:43:45.193091+0000","last_peered":"2024-04-17T05:43:45.193091+0000","last_clean":"2024-04-17T05:43:45.193091+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:43:45.193091+0000","last_undegraded":"2024-04-17T05:43:45.193091+0000","last_fullsized":"2024-04-17T05:43:45.193091+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:41.404295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'6","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.945897+0000","last_change":"2024-04-17T04:51:43.661076+0000","last_active":"2024-04-17T05:43:44.945897+0000","last_peered":"2024-04-17T05:43:44.945897+0000","last_clean":"2024-04-17T05:43:44.945897+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:43:44.945897+0000","last_undegraded":"2024-04-17T05:43:44.945897+0000","last_fullsized":"2024-04-17T05:43:44.945897+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:54.620750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48424,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":5216,"num_write":11,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159409+0000","last_change":"2024-04-17T04:51:39.633118+0000","last_active":"2024-04-17T05:43:40.159409+0000","last_peered":"2024-04-17T05:43:40.159409+0000","last_clean":"2024-04-17T05:43:40.159409+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:43:40.159409+0000","last_undegraded":"2024-04-17T05:43:40.159409+0000","last_fullsized":"2024-04-17T05:43:40.159409+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:25:34.341448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":102,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137165+0000","last_change":"2024-04-17T04:47:56.963262+0000","last_active":"2024-04-17T05:42:00.137165+0000","last_peered":"2024-04-17T05:42:00.137165+0000","last_clean":"2024-04-17T05:42:00.137165+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:42:00.137165+0000","last_undegraded":"2024-04-17T05:42:00.137165+0000","last_fullsized":"2024-04-17T05:42:00.137165+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:09.975510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.129755+0000","last_change":"2024-04-17T05:07:45.757192+0000","last_active":"2024-04-17T05:41:35.129755+0000","last_peered":"2024-04-17T05:41:35.129755+0000","last_clean":"2024-04-17T05:41:35.129755+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:41:35.129755+0000","last_undegraded":"2024-04-17T05:41:35.129755+0000","last_fullsized":"2024-04-17T05:41:35.129755+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:30:19.313475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'4","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.158953+0000","last_change":"2024-04-17T04:51:43.675078+0000","last_active":"2024-04-17T05:43:45.158953+0000","last_peered":"2024-04-17T05:43:45.158953+0000","last_clean":"2024-04-17T05:43:45.158953+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:43:45.158953+0000","last_undegraded":"2024-04-17T05:43:45.158953+0000","last_fullsized":"2024-04-17T05:43:45.158953+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:00.172403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164451+0000","last_change":"2024-04-17T04:51:39.618066+0000","last_active":"2024-04-17T05:43:40.164451+0000","last_peered":"2024-04-17T05:43:40.164451+0000","last_clean":"2024-04-17T05:43:40.164451+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:43:40.164451+0000","last_undegraded":"2024-04-17T05:43:40.164451+0000","last_fullsized":"2024-04-17T05:43:40.164451+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:51.504887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":286,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140125+0000","last_change":"2024-04-17T04:47:56.963049+0000","last_active":"2024-04-17T05:42:20.140125+0000","last_peered":"2024-04-17T05:42:20.140125+0000","last_clean":"2024-04-17T05:42:20.140125+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:42:20.140125+0000","last_undegraded":"2024-04-17T05:42:20.140125+0000","last_fullsized":"2024-04-17T05:42:20.140125+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:08.912141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159213+0000","last_change":"2024-04-17T05:07:45.754827+0000","last_active":"2024-04-17T05:43:45.159213+0000","last_peered":"2024-04-17T05:43:45.159213+0000","last_clean":"2024-04-17T05:43:45.159213+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:43:45.159213+0000","last_undegraded":"2024-04-17T05:43:45.159213+0000","last_fullsized":"2024-04-17T05:43:45.159213+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:12.463675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.162241+0000","last_change":"2024-04-17T04:51:43.655115+0000","last_active":"2024-04-17T05:43:30.162241+0000","last_peered":"2024-04-17T05:43:30.162241+0000","last_clean":"2024-04-17T05:43:30.162241+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:43:30.162241+0000","last_undegraded":"2024-04-17T05:43:30.162241+0000","last_fullsized":"2024-04-17T05:43:30.162241+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:20.469117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162125+0000","last_change":"2024-04-17T04:51:39.633266+0000","last_active":"2024-04-17T05:43:50.162125+0000","last_peered":"2024-04-17T05:43:50.162125+0000","last_clean":"2024-04-17T05:43:50.162125+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:43:50.162125+0000","last_undegraded":"2024-04-17T05:43:50.162125+0000","last_fullsized":"2024-04-17T05:43:50.162125+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:40.414020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.171752+0000","last_change":"2024-04-17T04:47:56.952507+0000","last_active":"2024-04-17T05:42:10.171752+0000","last_peered":"2024-04-17T05:42:10.171752+0000","last_clean":"2024-04-17T05:42:10.171752+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:42:10.171752+0000","last_undegraded":"2024-04-17T05:42:10.171752+0000","last_fullsized":"2024-04-17T05:42:10.171752+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:18.535221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.160987+0000","last_change":"2024-04-17T05:07:45.739696+0000","last_active":"2024-04-17T05:43:45.160987+0000","last_peered":"2024-04-17T05:43:45.160987+0000","last_clean":"2024-04-17T05:43:45.160987+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:43:45.160987+0000","last_undegraded":"2024-04-17T05:43:45.160987+0000","last_fullsized":"2024-04-17T05:43:45.160987+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:02:31.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.131990+0000","last_change":"2024-04-17T04:51:43.675773+0000","last_active":"2024-04-17T05:41:45.131990+0000","last_peered":"2024-04-17T05:41:45.131990+0000","last_clean":"2024-04-17T05:41:45.131990+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:41:45.131990+0000","last_undegraded":"2024-04-17T05:41:45.131990+0000","last_fullsized":"2024-04-17T05:41:45.131990+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:07.015940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:24.941710+0000","last_change":"2024-04-17T04:51:39.621153+0000","last_active":"2024-04-17T05:43:24.941710+0000","last_peered":"2024-04-17T05:43:24.941710+0000","last_clean":"2024-04-17T05:43:24.941710+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:43:24.941710+0000","last_undegraded":"2024-04-17T05:43:24.941710+0000","last_fullsized":"2024-04-17T05:43:24.941710+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:36.053793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":188,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138665+0000","last_change":"2024-04-17T04:47:56.966581+0000","last_active":"2024-04-17T05:42:10.138665+0000","last_peered":"2024-04-17T05:42:10.138665+0000","last_clean":"2024-04-17T05:42:10.138665+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:42:10.138665+0000","last_undegraded":"2024-04-17T05:42:10.138665+0000","last_fullsized":"2024-04-17T05:42:10.138665+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:35.900737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151057+0000","last_change":"2024-04-17T05:07:45.747152+0000","last_active":"2024-04-17T05:43:10.151057+0000","last_peered":"2024-04-17T05:43:10.151057+0000","last_clean":"2024-04-17T05:43:10.151057+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:43:10.151057+0000","last_undegraded":"2024-04-17T05:43:10.151057+0000","last_fullsized":"2024-04-17T05:43:10.151057+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:50:06.059672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136102+0000","last_change":"2024-04-17T04:51:43.682823+0000","last_active":"2024-04-17T05:41:55.136102+0000","last_peered":"2024-04-17T05:41:55.136102+0000","last_clean":"2024-04-17T05:41:55.136102+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:41:55.136102+0000","last_undegraded":"2024-04-17T05:41:55.136102+0000","last_fullsized":"2024-04-17T05:41:55.136102+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:56:21.798951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'7","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159240+0000","last_change":"2024-04-17T04:51:39.636037+0000","last_active":"2024-04-17T05:43:45.159240+0000","last_peered":"2024-04-17T05:43:45.159240+0000","last_clean":"2024-04-17T05:43:45.159240+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:43:45.159240+0000","last_undegraded":"2024-04-17T05:43:45.159240+0000","last_fullsized":"2024-04-17T05:43:45.159240+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:03.304591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":11291,"num_write":12,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":149,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139651+0000","last_change":"2024-04-17T04:47:56.956029+0000","last_active":"2024-04-17T05:42:10.139651+0000","last_peered":"2024-04-17T05:42:10.139651+0000","last_clean":"2024-04-17T05:42:10.139651+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:42:10.139651+0000","last_undegraded":"2024-04-17T05:42:10.139651+0000","last_fullsized":"2024-04-17T05:42:10.139651+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:21.889729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157001+0000","last_change":"2024-04-17T05:07:45.753495+0000","last_active":"2024-04-17T05:43:10.157001+0000","last_peered":"2024-04-17T05:43:10.157001+0000","last_clean":"2024-04-17T05:43:10.157001+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:43:10.157001+0000","last_undegraded":"2024-04-17T05:43:10.157001+0000","last_fullsized":"2024-04-17T05:43:10.157001+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:29:25.567114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162221+0000","last_change":"2024-04-17T04:51:43.681581+0000","last_active":"2024-04-17T05:43:50.162221+0000","last_peered":"2024-04-17T05:43:50.162221+0000","last_clean":"2024-04-17T05:43:50.162221+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:43:50.162221+0000","last_undegraded":"2024-04-17T05:43:50.162221+0000","last_fullsized":"2024-04-17T05:43:50.162221+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:53.170686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'2","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.165685+0000","last_change":"2024-04-17T04:51:39.634757+0000","last_active":"2024-04-17T05:43:55.165685+0000","last_peered":"2024-04-17T05:43:55.165685+0000","last_clean":"2024-04-17T05:43:55.165685+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:43:55.165685+0000","last_undegraded":"2024-04-17T05:43:55.165685+0000","last_fullsized":"2024-04-17T05:43:55.165685+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:38.980103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":196,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138819+0000","last_change":"2024-04-17T04:47:56.946324+0000","last_active":"2024-04-17T05:42:10.138819+0000","last_peered":"2024-04-17T05:42:10.138819+0000","last_clean":"2024-04-17T05:42:10.138819+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:42:10.138819+0000","last_undegraded":"2024-04-17T05:42:10.138819+0000","last_fullsized":"2024-04-17T05:42:10.138819+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:09.179720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167123+0000","last_change":"2024-04-17T05:07:45.736217+0000","last_active":"2024-04-17T05:43:50.167123+0000","last_peered":"2024-04-17T05:43:50.167123+0000","last_clean":"2024-04-17T05:43:50.167123+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:43:50.167123+0000","last_undegraded":"2024-04-17T05:43:50.167123+0000","last_fullsized":"2024-04-17T05:43:50.167123+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:04.504706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'17","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159324+0000","last_change":"2024-04-17T04:51:43.675854+0000","last_active":"2024-04-17T05:43:45.159324+0000","last_peered":"2024-04-17T05:43:45.159324+0000","last_clean":"2024-04-17T05:43:45.159324+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:43:45.159324+0000","last_undegraded":"2024-04-17T05:43:45.159324+0000","last_fullsized":"2024-04-17T05:43:45.159324+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:54.193750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":54274,"num_write":16,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'10","reported_seq":79,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163666+0000","last_change":"2024-04-17T04:51:39.635143+0000","last_active":"2024-04-17T05:43:40.163666+0000","last_peered":"2024-04-17T05:43:40.163666+0000","last_clean":"2024-04-17T05:43:40.163666+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:43:40.163666+0000","last_undegraded":"2024-04-17T05:43:40.163666+0000","last_fullsized":"2024-04-17T05:43:40.163666+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:53:08.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140249+0000","last_change":"2024-04-17T04:47:56.969411+0000","last_active":"2024-04-17T05:42:20.140249+0000","last_peered":"2024-04-17T05:42:20.140249+0000","last_clean":"2024-04-17T05:42:20.140249+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:42:20.140249+0000","last_undegraded":"2024-04-17T05:42:20.140249+0000","last_fullsized":"2024-04-17T05:42:20.140249+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:56.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938315+0000","last_change":"2024-04-17T05:07:45.749227+0000","last_active":"2024-04-17T05:43:09.938315+0000","last_peered":"2024-04-17T05:43:09.938315+0000","last_clean":"2024-04-17T05:43:09.938315+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:43:09.938315+0000","last_undegraded":"2024-04-17T05:43:09.938315+0000","last_fullsized":"2024-04-17T05:43:09.938315+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:15.680393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946093+0000","last_change":"2024-04-17T04:51:43.648648+0000","last_active":"2024-04-17T05:43:44.946093+0000","last_peered":"2024-04-17T05:43:44.946093+0000","last_clean":"2024-04-17T05:43:44.946093+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:43:44.946093+0000","last_undegraded":"2024-04-17T05:43:44.946093+0000","last_fullsized":"2024-04-17T05:43:44.946093+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:57:25.905644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.158015+0000","last_change":"2024-04-17T04:51:39.646180+0000","last_active":"2024-04-17T05:43:40.158015+0000","last_peered":"2024-04-17T05:43:40.158015+0000","last_clean":"2024-04-17T05:43:40.158015+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:43:40.158015+0000","last_undegraded":"2024-04-17T05:43:40.158015+0000","last_fullsized":"2024-04-17T05:43:40.158015+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:55.361466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":110,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143901+0000","last_change":"2024-04-17T04:47:56.947428+0000","last_active":"2024-04-17T05:42:10.143901+0000","last_peered":"2024-04-17T05:42:10.143901+0000","last_clean":"2024-04-17T05:42:10.143901+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:42:10.143901+0000","last_undegraded":"2024-04-17T05:42:10.143901+0000","last_fullsized":"2024-04-17T05:42:10.143901+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:57.355596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"263'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145978+0000","last_change":"2024-04-17T05:07:45.746243+0000","last_active":"2024-04-17T05:42:25.145978+0000","last_peered":"2024-04-17T05:42:25.145978+0000","last_clean":"2024-04-17T05:42:25.145978+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:42:25.145978+0000","last_undegraded":"2024-04-17T05:42:25.145978+0000","last_fullsized":"2024-04-17T05:42:25.145978+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:08:11.491152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'7","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167234+0000","last_change":"2024-04-17T04:51:43.675539+0000","last_active":"2024-04-17T05:43:50.167234+0000","last_peered":"2024-04-17T05:43:50.167234+0000","last_clean":"2024-04-17T05:43:50.167234+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:43:50.167234+0000","last_undegraded":"2024-04-17T05:43:50.167234+0000","last_fullsized":"2024-04-17T05:43:50.167234+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:29.255105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160253+0000","last_change":"2024-04-17T04:51:39.637181+0000","last_active":"2024-04-17T05:43:50.160253+0000","last_peered":"2024-04-17T05:43:50.160253+0000","last_clean":"2024-04-17T05:43:50.160253+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:43:50.160253+0000","last_undegraded":"2024-04-17T05:43:50.160253+0000","last_fullsized":"2024-04-17T05:43:50.160253+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:38.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143904+0000","last_change":"2024-04-17T04:47:56.949538+0000","last_active":"2024-04-17T05:42:10.143904+0000","last_peered":"2024-04-17T05:42:10.143904+0000","last_clean":"2024-04-17T05:42:10.143904+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:42:10.143904+0000","last_undegraded":"2024-04-17T05:42:10.143904+0000","last_fullsized":"2024-04-17T05:42:10.143904+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:29.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143092+0000","last_change":"2024-04-17T05:07:45.726938+0000","last_active":"2024-04-17T05:42:25.143092+0000","last_peered":"2024-04-17T05:42:25.143092+0000","last_clean":"2024-04-17T05:42:25.143092+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:42:25.143092+0000","last_undegraded":"2024-04-17T05:42:25.143092+0000","last_fullsized":"2024-04-17T05:42:25.143092+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:33.108430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.166629+0000","last_change":"2024-04-17T04:51:43.648636+0000","last_active":"2024-04-17T05:43:55.166629+0000","last_peered":"2024-04-17T05:43:55.166629+0000","last_clean":"2024-04-17T05:43:55.166629+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:43:55.166629+0000","last_undegraded":"2024-04-17T05:43:55.166629+0000","last_fullsized":"2024-04-17T05:43:55.166629+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:44.200548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159094+0000","last_change":"2024-04-17T04:51:39.636922+0000","last_active":"2024-04-17T05:43:45.159094+0000","last_peered":"2024-04-17T05:43:45.159094+0000","last_clean":"2024-04-17T05:43:45.159094+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:43:45.159094+0000","last_undegraded":"2024-04-17T05:43:45.159094+0000","last_fullsized":"2024-04-17T05:43:45.159094+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:38.269210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.129678+0000","last_change":"2024-04-17T04:47:56.968894+0000","last_active":"2024-04-17T05:42:10.129678+0000","last_peered":"2024-04-17T05:42:10.129678+0000","last_clean":"2024-04-17T05:42:10.129678+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:42:10.129678+0000","last_undegraded":"2024-04-17T05:42:10.129678+0000","last_fullsized":"2024-04-17T05:42:10.129678+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:47.848243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152961+0000","last_change":"2024-04-17T05:07:45.757247+0000","last_active":"2024-04-17T05:43:10.152961+0000","last_peered":"2024-04-17T05:43:10.152961+0000","last_clean":"2024-04-17T05:43:10.152961+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:43:10.152961+0000","last_undegraded":"2024-04-17T05:43:10.152961+0000","last_fullsized":"2024-04-17T05:43:10.152961+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:26.267898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160974+0000","last_change":"2024-04-17T04:51:43.682310+0000","last_active":"2024-04-17T05:43:50.160974+0000","last_peered":"2024-04-17T05:43:50.160974+0000","last_clean":"2024-04-17T05:43:50.160974+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:43:50.160974+0000","last_undegraded":"2024-04-17T05:43:50.160974+0000","last_fullsized":"2024-04-17T05:43:50.160974+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:14.153241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'1","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165820+0000","last_change":"2024-04-17T04:51:39.617751+0000","last_active":"2024-04-17T05:43:45.165820+0000","last_peered":"2024-04-17T05:43:45.165820+0000","last_clean":"2024-04-17T05:43:45.165820+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:43:45.165820+0000","last_undegraded":"2024-04-17T05:43:45.165820+0000","last_fullsized":"2024-04-17T05:43:45.165820+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:45.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":1,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":213,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138713+0000","last_change":"2024-04-17T04:47:56.952060+0000","last_active":"2024-04-17T05:42:10.138713+0000","last_peered":"2024-04-17T05:42:10.138713+0000","last_clean":"2024-04-17T05:42:10.138713+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:42:10.138713+0000","last_undegraded":"2024-04-17T05:42:10.138713+0000","last_fullsized":"2024-04-17T05:42:10.138713+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:52.024764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.152311+0000","last_change":"2024-04-17T05:07:45.752461+0000","last_active":"2024-04-17T05:43:15.152311+0000","last_peered":"2024-04-17T05:43:15.152311+0000","last_clean":"2024-04-17T05:43:15.152311+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:43:15.152311+0000","last_undegraded":"2024-04-17T05:43:15.152311+0000","last_fullsized":"2024-04-17T05:43:15.152311+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:43.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:49.947148+0000","last_change":"2024-04-17T04:51:43.651232+0000","last_active":"2024-04-17T05:43:49.947148+0000","last_peered":"2024-04-17T05:43:49.947148+0000","last_clean":"2024-04-17T05:43:49.947148+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:43:49.947148+0000","last_undegraded":"2024-04-17T05:43:49.947148+0000","last_fullsized":"2024-04-17T05:43:49.947148+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:48.905181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159752+0000","last_change":"2024-04-17T04:51:39.630310+0000","last_active":"2024-04-17T05:43:45.159752+0000","last_peered":"2024-04-17T05:43:45.159752+0000","last_clean":"2024-04-17T05:43:45.159752+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:43:45.159752+0000","last_undegraded":"2024-04-17T05:43:45.159752+0000","last_fullsized":"2024-04-17T05:43:45.159752+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:32:14.085186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6339880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6192,"num_write":3,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":127,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.137755+0000","last_change":"2024-04-17T04:47:56.963237+0000","last_active":"2024-04-17T05:42:10.137755+0000","last_peered":"2024-04-17T05:42:10.137755+0000","last_clean":"2024-04-17T05:42:10.137755+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:42:10.137755+0000","last_undegraded":"2024-04-17T05:42:10.137755+0000","last_fullsized":"2024-04-17T05:42:10.137755+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:11.681306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'285","reported_seq":946,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150137+0000","last_change":"2024-04-17T04:36:25.377106+0000","last_active":"2024-04-17T05:43:05.150137+0000","last_peered":"2024-04-17T05:43:05.150137+0000","last_clean":"2024-04-17T05:43:05.150137+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:43:05.150137+0000","last_undegraded":"2024-04-17T05:43:05.150137+0000","last_fullsized":"2024-04-17T05:43:05.150137+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:09.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53724,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":483,"num_read_kb":725,"num_write":1250,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":675,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134422+0000","last_change":"2024-04-17T04:36:27.389744+0000","last_active":"2024-04-17T05:41:30.134422+0000","last_peered":"2024-04-17T05:41:30.134422+0000","last_clean":"2024-04-17T05:41:30.134422+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:41:30.134422+0000","last_undegraded":"2024-04-17T05:41:30.134422+0000","last_fullsized":"2024-04-17T05:41:30.134422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:14.577671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'308","reported_seq":980,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157287+0000","last_change":"2024-04-17T04:36:25.357080+0000","last_active":"2024-04-17T05:43:05.157287+0000","last_peered":"2024-04-17T05:43:05.157287+0000","last_clean":"2024-04-17T05:43:05.157287+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:43:05.157287+0000","last_undegraded":"2024-04-17T05:43:05.157287+0000","last_fullsized":"2024-04-17T05:43:05.157287+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:21.403329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10755049,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":526,"num_read_kb":10797,"num_write":1432,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":706,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:39.489661+0000","last_change":"2024-04-17T04:36:27.382828+0000","last_active":"2024-04-17T05:42:39.489661+0000","last_peered":"2024-04-17T05:42:39.489661+0000","last_clean":"2024-04-17T05:42:39.489661+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:42:39.489661+0000","last_undegraded":"2024-04-17T05:42:39.489661+0000","last_fullsized":"2024-04-17T05:42:39.489661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:30.715656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'254","reported_seq":920,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151824+0000","last_change":"2024-04-17T04:36:25.329662+0000","last_active":"2024-04-17T05:43:05.151824+0000","last_peered":"2024-04-17T05:43:05.151824+0000","last_clean":"2024-04-17T05:43:05.151824+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:43:05.151824+0000","last_undegraded":"2024-04-17T05:43:05.151824+0000","last_fullsized":"2024-04-17T05:43:05.151824+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:42.966712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528901,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":417,"num_read_kb":58401,"num_write":1131,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120297+0000","last_change":"2024-04-17T04:36:27.385236+0000","last_active":"2024-04-17T05:41:30.120297+0000","last_peered":"2024-04-17T05:41:30.120297+0000","last_clean":"2024-04-17T05:41:30.120297+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:41:30.120297+0000","last_undegraded":"2024-04-17T05:41:30.120297+0000","last_fullsized":"2024-04-17T05:41:30.120297+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:55.872226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'241","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150409+0000","last_change":"2024-04-17T04:36:25.356234+0000","last_active":"2024-04-17T05:42:40.150409+0000","last_peered":"2024-04-17T05:42:40.150409+0000","last_clean":"2024-04-17T05:42:40.150409+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:42:40.150409+0000","last_undegraded":"2024-04-17T05:42:40.150409+0000","last_fullsized":"2024-04-17T05:42:40.150409+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:14.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5765916,"num_objects":48,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":417,"num_read_kb":37884,"num_write":1085,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'304","reported_seq":897,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.129356+0000","last_change":"2024-04-17T04:36:27.385726+0000","last_active":"2024-04-17T05:41:30.129356+0000","last_peered":"2024-04-17T05:41:30.129356+0000","last_clean":"2024-04-17T05:41:30.129356+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:41:30.129356+0000","last_undegraded":"2024-04-17T05:41:30.129356+0000","last_fullsized":"2024-04-17T05:41:30.129356+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:55:33.151184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'266","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.127332+0000","last_change":"2024-04-17T04:36:25.377714+0000","last_active":"2024-04-17T05:41:25.127332+0000","last_peered":"2024-04-17T05:41:25.127332+0000","last_clean":"2024-04-17T05:41:25.127332+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:41:25.127332+0000","last_undegraded":"2024-04-17T05:41:25.127332+0000","last_fullsized":"2024-04-17T05:41:25.127332+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:34.861649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19973206,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":458,"num_read_kb":13359,"num_write":1103,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5179,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.736719+0000","last_change":"2024-04-17T04:36:27.382622+0000","last_active":"2024-04-17T05:45:44.736719+0000","last_peered":"2024-04-17T05:45:44.736719+0000","last_clean":"2024-04-17T05:45:44.736719+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:45:44.736719+0000","last_undegraded":"2024-04-17T05:45:44.736719+0000","last_fullsized":"2024-04-17T05:45:44.736719+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:23.921181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'230","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:43.796816+0000","last_change":"2024-04-17T04:36:25.352114+0000","last_active":"2024-04-17T05:41:43.796816+0000","last_peered":"2024-04-17T05:41:43.796816+0000","last_clean":"2024-04-17T05:41:43.796816+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:41:43.796816+0000","last_undegraded":"2024-04-17T05:41:43.796816+0000","last_fullsized":"2024-04-17T05:41:43.796816+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:17.459792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11021151,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":513,"num_read_kb":64592,"num_write":906,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":774,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938531+0000","last_change":"2024-04-17T04:36:27.385407+0000","last_active":"2024-04-17T05:43:09.938531+0000","last_peered":"2024-04-17T05:43:09.938531+0000","last_clean":"2024-04-17T05:43:09.938531+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:43:09.938531+0000","last_undegraded":"2024-04-17T05:43:09.938531+0000","last_fullsized":"2024-04-17T05:43:09.938531+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:03.112120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'259","reported_seq":881,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:37.900457+0000","last_change":"2024-04-17T04:36:25.376858+0000","last_active":"2024-04-17T05:42:37.900457+0000","last_peered":"2024-04-17T05:42:37.900457+0000","last_clean":"2024-04-17T05:42:37.900457+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:42:37.900457+0000","last_undegraded":"2024-04-17T05:42:37.900457+0000","last_fullsized":"2024-04-17T05:42:37.900457+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:24.809459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3318739,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":393,"num_read_kb":9422,"num_write":1064,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157138+0000","last_change":"2024-04-17T04:36:27.376772+0000","last_active":"2024-04-17T05:43:10.157138+0000","last_peered":"2024-04-17T05:43:10.157138+0000","last_clean":"2024-04-17T05:43:10.157138+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:43:10.157138+0000","last_undegraded":"2024-04-17T05:43:10.157138+0000","last_fullsized":"2024-04-17T05:43:10.157138+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:07.055230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'260","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.718287+0000","last_change":"2024-04-17T04:36:25.343980+0000","last_active":"2024-04-17T05:43:39.718287+0000","last_peered":"2024-04-17T05:43:39.718287+0000","last_clean":"2024-04-17T05:43:39.718287+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:43:39.718287+0000","last_undegraded":"2024-04-17T05:43:39.718287+0000","last_fullsized":"2024-04-17T05:43:39.718287+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:21.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20922664,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":369,"num_read_kb":14992,"num_write":1106,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.156665+0000","last_change":"2024-04-17T04:36:27.379088+0000","last_active":"2024-04-17T05:43:10.156665+0000","last_peered":"2024-04-17T05:43:10.156665+0000","last_clean":"2024-04-17T05:43:10.156665+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:43:10.156665+0000","last_undegraded":"2024-04-17T05:43:10.156665+0000","last_fullsized":"2024-04-17T05:43:10.156665+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:05.867623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125191+0000","last_change":"2024-04-17T04:36:27.385695+0000","last_active":"2024-04-17T05:41:50.125191+0000","last_peered":"2024-04-17T05:41:50.125191+0000","last_clean":"2024-04-17T05:41:50.125191+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:41:50.125191+0000","last_undegraded":"2024-04-17T05:41:50.125191+0000","last_fullsized":"2024-04-17T05:41:50.125191+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:33.234173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'255","reported_seq":862,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.148032+0000","last_change":"2024-04-17T04:37:03.648715+0000","last_active":"2024-04-17T05:42:35.148032+0000","last_peered":"2024-04-17T05:42:35.148032+0000","last_clean":"2024-04-17T05:42:35.148032+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:42:35.148032+0000","last_undegraded":"2024-04-17T05:42:35.148032+0000","last_fullsized":"2024-04-17T05:42:35.148032+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:12.036980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8728170,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":386,"num_read_kb":14103,"num_write":1112,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'255","reported_seq":889,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.407437+0000","last_change":"2024-04-17T04:36:27.383282+0000","last_active":"2024-04-17T05:45:46.407437+0000","last_peered":"2024-04-17T05:45:46.407437+0000","last_clean":"2024-04-17T05:45:46.407437+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:45:46.407437+0000","last_undegraded":"2024-04-17T05:45:46.407437+0000","last_fullsized":"2024-04-17T05:45:46.407437+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:29.116974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'216","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:47.860045+0000","last_change":"2024-04-17T04:36:25.385368+0000","last_active":"2024-04-17T05:42:47.860045+0000","last_peered":"2024-04-17T05:42:47.860045+0000","last_clean":"2024-04-17T05:42:47.860045+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:42:47.860045+0000","last_undegraded":"2024-04-17T05:42:47.860045+0000","last_fullsized":"2024-04-17T05:42:47.860045+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:35.267010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761635,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":351,"num_read_kb":10549,"num_write":915,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":723,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142870+0000","last_change":"2024-04-17T04:36:27.376262+0000","last_active":"2024-04-17T05:42:05.142870+0000","last_peered":"2024-04-17T05:42:05.142870+0000","last_clean":"2024-04-17T05:42:05.142870+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:42:05.142870+0000","last_undegraded":"2024-04-17T05:42:05.142870+0000","last_fullsized":"2024-04-17T05:42:05.142870+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:02.524761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'248","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.390159+0000","last_change":"2024-04-17T04:36:25.385723+0000","last_active":"2024-04-17T05:43:39.390159+0000","last_peered":"2024-04-17T05:43:39.390159+0000","last_clean":"2024-04-17T05:43:39.390159+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:43:39.390159+0000","last_undegraded":"2024-04-17T05:43:39.390159+0000","last_fullsized":"2024-04-17T05:43:39.390159+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:51.669081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498677,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":402,"num_read_kb":160096,"num_write":1072,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136478+0000","last_change":"2024-04-17T04:36:27.383859+0000","last_active":"2024-04-17T05:41:45.136478+0000","last_peered":"2024-04-17T05:41:45.136478+0000","last_clean":"2024-04-17T05:41:45.136478+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:41:45.136478+0000","last_undegraded":"2024-04-17T05:41:45.136478+0000","last_fullsized":"2024-04-17T05:41:45.136478+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:14.617020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'242","reported_seq":947,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:52.574040+0000","last_change":"2024-04-17T04:36:25.361757+0000","last_active":"2024-04-17T05:41:52.574040+0000","last_peered":"2024-04-17T05:41:52.574040+0000","last_clean":"2024-04-17T05:41:52.574040+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:41:52.574040+0000","last_undegraded":"2024-04-17T05:41:52.574040+0000","last_fullsized":"2024-04-17T05:41:52.574040+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:21.146448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19295786,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":522,"num_read_kb":64725,"num_write":1051,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136549+0000","last_change":"2024-04-17T04:36:27.379593+0000","last_active":"2024-04-17T05:41:45.136549+0000","last_peered":"2024-04-17T05:41:45.136549+0000","last_clean":"2024-04-17T05:41:45.136549+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:41:45.136549+0000","last_undegraded":"2024-04-17T05:41:45.136549+0000","last_fullsized":"2024-04-17T05:41:45.136549+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:37.024479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'256","reported_seq":1026,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.874797+0000","last_change":"2024-04-17T04:36:25.374400+0000","last_active":"2024-04-17T05:42:57.874797+0000","last_peered":"2024-04-17T05:42:57.874797+0000","last_clean":"2024-04-17T05:42:57.874797+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:42:57.874797+0000","last_undegraded":"2024-04-17T05:42:57.874797+0000","last_fullsized":"2024-04-17T05:42:57.874797+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:25:10.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26759829,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":748,"num_read_kb":57779,"num_write":1204,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.501738+0000","last_change":"2024-04-17T04:36:27.385279+0000","last_active":"2024-04-17T05:41:55.501738+0000","last_peered":"2024-04-17T05:41:55.501738+0000","last_clean":"2024-04-17T05:41:55.501738+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:41:55.501738+0000","last_undegraded":"2024-04-17T05:41:55.501738+0000","last_fullsized":"2024-04-17T05:41:55.501738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:17.292970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'280","reported_seq":914,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135979+0000","last_change":"2024-04-17T04:36:25.355404+0000","last_active":"2024-04-17T05:41:55.135979+0000","last_peered":"2024-04-17T05:41:55.135979+0000","last_clean":"2024-04-17T05:41:55.135979+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:41:55.135979+0000","last_undegraded":"2024-04-17T05:41:55.135979+0000","last_fullsized":"2024-04-17T05:41:55.135979+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:47.270461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10613748,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":397,"num_read_kb":836,"num_write":1108,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'152","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.671918+0000","last_change":"2024-04-17T04:36:35.418351+0000","last_active":"2024-04-17T05:45:46.671918+0000","last_peered":"2024-04-17T05:45:46.671918+0000","last_clean":"2024-04-17T05:45:46.671918+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:45:46.671918+0000","last_undegraded":"2024-04-17T05:45:46.671918+0000","last_fullsized":"2024-04-17T05:45:46.671918+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:02.595419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:51.868791+0000","last_change":"2024-04-17T04:36:27.384993+0000","last_active":"2024-04-17T05:42:51.868791+0000","last_peered":"2024-04-17T05:42:51.868791+0000","last_clean":"2024-04-17T05:42:51.868791+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:42:51.868791+0000","last_undegraded":"2024-04-17T05:42:51.868791+0000","last_fullsized":"2024-04-17T05:42:51.868791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:24.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'258","reported_seq":882,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.719639+0000","last_change":"2024-04-17T04:36:25.368467+0000","last_active":"2024-04-17T05:42:50.719639+0000","last_peered":"2024-04-17T05:42:50.719639+0000","last_clean":"2024-04-17T05:42:50.719639+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:42:50.719639+0000","last_undegraded":"2024-04-17T05:42:50.719639+0000","last_fullsized":"2024-04-17T05:42:50.719639+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:23.161029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12579502,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":373,"num_read_kb":2527,"num_write":1113,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138168+0000","last_change":"2024-04-17T04:36:29.388609+0000","last_active":"2024-04-17T05:41:45.138168+0000","last_peered":"2024-04-17T05:41:45.138168+0000","last_clean":"2024-04-17T05:41:45.138168+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:41:45.138168+0000","last_undegraded":"2024-04-17T05:41:45.138168+0000","last_fullsized":"2024-04-17T05:41:45.138168+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:53.809254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'362","reported_seq":982,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.169232+0000","last_change":"2024-04-17T04:36:31.410990+0000","last_active":"2024-04-17T05:44:15.169232+0000","last_peered":"2024-04-17T05:44:15.169232+0000","last_clean":"2024-04-17T05:44:15.169232+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:44:15.169232+0000","last_undegraded":"2024-04-17T05:44:15.169232+0000","last_fullsized":"2024-04-17T05:44:15.169232+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":300,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:24.430923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58423,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":1138,"num_write":466,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133607+0000","last_change":"2024-04-17T04:36:33.425089+0000","last_active":"2024-04-17T05:41:50.133607+0000","last_peered":"2024-04-17T05:41:50.133607+0000","last_clean":"2024-04-17T05:41:50.133607+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:41:50.133607+0000","last_undegraded":"2024-04-17T05:41:50.133607+0000","last_fullsized":"2024-04-17T05:41:50.133607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:53.730080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'169","reported_seq":722,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.761057+0000","last_change":"2024-04-17T04:36:35.416351+0000","last_active":"2024-04-17T05:45:44.761057+0000","last_peered":"2024-04-17T05:45:44.761057+0000","last_clean":"2024-04-17T05:45:44.761057+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:45:44.761057+0000","last_undegraded":"2024-04-17T05:45:44.761057+0000","last_fullsized":"2024-04-17T05:45:44.761057+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:34.873231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":673,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138074+0000","last_change":"2024-04-17T04:36:27.389710+0000","last_active":"2024-04-17T05:41:45.138074+0000","last_peered":"2024-04-17T05:41:45.138074+0000","last_clean":"2024-04-17T05:41:45.138074+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:41:45.138074+0000","last_undegraded":"2024-04-17T05:41:45.138074+0000","last_fullsized":"2024-04-17T05:41:45.138074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:11:08.849990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'278","reported_seq":1049,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133284+0000","last_change":"2024-04-17T04:36:25.328153+0000","last_active":"2024-04-17T05:41:45.133284+0000","last_peered":"2024-04-17T05:41:45.133284+0000","last_clean":"2024-04-17T05:41:45.133284+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:41:45.133284+0000","last_undegraded":"2024-04-17T05:41:45.133284+0000","last_fullsized":"2024-04-17T05:41:45.133284+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:07.072313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21193393,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":693,"num_read_kb":61435,"num_write":1302,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.170670+0000","last_change":"2024-04-17T04:36:29.373876+0000","last_active":"2024-04-17T05:42:05.170670+0000","last_peered":"2024-04-17T05:42:05.170670+0000","last_clean":"2024-04-17T05:42:05.170670+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:42:05.170670+0000","last_undegraded":"2024-04-17T05:42:05.170670+0000","last_fullsized":"2024-04-17T05:42:05.170670+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:06.355032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'99","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125438+0000","last_change":"2024-04-17T04:36:31.395007+0000","last_active":"2024-04-17T05:41:50.125438+0000","last_peered":"2024-04-17T05:41:50.125438+0000","last_clean":"2024-04-17T05:41:50.125438+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:41:50.125438+0000","last_undegraded":"2024-04-17T05:41:50.125438+0000","last_fullsized":"2024-04-17T05:41:50.125438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:47.124749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125417+0000","last_change":"2024-04-17T04:36:33.414084+0000","last_active":"2024-04-17T05:41:50.125417+0000","last_peered":"2024-04-17T05:41:50.125417+0000","last_clean":"2024-04-17T05:41:50.125417+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:41:50.125417+0000","last_undegraded":"2024-04-17T05:41:50.125417+0000","last_fullsized":"2024-04-17T05:41:50.125417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:27.803230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'123","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.222678+0000","last_change":"2024-04-17T04:36:35.427530+0000","last_active":"2024-04-17T05:45:44.222678+0000","last_peered":"2024-04-17T05:45:44.222678+0000","last_clean":"2024-04-17T05:45:44.222678+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:45:44.222678+0000","last_undegraded":"2024-04-17T05:45:44.222678+0000","last_fullsized":"2024-04-17T05:45:44.222678+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:45.651840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":659,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:09.959182+0000","last_change":"2024-04-17T04:36:27.376107+0000","last_active":"2024-04-17T05:42:09.959182+0000","last_peered":"2024-04-17T05:42:09.959182+0000","last_clean":"2024-04-17T05:42:09.959182+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:42:09.959182+0000","last_undegraded":"2024-04-17T05:42:09.959182+0000","last_fullsized":"2024-04-17T05:42:09.959182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:13.603194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'276","reported_seq":1055,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.131908+0000","last_change":"2024-04-17T04:36:25.381456+0000","last_active":"2024-04-17T05:41:45.131908+0000","last_peered":"2024-04-17T05:41:45.131908+0000","last_clean":"2024-04-17T05:41:45.131908+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:41:45.131908+0000","last_undegraded":"2024-04-17T05:41:45.131908+0000","last_fullsized":"2024-04-17T05:41:45.131908+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:08.488003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22475342,"num_objects":61,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":615,"num_read_kb":75395,"num_write":1261,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165842+0000","last_change":"2024-04-17T04:36:29.359822+0000","last_active":"2024-04-17T05:41:45.165842+0000","last_peered":"2024-04-17T05:41:45.165842+0000","last_clean":"2024-04-17T05:41:45.165842+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:41:45.165842+0000","last_undegraded":"2024-04-17T05:41:45.165842+0000","last_fullsized":"2024-04-17T05:41:45.165842+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:41.948364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'137","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.141895+0000","last_change":"2024-04-17T04:36:31.391956+0000","last_active":"2024-04-17T05:42:10.141895+0000","last_peered":"2024-04-17T05:42:10.141895+0000","last_clean":"2024-04-17T05:42:10.141895+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:42:10.141895+0000","last_undegraded":"2024-04-17T05:42:10.141895+0000","last_fullsized":"2024-04-17T05:42:10.141895+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:12.846752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'512","reported_seq":2409,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.670706+0000","last_change":"2024-04-17T04:36:33.420806+0000","last_active":"2024-04-17T05:45:57.670706+0000","last_peered":"2024-04-17T05:45:57.670706+0000","last_clean":"2024-04-17T05:45:57.670706+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:45:57.670706+0000","last_undegraded":"2024-04-17T05:45:57.670706+0000","last_fullsized":"2024-04-17T05:45:57.670706+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:52.301226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'195","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.415632+0000","last_change":"2024-04-17T04:36:35.403573+0000","last_active":"2024-04-17T05:45:46.415632+0000","last_peered":"2024-04-17T05:45:46.415632+0000","last_clean":"2024-04-17T05:45:46.415632+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:45:46.415632+0000","last_undegraded":"2024-04-17T05:45:46.415632+0000","last_fullsized":"2024-04-17T05:45:46.415632+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:09.033539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:07.139896+0000","last_change":"2024-04-17T04:36:27.395613+0000","last_active":"2024-04-17T05:43:07.139896+0000","last_peered":"2024-04-17T05:43:07.139896+0000","last_clean":"2024-04-17T05:43:07.139896+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:43:07.139896+0000","last_undegraded":"2024-04-17T05:43:07.139896+0000","last_fullsized":"2024-04-17T05:43:07.139896+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:32:19.325576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'235","reported_seq":881,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133330+0000","last_change":"2024-04-17T04:36:25.384743+0000","last_active":"2024-04-17T05:41:45.133330+0000","last_peered":"2024-04-17T05:41:45.133330+0000","last_clean":"2024-04-17T05:41:45.133330+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:41:45.133330+0000","last_undegraded":"2024-04-17T05:41:45.133330+0000","last_fullsized":"2024-04-17T05:41:45.133330+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:05.451652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727402,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":395,"num_read_kb":48921,"num_write":1085,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138271+0000","last_change":"2024-04-17T04:36:29.391920+0000","last_active":"2024-04-17T05:41:45.138271+0000","last_peered":"2024-04-17T05:41:45.138271+0000","last_clean":"2024-04-17T05:41:45.138271+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:41:45.138271+0000","last_undegraded":"2024-04-17T05:41:45.138271+0000","last_fullsized":"2024-04-17T05:41:45.138271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:00:30.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'274","reported_seq":854,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166406+0000","last_change":"2024-04-17T04:36:31.422728+0000","last_active":"2024-04-17T05:44:15.166406+0000","last_peered":"2024-04-17T05:44:15.166406+0000","last_clean":"2024-04-17T05:44:15.166406+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:44:15.166406+0000","last_undegraded":"2024-04-17T05:44:15.166406+0000","last_fullsized":"2024-04-17T05:44:15.166406+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:05.940948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62885,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":973,"num_write":370,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'529","reported_seq":2297,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.662523+0000","last_change":"2024-04-17T04:36:33.422550+0000","last_active":"2024-04-17T05:45:57.662523+0000","last_peered":"2024-04-17T05:45:57.662523+0000","last_clean":"2024-04-17T05:45:57.662523+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:45:57.662523+0000","last_undegraded":"2024-04-17T05:45:57.662523+0000","last_fullsized":"2024-04-17T05:45:57.662523+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":500,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:46.514333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'121","reported_seq":720,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163455+0000","last_change":"2024-04-17T04:36:27.398309+0000","last_active":"2024-04-17T05:43:40.163455+0000","last_peered":"2024-04-17T05:43:40.163455+0000","last_clean":"2024-04-17T05:43:40.163455+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:43:40.163455+0000","last_undegraded":"2024-04-17T05:43:40.163455+0000","last_fullsized":"2024-04-17T05:43:40.163455+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:23.743688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'230","reported_seq":863,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133579+0000","last_change":"2024-04-17T04:36:25.370799+0000","last_active":"2024-04-17T05:41:50.133579+0000","last_peered":"2024-04-17T05:41:50.133579+0000","last_clean":"2024-04-17T05:41:50.133579+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:41:50.133579+0000","last_undegraded":"2024-04-17T05:41:50.133579+0000","last_fullsized":"2024-04-17T05:41:50.133579+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:55.146915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15464766,"num_objects":51,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":386,"num_read_kb":24078,"num_write":1075,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.157843+0000","last_change":"2024-04-17T04:36:29.389738+0000","last_active":"2024-04-17T05:43:40.157843+0000","last_peered":"2024-04-17T05:43:40.157843+0000","last_clean":"2024-04-17T05:43:40.157843+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:43:40.157843+0000","last_undegraded":"2024-04-17T05:43:40.157843+0000","last_fullsized":"2024-04-17T05:43:40.157843+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:52.759093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'377","reported_seq":1009,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.167892+0000","last_change":"2024-04-17T04:36:31.422910+0000","last_active":"2024-04-17T05:44:00.167892+0000","last_peered":"2024-04-17T05:44:00.167892+0000","last_clean":"2024-04-17T05:44:00.167892+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:44:00.167892+0000","last_undegraded":"2024-04-17T05:44:00.167892+0000","last_fullsized":"2024-04-17T05:44:00.167892+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:47.096298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54931,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":606,"num_read_kb":1124,"num_write":470,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139387+0000","last_change":"2024-04-17T04:36:33.408109+0000","last_active":"2024-04-17T05:41:50.139387+0000","last_peered":"2024-04-17T05:41:50.139387+0000","last_clean":"2024-04-17T05:41:50.139387+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:41:50.139387+0000","last_undegraded":"2024-04-17T05:41:50.139387+0000","last_fullsized":"2024-04-17T05:41:50.139387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:19.914865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'560","reported_seq":1384,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.656195+0000","last_change":"2024-04-17T04:36:35.409093+0000","last_active":"2024-04-17T05:45:54.656195+0000","last_peered":"2024-04-17T05:45:54.656195+0000","last_clean":"2024-04-17T05:45:54.656195+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:45:54.656195+0000","last_undegraded":"2024-04-17T05:45:54.656195+0000","last_fullsized":"2024-04-17T05:45:54.656195+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":500,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:27.394663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":865,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.944851+0000","last_change":"2024-04-17T04:36:27.379851+0000","last_active":"2024-04-17T05:43:39.944851+0000","last_peered":"2024-04-17T05:43:39.944851+0000","last_clean":"2024-04-17T05:43:39.944851+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:43:39.944851+0000","last_undegraded":"2024-04-17T05:43:39.944851+0000","last_fullsized":"2024-04-17T05:43:39.944851+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:27.808945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'312","reported_seq":972,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:12.983824+0000","last_change":"2024-04-17T04:36:25.371549+0000","last_active":"2024-04-17T05:42:12.983824+0000","last_peered":"2024-04-17T05:42:12.983824+0000","last_clean":"2024-04-17T05:42:12.983824+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:42:12.983824+0000","last_undegraded":"2024-04-17T05:42:12.983824+0000","last_fullsized":"2024-04-17T05:42:12.983824+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:35.252546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7755476,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":488,"num_read_kb":14712,"num_write":1238,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162258+0000","last_change":"2024-04-17T04:36:29.371794+0000","last_active":"2024-04-17T05:43:40.162258+0000","last_peered":"2024-04-17T05:43:40.162258+0000","last_clean":"2024-04-17T05:43:40.162258+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:43:40.162258+0000","last_undegraded":"2024-04-17T05:43:40.162258+0000","last_fullsized":"2024-04-17T05:43:40.162258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:11.383775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'264","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198231+0000","last_change":"2024-04-17T04:36:31.392065+0000","last_active":"2024-04-17T05:44:05.198231+0000","last_peered":"2024-04-17T05:44:05.198231+0000","last_clean":"2024-04-17T05:44:05.198231+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:44:05.198231+0000","last_undegraded":"2024-04-17T05:44:05.198231+0000","last_fullsized":"2024-04-17T05:44:05.198231+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:36.246942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45337,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":414,"num_read_kb":838,"num_write":330,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166729+0000","last_change":"2024-04-17T04:36:33.416143+0000","last_active":"2024-04-17T05:41:50.166729+0000","last_peered":"2024-04-17T05:41:50.166729+0000","last_clean":"2024-04-17T05:41:50.166729+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:41:50.166729+0000","last_undegraded":"2024-04-17T05:41:50.166729+0000","last_fullsized":"2024-04-17T05:41:50.166729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:51.569882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2371","reported_seq":4173,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.067191+0000","last_change":"2024-04-17T04:36:35.418228+0000","last_active":"2024-04-17T05:45:40.067191+0000","last_peered":"2024-04-17T05:45:40.067191+0000","last_clean":"2024-04-17T05:45:40.067191+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:45:40.067191+0000","last_undegraded":"2024-04-17T05:45:40.067191+0000","last_fullsized":"2024-04-17T05:45:40.067191+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":2300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.125017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160924+0000","last_change":"2024-04-17T04:36:27.376914+0000","last_active":"2024-04-17T05:43:50.160924+0000","last_peered":"2024-04-17T05:43:50.160924+0000","last_clean":"2024-04-17T05:43:50.160924+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:43:50.160924+0000","last_undegraded":"2024-04-17T05:43:50.160924+0000","last_fullsized":"2024-04-17T05:43:50.160924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:06.533166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'288","reported_seq":962,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:12.960895+0000","last_change":"2024-04-17T04:36:25.386734+0000","last_active":"2024-04-17T05:42:12.960895+0000","last_peered":"2024-04-17T05:42:12.960895+0000","last_clean":"2024-04-17T05:42:12.960895+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:42:12.960895+0000","last_undegraded":"2024-04-17T05:42:12.960895+0000","last_fullsized":"2024-04-17T05:42:12.960895+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:49.654467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7146143,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":536,"num_read_kb":65373,"num_write":1251,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136503+0000","last_change":"2024-04-17T04:36:29.372163+0000","last_active":"2024-04-17T05:41:45.136503+0000","last_peered":"2024-04-17T05:41:45.136503+0000","last_clean":"2024-04-17T05:41:45.136503+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:41:45.136503+0000","last_undegraded":"2024-04-17T05:41:45.136503+0000","last_fullsized":"2024-04-17T05:41:45.136503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:29.815813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'330","reported_seq":941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.632269+0000","last_change":"2024-04-17T04:36:31.380567+0000","last_active":"2024-04-17T05:45:54.632269+0000","last_peered":"2024-04-17T05:45:54.632269+0000","last_clean":"2024-04-17T05:45:54.632269+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:45:54.632269+0000","last_undegraded":"2024-04-17T05:45:54.632269+0000","last_fullsized":"2024-04-17T05:45:54.632269+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:55.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":530,"num_read_kb":1046,"num_write":422,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'534","reported_seq":4610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.650496+0000","last_change":"2024-04-17T04:36:33.411578+0000","last_active":"2024-04-17T05:45:57.650496+0000","last_peered":"2024-04-17T05:45:57.650496+0000","last_clean":"2024-04-17T05:45:57.650496+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:45:57.650496+0000","last_undegraded":"2024-04-17T05:45:57.650496+0000","last_fullsized":"2024-04-17T05:45:57.650496+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.576073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161111+0000","last_change":"2024-04-17T04:36:35.427581+0000","last_active":"2024-04-17T05:43:45.161111+0000","last_peered":"2024-04-17T05:43:45.161111+0000","last_clean":"2024-04-17T05:43:45.161111+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:43:45.161111+0000","last_undegraded":"2024-04-17T05:43:45.161111+0000","last_fullsized":"2024-04-17T05:43:45.161111+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:13.700005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'95","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.081384+0000","last_change":"2024-04-17T04:36:27.370100+0000","last_active":"2024-04-17T05:42:57.081384+0000","last_peered":"2024-04-17T05:42:57.081384+0000","last_clean":"2024-04-17T05:42:57.081384+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:42:57.081384+0000","last_undegraded":"2024-04-17T05:42:57.081384+0000","last_fullsized":"2024-04-17T05:42:57.081384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:31.666063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'275","reported_seq":974,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144551+0000","last_change":"2024-04-17T04:37:03.648700+0000","last_active":"2024-04-17T05:42:20.144551+0000","last_peered":"2024-04-17T05:42:20.144551+0000","last_clean":"2024-04-17T05:42:20.144551+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:42:20.144551+0000","last_undegraded":"2024-04-17T05:42:20.144551+0000","last_fullsized":"2024-04-17T05:42:20.144551+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:49:08.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6559602,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":598,"num_read_kb":24373,"num_write":1106,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136436+0000","last_change":"2024-04-17T04:36:29.375596+0000","last_active":"2024-04-17T05:41:45.136436+0000","last_peered":"2024-04-17T05:41:45.136436+0000","last_clean":"2024-04-17T05:41:45.136436+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:41:45.136436+0000","last_undegraded":"2024-04-17T05:41:45.136436+0000","last_fullsized":"2024-04-17T05:41:45.136436+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:20.886949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'462","reported_seq":1109,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.633698+0000","last_change":"2024-04-17T04:36:31.409877+0000","last_active":"2024-04-17T05:45:54.633698+0000","last_peered":"2024-04-17T05:45:54.633698+0000","last_clean":"2024-04-17T05:45:54.633698+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:45:54.633698+0000","last_undegraded":"2024-04-17T05:45:54.633698+0000","last_fullsized":"2024-04-17T05:45:54.633698+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:19.065697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101390,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":805,"num_read_kb":1767,"num_write":654,"num_write_kb":532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.163452+0000","last_change":"2024-04-17T04:36:33.420378+0000","last_active":"2024-04-17T05:43:45.163452+0000","last_peered":"2024-04-17T05:43:45.163452+0000","last_clean":"2024-04-17T05:43:45.163452+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:43:45.163452+0000","last_undegraded":"2024-04-17T05:43:45.163452+0000","last_fullsized":"2024-04-17T05:43:45.163452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:59.519269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'148","reported_seq":689,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165985+0000","last_change":"2024-04-17T04:36:35.417260+0000","last_active":"2024-04-17T05:43:45.165985+0000","last_peered":"2024-04-17T05:43:45.165985+0000","last_clean":"2024-04-17T05:43:45.165985+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:43:45.165985+0000","last_undegraded":"2024-04-17T05:43:45.165985+0000","last_fullsized":"2024-04-17T05:43:45.165985+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:41.967458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124135+0000","last_change":"2024-04-17T04:36:27.393949+0000","last_active":"2024-04-17T05:41:45.124135+0000","last_peered":"2024-04-17T05:41:45.124135+0000","last_clean":"2024-04-17T05:41:45.124135+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:41:45.124135+0000","last_undegraded":"2024-04-17T05:41:45.124135+0000","last_fullsized":"2024-04-17T05:41:45.124135+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:12.169777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'287","reported_seq":955,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164840+0000","last_change":"2024-04-17T04:36:25.379472+0000","last_active":"2024-04-17T05:41:40.164840+0000","last_peered":"2024-04-17T05:41:40.164840+0000","last_clean":"2024-04-17T05:41:40.164840+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:41:40.164840+0000","last_undegraded":"2024-04-17T05:41:40.164840+0000","last_fullsized":"2024-04-17T05:41:40.164840+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:42:53.335830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2019600,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":501,"num_read_kb":9569,"num_write":1164,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136613+0000","last_change":"2024-04-17T04:36:29.374168+0000","last_active":"2024-04-17T05:41:45.136613+0000","last_peered":"2024-04-17T05:41:45.136613+0000","last_clean":"2024-04-17T05:41:45.136613+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:41:45.136613+0000","last_undegraded":"2024-04-17T05:41:45.136613+0000","last_fullsized":"2024-04-17T05:41:45.136613+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:22.063423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'308","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.158285+0000","last_change":"2024-04-17T04:36:31.392925+0000","last_active":"2024-04-17T05:44:05.158285+0000","last_peered":"2024-04-17T05:44:05.158285+0000","last_clean":"2024-04-17T05:44:05.158285+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:44:05.158285+0000","last_undegraded":"2024-04-17T05:44:05.158285+0000","last_fullsized":"2024-04-17T05:44:05.158285+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:08.285743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56427,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":522,"num_read_kb":1053,"num_write":466,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":494,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137350+0000","last_change":"2024-04-17T04:36:33.430909+0000","last_active":"2024-04-17T05:42:00.137350+0000","last_peered":"2024-04-17T05:42:00.137350+0000","last_clean":"2024-04-17T05:42:00.137350+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:42:00.137350+0000","last_undegraded":"2024-04-17T05:42:00.137350+0000","last_fullsized":"2024-04-17T05:42:00.137350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:54.144240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'129","reported_seq":636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.818089+0000","last_change":"2024-04-17T04:36:35.423084+0000","last_active":"2024-04-17T05:45:40.818089+0000","last_peered":"2024-04-17T05:45:40.818089+0000","last_clean":"2024-04-17T05:45:40.818089+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:45:40.818089+0000","last_undegraded":"2024-04-17T05:45:40.818089+0000","last_fullsized":"2024-04-17T05:45:40.818089+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:20:19.232177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:53.923395+0000","last_change":"2024-04-17T04:36:27.383527+0000","last_active":"2024-04-17T05:41:53.923395+0000","last_peered":"2024-04-17T05:41:53.923395+0000","last_clean":"2024-04-17T05:41:53.923395+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:41:53.923395+0000","last_undegraded":"2024-04-17T05:41:53.923395+0000","last_fullsized":"2024-04-17T05:41:53.923395+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:29.969137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'301","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159689+0000","last_change":"2024-04-17T04:36:25.371674+0000","last_active":"2024-04-17T05:43:45.159689+0000","last_peered":"2024-04-17T05:43:45.159689+0000","last_clean":"2024-04-17T05:43:45.159689+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:43:45.159689+0000","last_undegraded":"2024-04-17T05:43:45.159689+0000","last_fullsized":"2024-04-17T05:43:45.159689+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:54.700946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6845506,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":545,"num_read_kb":15072,"num_write":1194,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:29.359143+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:37.440920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'287","reported_seq":891,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.171451+0000","last_change":"2024-04-17T04:36:31.397866+0000","last_active":"2024-04-17T05:44:20.171451+0000","last_peered":"2024-04-17T05:44:20.171451+0000","last_clean":"2024-04-17T05:44:20.171451+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:44:20.171451+0000","last_undegraded":"2024-04-17T05:44:20.171451+0000","last_fullsized":"2024-04-17T05:44:20.171451+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:42.568179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51070,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":477,"num_read_kb":941,"num_write":374,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'508","reported_seq":2439,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.663877+0000","last_change":"2024-04-17T04:36:33.408560+0000","last_active":"2024-04-17T05:45:57.663877+0000","last_peered":"2024-04-17T05:45:57.663877+0000","last_clean":"2024-04-17T05:45:57.663877+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:45:57.663877+0000","last_undegraded":"2024-04-17T05:45:57.663877+0000","last_fullsized":"2024-04-17T05:45:57.663877+0000","mapping_epoch":31,"log_start":"261'400","ondisk_log_start":"261'400","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":400,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:41.906428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'96","reported_seq":602,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:42.569922+0000","last_change":"2024-04-17T04:36:35.428863+0000","last_active":"2024-04-17T05:45:42.569922+0000","last_peered":"2024-04-17T05:45:42.569922+0000","last_clean":"2024-04-17T05:45:42.569922+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:45:42.569922+0000","last_undegraded":"2024-04-17T05:45:42.569922+0000","last_fullsized":"2024-04-17T05:45:42.569922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:09.522109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.169748+0000","last_change":"2024-04-17T04:36:27.393058+0000","last_active":"2024-04-17T05:42:30.169748+0000","last_peered":"2024-04-17T05:42:30.169748+0000","last_clean":"2024-04-17T05:42:30.169748+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:42:30.169748+0000","last_undegraded":"2024-04-17T05:42:30.169748+0000","last_fullsized":"2024-04-17T05:42:30.169748+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:22.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'245","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:34.943708+0000","last_change":"2024-04-17T04:36:25.356100+0000","last_active":"2024-04-17T05:43:34.943708+0000","last_peered":"2024-04-17T05:43:34.943708+0000","last_clean":"2024-04-17T05:43:34.943708+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:43:34.943708+0000","last_undegraded":"2024-04-17T05:43:34.943708+0000","last_fullsized":"2024-04-17T05:43:34.943708+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:10.485745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10486600,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":525,"num_read_kb":5671,"num_write":1142,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"28'2","reported_seq":1150,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.527318+0000","last_change":"2024-04-17T04:36:29.389132+0000","last_active":"2024-04-17T05:45:54.527318+0000","last_peered":"2024-04-17T05:45:54.527318+0000","last_clean":"2024-04-17T05:45:54.527318+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:45:54.527318+0000","last_undegraded":"2024-04-17T05:45:54.527318+0000","last_fullsized":"2024-04-17T05:45:54.527318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:09:43.654615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'97","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166974+0000","last_change":"2024-04-17T04:36:31.421130+0000","last_active":"2024-04-17T05:41:50.166974+0000","last_peered":"2024-04-17T05:41:50.166974+0000","last_clean":"2024-04-17T05:41:50.166974+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:41:50.166974+0000","last_undegraded":"2024-04-17T05:41:50.166974+0000","last_fullsized":"2024-04-17T05:41:50.166974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:38.963408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'535","reported_seq":2243,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.671105+0000","last_change":"2024-04-17T04:36:33.421354+0000","last_active":"2024-04-17T05:45:57.671105+0000","last_peered":"2024-04-17T05:45:57.671105+0000","last_clean":"2024-04-17T05:45:57.671105+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:45:57.671105+0000","last_undegraded":"2024-04-17T05:45:57.671105+0000","last_fullsized":"2024-04-17T05:45:57.671105+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":500,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:27.382198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'97","reported_seq":600,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939559+0000","last_change":"2024-04-17T04:36:35.427123+0000","last_active":"2024-04-17T05:43:14.939559+0000","last_peered":"2024-04-17T05:43:14.939559+0000","last_clean":"2024-04-17T05:43:14.939559+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:43:14.939559+0000","last_undegraded":"2024-04-17T05:43:14.939559+0000","last_fullsized":"2024-04-17T05:43:14.939559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:26.157064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":756,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141817+0000","last_change":"2024-04-17T04:36:27.377947+0000","last_active":"2024-04-17T05:42:00.141817+0000","last_peered":"2024-04-17T05:42:00.141817+0000","last_clean":"2024-04-17T05:42:00.141817+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:42:00.141817+0000","last_undegraded":"2024-04-17T05:42:00.141817+0000","last_fullsized":"2024-04-17T05:42:00.141817+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:47.608940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'228","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.190870+0000","last_change":"2024-04-17T04:36:25.353720+0000","last_active":"2024-04-17T05:43:35.190870+0000","last_peered":"2024-04-17T05:43:35.190870+0000","last_clean":"2024-04-17T05:43:35.190870+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:43:35.190870+0000","last_undegraded":"2024-04-17T05:43:35.190870+0000","last_fullsized":"2024-04-17T05:43:35.190870+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:24.629387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15899766,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":342,"num_read_kb":36760,"num_write":925,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124238+0000","last_change":"2024-04-17T04:36:29.389735+0000","last_active":"2024-04-17T05:41:45.124238+0000","last_peered":"2024-04-17T05:41:45.124238+0000","last_clean":"2024-04-17T05:41:45.124238+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:41:45.124238+0000","last_undegraded":"2024-04-17T05:41:45.124238+0000","last_fullsized":"2024-04-17T05:41:45.124238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:19.695221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'347","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.169175+0000","last_change":"2024-04-17T04:36:31.398102+0000","last_active":"2024-04-17T05:44:05.169175+0000","last_peered":"2024-04-17T05:44:05.169175+0000","last_clean":"2024-04-17T05:44:05.169175+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:44:05.169175+0000","last_undegraded":"2024-04-17T05:44:05.169175+0000","last_fullsized":"2024-04-17T05:44:05.169175+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:41.468361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":568,"num_read_kb":1054,"num_write":442,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134628+0000","last_change":"2024-04-17T04:36:33.419643+0000","last_active":"2024-04-17T05:41:50.134628+0000","last_peered":"2024-04-17T05:41:50.134628+0000","last_clean":"2024-04-17T05:41:50.134628+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:41:50.134628+0000","last_undegraded":"2024-04-17T05:41:50.134628+0000","last_fullsized":"2024-04-17T05:41:50.134628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:51.100488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'157","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.190550+0000","last_change":"2024-04-17T04:36:35.408247+0000","last_active":"2024-04-17T05:45:20.190550+0000","last_peered":"2024-04-17T05:45:20.190550+0000","last_clean":"2024-04-17T05:45:20.190550+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:45:20.190550+0000","last_undegraded":"2024-04-17T05:45:20.190550+0000","last_fullsized":"2024-04-17T05:45:20.190550+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:26.932136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":695,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.154250+0000","last_change":"2024-04-17T04:36:27.387349+0000","last_active":"2024-04-17T05:43:20.154250+0000","last_peered":"2024-04-17T05:43:20.154250+0000","last_clean":"2024-04-17T05:43:20.154250+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:43:20.154250+0000","last_undegraded":"2024-04-17T05:43:20.154250+0000","last_fullsized":"2024-04-17T05:43:20.154250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:39.509316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":680,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141161+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T05:42:25.141161+0000","last_peered":"2024-04-17T05:42:25.141161+0000","last_clean":"2024-04-17T05:42:25.141161+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:42:25.141161+0000","last_undegraded":"2024-04-17T05:42:25.141161+0000","last_fullsized":"2024-04-17T05:42:25.141161+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:24:00.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154818+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T05:43:00.154818+0000","last_peered":"2024-04-17T05:43:00.154818+0000","last_clean":"2024-04-17T05:43:00.154818+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:43:00.154818+0000","last_undegraded":"2024-04-17T05:43:00.154818+0000","last_fullsized":"2024-04-17T05:43:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:13.926220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'276","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.184874+0000","last_change":"2024-04-17T04:36:25.364067+0000","last_active":"2024-04-17T05:43:05.184874+0000","last_peered":"2024-04-17T05:43:05.184874+0000","last_clean":"2024-04-17T05:43:05.184874+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:43:05.184874+0000","last_undegraded":"2024-04-17T05:43:05.184874+0000","last_fullsized":"2024-04-17T05:43:05.184874+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:41.272537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346866,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":496,"num_read_kb":2736,"num_write":1164,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151134+0000","last_change":"2024-04-17T04:36:29.381453+0000","last_active":"2024-04-17T05:43:10.151134+0000","last_peered":"2024-04-17T05:43:10.151134+0000","last_clean":"2024-04-17T05:43:10.151134+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:43:10.151134+0000","last_undegraded":"2024-04-17T05:43:10.151134+0000","last_fullsized":"2024-04-17T05:43:10.151134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:32.455502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'709","reported_seq":1415,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.648327+0000","last_change":"2024-04-17T04:36:31.400231+0000","last_active":"2024-04-17T05:45:54.648327+0000","last_peered":"2024-04-17T05:45:54.648327+0000","last_clean":"2024-04-17T05:45:54.648327+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:45:54.648327+0000","last_undegraded":"2024-04-17T05:45:54.648327+0000","last_fullsized":"2024-04-17T05:45:54.648327+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":600,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:02.459768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155640,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1372,"num_read_kb":2849,"num_write":1330,"num_write_kb":1169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161079+0000","last_change":"2024-04-17T04:36:33.430394+0000","last_active":"2024-04-17T05:43:45.161079+0000","last_peered":"2024-04-17T05:43:45.161079+0000","last_clean":"2024-04-17T05:43:45.161079+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:43:45.161079+0000","last_undegraded":"2024-04-17T05:43:45.161079+0000","last_fullsized":"2024-04-17T05:43:45.161079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:45.115863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'220","reported_seq":913,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.519743+0000","last_change":"2024-04-17T04:36:35.421183+0000","last_active":"2024-04-17T05:45:46.519743+0000","last_peered":"2024-04-17T05:45:46.519743+0000","last_clean":"2024-04-17T05:45:46.519743+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:45:46.519743+0000","last_undegraded":"2024-04-17T05:45:46.519743+0000","last_fullsized":"2024-04-17T05:45:46.519743+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:30.565325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159648+0000","last_change":"2024-04-17T04:36:27.365454+0000","last_active":"2024-04-17T05:43:40.159648+0000","last_peered":"2024-04-17T05:43:40.159648+0000","last_clean":"2024-04-17T05:43:40.159648+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:43:40.159648+0000","last_undegraded":"2024-04-17T05:43:40.159648+0000","last_fullsized":"2024-04-17T05:43:40.159648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:16.200473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":524,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142294+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T05:42:30.142294+0000","last_peered":"2024-04-17T05:42:30.142294+0000","last_clean":"2024-04-17T05:42:30.142294+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:42:30.142294+0000","last_undegraded":"2024-04-17T05:42:30.142294+0000","last_fullsized":"2024-04-17T05:42:30.142294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:06.659093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'280","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151677+0000","last_change":"2024-04-17T04:36:25.355561+0000","last_active":"2024-04-17T05:43:05.151677+0000","last_peered":"2024-04-17T05:43:05.151677+0000","last_clean":"2024-04-17T05:43:05.151677+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:43:05.151677+0000","last_undegraded":"2024-04-17T05:43:05.151677+0000","last_fullsized":"2024-04-17T05:43:05.151677+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:21.131078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12803559,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":507,"num_read_kb":3714,"num_write":1174,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163722+0000","last_change":"2024-04-17T04:36:29.393231+0000","last_active":"2024-04-17T05:43:40.163722+0000","last_peered":"2024-04-17T05:43:40.163722+0000","last_clean":"2024-04-17T05:43:40.163722+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:43:40.163722+0000","last_undegraded":"2024-04-17T05:43:40.163722+0000","last_fullsized":"2024-04-17T05:43:40.163722+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:30.045764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1084","reported_seq":2786,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.664430+0000","last_change":"2024-04-17T04:36:31.398322+0000","last_active":"2024-04-17T05:45:57.664430+0000","last_peered":"2024-04-17T05:45:57.664430+0000","last_clean":"2024-04-17T05:45:57.664430+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:45:57.664430+0000","last_undegraded":"2024-04-17T05:45:57.664430+0000","last_fullsized":"2024-04-17T05:45:57.664430+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":1000,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.046142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":178612,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1777,"num_read_kb":3331,"num_write":1104,"num_write_kb":931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154560+0000","last_change":"2024-04-17T04:36:33.422428+0000","last_active":"2024-04-17T05:43:25.154560+0000","last_peered":"2024-04-17T05:43:25.154560+0000","last_clean":"2024-04-17T05:43:25.154560+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:43:25.154560+0000","last_undegraded":"2024-04-17T05:43:25.154560+0000","last_fullsized":"2024-04-17T05:43:25.154560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:00.479531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'112","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.389082+0000","last_change":"2024-04-17T04:36:35.402669+0000","last_active":"2024-04-17T05:45:46.389082+0000","last_peered":"2024-04-17T05:45:46.389082+0000","last_clean":"2024-04-17T05:45:46.389082+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:45:46.389082+0000","last_undegraded":"2024-04-17T05:45:46.389082+0000","last_fullsized":"2024-04-17T05:45:46.389082+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:05.019981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164737+0000","last_change":"2024-04-17T04:36:27.376154+0000","last_active":"2024-04-17T05:43:40.164737+0000","last_peered":"2024-04-17T05:43:40.164737+0000","last_clean":"2024-04-17T05:43:40.164737+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:43:40.164737+0000","last_undegraded":"2024-04-17T05:43:40.164737+0000","last_fullsized":"2024-04-17T05:43:40.164737+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:53.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":530,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.151409+0000","last_change":"2024-04-17T04:36:17.752374+0000","last_active":"2024-04-17T05:42:55.151409+0000","last_peered":"2024-04-17T05:42:55.151409+0000","last_clean":"2024-04-17T05:42:55.151409+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:42:55.151409+0000","last_undegraded":"2024-04-17T05:42:55.151409+0000","last_fullsized":"2024-04-17T05:42:55.151409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:41.044005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'243","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150111+0000","last_change":"2024-04-17T04:36:25.343663+0000","last_active":"2024-04-17T05:42:40.150111+0000","last_peered":"2024-04-17T05:42:40.150111+0000","last_clean":"2024-04-17T05:42:40.150111+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:42:40.150111+0000","last_undegraded":"2024-04-17T05:42:40.150111+0000","last_fullsized":"2024-04-17T05:42:40.150111+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:49.114178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5703040,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":424,"num_read_kb":21971,"num_write":1062,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"28'1","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162452+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:43:40.162452+0000","last_peered":"2024-04-17T05:43:40.162452+0000","last_clean":"2024-04-17T05:43:40.162452+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:43:40.162452+0000","last_undegraded":"2024-04-17T05:43:40.162452+0000","last_fullsized":"2024-04-17T05:43:40.162452+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:01.144890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'234","reported_seq":1618,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:52.541681+0000","last_change":"2024-04-17T04:36:31.427048+0000","last_active":"2024-04-17T05:45:52.541681+0000","last_peered":"2024-04-17T05:45:52.541681+0000","last_clean":"2024-04-17T05:45:52.541681+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:45:52.541681+0000","last_undegraded":"2024-04-17T05:45:52.541681+0000","last_fullsized":"2024-04-17T05:45:52.541681+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:49:05.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153957+0000","last_change":"2024-04-17T04:36:33.426587+0000","last_active":"2024-04-17T05:43:15.153957+0000","last_peered":"2024-04-17T05:43:15.153957+0000","last_clean":"2024-04-17T05:43:15.153957+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:43:15.153957+0000","last_undegraded":"2024-04-17T05:43:15.153957+0000","last_fullsized":"2024-04-17T05:43:15.153957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:52:34.456097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'262","reported_seq":866,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.184123+0000","last_change":"2024-04-17T04:36:35.408138+0000","last_active":"2024-04-17T05:45:45.184123+0000","last_peered":"2024-04-17T05:45:45.184123+0000","last_clean":"2024-04-17T05:45:45.184123+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:45:45.184123+0000","last_undegraded":"2024-04-17T05:45:45.184123+0000","last_fullsized":"2024-04-17T05:45:45.184123+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:09.321611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158227+0000","last_change":"2024-04-17T04:36:27.361706+0000","last_active":"2024-04-17T05:43:10.158227+0000","last_peered":"2024-04-17T05:43:10.158227+0000","last_clean":"2024-04-17T05:43:10.158227+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:43:10.158227+0000","last_undegraded":"2024-04-17T05:43:10.158227+0000","last_fullsized":"2024-04-17T05:43:10.158227+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:25.186757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154761+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T05:43:00.154761+0000","last_peered":"2024-04-17T05:43:00.154761+0000","last_clean":"2024-04-17T05:43:00.154761+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:43:00.154761+0000","last_undegraded":"2024-04-17T05:43:00.154761+0000","last_fullsized":"2024-04-17T05:43:00.154761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:44:32.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'278","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:04.937303+0000","last_change":"2024-04-17T04:36:25.356426+0000","last_active":"2024-04-17T05:43:04.937303+0000","last_peered":"2024-04-17T05:43:04.937303+0000","last_clean":"2024-04-17T05:43:04.937303+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:43:04.937303+0000","last_undegraded":"2024-04-17T05:43:04.937303+0000","last_fullsized":"2024-04-17T05:43:04.937303+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:48.430125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11586207,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":563,"num_read_kb":36553,"num_write":1174,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.155027+0000","last_change":"2024-04-17T04:36:29.371781+0000","last_active":"2024-04-17T05:43:20.155027+0000","last_peered":"2024-04-17T05:43:20.155027+0000","last_clean":"2024-04-17T05:43:20.155027+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:43:20.155027+0000","last_undegraded":"2024-04-17T05:43:20.155027+0000","last_fullsized":"2024-04-17T05:43:20.155027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:40:17.404012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'139","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193023+0000","last_change":"2024-04-17T04:36:31.400012+0000","last_active":"2024-04-17T05:43:45.193023+0000","last_peered":"2024-04-17T05:43:45.193023+0000","last_clean":"2024-04-17T05:43:45.193023+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:43:45.193023+0000","last_undegraded":"2024-04-17T05:43:45.193023+0000","last_fullsized":"2024-04-17T05:43:45.193023+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:30:23.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159837+0000","last_change":"2024-04-17T04:36:33.416438+0000","last_active":"2024-04-17T05:43:45.159837+0000","last_peered":"2024-04-17T05:43:45.159837+0000","last_clean":"2024-04-17T05:43:45.159837+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:43:45.159837+0000","last_undegraded":"2024-04-17T05:43:45.159837+0000","last_fullsized":"2024-04-17T05:43:45.159837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:15:47.410813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'130","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.164559+0000","last_change":"2024-04-17T04:36:35.411210+0000","last_active":"2024-04-17T05:43:45.164559+0000","last_peered":"2024-04-17T05:43:45.164559+0000","last_clean":"2024-04-17T05:43:45.164559+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:43:45.164559+0000","last_undegraded":"2024-04-17T05:43:45.164559+0000","last_fullsized":"2024-04-17T05:43:45.164559+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:26.503048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158134+0000","last_change":"2024-04-17T04:36:27.376368+0000","last_active":"2024-04-17T05:43:10.158134+0000","last_peered":"2024-04-17T05:43:10.158134+0000","last_clean":"2024-04-17T05:43:10.158134+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:43:10.158134+0000","last_undegraded":"2024-04-17T05:43:10.158134+0000","last_fullsized":"2024-04-17T05:43:10.158134+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:23:39.170608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'231","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.159243+0000","last_change":"2024-04-17T04:36:25.368624+0000","last_active":"2024-04-17T05:43:20.159243+0000","last_peered":"2024-04-17T05:43:20.159243+0000","last_clean":"2024-04-17T05:43:20.159243+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:43:20.159243+0000","last_undegraded":"2024-04-17T05:43:20.159243+0000","last_fullsized":"2024-04-17T05:43:20.159243+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:51.630737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84605,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":406,"num_read_kb":1448,"num_write":1062,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.156128+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T05:43:00.156128+0000","last_peered":"2024-04-17T05:43:00.156128+0000","last_clean":"2024-04-17T05:43:00.156128+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:43:00.156128+0000","last_undegraded":"2024-04-17T05:43:00.156128+0000","last_fullsized":"2024-04-17T05:43:00.156128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:16.202650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152928+0000","last_change":"2024-04-17T04:36:29.384820+0000","last_active":"2024-04-17T05:43:10.152928+0000","last_peered":"2024-04-17T05:43:10.152928+0000","last_clean":"2024-04-17T05:43:10.152928+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:43:10.152928+0000","last_undegraded":"2024-04-17T05:43:10.152928+0000","last_fullsized":"2024-04-17T05:43:10.152928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:27.497471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'610","reported_seq":1296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.603615+0000","last_change":"2024-04-17T04:36:31.411434+0000","last_active":"2024-04-17T05:45:54.603615+0000","last_peered":"2024-04-17T05:45:54.603615+0000","last_clean":"2024-04-17T05:45:54.603615+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:45:54.603615+0000","last_undegraded":"2024-04-17T05:45:54.603615+0000","last_fullsized":"2024-04-17T05:45:54.603615+0000","mapping_epoch":29,"log_start":"264'500","ondisk_log_start":"264'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:30.011121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101724,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1633,"num_read_kb":2600,"num_write":2162,"num_write_kb":1969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939512+0000","last_change":"2024-04-17T04:36:33.407911+0000","last_active":"2024-04-17T05:43:14.939512+0000","last_peered":"2024-04-17T05:43:14.939512+0000","last_clean":"2024-04-17T05:43:14.939512+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:43:14.939512+0000","last_undegraded":"2024-04-17T05:43:14.939512+0000","last_fullsized":"2024-04-17T05:43:14.939512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:49:35.539455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'82","reported_seq":584,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.166476+0000","last_change":"2024-04-17T04:36:35.429331+0000","last_active":"2024-04-17T05:44:05.166476+0000","last_peered":"2024-04-17T05:44:05.166476+0000","last_clean":"2024-04-17T05:44:05.166476+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:44:05.166476+0000","last_undegraded":"2024-04-17T05:44:05.166476+0000","last_fullsized":"2024-04-17T05:44:05.166476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:03.344843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":592,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152888+0000","last_change":"2024-04-17T04:36:27.376396+0000","last_active":"2024-04-17T05:43:10.152888+0000","last_peered":"2024-04-17T05:43:10.152888+0000","last_clean":"2024-04-17T05:43:10.152888+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:43:10.152888+0000","last_undegraded":"2024-04-17T05:43:10.152888+0000","last_fullsized":"2024-04-17T05:43:10.152888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:36.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'203","reported_seq":824,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157227+0000","last_change":"2024-04-17T04:36:25.345048+0000","last_active":"2024-04-17T05:43:05.157227+0000","last_peered":"2024-04-17T05:43:05.157227+0000","last_clean":"2024-04-17T05:43:05.157227+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:43:05.157227+0000","last_undegraded":"2024-04-17T05:43:05.157227+0000","last_fullsized":"2024-04-17T05:43:05.157227+0000","mapping_epoch":23,"log_start":"264'100","ondisk_log_start":"264'100","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:40:34.714988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9471022,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":330,"num_read_kb":35211,"num_write":775,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":523,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151170+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T05:43:10.151170+0000","last_peered":"2024-04-17T05:43:10.151170+0000","last_clean":"2024-04-17T05:43:10.151170+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:43:10.151170+0000","last_undegraded":"2024-04-17T05:43:10.151170+0000","last_fullsized":"2024-04-17T05:43:10.151170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:33.083623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151795+0000","last_change":"2024-04-17T04:36:29.375400+0000","last_active":"2024-04-17T05:43:10.151795+0000","last_peered":"2024-04-17T05:43:10.151795+0000","last_clean":"2024-04-17T05:43:10.151795+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:43:10.151795+0000","last_undegraded":"2024-04-17T05:43:10.151795+0000","last_fullsized":"2024-04-17T05:43:10.151795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:16.669890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'199","reported_seq":792,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.159277+0000","last_change":"2024-04-17T04:36:31.420876+0000","last_active":"2024-04-17T05:43:15.159277+0000","last_peered":"2024-04-17T05:43:15.159277+0000","last_clean":"2024-04-17T05:43:15.159277+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:43:15.159277+0000","last_undegraded":"2024-04-17T05:43:15.159277+0000","last_fullsized":"2024-04-17T05:43:15.159277+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:23:18.629160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'536","reported_seq":4354,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.668605+0000","last_change":"2024-04-17T04:36:33.407900+0000","last_active":"2024-04-17T05:45:57.668605+0000","last_peered":"2024-04-17T05:45:57.668605+0000","last_clean":"2024-04-17T05:45:57.668605+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:45:57.668605+0000","last_undegraded":"2024-04-17T05:45:57.668605+0000","last_fullsized":"2024-04-17T05:45:57.668605+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:36:50.014322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'156","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.202363+0000","last_change":"2024-04-17T04:36:35.418551+0000","last_active":"2024-04-17T05:44:20.202363+0000","last_peered":"2024-04-17T05:44:20.202363+0000","last_clean":"2024-04-17T05:44:20.202363+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:44:20.202363+0000","last_undegraded":"2024-04-17T05:44:20.202363+0000","last_fullsized":"2024-04-17T05:44:20.202363+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:08.177362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":799,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180216+0000","last_change":"2024-04-17T04:36:27.395791+0000","last_active":"2024-04-17T05:42:45.180216+0000","last_peered":"2024-04-17T05:42:45.180216+0000","last_clean":"2024-04-17T05:42:45.180216+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:42:45.180216+0000","last_undegraded":"2024-04-17T05:42:45.180216+0000","last_fullsized":"2024-04-17T05:42:45.180216+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:29:33.119512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'309","reported_seq":966,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191121+0000","last_change":"2024-04-17T04:36:25.347483+0000","last_active":"2024-04-17T05:43:35.191121+0000","last_peered":"2024-04-17T05:43:35.191121+0000","last_clean":"2024-04-17T05:43:35.191121+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:43:35.191121+0000","last_undegraded":"2024-04-17T05:43:35.191121+0000","last_fullsized":"2024-04-17T05:43:35.191121+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:52:07.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290300,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":521,"num_read_kb":5842,"num_write":1316,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":527,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.161303+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T05:43:25.161303+0000","last_peered":"2024-04-17T05:43:25.161303+0000","last_clean":"2024-04-17T05:43:25.161303+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:43:25.161303+0000","last_undegraded":"2024-04-17T05:43:25.161303+0000","last_fullsized":"2024-04-17T05:43:25.161303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:38.757105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938398+0000","last_change":"2024-04-17T04:36:29.375601+0000","last_active":"2024-04-17T05:43:09.938398+0000","last_peered":"2024-04-17T05:43:09.938398+0000","last_clean":"2024-04-17T05:43:09.938398+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:43:09.938398+0000","last_undegraded":"2024-04-17T05:43:09.938398+0000","last_fullsized":"2024-04-17T05:43:09.938398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:57.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'136","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939607+0000","last_change":"2024-04-17T04:36:31.400243+0000","last_active":"2024-04-17T05:43:14.939607+0000","last_peered":"2024-04-17T05:43:14.939607+0000","last_clean":"2024-04-17T05:43:14.939607+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:43:14.939607+0000","last_undegraded":"2024-04-17T05:43:14.939607+0000","last_fullsized":"2024-04-17T05:43:14.939607+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:41.986999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'528","reported_seq":2299,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.662141+0000","last_change":"2024-04-17T04:36:33.421012+0000","last_active":"2024-04-17T05:45:57.662141+0000","last_peered":"2024-04-17T05:45:57.662141+0000","last_clean":"2024-04-17T05:45:57.662141+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:45:57.662141+0000","last_undegraded":"2024-04-17T05:45:57.662141+0000","last_fullsized":"2024-04-17T05:45:57.662141+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":500,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:52.921022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'218","reported_seq":734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.976785+0000","last_change":"2024-04-17T04:36:35.431849+0000","last_active":"2024-04-17T05:45:45.976785+0000","last_peered":"2024-04-17T05:45:45.976785+0000","last_clean":"2024-04-17T05:45:45.976785+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:45:45.976785+0000","last_undegraded":"2024-04-17T05:45:45.976785+0000","last_fullsized":"2024-04-17T05:45:45.976785+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:18.340086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4093","reported_seq":4724,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938365+0000","last_change":"2024-04-17T04:36:27.378432+0000","last_active":"2024-04-17T05:43:09.938365+0000","last_peered":"2024-04-17T05:43:09.938365+0000","last_clean":"2024-04-17T05:43:09.938365+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:43:09.938365+0000","last_undegraded":"2024-04-17T05:43:09.938365+0000","last_fullsized":"2024-04-17T05:43:09.938365+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":3000,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:55.591614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'291","reported_seq":908,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158359+0000","last_change":"2024-04-17T04:36:25.354707+0000","last_active":"2024-04-17T05:43:35.158359+0000","last_peered":"2024-04-17T05:43:35.158359+0000","last_clean":"2024-04-17T05:43:35.158359+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:43:35.158359+0000","last_undegraded":"2024-04-17T05:43:35.158359+0000","last_fullsized":"2024-04-17T05:43:35.158359+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:46.297295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4228470,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":401,"num_read_kb":7519,"num_write":1219,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.156524+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T05:43:30.156524+0000","last_peered":"2024-04-17T05:43:30.156524+0000","last_clean":"2024-04-17T05:43:30.156524+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:43:30.156524+0000","last_undegraded":"2024-04-17T05:43:30.156524+0000","last_fullsized":"2024-04-17T05:43:30.156524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:24.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180151+0000","last_change":"2024-04-17T04:36:29.363086+0000","last_active":"2024-04-17T05:42:45.180151+0000","last_peered":"2024-04-17T05:42:45.180151+0000","last_clean":"2024-04-17T05:42:45.180151+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:42:45.180151+0000","last_undegraded":"2024-04-17T05:42:45.180151+0000","last_fullsized":"2024-04-17T05:42:45.180151+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:48.241347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'715","reported_seq":1444,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.600048+0000","last_change":"2024-04-17T04:36:31.422637+0000","last_active":"2024-04-17T05:45:54.600048+0000","last_peered":"2024-04-17T05:45:54.600048+0000","last_clean":"2024-04-17T05:45:54.600048+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:45:54.600048+0000","last_undegraded":"2024-04-17T05:45:54.600048+0000","last_fullsized":"2024-04-17T05:45:54.600048+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":700,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:37.277476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156924,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1257,"num_read_kb":2730,"num_write":1020,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187164+0000","last_change":"2024-04-17T04:36:33.415987+0000","last_active":"2024-04-17T05:43:15.187164+0000","last_peered":"2024-04-17T05:43:15.187164+0000","last_clean":"2024-04-17T05:43:15.187164+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:43:15.187164+0000","last_undegraded":"2024-04-17T05:43:15.187164+0000","last_fullsized":"2024-04-17T05:43:15.187164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:25.307133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'102","reported_seq":591,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.397345+0000","last_change":"2024-04-17T04:36:35.424792+0000","last_active":"2024-04-17T05:42:25.397345+0000","last_peered":"2024-04-17T05:42:25.397345+0000","last_clean":"2024-04-17T05:42:25.397345+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:42:25.397345+0000","last_undegraded":"2024-04-17T05:42:25.397345+0000","last_fullsized":"2024-04-17T05:42:25.397345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:27.783198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'149","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.086388+0000","last_change":"2024-04-17T04:36:35.425561+0000","last_active":"2024-04-17T05:45:45.086388+0000","last_peered":"2024-04-17T05:45:45.086388+0000","last_clean":"2024-04-17T05:45:45.086388+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:45:45.086388+0000","last_undegraded":"2024-04-17T05:45:45.086388+0000","last_fullsized":"2024-04-17T05:45:45.086388+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:48.063997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'73","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125289+0000","last_change":"2024-04-17T04:36:27.400403+0000","last_active":"2024-04-17T05:41:50.125289+0000","last_peered":"2024-04-17T05:41:50.125289+0000","last_clean":"2024-04-17T05:41:50.125289+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:41:50.125289+0000","last_undegraded":"2024-04-17T05:41:50.125289+0000","last_fullsized":"2024-04-17T05:41:50.125289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:03.426737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'220","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.706350+0000","last_change":"2024-04-17T04:36:25.357052+0000","last_active":"2024-04-17T05:42:50.706350+0000","last_peered":"2024-04-17T05:42:50.706350+0000","last_clean":"2024-04-17T05:42:50.706350+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:42:50.706350+0000","last_undegraded":"2024-04-17T05:42:50.706350+0000","last_fullsized":"2024-04-17T05:42:50.706350+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:07:39.562019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502965,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":370,"num_read_kb":11105,"num_write":918,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139292+0000","last_change":"2024-04-17T04:36:29.391641+0000","last_active":"2024-04-17T05:41:50.139292+0000","last_peered":"2024-04-17T05:41:50.139292+0000","last_clean":"2024-04-17T05:41:50.139292+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:41:50.139292+0000","last_undegraded":"2024-04-17T05:41:50.139292+0000","last_fullsized":"2024-04-17T05:41:50.139292+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:29.991489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'296","reported_seq":886,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.200972+0000","last_change":"2024-04-17T04:36:31.406561+0000","last_active":"2024-04-17T05:44:15.200972+0000","last_peered":"2024-04-17T05:44:15.200972+0000","last_clean":"2024-04-17T05:44:15.200972+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:44:15.200972+0000","last_undegraded":"2024-04-17T05:44:15.200972+0000","last_fullsized":"2024-04-17T05:44:15.200972+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:50:50.884998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62692,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":511,"num_read_kb":1084,"num_write":418,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168016+0000","last_change":"2024-04-17T04:36:33.420539+0000","last_active":"2024-04-17T05:41:55.168016+0000","last_peered":"2024-04-17T05:41:55.168016+0000","last_clean":"2024-04-17T05:41:55.168016+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:41:55.168016+0000","last_undegraded":"2024-04-17T05:41:55.168016+0000","last_fullsized":"2024-04-17T05:41:55.168016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:42.375895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'98","reported_seq":603,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.713280+0000","last_change":"2024-04-17T04:36:35.430479+0000","last_active":"2024-04-17T05:45:44.713280+0000","last_peered":"2024-04-17T05:45:44.713280+0000","last_clean":"2024-04-17T05:45:44.713280+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:45:44.713280+0000","last_undegraded":"2024-04-17T05:45:44.713280+0000","last_fullsized":"2024-04-17T05:45:44.713280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:21:02.348756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'126","reported_seq":737,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:17.594052+0000","last_change":"2024-04-17T04:36:27.389126+0000","last_active":"2024-04-17T05:42:17.594052+0000","last_peered":"2024-04-17T05:42:17.594052+0000","last_clean":"2024-04-17T05:42:17.594052+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:42:17.594052+0000","last_undegraded":"2024-04-17T05:42:17.594052+0000","last_fullsized":"2024-04-17T05:42:17.594052+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:40.137919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'309","reported_seq":961,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144594+0000","last_change":"2024-04-17T04:37:03.635652+0000","last_active":"2024-04-17T05:42:20.144594+0000","last_peered":"2024-04-17T05:42:20.144594+0000","last_clean":"2024-04-17T05:42:20.144594+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:42:20.144594+0000","last_undegraded":"2024-04-17T05:42:20.144594+0000","last_fullsized":"2024-04-17T05:42:20.144594+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:33.722634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3797882,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":561,"num_read_kb":15946,"num_write":1479,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166897+0000","last_change":"2024-04-17T04:36:29.371357+0000","last_active":"2024-04-17T05:41:50.166897+0000","last_peered":"2024-04-17T05:41:50.166897+0000","last_clean":"2024-04-17T05:41:50.166897+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:41:50.166897+0000","last_undegraded":"2024-04-17T05:41:50.166897+0000","last_fullsized":"2024-04-17T05:41:50.166897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:50.443422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'56","reported_seq":567,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168798+0000","last_change":"2024-04-17T04:36:31.404202+0000","last_active":"2024-04-17T05:44:15.168798+0000","last_peered":"2024-04-17T05:44:15.168798+0000","last_clean":"2024-04-17T05:44:15.168798+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:44:15.168798+0000","last_undegraded":"2024-04-17T05:44:15.168798+0000","last_fullsized":"2024-04-17T05:44:15.168798+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:41.388160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134552+0000","last_change":"2024-04-17T04:36:33.416154+0000","last_active":"2024-04-17T05:41:55.134552+0000","last_peered":"2024-04-17T05:41:55.134552+0000","last_clean":"2024-04-17T05:41:55.134552+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:41:55.134552+0000","last_undegraded":"2024-04-17T05:41:55.134552+0000","last_fullsized":"2024-04-17T05:41:55.134552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:21.721127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'217","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.789782+0000","last_change":"2024-04-17T04:36:35.428428+0000","last_active":"2024-04-17T05:45:44.789782+0000","last_peered":"2024-04-17T05:45:44.789782+0000","last_clean":"2024-04-17T05:45:44.789782+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:45:44.789782+0000","last_undegraded":"2024-04-17T05:45:44.789782+0000","last_fullsized":"2024-04-17T05:45:44.789782+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:55.027046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166680+0000","last_change":"2024-04-17T04:36:27.395464+0000","last_active":"2024-04-17T05:41:50.166680+0000","last_peered":"2024-04-17T05:41:50.166680+0000","last_clean":"2024-04-17T05:41:50.166680+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:41:50.166680+0000","last_undegraded":"2024-04-17T05:41:50.166680+0000","last_fullsized":"2024-04-17T05:41:50.166680+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:23.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'286","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:43.075764+0000","last_change":"2024-04-17T04:36:25.338779+0000","last_active":"2024-04-17T05:42:43.075764+0000","last_peered":"2024-04-17T05:42:43.075764+0000","last_clean":"2024-04-17T05:42:43.075764+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:42:43.075764+0000","last_undegraded":"2024-04-17T05:42:43.075764+0000","last_fullsized":"2024-04-17T05:42:43.075764+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:59.502139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56924,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":518,"num_read_kb":11343,"num_write":1233,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133249+0000","last_change":"2024-04-17T04:36:29.381180+0000","last_active":"2024-04-17T05:41:50.133249+0000","last_peered":"2024-04-17T05:41:50.133249+0000","last_clean":"2024-04-17T05:41:50.133249+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:41:50.133249+0000","last_undegraded":"2024-04-17T05:41:50.133249+0000","last_fullsized":"2024-04-17T05:41:50.133249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:01.625306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'335","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.201110+0000","last_change":"2024-04-17T04:36:31.406900+0000","last_active":"2024-04-17T05:44:15.201110+0000","last_peered":"2024-04-17T05:44:15.201110+0000","last_clean":"2024-04-17T05:44:15.201110+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:44:15.201110+0000","last_undegraded":"2024-04-17T05:44:15.201110+0000","last_fullsized":"2024-04-17T05:44:15.201110+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:47.400810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62421,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":539,"num_read_kb":1090,"num_write":434,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":490,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135689+0000","last_change":"2024-04-17T04:36:33.412913+0000","last_active":"2024-04-17T05:41:55.135689+0000","last_peered":"2024-04-17T05:41:55.135689+0000","last_clean":"2024-04-17T05:41:55.135689+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:41:55.135689+0000","last_undegraded":"2024-04-17T05:41:55.135689+0000","last_fullsized":"2024-04-17T05:41:55.135689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:56.825814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'112","reported_seq":599,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.542927+0000","last_change":"2024-04-17T04:36:35.424722+0000","last_active":"2024-04-17T05:41:51.542927+0000","last_peered":"2024-04-17T05:41:51.542927+0000","last_clean":"2024-04-17T05:41:51.542927+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:41:51.542927+0000","last_undegraded":"2024-04-17T05:41:51.542927+0000","last_fullsized":"2024-04-17T05:41:51.542927+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:15:26.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":759,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166609+0000","last_change":"2024-04-17T04:36:27.378417+0000","last_active":"2024-04-17T05:41:50.166609+0000","last_peered":"2024-04-17T05:41:50.166609+0000","last_clean":"2024-04-17T05:41:50.166609+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:41:50.166609+0000","last_undegraded":"2024-04-17T05:41:50.166609+0000","last_fullsized":"2024-04-17T05:41:50.166609+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:17.606187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'245","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.397517+0000","last_change":"2024-04-17T04:36:25.344700+0000","last_active":"2024-04-17T05:43:39.397517+0000","last_peered":"2024-04-17T05:43:39.397517+0000","last_clean":"2024-04-17T05:43:39.397517+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:43:39.397517+0000","last_undegraded":"2024-04-17T05:43:39.397517+0000","last_fullsized":"2024-04-17T05:43:39.397517+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:28:58.108460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18113654,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":380,"num_read_kb":84514,"num_write":925,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134534+0000","last_change":"2024-04-17T04:36:29.384475+0000","last_active":"2024-04-17T05:41:50.134534+0000","last_peered":"2024-04-17T05:41:50.134534+0000","last_clean":"2024-04-17T05:41:50.134534+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:41:50.134534+0000","last_undegraded":"2024-04-17T05:41:50.134534+0000","last_fullsized":"2024-04-17T05:41:50.134534+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:57.583947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'253","reported_seq":828,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166282+0000","last_change":"2024-04-17T04:36:31.380383+0000","last_active":"2024-04-17T05:44:15.166282+0000","last_peered":"2024-04-17T05:44:15.166282+0000","last_clean":"2024-04-17T05:44:15.166282+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:44:15.166282+0000","last_undegraded":"2024-04-17T05:44:15.166282+0000","last_fullsized":"2024-04-17T05:44:15.166282+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:45.175550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51759,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":419,"num_read_kb":889,"num_write":336,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136027+0000","last_change":"2024-04-17T04:36:33.429804+0000","last_active":"2024-04-17T05:41:55.136027+0000","last_peered":"2024-04-17T05:41:55.136027+0000","last_clean":"2024-04-17T05:41:55.136027+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:41:55.136027+0000","last_undegraded":"2024-04-17T05:41:55.136027+0000","last_fullsized":"2024-04-17T05:41:55.136027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:10.357384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'289","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151938+0000","last_change":"2024-04-17T04:36:27.387575+0000","last_active":"2024-04-17T05:43:10.151938+0000","last_peered":"2024-04-17T05:43:10.151938+0000","last_clean":"2024-04-17T05:43:10.151938+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:43:10.151938+0000","last_undegraded":"2024-04-17T05:43:10.151938+0000","last_fullsized":"2024-04-17T05:43:10.151938+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:05.627702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'242","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164601+0000","last_change":"2024-04-17T04:36:25.326760+0000","last_active":"2024-04-17T05:41:40.164601+0000","last_peered":"2024-04-17T05:41:40.164601+0000","last_clean":"2024-04-17T05:41:40.164601+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:41:40.164601+0000","last_undegraded":"2024-04-17T05:41:40.164601+0000","last_fullsized":"2024-04-17T05:41:40.164601+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:25.101546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7321484,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":471,"num_read_kb":33047,"num_write":1108,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146569+0000","last_change":"2024-04-17T04:36:29.373251+0000","last_active":"2024-04-17T05:42:40.146569+0000","last_peered":"2024-04-17T05:42:40.146569+0000","last_clean":"2024-04-17T05:42:40.146569+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:42:40.146569+0000","last_undegraded":"2024-04-17T05:42:40.146569+0000","last_fullsized":"2024-04-17T05:42:40.146569+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:18:53.682774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'466","reported_seq":1107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.641533+0000","last_change":"2024-04-17T04:36:31.397750+0000","last_active":"2024-04-17T05:45:54.641533+0000","last_peered":"2024-04-17T05:45:54.641533+0000","last_clean":"2024-04-17T05:45:54.641533+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:45:54.641533+0000","last_undegraded":"2024-04-17T05:45:54.641533+0000","last_fullsized":"2024-04-17T05:45:54.641533+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":400,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:14.506372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103130,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":820,"num_read_kb":1787,"num_write":708,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.135785+0000","last_change":"2024-04-17T04:36:33.418783+0000","last_active":"2024-04-17T05:41:35.135785+0000","last_peered":"2024-04-17T05:41:35.135785+0000","last_clean":"2024-04-17T05:41:35.135785+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:41:35.135785+0000","last_undegraded":"2024-04-17T05:41:35.135785+0000","last_fullsized":"2024-04-17T05:41:35.135785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:38:46.129082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'171","reported_seq":678,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168149+0000","last_change":"2024-04-17T04:36:35.419329+0000","last_active":"2024-04-17T05:41:55.168149+0000","last_peered":"2024-04-17T05:41:55.168149+0000","last_clean":"2024-04-17T05:41:55.168149+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:41:55.168149+0000","last_undegraded":"2024-04-17T05:41:55.168149+0000","last_fullsized":"2024-04-17T05:41:55.168149+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:14.062860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":709,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146514+0000","last_change":"2024-04-17T04:36:27.386903+0000","last_active":"2024-04-17T05:42:40.146514+0000","last_peered":"2024-04-17T05:42:40.146514+0000","last_clean":"2024-04-17T05:42:40.146514+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:42:40.146514+0000","last_undegraded":"2024-04-17T05:42:40.146514+0000","last_fullsized":"2024-04-17T05:42:40.146514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:46:53.605066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'252","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.140440+0000","last_change":"2024-04-17T04:36:25.357464+0000","last_active":"2024-04-17T05:41:55.140440+0000","last_peered":"2024-04-17T05:41:55.140440+0000","last_clean":"2024-04-17T05:41:55.140440+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:41:55.140440+0000","last_undegraded":"2024-04-17T05:41:55.140440+0000","last_fullsized":"2024-04-17T05:41:55.140440+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:12.420278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6367399,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":447,"num_read_kb":112378,"num_write":1170,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151910+0000","last_change":"2024-04-17T04:36:29.376006+0000","last_active":"2024-04-17T05:43:10.151910+0000","last_peered":"2024-04-17T05:43:10.151910+0000","last_clean":"2024-04-17T05:43:10.151910+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:43:10.151910+0000","last_undegraded":"2024-04-17T05:43:10.151910+0000","last_fullsized":"2024-04-17T05:43:10.151910+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.144787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'310","reported_seq":904,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.163993+0000","last_change":"2024-04-17T04:36:31.402737+0000","last_active":"2024-04-17T05:44:05.163993+0000","last_peered":"2024-04-17T05:44:05.163993+0000","last_clean":"2024-04-17T05:44:05.163993+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:44:05.163993+0000","last_undegraded":"2024-04-17T05:44:05.163993+0000","last_fullsized":"2024-04-17T05:44:05.163993+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:20.700900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52833,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":523,"num_read_kb":1031,"num_write":436,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.136274+0000","last_change":"2024-04-17T04:36:33.415254+0000","last_active":"2024-04-17T05:41:35.136274+0000","last_peered":"2024-04-17T05:41:35.136274+0000","last_clean":"2024-04-17T05:41:35.136274+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:41:35.136274+0000","last_undegraded":"2024-04-17T05:41:35.136274+0000","last_fullsized":"2024-04-17T05:41:35.136274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:37.629549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'175","reported_seq":711,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166706+0000","last_change":"2024-04-17T04:36:35.424191+0000","last_active":"2024-04-17T05:44:15.166706+0000","last_peered":"2024-04-17T05:44:15.166706+0000","last_clean":"2024-04-17T05:44:15.166706+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:44:15.166706+0000","last_undegraded":"2024-04-17T05:44:15.166706+0000","last_fullsized":"2024-04-17T05:44:15.166706+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:48.892746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'114","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.162530+0000","last_change":"2024-04-17T04:36:27.378054+0000","last_active":"2024-04-17T05:41:30.162530+0000","last_peered":"2024-04-17T05:41:30.162530+0000","last_clean":"2024-04-17T05:41:30.162530+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:41:30.162530+0000","last_undegraded":"2024-04-17T05:41:30.162530+0000","last_fullsized":"2024-04-17T05:41:30.162530+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:41.230833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'297","reported_seq":981,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:38.607179+0000","last_change":"2024-04-17T04:36:25.356583+0000","last_active":"2024-04-17T05:41:38.607179+0000","last_peered":"2024-04-17T05:41:38.607179+0000","last_clean":"2024-04-17T05:41:38.607179+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:41:38.607179+0000","last_undegraded":"2024-04-17T05:41:38.607179+0000","last_fullsized":"2024-04-17T05:41:38.607179+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:38.408649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5881433,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":515,"num_read_kb":10795,"num_write":1289,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":831,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.514388+0000","last_change":"2024-04-17T04:36:29.370143+0000","last_active":"2024-04-17T05:45:54.514388+0000","last_peered":"2024-04-17T05:45:54.514388+0000","last_clean":"2024-04-17T05:45:54.514388+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:45:54.514388+0000","last_undegraded":"2024-04-17T05:45:54.514388+0000","last_fullsized":"2024-04-17T05:45:54.514388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:33.209495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'293","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.216972+0000","last_change":"2024-04-17T04:36:31.421343+0000","last_active":"2024-04-17T05:45:20.216972+0000","last_peered":"2024-04-17T05:45:20.216972+0000","last_clean":"2024-04-17T05:45:20.216972+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:45:20.216972+0000","last_undegraded":"2024-04-17T05:45:20.216972+0000","last_fullsized":"2024-04-17T05:45:20.216972+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:50.024878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58684,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":490,"num_read_kb":1030,"num_write":398,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.152311+0000","last_change":"2024-04-17T04:36:33.415113+0000","last_active":"2024-04-17T05:42:50.152311+0000","last_peered":"2024-04-17T05:42:50.152311+0000","last_clean":"2024-04-17T05:42:50.152311+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:42:50.152311+0000","last_undegraded":"2024-04-17T05:42:50.152311+0000","last_fullsized":"2024-04-17T05:42:50.152311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:41.644384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'102","reported_seq":601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.654720+0000","last_change":"2024-04-17T04:36:35.428333+0000","last_active":"2024-04-17T05:45:54.654720+0000","last_peered":"2024-04-17T05:45:54.654720+0000","last_clean":"2024-04-17T05:45:54.654720+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:45:54.654720+0000","last_undegraded":"2024-04-17T05:45:54.654720+0000","last_fullsized":"2024-04-17T05:45:54.654720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:39.420180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.133409+0000","last_change":"2024-04-17T04:36:27.356010+0000","last_active":"2024-04-17T05:41:30.133409+0000","last_peered":"2024-04-17T05:41:30.133409+0000","last_clean":"2024-04-17T05:41:30.133409+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:41:30.133409+0000","last_undegraded":"2024-04-17T05:41:30.133409+0000","last_fullsized":"2024-04-17T05:41:30.133409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:42.969449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'255","reported_seq":1080,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.128114+0000","last_change":"2024-04-17T04:36:25.354574+0000","last_active":"2024-04-17T05:41:25.128114+0000","last_peered":"2024-04-17T05:41:25.128114+0000","last_clean":"2024-04-17T05:41:25.128114+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:41:25.128114+0000","last_undegraded":"2024-04-17T05:41:25.128114+0000","last_fullsized":"2024-04-17T05:41:25.128114+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:55.589797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21838700,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":777,"num_read_kb":11279,"num_write":1138,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.135309+0000","last_change":"2024-04-17T04:36:29.365578+0000","last_active":"2024-04-17T05:41:30.135309+0000","last_peered":"2024-04-17T05:41:30.135309+0000","last_clean":"2024-04-17T05:41:30.135309+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:41:30.135309+0000","last_undegraded":"2024-04-17T05:41:30.135309+0000","last_fullsized":"2024-04-17T05:41:30.135309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:29.189344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'119","reported_seq":666,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.137220+0000","last_change":"2024-04-17T04:36:31.422491+0000","last_active":"2024-04-17T05:42:45.137220+0000","last_peered":"2024-04-17T05:42:45.137220+0000","last_clean":"2024-04-17T05:42:45.137220+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:42:45.137220+0000","last_undegraded":"2024-04-17T05:42:45.137220+0000","last_fullsized":"2024-04-17T05:42:45.137220+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:06.145178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'535","reported_seq":2310,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.671073+0000","last_change":"2024-04-17T04:36:33.404252+0000","last_active":"2024-04-17T05:45:57.671073+0000","last_peered":"2024-04-17T05:45:57.671073+0000","last_clean":"2024-04-17T05:45:57.671073+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:45:57.671073+0000","last_undegraded":"2024-04-17T05:45:57.671073+0000","last_fullsized":"2024-04-17T05:45:57.671073+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":500,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:11.625831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'114","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134497+0000","last_change":"2024-04-17T04:36:35.413901+0000","last_active":"2024-04-17T05:41:55.134497+0000","last_peered":"2024-04-17T05:41:55.134497+0000","last_clean":"2024-04-17T05:41:55.134497+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:41:55.134497+0000","last_undegraded":"2024-04-17T05:41:55.134497+0000","last_fullsized":"2024-04-17T05:41:55.134497+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:00:54.978078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'229","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120332+0000","last_change":"2024-04-17T04:36:27.394864+0000","last_active":"2024-04-17T05:41:30.120332+0000","last_peered":"2024-04-17T05:41:30.120332+0000","last_clean":"2024-04-17T05:41:30.120332+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:41:30.120332+0000","last_undegraded":"2024-04-17T05:41:30.120332+0000","last_fullsized":"2024-04-17T05:41:30.120332+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:50:03.593045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'268","reported_seq":867,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.133458+0000","last_change":"2024-04-17T04:36:25.369133+0000","last_active":"2024-04-17T05:41:25.133458+0000","last_peered":"2024-04-17T05:41:25.133458+0000","last_clean":"2024-04-17T05:41:25.133458+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:41:25.133458+0000","last_undegraded":"2024-04-17T05:41:25.133458+0000","last_fullsized":"2024-04-17T05:41:25.133458+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:22.406331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10495834,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":376,"num_read_kb":414,"num_write":1132,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.128372+0000","last_change":"2024-04-17T04:36:29.379152+0000","last_active":"2024-04-17T05:41:30.128372+0000","last_peered":"2024-04-17T05:41:30.128372+0000","last_clean":"2024-04-17T05:41:30.128372+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:41:30.128372+0000","last_undegraded":"2024-04-17T05:41:30.128372+0000","last_fullsized":"2024-04-17T05:41:30.128372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:52.689910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'336","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.458700+0000","last_change":"2024-04-17T04:36:31.396219+0000","last_active":"2024-04-17T05:45:46.458700+0000","last_peered":"2024-04-17T05:45:46.458700+0000","last_clean":"2024-04-17T05:45:46.458700+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:45:46.458700+0000","last_undegraded":"2024-04-17T05:45:46.458700+0000","last_fullsized":"2024-04-17T05:45:46.458700+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:39:05.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":534,"num_read_kb":1021,"num_write":420,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134718+0000","last_change":"2024-04-17T04:36:33.430639+0000","last_active":"2024-04-17T05:41:50.134718+0000","last_peered":"2024-04-17T05:41:50.134718+0000","last_clean":"2024-04-17T05:41:50.134718+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:41:50.134718+0000","last_undegraded":"2024-04-17T05:41:50.134718+0000","last_fullsized":"2024-04-17T05:41:50.134718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.795912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'127","reported_seq":629,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166471+0000","last_change":"2024-04-17T04:36:35.436213+0000","last_active":"2024-04-17T05:44:15.166471+0000","last_peered":"2024-04-17T05:44:15.166471+0000","last_clean":"2024-04-17T05:44:15.166471+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:44:15.166471+0000","last_undegraded":"2024-04-17T05:44:15.166471+0000","last_fullsized":"2024-04-17T05:44:15.166471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:44.113556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.536900+0000","last_change":"2024-04-17T04:36:27.378160+0000","last_active":"2024-04-17T05:41:51.536900+0000","last_peered":"2024-04-17T05:41:51.536900+0000","last_clean":"2024-04-17T05:41:51.536900+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:41:51.536900+0000","last_undegraded":"2024-04-17T05:41:51.536900+0000","last_fullsized":"2024-04-17T05:41:51.536900+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:53.663348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'275","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.133518+0000","last_change":"2024-04-17T04:36:25.365976+0000","last_active":"2024-04-17T05:41:25.133518+0000","last_peered":"2024-04-17T05:41:25.133518+0000","last_clean":"2024-04-17T05:41:25.133518+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:41:25.133518+0000","last_undegraded":"2024-04-17T05:41:25.133518+0000","last_fullsized":"2024-04-17T05:41:25.133518+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:05.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267001,"num_objects":79,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":484,"num_read_kb":31239,"num_write":1200,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134570+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:41:30.134570+0000","last_peered":"2024-04-17T05:41:30.134570+0000","last_clean":"2024-04-17T05:41:30.134570+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:41:30.134570+0000","last_undegraded":"2024-04-17T05:41:30.134570+0000","last_fullsized":"2024-04-17T05:41:30.134570+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:39.416117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'268","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168845+0000","last_change":"2024-04-17T04:36:31.403334+0000","last_active":"2024-04-17T05:44:15.168845+0000","last_peered":"2024-04-17T05:44:15.168845+0000","last_clean":"2024-04-17T05:44:15.168845+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:44:15.168845+0000","last_undegraded":"2024-04-17T05:44:15.168845+0000","last_fullsized":"2024-04-17T05:44:15.168845+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:25.865066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":448,"num_read_kb":958,"num_write":362,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136680+0000","last_change":"2024-04-17T04:36:33.420313+0000","last_active":"2024-04-17T05:42:00.136680+0000","last_peered":"2024-04-17T05:42:00.136680+0000","last_clean":"2024-04-17T05:42:00.136680+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:42:00.136680+0000","last_undegraded":"2024-04-17T05:42:00.136680+0000","last_fullsized":"2024-04-17T05:42:00.136680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:55.964612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'134","reported_seq":655,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.071549+0000","last_change":"2024-04-17T04:36:35.422747+0000","last_active":"2024-04-17T05:45:46.071549+0000","last_peered":"2024-04-17T05:45:46.071549+0000","last_clean":"2024-04-17T05:45:46.071549+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:45:46.071549+0000","last_undegraded":"2024-04-17T05:45:46.071549+0000","last_fullsized":"2024-04-17T05:45:46.071549+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:32.269106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'203","reported_seq":810,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:14.025767+0000","last_change":"2024-04-17T04:36:27.381423+0000","last_active":"2024-04-17T05:42:14.025767+0000","last_peered":"2024-04-17T05:42:14.025767+0000","last_clean":"2024-04-17T05:42:14.025767+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:42:14.025767+0000","last_undegraded":"2024-04-17T05:42:14.025767+0000","last_fullsized":"2024-04-17T05:42:14.025767+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:04.091357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'307","reported_seq":970,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.179182+0000","last_change":"2024-04-17T04:36:25.374623+0000","last_active":"2024-04-17T05:42:40.179182+0000","last_peered":"2024-04-17T05:42:40.179182+0000","last_clean":"2024-04-17T05:42:40.179182+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:42:40.179182+0000","last_undegraded":"2024-04-17T05:42:40.179182+0000","last_fullsized":"2024-04-17T05:42:40.179182+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:52.802493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13343203,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":445,"num_read_kb":19835,"num_write":1327,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123902+0000","last_change":"2024-04-17T04:36:29.377026+0000","last_active":"2024-04-17T05:41:45.123902+0000","last_peered":"2024-04-17T05:41:45.123902+0000","last_clean":"2024-04-17T05:41:45.123902+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:41:45.123902+0000","last_undegraded":"2024-04-17T05:41:45.123902+0000","last_fullsized":"2024-04-17T05:41:45.123902+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:26:46.743881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'255","reported_seq":822,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198320+0000","last_change":"2024-04-17T04:36:31.398159+0000","last_active":"2024-04-17T05:44:05.198320+0000","last_peered":"2024-04-17T05:44:05.198320+0000","last_clean":"2024-04-17T05:44:05.198320+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:44:05.198320+0000","last_undegraded":"2024-04-17T05:44:05.198320+0000","last_fullsized":"2024-04-17T05:44:05.198320+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:04.571296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56291,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":432,"num_read_kb":970,"num_write":360,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.121505+0000","last_change":"2024-04-17T04:36:33.401472+0000","last_active":"2024-04-17T05:41:35.121505+0000","last_peered":"2024-04-17T05:41:35.121505+0000","last_clean":"2024-04-17T05:41:35.121505+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:41:35.121505+0000","last_undegraded":"2024-04-17T05:41:35.121505+0000","last_fullsized":"2024-04-17T05:41:35.121505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:49.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'141","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.700107+0000","last_change":"2024-04-17T04:36:35.423654+0000","last_active":"2024-04-17T05:45:40.700107+0000","last_peered":"2024-04-17T05:45:40.700107+0000","last_clean":"2024-04-17T05:45:40.700107+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:45:40.700107+0000","last_undegraded":"2024-04-17T05:45:40.700107+0000","last_fullsized":"2024-04-17T05:45:40.700107+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:17:36.321043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'297","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123940+0000","last_change":"2024-04-17T04:36:27.400669+0000","last_active":"2024-04-17T05:41:45.123940+0000","last_peered":"2024-04-17T05:41:45.123940+0000","last_clean":"2024-04-17T05:41:45.123940+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:41:45.123940+0000","last_undegraded":"2024-04-17T05:41:45.123940+0000","last_fullsized":"2024-04-17T05:41:45.123940+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:59.138022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'362","reported_seq":1068,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151944+0000","last_change":"2024-04-17T04:36:25.356509+0000","last_active":"2024-04-17T05:43:05.151944+0000","last_peered":"2024-04-17T05:43:05.151944+0000","last_clean":"2024-04-17T05:43:05.151944+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:43:05.151944+0000","last_undegraded":"2024-04-17T05:43:05.151944+0000","last_fullsized":"2024-04-17T05:43:05.151944+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":300,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:14.238916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16573947,"num_objects":81,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":588,"num_read_kb":71297,"num_write":1611,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"35'6","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.508087+0000","last_change":"2024-04-17T04:36:29.376775+0000","last_active":"2024-04-17T05:45:54.508087+0000","last_peered":"2024-04-17T05:45:54.508087+0000","last_clean":"2024-04-17T05:45:54.508087+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:45:54.508087+0000","last_undegraded":"2024-04-17T05:45:54.508087+0000","last_fullsized":"2024-04-17T05:45:54.508087+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:09.589509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":498,"num_read_kb":809,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'314","reported_seq":919,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.158024+0000","last_change":"2024-04-17T04:36:31.391158+0000","last_active":"2024-04-17T05:44:05.158024+0000","last_peered":"2024-04-17T05:44:05.158024+0000","last_clean":"2024-04-17T05:44:05.158024+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:44:05.158024+0000","last_undegraded":"2024-04-17T05:44:05.158024+0000","last_fullsized":"2024-04-17T05:44:05.158024+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:43.696307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58738,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":517,"num_read_kb":1039,"num_write":414,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130642+0000","last_change":"2024-04-17T04:36:33.430195+0000","last_active":"2024-04-17T05:41:35.130642+0000","last_peered":"2024-04-17T05:41:35.130642+0000","last_clean":"2024-04-17T05:41:35.130642+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:41:35.130642+0000","last_undegraded":"2024-04-17T05:41:35.130642+0000","last_fullsized":"2024-04-17T05:41:35.130642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:37.038297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'138","reported_seq":637,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.163062+0000","last_change":"2024-04-17T04:36:35.427734+0000","last_active":"2024-04-17T05:44:25.163062+0000","last_peered":"2024-04-17T05:44:25.163062+0000","last_clean":"2024-04-17T05:44:25.163062+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:44:25.163062+0000","last_undegraded":"2024-04-17T05:44:25.163062+0000","last_fullsized":"2024-04-17T05:44:25.163062+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:48:29.121374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'154","reported_seq":788,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164804+0000","last_change":"2024-04-17T04:36:27.397669+0000","last_active":"2024-04-17T05:43:40.164804+0000","last_peered":"2024-04-17T05:43:40.164804+0000","last_clean":"2024-04-17T05:43:40.164804+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:43:40.164804+0000","last_undegraded":"2024-04-17T05:43:40.164804+0000","last_fullsized":"2024-04-17T05:43:40.164804+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:46:34.375349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'355","reported_seq":1073,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150519+0000","last_change":"2024-04-17T04:36:25.383690+0000","last_active":"2024-04-17T05:43:05.150519+0000","last_peered":"2024-04-17T05:43:05.150519+0000","last_clean":"2024-04-17T05:43:05.150519+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:43:05.150519+0000","last_undegraded":"2024-04-17T05:43:05.150519+0000","last_fullsized":"2024-04-17T05:43:05.150519+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:14.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993920,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":770,"num_read_kb":6303,"num_write":1311,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":648,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142976+0000","last_change":"2024-04-17T04:37:03.630121+0000","last_active":"2024-04-17T05:42:05.142976+0000","last_peered":"2024-04-17T05:42:05.142976+0000","last_clean":"2024-04-17T05:42:05.142976+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:42:05.142976+0000","last_undegraded":"2024-04-17T05:42:05.142976+0000","last_fullsized":"2024-04-17T05:42:05.142976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:09:56.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'293","reported_seq":959,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.153334+0000","last_change":"2024-04-17T04:36:25.355361+0000","last_active":"2024-04-17T05:43:05.153334+0000","last_peered":"2024-04-17T05:43:05.153334+0000","last_clean":"2024-04-17T05:43:05.153334+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:43:05.153334+0000","last_undegraded":"2024-04-17T05:43:05.153334+0000","last_fullsized":"2024-04-17T05:43:05.153334+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:15.756630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11689993,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":496,"num_read_kb":39832,"num_write":1126,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'148","reported_seq":762,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163630+0000","last_change":"2024-04-17T04:36:27.388520+0000","last_active":"2024-04-17T05:43:40.163630+0000","last_peered":"2024-04-17T05:43:40.163630+0000","last_clean":"2024-04-17T05:43:40.163630+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:43:40.163630+0000","last_undegraded":"2024-04-17T05:43:40.163630+0000","last_fullsized":"2024-04-17T05:43:40.163630+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:31.868037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'250","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.155879+0000","last_change":"2024-04-17T04:36:25.365759+0000","last_active":"2024-04-17T05:43:05.155879+0000","last_peered":"2024-04-17T05:43:05.155879+0000","last_clean":"2024-04-17T05:43:05.155879+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:43:05.155879+0000","last_undegraded":"2024-04-17T05:43:05.155879+0000","last_fullsized":"2024-04-17T05:43:05.155879+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:25:48.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4343917,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":379,"num_read_kb":10784,"num_write":1108,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'94","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164612+0000","last_change":"2024-04-17T04:36:27.362737+0000","last_active":"2024-04-17T05:43:40.164612+0000","last_peered":"2024-04-17T05:43:40.164612+0000","last_clean":"2024-04-17T05:43:40.164612+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:43:40.164612+0000","last_undegraded":"2024-04-17T05:43:40.164612+0000","last_fullsized":"2024-04-17T05:43:40.164612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:24:50.502747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'241","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153889+0000","last_change":"2024-04-17T04:36:25.355263+0000","last_active":"2024-04-17T05:43:15.153889+0000","last_peered":"2024-04-17T05:43:15.153889+0000","last_clean":"2024-04-17T05:43:15.153889+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:43:15.153889+0000","last_undegraded":"2024-04-17T05:43:15.153889+0000","last_fullsized":"2024-04-17T05:43:15.153889+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:43.114752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11863036,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":642,"num_read_kb":32422,"num_write":1055,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":635,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.189253+0000","last_change":"2024-04-17T04:36:27.395680+0000","last_active":"2024-04-17T05:43:25.189253+0000","last_peered":"2024-04-17T05:43:25.189253+0000","last_clean":"2024-04-17T05:43:25.189253+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:43:25.189253+0000","last_undegraded":"2024-04-17T05:43:25.189253+0000","last_fullsized":"2024-04-17T05:43:25.189253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:04.849323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'223","reported_seq":835,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191011+0000","last_change":"2024-04-17T04:37:03.637190+0000","last_active":"2024-04-17T05:43:35.191011+0000","last_peered":"2024-04-17T05:43:35.191011+0000","last_clean":"2024-04-17T05:43:35.191011+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:43:35.191011+0000","last_undegraded":"2024-04-17T05:43:35.191011+0000","last_fullsized":"2024-04-17T05:43:35.191011+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:24:25.534759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5542945,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":301,"num_read_kb":5919,"num_write":935,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938441+0000","last_change":"2024-04-17T04:36:27.395504+0000","last_active":"2024-04-17T05:43:09.938441+0000","last_peered":"2024-04-17T05:43:09.938441+0000","last_clean":"2024-04-17T05:43:09.938441+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:43:09.938441+0000","last_undegraded":"2024-04-17T05:43:09.938441+0000","last_fullsized":"2024-04-17T05:43:09.938441+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:27.120391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'257","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163317+0000","last_change":"2024-04-17T04:36:25.342748+0000","last_active":"2024-04-17T05:43:35.163317+0000","last_peered":"2024-04-17T05:43:35.163317+0000","last_clean":"2024-04-17T05:43:35.163317+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:43:35.163317+0000","last_undegraded":"2024-04-17T05:43:35.163317+0000","last_fullsized":"2024-04-17T05:43:35.163317+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:25.415287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173927,"num_objects":65,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":430,"num_read_kb":70994,"num_write":1129,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'215","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:36.452977+0000","last_change":"2024-04-17T04:36:27.384524+0000","last_active":"2024-04-17T05:43:36.452977+0000","last_peered":"2024-04-17T05:43:36.452977+0000","last_clean":"2024-04-17T05:43:36.452977+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:43:36.452977+0000","last_undegraded":"2024-04-17T05:43:36.452977+0000","last_fullsized":"2024-04-17T05:43:36.452977+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:35.098376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'314","reported_seq":995,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.157600+0000","last_change":"2024-04-17T04:36:25.386085+0000","last_active":"2024-04-17T05:43:35.157600+0000","last_peered":"2024-04-17T05:43:35.157600+0000","last_clean":"2024-04-17T05:43:35.157600+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:43:35.157600+0000","last_undegraded":"2024-04-17T05:43:35.157600+0000","last_fullsized":"2024-04-17T05:43:35.157600+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:09.994649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12903063,"num_objects":57,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":597,"num_read_kb":34079,"num_write":1419,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151711+0000","last_change":"2024-04-17T04:36:27.387044+0000","last_active":"2024-04-17T05:43:10.151711+0000","last_peered":"2024-04-17T05:43:10.151711+0000","last_clean":"2024-04-17T05:43:10.151711+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:43:10.151711+0000","last_undegraded":"2024-04-17T05:43:10.151711+0000","last_fullsized":"2024-04-17T05:43:10.151711+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:47.002935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'266","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191154+0000","last_change":"2024-04-17T04:36:25.379835+0000","last_active":"2024-04-17T05:43:35.191154+0000","last_peered":"2024-04-17T05:43:35.191154+0000","last_clean":"2024-04-17T05:43:35.191154+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:43:35.191154+0000","last_undegraded":"2024-04-17T05:43:35.191154+0000","last_fullsized":"2024-04-17T05:43:35.191154+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:54:10.099587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16832221,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":477,"num_read_kb":42709,"num_write":1222,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":738,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:49.022874+0000","last_change":"2024-04-17T04:36:27.373717+0000","last_active":"2024-04-17T05:42:49.022874+0000","last_peered":"2024-04-17T05:42:49.022874+0000","last_clean":"2024-04-17T05:42:49.022874+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:42:49.022874+0000","last_undegraded":"2024-04-17T05:42:49.022874+0000","last_fullsized":"2024-04-17T05:42:49.022874+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:38:49.102110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'237","reported_seq":871,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.165564+0000","last_change":"2024-04-17T04:36:25.361686+0000","last_active":"2024-04-17T05:43:50.165564+0000","last_peered":"2024-04-17T05:43:50.165564+0000","last_clean":"2024-04-17T05:43:50.165564+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:43:50.165564+0000","last_undegraded":"2024-04-17T05:43:50.165564+0000","last_fullsized":"2024-04-17T05:43:50.165564+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:40.663817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5566546,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":393,"num_read_kb":545,"num_write":1054,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":754,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125394+0000","last_change":"2024-04-17T04:36:27.378398+0000","last_active":"2024-04-17T05:41:50.125394+0000","last_peered":"2024-04-17T05:41:50.125394+0000","last_clean":"2024-04-17T05:41:50.125394+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:41:50.125394+0000","last_undegraded":"2024-04-17T05:41:50.125394+0000","last_fullsized":"2024-04-17T05:41:50.125394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:49.831818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'237","reported_seq":886,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.350982+0000","last_change":"2024-04-17T04:36:25.353469+0000","last_active":"2024-04-17T05:43:39.350982+0000","last_peered":"2024-04-17T05:43:39.350982+0000","last_clean":"2024-04-17T05:43:39.350982+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:43:39.350982+0000","last_undegraded":"2024-04-17T05:43:39.350982+0000","last_fullsized":"2024-04-17T05:43:39.350982+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2315115,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":391,"num_read_kb":4352,"num_write":1033,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":837,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139934+0000","last_change":"2024-04-17T04:36:27.398568+0000","last_active":"2024-04-17T05:42:00.139934+0000","last_peered":"2024-04-17T05:42:00.139934+0000","last_clean":"2024-04-17T05:42:00.139934+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:42:00.139934+0000","last_undegraded":"2024-04-17T05:42:00.139934+0000","last_fullsized":"2024-04-17T05:42:00.139934+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:48.128020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'247","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144425+0000","last_change":"2024-04-17T04:37:03.632788+0000","last_active":"2024-04-17T05:42:20.144425+0000","last_peered":"2024-04-17T05:42:20.144425+0000","last_clean":"2024-04-17T05:42:20.144425+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:42:20.144425+0000","last_undegraded":"2024-04-17T05:42:20.144425+0000","last_fullsized":"2024-04-17T05:42:20.144425+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:33.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5338870,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":373,"num_read_kb":16015,"num_write":997,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'95","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124087+0000","last_change":"2024-04-17T04:36:27.402182+0000","last_active":"2024-04-17T05:41:45.124087+0000","last_peered":"2024-04-17T05:41:45.124087+0000","last_clean":"2024-04-17T05:41:45.124087+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:41:45.124087+0000","last_undegraded":"2024-04-17T05:41:45.124087+0000","last_fullsized":"2024-04-17T05:41:45.124087+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:59.716958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'300","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132046+0000","last_change":"2024-04-17T04:36:25.333011+0000","last_active":"2024-04-17T05:41:40.132046+0000","last_peered":"2024-04-17T05:41:40.132046+0000","last_clean":"2024-04-17T05:41:40.132046+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:41:40.132046+0000","last_undegraded":"2024-04-17T05:41:40.132046+0000","last_fullsized":"2024-04-17T05:41:40.132046+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:22.239383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071966,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":686,"num_read_kb":12134,"num_write":1196,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":686,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.132122+0000","last_change":"2024-04-17T04:36:27.388791+0000","last_active":"2024-04-17T05:41:45.132122+0000","last_peered":"2024-04-17T05:41:45.132122+0000","last_clean":"2024-04-17T05:41:45.132122+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:41:45.132122+0000","last_undegraded":"2024-04-17T05:41:45.132122+0000","last_fullsized":"2024-04-17T05:41:45.132122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:21.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'281","reported_seq":917,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:54.078754+0000","last_change":"2024-04-17T04:36:25.373417+0000","last_active":"2024-04-17T05:42:54.078754+0000","last_peered":"2024-04-17T05:42:54.078754+0000","last_clean":"2024-04-17T05:42:54.078754+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:42:54.078754+0000","last_undegraded":"2024-04-17T05:42:54.078754+0000","last_fullsized":"2024-04-17T05:42:54.078754+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:11.729276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30815,"num_objects":79,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":425,"num_read_kb":657,"num_write":1176,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'221","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133347+0000","last_change":"2024-04-17T04:36:27.372140+0000","last_active":"2024-04-17T05:41:45.133347+0000","last_peered":"2024-04-17T05:41:45.133347+0000","last_clean":"2024-04-17T05:41:45.133347+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:41:45.133347+0000","last_undegraded":"2024-04-17T05:41:45.133347+0000","last_fullsized":"2024-04-17T05:41:45.133347+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.338570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'355","reported_seq":1093,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133443+0000","last_change":"2024-04-17T04:36:25.386307+0000","last_active":"2024-04-17T05:41:50.133443+0000","last_peered":"2024-04-17T05:41:50.133443+0000","last_clean":"2024-04-17T05:41:50.133443+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:41:50.133443+0000","last_undegraded":"2024-04-17T05:41:50.133443+0000","last_fullsized":"2024-04-17T05:41:50.133443+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:43.522371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18382422,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":829,"num_read_kb":32478,"num_write":1352,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165668+0000","last_change":"2024-04-17T04:36:27.385349+0000","last_active":"2024-04-17T05:41:45.165668+0000","last_peered":"2024-04-17T05:41:45.165668+0000","last_clean":"2024-04-17T05:41:45.165668+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:41:45.165668+0000","last_undegraded":"2024-04-17T05:41:45.165668+0000","last_fullsized":"2024-04-17T05:41:45.165668+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:27:44.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'264","reported_seq":940,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136676+0000","last_change":"2024-04-17T04:36:25.355339+0000","last_active":"2024-04-17T05:41:45.136676+0000","last_peered":"2024-04-17T05:41:45.136676+0000","last_clean":"2024-04-17T05:41:45.136676+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:41:45.136676+0000","last_undegraded":"2024-04-17T05:41:45.136676+0000","last_fullsized":"2024-04-17T05:41:45.136676+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:47:52.068393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7660588,"num_objects":55,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":476,"num_read_kb":46303,"num_write":1246,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'321","reported_seq":1043,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160308+0000","last_change":"2024-04-17T04:37:03.628711+0000","last_active":"2024-04-17T05:43:50.160308+0000","last_peered":"2024-04-17T05:43:50.160308+0000","last_clean":"2024-04-17T05:43:50.160308+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:43:50.160308+0000","last_undegraded":"2024-04-17T05:43:50.160308+0000","last_fullsized":"2024-04-17T05:43:50.160308+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:45.773023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'262","reported_seq":895,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:25.361750+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:44.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23096680,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":400,"num_read_kb":49597,"num_write":1165,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'238","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.137869+0000","last_change":"2024-04-17T04:36:25.345113+0000","last_active":"2024-04-17T05:41:45.137869+0000","last_peered":"2024-04-17T05:41:45.137869+0000","last_clean":"2024-04-17T05:41:45.137869+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:41:45.137869+0000","last_undegraded":"2024-04-17T05:41:45.137869+0000","last_fullsized":"2024-04-17T05:41:45.137869+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:36.604225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584415,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":354,"num_read_kb":7661,"num_write":1028,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'243","reported_seq":842,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136202+0000","last_change":"2024-04-17T04:36:27.379068+0000","last_active":"2024-04-17T05:41:55.136202+0000","last_peered":"2024-04-17T05:41:55.136202+0000","last_clean":"2024-04-17T05:41:55.136202+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:41:55.136202+0000","last_undegraded":"2024-04-17T05:41:55.136202+0000","last_fullsized":"2024-04-17T05:41:55.136202+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:27.386167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'227","reported_seq":870,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:24.492781+0000","last_change":"2024-04-17T04:36:25.357445+0000","last_active":"2024-04-17T05:42:24.492781+0000","last_peered":"2024-04-17T05:42:24.492781+0000","last_clean":"2024-04-17T05:42:24.492781+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:42:24.492781+0000","last_undegraded":"2024-04-17T05:42:24.492781+0000","last_fullsized":"2024-04-17T05:42:24.492781+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:12.816116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10515588,"num_objects":59,"num_object_clones":0,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":397,"num_read_kb":20933,"num_write":1073,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":819,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138230+0000","last_change":"2024-04-17T04:36:27.378695+0000","last_active":"2024-04-17T05:41:45.138230+0000","last_peered":"2024-04-17T05:41:45.138230+0000","last_clean":"2024-04-17T05:41:45.138230+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:41:45.138230+0000","last_undegraded":"2024-04-17T05:41:45.138230+0000","last_fullsized":"2024-04-17T05:41:45.138230+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:37.636571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1280,"ondisk_log_size":1280,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1676,"ondisk_log_size":1676,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1746118,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":17591,"num_read_kb":33647,"num_write":14851,"num_write_kb":11154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3284992,"data_stored":2735018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2071,"ondisk_log_size":2071,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1642,"num_read_kb":1951,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3948,"ondisk_log_size":3948,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":648130815,"num_objects":4413,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":30775,"num_read_kb":1864049,"num_write":73496,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930250752,"data_stored":930250752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":100760153,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":130,"num_read_kb":221363,"num_write":141,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201523200,"data_stored":201520306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":100760153,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":125,"num_read_kb":210051,"num_write":141,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201523200,"data_stored":201520306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575690,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451628,"kb_used_data":167780,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93920212,"statfs":{"total":96636764160,"available":96174297088,"internally_reserved":0,"allocated":171806720,"data_stored":169745941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":290650570},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575691,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474680,"kb_used_data":168624,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93897160,"statfs":{"total":96636764160,"available":96150691840,"internally_reserved":0,"allocated":172670976,"data_stored":170683181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":313391559},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575691,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314860,"kb_used_data":175460,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94056980,"statfs":{"total":96636764160,"available":96314347520,"internally_reserved":0,"allocated":179671040,"data_stored":177571130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575693,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":262200,"kb_used_data":158832,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94109640,"statfs":{"total":96636764160,"available":96368271360,"internally_reserved":0,"allocated":162643968,"data_stored":160677607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575691,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445376,"kb_used_data":201848,"kb_used_omap":1,"kb_used_meta":243518,"kb_avail":93926464,"statfs":{"total":96636764160,"available":96180699136,"internally_reserved":0,"allocated":206692352,"data_stored":204603759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":249362887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575695,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":277500,"kb_used_data":165876,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94094340,"statfs":{"total":96636764160,"available":96352604160,"internally_reserved":0,"allocated":169857024,"data_stored":167899459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114293189},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575693,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288536,"kb_used_data":149136,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94083304,"statfs":{"total":96636764160,"available":96341303296,"internally_reserved":0,"allocated":152715264,"data_stored":150702994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575691,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425908,"kb_used_data":151852,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93945932,"statfs":{"total":96636764160,"available":96200634368,"internally_reserved":0,"allocated":155496448,"data_stored":153432781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":126107648,"data_stored":126107648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111304704,"data_stored":111304704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93118464,"data_stored":93118464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":122077184,"data_stored":122077184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":120524800,"data_stored":120524800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":121225216,"data_stored":121225216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":118833152,"data_stored":118833152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":117059584,"data_stored":117059584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":352781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":385826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":117807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":472766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":92189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":555856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":256028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":501765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14680064,"data_stored":14680064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25214976,"data_stored":25214257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":40894464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28360704,"data_stored":28359976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21020672,"data_stored":21019944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15777792,"data_stored":15777073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31506432,"data_stored":31505713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46235648,"data_stored":46234201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15777792,"data_stored":15777064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:46:04.352 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:46:04.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:04.718 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:46:04.730 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2353,"stamp":"2024-04-17T05:46:03.280712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":852009202,"num_objects":4832,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":156306,"num_read_kb":2419174,"num_write":142134,"num_write_kb":2463295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13386,"ondisk_log_size":13386,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2940688,"kb_used_data":1339408,"kb_used_omap":12,"kb_used_meta":1601203,"kb_avail":752034032,"statfs":{"total":773094113280,"available":770082848768,"internally_reserved":0,"allocated":1371553792,"data_stored":1355316852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1639632433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008357"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151550+0000","last_change":"2024-04-17T05:07:45.742645+0000","last_active":"2024-04-17T05:42:45.151550+0000","last_peered":"2024-04-17T05:42:45.151550+0000","last_clean":"2024-04-17T05:42:45.151550+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:42:45.151550+0000","last_undegraded":"2024-04-17T05:42:45.151550+0000","last_fullsized":"2024-04-17T05:42:45.151550+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:06.072659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138115+0000","last_change":"2024-04-17T04:51:43.682048+0000","last_active":"2024-04-17T05:41:45.138115+0000","last_peered":"2024-04-17T05:41:45.138115+0000","last_clean":"2024-04-17T05:41:45.138115+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:41:45.138115+0000","last_undegraded":"2024-04-17T05:41:45.138115+0000","last_fullsized":"2024-04-17T05:41:45.138115+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:06.493069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164913+0000","last_change":"2024-04-17T04:51:39.638879+0000","last_active":"2024-04-17T05:41:40.164913+0000","last_peered":"2024-04-17T05:41:40.164913+0000","last_clean":"2024-04-17T05:41:40.164913+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:41:40.164913+0000","last_undegraded":"2024-04-17T05:41:40.164913+0000","last_fullsized":"2024-04-17T05:41:40.164913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:57:53.046398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":148,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139601+0000","last_change":"2024-04-17T04:47:56.968120+0000","last_active":"2024-04-17T05:42:10.139601+0000","last_peered":"2024-04-17T05:42:10.139601+0000","last_clean":"2024-04-17T05:42:10.139601+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:42:10.139601+0000","last_undegraded":"2024-04-17T05:42:10.139601+0000","last_fullsized":"2024-04-17T05:42:10.139601+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:39.089673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149910+0000","last_change":"2024-04-17T05:07:45.746586+0000","last_active":"2024-04-17T05:42:45.149910+0000","last_peered":"2024-04-17T05:42:45.149910+0000","last_clean":"2024-04-17T05:42:45.149910+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:42:45.149910+0000","last_undegraded":"2024-04-17T05:42:45.149910+0000","last_fullsized":"2024-04-17T05:42:45.149910+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:02.041624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'2","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124114+0000","last_change":"2024-04-17T04:51:43.661871+0000","last_active":"2024-04-17T05:41:45.124114+0000","last_peered":"2024-04-17T05:41:45.124114+0000","last_clean":"2024-04-17T05:41:45.124114+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:41:45.124114+0000","last_undegraded":"2024-04-17T05:41:45.124114+0000","last_fullsized":"2024-04-17T05:41:45.124114+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:46.906315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.137511+0000","last_change":"2024-04-17T04:51:39.640822+0000","last_active":"2024-04-17T05:41:40.137511+0000","last_peered":"2024-04-17T05:41:40.137511+0000","last_clean":"2024-04-17T05:41:40.137511+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:41:40.137511+0000","last_undegraded":"2024-04-17T05:41:40.137511+0000","last_fullsized":"2024-04-17T05:41:40.137511+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:51.213156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'96","reported_seq":350,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.143271+0000","last_change":"2024-04-17T04:47:56.937898+0000","last_active":"2024-04-17T05:42:15.143271+0000","last_peered":"2024-04-17T05:42:15.143271+0000","last_clean":"2024-04-17T05:42:15.143271+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:42:15.143271+0000","last_undegraded":"2024-04-17T05:42:15.143271+0000","last_fullsized":"2024-04-17T05:42:15.143271+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:02:44.307495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130858+0000","last_change":"2024-04-17T05:07:45.739412+0000","last_active":"2024-04-17T05:41:35.130858+0000","last_peered":"2024-04-17T05:41:35.130858+0000","last_clean":"2024-04-17T05:41:35.130858+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:41:35.130858+0000","last_undegraded":"2024-04-17T05:41:35.130858+0000","last_fullsized":"2024-04-17T05:41:35.130858+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:26.784407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162336+0000","last_change":"2024-04-17T04:51:43.682620+0000","last_active":"2024-04-17T05:43:50.162336+0000","last_peered":"2024-04-17T05:43:50.162336+0000","last_clean":"2024-04-17T05:43:50.162336+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:43:50.162336+0000","last_undegraded":"2024-04-17T05:43:50.162336+0000","last_fullsized":"2024-04-17T05:43:50.162336+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:29.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'2","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.164674+0000","last_change":"2024-04-17T04:51:39.605291+0000","last_active":"2024-04-17T05:43:50.164674+0000","last_peered":"2024-04-17T05:43:50.164674+0000","last_clean":"2024-04-17T05:43:50.164674+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:43:50.164674+0000","last_undegraded":"2024-04-17T05:43:50.164674+0000","last_fullsized":"2024-04-17T05:43:50.164674+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:40.774979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":121,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.150706+0000","last_change":"2024-04-17T04:47:56.953281+0000","last_active":"2024-04-17T05:43:00.150706+0000","last_peered":"2024-04-17T05:43:00.150706+0000","last_clean":"2024-04-17T05:43:00.150706+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:43:00.150706+0000","last_undegraded":"2024-04-17T05:43:00.150706+0000","last_fullsized":"2024-04-17T05:43:00.150706+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:02.682710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164774+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:41:40.164774+0000","last_peered":"2024-04-17T05:41:40.164774+0000","last_clean":"2024-04-17T05:41:40.164774+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:41:40.164774+0000","last_undegraded":"2024-04-17T05:41:40.164774+0000","last_fullsized":"2024-04-17T05:41:40.164774+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:05.623921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.195511+0000","last_change":"2024-04-17T04:51:43.676613+0000","last_active":"2024-04-17T05:43:55.195511+0000","last_peered":"2024-04-17T05:43:55.195511+0000","last_clean":"2024-04-17T05:43:55.195511+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:43:55.195511+0000","last_undegraded":"2024-04-17T05:43:55.195511+0000","last_fullsized":"2024-04-17T05:43:55.195511+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:26.269622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946142+0000","last_change":"2024-04-17T04:51:39.615233+0000","last_active":"2024-04-17T05:43:44.946142+0000","last_peered":"2024-04-17T05:43:44.946142+0000","last_clean":"2024-04-17T05:43:44.946142+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:43:44.946142+0000","last_undegraded":"2024-04-17T05:43:44.946142+0000","last_fullsized":"2024-04-17T05:43:44.946142+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:02.139603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":160,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183795+0000","last_change":"2024-04-17T04:47:56.969273+0000","last_active":"2024-04-17T05:43:00.183795+0000","last_peered":"2024-04-17T05:43:00.183795+0000","last_clean":"2024-04-17T05:43:00.183795+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:43:00.183795+0000","last_undegraded":"2024-04-17T05:43:00.183795+0000","last_fullsized":"2024-04-17T05:43:00.183795+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:24.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.194209+0000","last_change":"2024-04-17T05:07:45.745164+0000","last_active":"2024-04-17T05:43:50.194209+0000","last_peered":"2024-04-17T05:43:50.194209+0000","last_clean":"2024-04-17T05:43:50.194209+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:43:50.194209+0000","last_undegraded":"2024-04-17T05:43:50.194209+0000","last_fullsized":"2024-04-17T05:43:50.194209+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:44.184077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'1","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150275+0000","last_change":"2024-04-17T04:51:43.666389+0000","last_active":"2024-04-17T05:42:40.150275+0000","last_peered":"2024-04-17T05:42:40.150275+0000","last_clean":"2024-04-17T05:42:40.150275+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:42:40.150275+0000","last_undegraded":"2024-04-17T05:42:40.150275+0000","last_fullsized":"2024-04-17T05:42:40.150275+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:10.966058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.142876+0000","last_change":"2024-04-17T04:51:39.625539+0000","last_active":"2024-04-17T05:42:25.142876+0000","last_peered":"2024-04-17T05:42:25.142876+0000","last_clean":"2024-04-17T05:42:25.142876+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:42:25.142876+0000","last_undegraded":"2024-04-17T05:42:25.142876+0000","last_fullsized":"2024-04-17T05:42:25.142876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:55.423515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":200,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183684+0000","last_change":"2024-04-17T04:47:56.967026+0000","last_active":"2024-04-17T05:43:00.183684+0000","last_peered":"2024-04-17T05:43:00.183684+0000","last_clean":"2024-04-17T05:43:00.183684+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:43:00.183684+0000","last_undegraded":"2024-04-17T05:43:00.183684+0000","last_fullsized":"2024-04-17T05:43:00.183684+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:26.424709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158284+0000","last_change":"2024-04-17T05:07:45.754178+0000","last_active":"2024-04-17T05:43:10.158284+0000","last_peered":"2024-04-17T05:43:10.158284+0000","last_clean":"2024-04-17T05:43:10.158284+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:43:10.158284+0000","last_undegraded":"2024-04-17T05:43:10.158284+0000","last_fullsized":"2024-04-17T05:43:10.158284+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:49.281995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'4","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142841+0000","last_change":"2024-04-17T04:51:43.664767+0000","last_active":"2024-04-17T05:42:30.142841+0000","last_peered":"2024-04-17T05:42:30.142841+0000","last_clean":"2024-04-17T05:42:30.142841+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:42:30.142841+0000","last_undegraded":"2024-04-17T05:42:30.142841+0000","last_fullsized":"2024-04-17T05:42:30.142841+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:36.530590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":22528,"num_write":9,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"261'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.134896+0000","last_change":"2024-04-17T04:51:39.611306+0000","last_active":"2024-04-17T05:42:35.134896+0000","last_peered":"2024-04-17T05:42:35.134896+0000","last_clean":"2024-04-17T05:42:35.134896+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:42:35.134896+0000","last_undegraded":"2024-04-17T05:42:35.134896+0000","last_fullsized":"2024-04-17T05:42:35.134896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:51.035427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'74","reported_seq":283,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149482+0000","last_change":"2024-04-17T04:47:56.952081+0000","last_active":"2024-04-17T05:43:00.149482+0000","last_peered":"2024-04-17T05:43:00.149482+0000","last_clean":"2024-04-17T05:43:00.149482+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:43:00.149482+0000","last_undegraded":"2024-04-17T05:43:00.149482+0000","last_fullsized":"2024-04-17T05:43:00.149482+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:08:06.903329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145686+0000","last_change":"2024-04-17T05:07:45.754885+0000","last_active":"2024-04-17T05:42:45.145686+0000","last_peered":"2024-04-17T05:42:45.145686+0000","last_clean":"2024-04-17T05:42:45.145686+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:42:45.145686+0000","last_undegraded":"2024-04-17T05:42:45.145686+0000","last_fullsized":"2024-04-17T05:42:45.145686+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:02.913191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133462+0000","last_change":"2024-04-17T04:51:43.664884+0000","last_active":"2024-04-17T05:41:45.133462+0000","last_peered":"2024-04-17T05:41:45.133462+0000","last_clean":"2024-04-17T05:41:45.133462+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:41:45.133462+0000","last_undegraded":"2024-04-17T05:41:45.133462+0000","last_fullsized":"2024-04-17T05:41:45.133462+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:04.163089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'8","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122767+0000","last_change":"2024-04-17T04:51:39.621614+0000","last_active":"2024-04-17T05:41:40.122767+0000","last_peered":"2024-04-17T05:41:40.122767+0000","last_clean":"2024-04-17T05:41:40.122767+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:41:40.122767+0000","last_undegraded":"2024-04-17T05:41:40.122767+0000","last_fullsized":"2024-04-17T05:41:40.122767+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:01.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":23,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":74,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:20.127064+0000","last_change":"2024-04-17T04:47:56.955568+0000","last_active":"2024-04-17T05:41:20.127064+0000","last_peered":"2024-04-17T05:41:20.127064+0000","last_clean":"2024-04-17T05:41:20.127064+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:41:20.127064+0000","last_undegraded":"2024-04-17T05:41:20.127064+0000","last_fullsized":"2024-04-17T05:41:20.127064+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:09:24.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145754+0000","last_change":"2024-04-17T05:07:45.757519+0000","last_active":"2024-04-17T05:42:45.145754+0000","last_peered":"2024-04-17T05:42:45.145754+0000","last_clean":"2024-04-17T05:42:45.145754+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:42:45.145754+0000","last_undegraded":"2024-04-17T05:42:45.145754+0000","last_fullsized":"2024-04-17T05:42:45.145754+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:27.671557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165700+0000","last_change":"2024-04-17T04:51:43.686621+0000","last_active":"2024-04-17T05:41:45.165700+0000","last_peered":"2024-04-17T05:41:45.165700+0000","last_clean":"2024-04-17T05:41:45.165700+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:41:45.165700+0000","last_undegraded":"2024-04-17T05:41:45.165700+0000","last_fullsized":"2024-04-17T05:41:45.165700+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:36.829860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132003+0000","last_change":"2024-04-17T04:51:39.643524+0000","last_active":"2024-04-17T05:41:40.132003+0000","last_peered":"2024-04-17T05:41:40.132003+0000","last_clean":"2024-04-17T05:41:40.132003+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:41:40.132003+0000","last_undegraded":"2024-04-17T05:41:40.132003+0000","last_fullsized":"2024-04-17T05:41:40.132003+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:01.434777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":90,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163169+0000","last_change":"2024-04-17T04:47:56.932780+0000","last_active":"2024-04-17T05:43:35.163169+0000","last_peered":"2024-04-17T05:43:35.163169+0000","last_clean":"2024-04-17T05:43:35.163169+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:43:35.163169+0000","last_undegraded":"2024-04-17T05:43:35.163169+0000","last_fullsized":"2024-04-17T05:43:35.163169+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:07.644614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151106+0000","last_change":"2024-04-17T05:07:45.743105+0000","last_active":"2024-04-17T05:42:45.151106+0000","last_peered":"2024-04-17T05:42:45.151106+0000","last_clean":"2024-04-17T05:42:45.151106+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:42:45.151106+0000","last_undegraded":"2024-04-17T05:42:45.151106+0000","last_fullsized":"2024-04-17T05:42:45.151106+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:30.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176828+0000","last_change":"2024-04-17T04:51:43.664814+0000","last_active":"2024-04-17T05:42:30.176828+0000","last_peered":"2024-04-17T05:42:30.176828+0000","last_clean":"2024-04-17T05:42:30.176828+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:42:30.176828+0000","last_undegraded":"2024-04-17T05:42:30.176828+0000","last_fullsized":"2024-04-17T05:42:30.176828+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:39.897527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157053+0000","last_change":"2024-04-17T04:51:39.626132+0000","last_active":"2024-04-17T05:43:10.157053+0000","last_peered":"2024-04-17T05:43:10.157053+0000","last_clean":"2024-04-17T05:43:10.157053+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:43:10.157053+0000","last_undegraded":"2024-04-17T05:43:10.157053+0000","last_fullsized":"2024-04-17T05:43:10.157053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:23.450299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":80,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158447+0000","last_change":"2024-04-17T04:47:56.956719+0000","last_active":"2024-04-17T05:43:35.158447+0000","last_peered":"2024-04-17T05:43:35.158447+0000","last_clean":"2024-04-17T05:43:35.158447+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:43:35.158447+0000","last_undegraded":"2024-04-17T05:43:35.158447+0000","last_fullsized":"2024-04-17T05:43:35.158447+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:10.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149849+0000","last_change":"2024-04-17T05:07:45.738001+0000","last_active":"2024-04-17T05:42:45.149849+0000","last_peered":"2024-04-17T05:42:45.149849+0000","last_clean":"2024-04-17T05:42:45.149849+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:42:45.149849+0000","last_undegraded":"2024-04-17T05:42:45.149849+0000","last_fullsized":"2024-04-17T05:42:45.149849+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.048242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187309+0000","last_change":"2024-04-17T04:51:43.682839+0000","last_active":"2024-04-17T05:43:15.187309+0000","last_peered":"2024-04-17T05:43:15.187309+0000","last_clean":"2024-04-17T05:43:15.187309+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:43:15.187309+0000","last_undegraded":"2024-04-17T05:43:15.187309+0000","last_fullsized":"2024-04-17T05:43:15.187309+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:32.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'1","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141354+0000","last_change":"2024-04-17T04:51:39.639685+0000","last_active":"2024-04-17T05:42:25.141354+0000","last_peered":"2024-04-17T05:42:25.141354+0000","last_clean":"2024-04-17T05:42:25.141354+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:42:25.141354+0000","last_undegraded":"2024-04-17T05:42:25.141354+0000","last_fullsized":"2024-04-17T05:42:25.141354+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:41.954023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191081+0000","last_change":"2024-04-17T04:47:56.966803+0000","last_active":"2024-04-17T05:43:35.191081+0000","last_peered":"2024-04-17T05:43:35.191081+0000","last_clean":"2024-04-17T05:43:35.191081+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:43:35.191081+0000","last_undegraded":"2024-04-17T05:43:35.191081+0000","last_fullsized":"2024-04-17T05:43:35.191081+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:27.144075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145538+0000","last_change":"2024-04-17T05:07:45.747162+0000","last_active":"2024-04-17T05:42:45.145538+0000","last_peered":"2024-04-17T05:42:45.145538+0000","last_clean":"2024-04-17T05:42:45.145538+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:42:45.145538+0000","last_undegraded":"2024-04-17T05:42:45.145538+0000","last_fullsized":"2024-04-17T05:42:45.145538+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:25.853530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'3","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176921+0000","last_change":"2024-04-17T04:51:43.671647+0000","last_active":"2024-04-17T05:42:30.176921+0000","last_peered":"2024-04-17T05:42:30.176921+0000","last_clean":"2024-04-17T05:42:30.176921+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:42:30.176921+0000","last_undegraded":"2024-04-17T05:42:30.176921+0000","last_fullsized":"2024-04-17T05:42:30.176921+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:09:37.852227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5291313,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":8,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'10","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145913+0000","last_change":"2024-04-17T04:51:39.612849+0000","last_active":"2024-04-17T05:42:25.145913+0000","last_peered":"2024-04-17T05:42:25.145913+0000","last_clean":"2024-04-17T05:42:25.145913+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:42:25.145913+0000","last_undegraded":"2024-04-17T05:42:25.145913+0000","last_fullsized":"2024-04-17T05:42:25.145913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:41.680996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":46082,"num_write":14,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.161351+0000","last_change":"2024-04-17T04:47:56.937214+0000","last_active":"2024-04-17T05:43:35.161351+0000","last_peered":"2024-04-17T05:43:35.161351+0000","last_clean":"2024-04-17T05:43:35.161351+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:43:35.161351+0000","last_undegraded":"2024-04-17T05:43:35.161351+0000","last_fullsized":"2024-04-17T05:43:35.161351+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:02.809448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151449+0000","last_change":"2024-04-17T05:07:45.744163+0000","last_active":"2024-04-17T05:42:45.151449+0000","last_peered":"2024-04-17T05:42:45.151449+0000","last_clean":"2024-04-17T05:42:45.151449+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:42:45.151449+0000","last_undegraded":"2024-04-17T05:42:45.151449+0000","last_fullsized":"2024-04-17T05:42:45.151449+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:42.169974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'10","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.133958+0000","last_change":"2024-04-17T04:51:43.667351+0000","last_active":"2024-04-17T05:42:30.133958+0000","last_peered":"2024-04-17T05:42:30.133958+0000","last_clean":"2024-04-17T05:42:30.133958+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:42:30.133958+0000","last_undegraded":"2024-04-17T05:42:30.133958+0000","last_fullsized":"2024-04-17T05:42:30.133958+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:27.804721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":37890,"num_write":19,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143121+0000","last_change":"2024-04-17T04:51:39.639400+0000","last_active":"2024-04-17T05:42:25.143121+0000","last_peered":"2024-04-17T05:42:25.143121+0000","last_clean":"2024-04-17T05:42:25.143121+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:42:25.143121+0000","last_undegraded":"2024-04-17T05:42:25.143121+0000","last_fullsized":"2024-04-17T05:42:25.143121+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:39.092327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":211,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133386+0000","last_change":"2024-04-17T04:47:56.964248+0000","last_active":"2024-04-17T05:41:50.133386+0000","last_peered":"2024-04-17T05:41:50.133386+0000","last_clean":"2024-04-17T05:41:50.133386+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:41:50.133386+0000","last_undegraded":"2024-04-17T05:41:50.133386+0000","last_fullsized":"2024-04-17T05:41:50.133386+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:19.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163617+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:41:35.163617+0000","last_peered":"2024-04-17T05:41:35.163617+0000","last_clean":"2024-04-17T05:41:35.163617+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:41:35.163617+0000","last_undegraded":"2024-04-17T05:41:35.163617+0000","last_fullsized":"2024-04-17T05:41:35.163617+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:45.333124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'5","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157158+0000","last_change":"2024-04-17T04:51:43.670407+0000","last_active":"2024-04-17T05:43:05.157158+0000","last_peered":"2024-04-17T05:43:05.157158+0000","last_clean":"2024-04-17T05:43:05.157158+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:43:05.157158+0000","last_undegraded":"2024-04-17T05:43:05.157158+0000","last_fullsized":"2024-04-17T05:43:05.157158+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:31.623818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:59.936150+0000","last_change":"2024-04-17T04:51:39.613038+0000","last_active":"2024-04-17T05:42:59.936150+0000","last_peered":"2024-04-17T05:42:59.936150+0000","last_clean":"2024-04-17T05:42:59.936150+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:42:59.936150+0000","last_undegraded":"2024-04-17T05:42:59.936150+0000","last_fullsized":"2024-04-17T05:42:59.936150+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:51.929471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.130770+0000","last_change":"2024-04-17T04:47:56.962330+0000","last_active":"2024-04-17T05:41:40.130770+0000","last_peered":"2024-04-17T05:41:40.130770+0000","last_clean":"2024-04-17T05:41:40.130770+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:41:40.130770+0000","last_undegraded":"2024-04-17T05:41:40.130770+0000","last_fullsized":"2024-04-17T05:41:40.130770+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:38.049482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130958+0000","last_change":"2024-04-17T05:07:45.749757+0000","last_active":"2024-04-17T05:41:35.130958+0000","last_peered":"2024-04-17T05:41:35.130958+0000","last_clean":"2024-04-17T05:41:35.130958+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:41:35.130958+0000","last_undegraded":"2024-04-17T05:41:35.130958+0000","last_fullsized":"2024-04-17T05:41:35.130958+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:50:18.794745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'1","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157104+0000","last_change":"2024-04-17T04:51:43.650369+0000","last_active":"2024-04-17T05:43:05.157104+0000","last_peered":"2024-04-17T05:43:05.157104+0000","last_clean":"2024-04-17T05:43:05.157104+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:43:05.157104+0000","last_undegraded":"2024-04-17T05:43:05.157104+0000","last_fullsized":"2024-04-17T05:43:05.157104+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:16:08.661428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183604+0000","last_change":"2024-04-17T04:51:39.631783+0000","last_active":"2024-04-17T05:43:00.183604+0000","last_peered":"2024-04-17T05:43:00.183604+0000","last_clean":"2024-04-17T05:43:00.183604+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:43:00.183604+0000","last_undegraded":"2024-04-17T05:43:00.183604+0000","last_fullsized":"2024-04-17T05:43:00.183604+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:32.873589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":132,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.131963+0000","last_change":"2024-04-17T04:47:56.967539+0000","last_active":"2024-04-17T05:41:40.131963+0000","last_peered":"2024-04-17T05:41:40.131963+0000","last_clean":"2024-04-17T05:41:40.131963+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:41:40.131963+0000","last_undegraded":"2024-04-17T05:41:40.131963+0000","last_fullsized":"2024-04-17T05:41:40.131963+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:50.899642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"263'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.149607+0000","last_change":"2024-04-17T05:07:45.748765+0000","last_active":"2024-04-17T05:42:55.149607+0000","last_peered":"2024-04-17T05:42:55.149607+0000","last_clean":"2024-04-17T05:42:55.149607+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:42:55.149607+0000","last_undegraded":"2024-04-17T05:42:55.149607+0000","last_fullsized":"2024-04-17T05:42:55.149607+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:21.732472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'2","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.184978+0000","last_change":"2024-04-17T04:51:43.684076+0000","last_active":"2024-04-17T05:43:05.184978+0000","last_peered":"2024-04-17T05:43:05.184978+0000","last_clean":"2024-04-17T05:43:05.184978+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:43:05.184978+0000","last_undegraded":"2024-04-17T05:43:05.184978+0000","last_fullsized":"2024-04-17T05:43:05.184978+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.261686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":10240,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149546+0000","last_change":"2024-04-17T04:51:39.626930+0000","last_active":"2024-04-17T05:43:00.149546+0000","last_peered":"2024-04-17T05:43:00.149546+0000","last_clean":"2024-04-17T05:43:00.149546+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:43:00.149546+0000","last_undegraded":"2024-04-17T05:43:00.149546+0000","last_fullsized":"2024-04-17T05:43:00.149546+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:24:11.581902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122667+0000","last_change":"2024-04-17T04:47:56.946016+0000","last_active":"2024-04-17T05:41:40.122667+0000","last_peered":"2024-04-17T05:41:40.122667+0000","last_clean":"2024-04-17T05:41:40.122667+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:41:40.122667+0000","last_undegraded":"2024-04-17T05:41:40.122667+0000","last_fullsized":"2024-04-17T05:41:40.122667+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:39.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163578+0000","last_change":"2024-04-17T05:07:45.741199+0000","last_active":"2024-04-17T05:41:35.163578+0000","last_peered":"2024-04-17T05:41:35.163578+0000","last_clean":"2024-04-17T05:41:35.163578+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:41:35.163578+0000","last_undegraded":"2024-04-17T05:41:35.163578+0000","last_fullsized":"2024-04-17T05:41:35.163578+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:14:05.831006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150592+0000","last_change":"2024-04-17T04:51:43.673701+0000","last_active":"2024-04-17T05:43:05.150592+0000","last_peered":"2024-04-17T05:43:05.150592+0000","last_clean":"2024-04-17T05:43:05.150592+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:43:05.150592+0000","last_undegraded":"2024-04-17T05:43:05.150592+0000","last_fullsized":"2024-04-17T05:43:05.150592+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:00.395599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154845+0000","last_change":"2024-04-17T04:51:39.625916+0000","last_active":"2024-04-17T05:43:00.154845+0000","last_peered":"2024-04-17T05:43:00.154845+0000","last_clean":"2024-04-17T05:43:00.154845+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:43:00.154845+0000","last_undegraded":"2024-04-17T05:43:00.154845+0000","last_fullsized":"2024-04-17T05:43:00.154845+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:35:16.145280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":195,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164555+0000","last_change":"2024-04-17T04:47:56.960822+0000","last_active":"2024-04-17T05:41:40.164555+0000","last_peered":"2024-04-17T05:41:40.164555+0000","last_clean":"2024-04-17T05:41:40.164555+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:41:40.164555+0000","last_undegraded":"2024-04-17T05:41:40.164555+0000","last_fullsized":"2024-04-17T05:41:40.164555+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:26.089711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.147879+0000","last_change":"2024-04-17T05:07:45.745113+0000","last_active":"2024-04-17T05:42:35.147879+0000","last_peered":"2024-04-17T05:42:35.147879+0000","last_clean":"2024-04-17T05:42:35.147879+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:42:35.147879+0000","last_undegraded":"2024-04-17T05:42:35.147879+0000","last_fullsized":"2024-04-17T05:42:35.147879+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:03.124310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'4","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.190008+0000","last_change":"2024-04-17T04:51:43.682734+0000","last_active":"2024-04-17T05:43:30.190008+0000","last_peered":"2024-04-17T05:43:30.190008+0000","last_clean":"2024-04-17T05:43:30.190008+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:43:30.190008+0000","last_undegraded":"2024-04-17T05:43:30.190008+0000","last_fullsized":"2024-04-17T05:43:30.190008+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:32.531568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":9,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154462+0000","last_change":"2024-04-17T04:51:39.645602+0000","last_active":"2024-04-17T05:43:25.154462+0000","last_peered":"2024-04-17T05:43:25.154462+0000","last_clean":"2024-04-17T05:43:25.154462+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:43:25.154462+0000","last_undegraded":"2024-04-17T05:43:25.154462+0000","last_fullsized":"2024-04-17T05:43:25.154462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:02.515138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.173510+0000","last_change":"2024-04-17T04:47:56.962048+0000","last_active":"2024-04-17T05:42:15.173510+0000","last_peered":"2024-04-17T05:42:15.173510+0000","last_clean":"2024-04-17T05:42:15.173510+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:42:15.173510+0000","last_undegraded":"2024-04-17T05:42:15.173510+0000","last_fullsized":"2024-04-17T05:42:15.173510+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:15.157938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159170+0000","last_change":"2024-04-17T05:07:45.751549+0000","last_active":"2024-04-17T05:43:45.159170+0000","last_peered":"2024-04-17T05:43:45.159170+0000","last_clean":"2024-04-17T05:43:45.159170+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:43:45.159170+0000","last_undegraded":"2024-04-17T05:43:45.159170+0000","last_fullsized":"2024-04-17T05:43:45.159170+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:47.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.189964+0000","last_change":"2024-04-17T04:51:43.688891+0000","last_active":"2024-04-17T05:43:30.189964+0000","last_peered":"2024-04-17T05:43:30.189964+0000","last_clean":"2024-04-17T05:43:30.189964+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:43:30.189964+0000","last_undegraded":"2024-04-17T05:43:30.189964+0000","last_fullsized":"2024-04-17T05:43:30.189964+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:35.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":9216,"num_write":3,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'4","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.159519+0000","last_change":"2024-04-17T04:51:39.634707+0000","last_active":"2024-04-17T05:43:25.159519+0000","last_peered":"2024-04-17T05:43:25.159519+0000","last_clean":"2024-04-17T05:43:25.159519+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:43:25.159519+0000","last_undegraded":"2024-04-17T05:43:25.159519+0000","last_fullsized":"2024-04-17T05:43:25.159519+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:09.043206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":11264,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141762+0000","last_change":"2024-04-17T04:47:56.954332+0000","last_active":"2024-04-17T05:42:00.141762+0000","last_peered":"2024-04-17T05:42:00.141762+0000","last_clean":"2024-04-17T05:42:00.141762+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:42:00.141762+0000","last_undegraded":"2024-04-17T05:42:00.141762+0000","last_fullsized":"2024-04-17T05:42:00.141762+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:07.662187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141842+0000","last_change":"2024-04-17T05:07:45.741143+0000","last_active":"2024-04-17T05:42:25.141842+0000","last_peered":"2024-04-17T05:42:25.141842+0000","last_clean":"2024-04-17T05:42:25.141842+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:42:25.141842+0000","last_undegraded":"2024-04-17T05:42:25.141842+0000","last_fullsized":"2024-04-17T05:42:25.141842+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:50.193462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164848+0000","last_change":"2024-04-17T04:51:43.657675+0000","last_active":"2024-04-17T05:43:40.164848+0000","last_peered":"2024-04-17T05:43:40.164848+0000","last_clean":"2024-04-17T05:43:40.164848+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:43:40.164848+0000","last_undegraded":"2024-04-17T05:43:40.164848+0000","last_fullsized":"2024-04-17T05:43:40.164848+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:59:28.152355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'2","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.156097+0000","last_change":"2024-04-17T04:51:39.634454+0000","last_active":"2024-04-17T05:43:25.156097+0000","last_peered":"2024-04-17T05:43:25.156097+0000","last_clean":"2024-04-17T05:43:25.156097+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:43:25.156097+0000","last_undegraded":"2024-04-17T05:43:25.156097+0000","last_fullsized":"2024-04-17T05:43:25.156097+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:48.747740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":123,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136769+0000","last_change":"2024-04-17T04:47:56.963915+0000","last_active":"2024-04-17T05:42:00.136769+0000","last_peered":"2024-04-17T05:42:00.136769+0000","last_clean":"2024-04-17T05:42:00.136769+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:42:00.136769+0000","last_undegraded":"2024-04-17T05:42:00.136769+0000","last_fullsized":"2024-04-17T05:42:00.136769+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:58.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141220+0000","last_change":"2024-04-17T05:07:45.757127+0000","last_active":"2024-04-17T05:42:25.141220+0000","last_peered":"2024-04-17T05:42:25.141220+0000","last_clean":"2024-04-17T05:42:25.141220+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:42:25.141220+0000","last_undegraded":"2024-04-17T05:42:25.141220+0000","last_fullsized":"2024-04-17T05:42:25.141220+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:42.585132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130813+0000","last_change":"2024-04-17T04:52:04.726201+0000","last_active":"2024-04-17T05:41:35.130813+0000","last_peered":"2024-04-17T05:41:35.130813+0000","last_clean":"2024-04-17T05:41:35.130813+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:41:35.130813+0000","last_undegraded":"2024-04-17T05:41:35.130813+0000","last_fullsized":"2024-04-17T05:41:35.130813+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:14:03.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163394+0000","last_change":"2024-04-17T04:51:39.618531+0000","last_active":"2024-04-17T05:43:35.163394+0000","last_peered":"2024-04-17T05:43:35.163394+0000","last_clean":"2024-04-17T05:43:35.163394+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:43:35.163394+0000","last_undegraded":"2024-04-17T05:43:35.163394+0000","last_fullsized":"2024-04-17T05:43:35.163394+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:52.911539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":128,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139900+0000","last_change":"2024-04-17T04:47:56.962022+0000","last_active":"2024-04-17T05:42:00.139900+0000","last_peered":"2024-04-17T05:42:00.139900+0000","last_clean":"2024-04-17T05:42:00.139900+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:42:00.139900+0000","last_undegraded":"2024-04-17T05:42:00.139900+0000","last_fullsized":"2024-04-17T05:42:00.139900+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:04.015930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193091+0000","last_change":"2024-04-17T05:07:45.744264+0000","last_active":"2024-04-17T05:43:45.193091+0000","last_peered":"2024-04-17T05:43:45.193091+0000","last_clean":"2024-04-17T05:43:45.193091+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:43:45.193091+0000","last_undegraded":"2024-04-17T05:43:45.193091+0000","last_fullsized":"2024-04-17T05:43:45.193091+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:41.404295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'6","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.945897+0000","last_change":"2024-04-17T04:51:43.661076+0000","last_active":"2024-04-17T05:43:44.945897+0000","last_peered":"2024-04-17T05:43:44.945897+0000","last_clean":"2024-04-17T05:43:44.945897+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:43:44.945897+0000","last_undegraded":"2024-04-17T05:43:44.945897+0000","last_fullsized":"2024-04-17T05:43:44.945897+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:54.620750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48424,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":5216,"num_write":11,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159409+0000","last_change":"2024-04-17T04:51:39.633118+0000","last_active":"2024-04-17T05:43:40.159409+0000","last_peered":"2024-04-17T05:43:40.159409+0000","last_clean":"2024-04-17T05:43:40.159409+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:43:40.159409+0000","last_undegraded":"2024-04-17T05:43:40.159409+0000","last_fullsized":"2024-04-17T05:43:40.159409+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:25:34.341448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":102,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137165+0000","last_change":"2024-04-17T04:47:56.963262+0000","last_active":"2024-04-17T05:42:00.137165+0000","last_peered":"2024-04-17T05:42:00.137165+0000","last_clean":"2024-04-17T05:42:00.137165+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:42:00.137165+0000","last_undegraded":"2024-04-17T05:42:00.137165+0000","last_fullsized":"2024-04-17T05:42:00.137165+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:09.975510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.129755+0000","last_change":"2024-04-17T05:07:45.757192+0000","last_active":"2024-04-17T05:41:35.129755+0000","last_peered":"2024-04-17T05:41:35.129755+0000","last_clean":"2024-04-17T05:41:35.129755+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:41:35.129755+0000","last_undegraded":"2024-04-17T05:41:35.129755+0000","last_fullsized":"2024-04-17T05:41:35.129755+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:30:19.313475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'4","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.158953+0000","last_change":"2024-04-17T04:51:43.675078+0000","last_active":"2024-04-17T05:43:45.158953+0000","last_peered":"2024-04-17T05:43:45.158953+0000","last_clean":"2024-04-17T05:43:45.158953+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:43:45.158953+0000","last_undegraded":"2024-04-17T05:43:45.158953+0000","last_fullsized":"2024-04-17T05:43:45.158953+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:00.172403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164451+0000","last_change":"2024-04-17T04:51:39.618066+0000","last_active":"2024-04-17T05:43:40.164451+0000","last_peered":"2024-04-17T05:43:40.164451+0000","last_clean":"2024-04-17T05:43:40.164451+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:43:40.164451+0000","last_undegraded":"2024-04-17T05:43:40.164451+0000","last_fullsized":"2024-04-17T05:43:40.164451+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:51.504887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":286,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140125+0000","last_change":"2024-04-17T04:47:56.963049+0000","last_active":"2024-04-17T05:42:20.140125+0000","last_peered":"2024-04-17T05:42:20.140125+0000","last_clean":"2024-04-17T05:42:20.140125+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:42:20.140125+0000","last_undegraded":"2024-04-17T05:42:20.140125+0000","last_fullsized":"2024-04-17T05:42:20.140125+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:08.912141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159213+0000","last_change":"2024-04-17T05:07:45.754827+0000","last_active":"2024-04-17T05:43:45.159213+0000","last_peered":"2024-04-17T05:43:45.159213+0000","last_clean":"2024-04-17T05:43:45.159213+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:43:45.159213+0000","last_undegraded":"2024-04-17T05:43:45.159213+0000","last_fullsized":"2024-04-17T05:43:45.159213+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:12.463675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.162241+0000","last_change":"2024-04-17T04:51:43.655115+0000","last_active":"2024-04-17T05:43:30.162241+0000","last_peered":"2024-04-17T05:43:30.162241+0000","last_clean":"2024-04-17T05:43:30.162241+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:43:30.162241+0000","last_undegraded":"2024-04-17T05:43:30.162241+0000","last_fullsized":"2024-04-17T05:43:30.162241+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:20.469117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162125+0000","last_change":"2024-04-17T04:51:39.633266+0000","last_active":"2024-04-17T05:43:50.162125+0000","last_peered":"2024-04-17T05:43:50.162125+0000","last_clean":"2024-04-17T05:43:50.162125+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:43:50.162125+0000","last_undegraded":"2024-04-17T05:43:50.162125+0000","last_fullsized":"2024-04-17T05:43:50.162125+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:40.414020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.171752+0000","last_change":"2024-04-17T04:47:56.952507+0000","last_active":"2024-04-17T05:42:10.171752+0000","last_peered":"2024-04-17T05:42:10.171752+0000","last_clean":"2024-04-17T05:42:10.171752+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:42:10.171752+0000","last_undegraded":"2024-04-17T05:42:10.171752+0000","last_fullsized":"2024-04-17T05:42:10.171752+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:18.535221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.160987+0000","last_change":"2024-04-17T05:07:45.739696+0000","last_active":"2024-04-17T05:43:45.160987+0000","last_peered":"2024-04-17T05:43:45.160987+0000","last_clean":"2024-04-17T05:43:45.160987+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:43:45.160987+0000","last_undegraded":"2024-04-17T05:43:45.160987+0000","last_fullsized":"2024-04-17T05:43:45.160987+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:02:31.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.131990+0000","last_change":"2024-04-17T04:51:43.675773+0000","last_active":"2024-04-17T05:41:45.131990+0000","last_peered":"2024-04-17T05:41:45.131990+0000","last_clean":"2024-04-17T05:41:45.131990+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:41:45.131990+0000","last_undegraded":"2024-04-17T05:41:45.131990+0000","last_fullsized":"2024-04-17T05:41:45.131990+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:07.015940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:24.941710+0000","last_change":"2024-04-17T04:51:39.621153+0000","last_active":"2024-04-17T05:43:24.941710+0000","last_peered":"2024-04-17T05:43:24.941710+0000","last_clean":"2024-04-17T05:43:24.941710+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:43:24.941710+0000","last_undegraded":"2024-04-17T05:43:24.941710+0000","last_fullsized":"2024-04-17T05:43:24.941710+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:36.053793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":188,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138665+0000","last_change":"2024-04-17T04:47:56.966581+0000","last_active":"2024-04-17T05:42:10.138665+0000","last_peered":"2024-04-17T05:42:10.138665+0000","last_clean":"2024-04-17T05:42:10.138665+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:42:10.138665+0000","last_undegraded":"2024-04-17T05:42:10.138665+0000","last_fullsized":"2024-04-17T05:42:10.138665+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:35.900737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151057+0000","last_change":"2024-04-17T05:07:45.747152+0000","last_active":"2024-04-17T05:43:10.151057+0000","last_peered":"2024-04-17T05:43:10.151057+0000","last_clean":"2024-04-17T05:43:10.151057+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:43:10.151057+0000","last_undegraded":"2024-04-17T05:43:10.151057+0000","last_fullsized":"2024-04-17T05:43:10.151057+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:50:06.059672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136102+0000","last_change":"2024-04-17T04:51:43.682823+0000","last_active":"2024-04-17T05:41:55.136102+0000","last_peered":"2024-04-17T05:41:55.136102+0000","last_clean":"2024-04-17T05:41:55.136102+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:41:55.136102+0000","last_undegraded":"2024-04-17T05:41:55.136102+0000","last_fullsized":"2024-04-17T05:41:55.136102+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:56:21.798951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'7","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159240+0000","last_change":"2024-04-17T04:51:39.636037+0000","last_active":"2024-04-17T05:43:45.159240+0000","last_peered":"2024-04-17T05:43:45.159240+0000","last_clean":"2024-04-17T05:43:45.159240+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:43:45.159240+0000","last_undegraded":"2024-04-17T05:43:45.159240+0000","last_fullsized":"2024-04-17T05:43:45.159240+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:03.304591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":11291,"num_write":12,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":149,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139651+0000","last_change":"2024-04-17T04:47:56.956029+0000","last_active":"2024-04-17T05:42:10.139651+0000","last_peered":"2024-04-17T05:42:10.139651+0000","last_clean":"2024-04-17T05:42:10.139651+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:42:10.139651+0000","last_undegraded":"2024-04-17T05:42:10.139651+0000","last_fullsized":"2024-04-17T05:42:10.139651+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:21.889729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157001+0000","last_change":"2024-04-17T05:07:45.753495+0000","last_active":"2024-04-17T05:43:10.157001+0000","last_peered":"2024-04-17T05:43:10.157001+0000","last_clean":"2024-04-17T05:43:10.157001+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:43:10.157001+0000","last_undegraded":"2024-04-17T05:43:10.157001+0000","last_fullsized":"2024-04-17T05:43:10.157001+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:29:25.567114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162221+0000","last_change":"2024-04-17T04:51:43.681581+0000","last_active":"2024-04-17T05:43:50.162221+0000","last_peered":"2024-04-17T05:43:50.162221+0000","last_clean":"2024-04-17T05:43:50.162221+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:43:50.162221+0000","last_undegraded":"2024-04-17T05:43:50.162221+0000","last_fullsized":"2024-04-17T05:43:50.162221+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:53.170686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'2","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.165685+0000","last_change":"2024-04-17T04:51:39.634757+0000","last_active":"2024-04-17T05:43:55.165685+0000","last_peered":"2024-04-17T05:43:55.165685+0000","last_clean":"2024-04-17T05:43:55.165685+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:43:55.165685+0000","last_undegraded":"2024-04-17T05:43:55.165685+0000","last_fullsized":"2024-04-17T05:43:55.165685+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:38.980103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":196,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138819+0000","last_change":"2024-04-17T04:47:56.946324+0000","last_active":"2024-04-17T05:42:10.138819+0000","last_peered":"2024-04-17T05:42:10.138819+0000","last_clean":"2024-04-17T05:42:10.138819+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:42:10.138819+0000","last_undegraded":"2024-04-17T05:42:10.138819+0000","last_fullsized":"2024-04-17T05:42:10.138819+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:09.179720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167123+0000","last_change":"2024-04-17T05:07:45.736217+0000","last_active":"2024-04-17T05:43:50.167123+0000","last_peered":"2024-04-17T05:43:50.167123+0000","last_clean":"2024-04-17T05:43:50.167123+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:43:50.167123+0000","last_undegraded":"2024-04-17T05:43:50.167123+0000","last_fullsized":"2024-04-17T05:43:50.167123+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:04.504706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'17","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159324+0000","last_change":"2024-04-17T04:51:43.675854+0000","last_active":"2024-04-17T05:43:45.159324+0000","last_peered":"2024-04-17T05:43:45.159324+0000","last_clean":"2024-04-17T05:43:45.159324+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:43:45.159324+0000","last_undegraded":"2024-04-17T05:43:45.159324+0000","last_fullsized":"2024-04-17T05:43:45.159324+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:54.193750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":54274,"num_write":16,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'10","reported_seq":79,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163666+0000","last_change":"2024-04-17T04:51:39.635143+0000","last_active":"2024-04-17T05:43:40.163666+0000","last_peered":"2024-04-17T05:43:40.163666+0000","last_clean":"2024-04-17T05:43:40.163666+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:43:40.163666+0000","last_undegraded":"2024-04-17T05:43:40.163666+0000","last_fullsized":"2024-04-17T05:43:40.163666+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:53:08.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140249+0000","last_change":"2024-04-17T04:47:56.969411+0000","last_active":"2024-04-17T05:42:20.140249+0000","last_peered":"2024-04-17T05:42:20.140249+0000","last_clean":"2024-04-17T05:42:20.140249+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:42:20.140249+0000","last_undegraded":"2024-04-17T05:42:20.140249+0000","last_fullsized":"2024-04-17T05:42:20.140249+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:56.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938315+0000","last_change":"2024-04-17T05:07:45.749227+0000","last_active":"2024-04-17T05:43:09.938315+0000","last_peered":"2024-04-17T05:43:09.938315+0000","last_clean":"2024-04-17T05:43:09.938315+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:43:09.938315+0000","last_undegraded":"2024-04-17T05:43:09.938315+0000","last_fullsized":"2024-04-17T05:43:09.938315+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:15.680393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946093+0000","last_change":"2024-04-17T04:51:43.648648+0000","last_active":"2024-04-17T05:43:44.946093+0000","last_peered":"2024-04-17T05:43:44.946093+0000","last_clean":"2024-04-17T05:43:44.946093+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:43:44.946093+0000","last_undegraded":"2024-04-17T05:43:44.946093+0000","last_fullsized":"2024-04-17T05:43:44.946093+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:57:25.905644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.158015+0000","last_change":"2024-04-17T04:51:39.646180+0000","last_active":"2024-04-17T05:43:40.158015+0000","last_peered":"2024-04-17T05:43:40.158015+0000","last_clean":"2024-04-17T05:43:40.158015+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:43:40.158015+0000","last_undegraded":"2024-04-17T05:43:40.158015+0000","last_fullsized":"2024-04-17T05:43:40.158015+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:55.361466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":110,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143901+0000","last_change":"2024-04-17T04:47:56.947428+0000","last_active":"2024-04-17T05:42:10.143901+0000","last_peered":"2024-04-17T05:42:10.143901+0000","last_clean":"2024-04-17T05:42:10.143901+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:42:10.143901+0000","last_undegraded":"2024-04-17T05:42:10.143901+0000","last_fullsized":"2024-04-17T05:42:10.143901+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:57.355596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"263'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145978+0000","last_change":"2024-04-17T05:07:45.746243+0000","last_active":"2024-04-17T05:42:25.145978+0000","last_peered":"2024-04-17T05:42:25.145978+0000","last_clean":"2024-04-17T05:42:25.145978+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:42:25.145978+0000","last_undegraded":"2024-04-17T05:42:25.145978+0000","last_fullsized":"2024-04-17T05:42:25.145978+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:08:11.491152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'7","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167234+0000","last_change":"2024-04-17T04:51:43.675539+0000","last_active":"2024-04-17T05:43:50.167234+0000","last_peered":"2024-04-17T05:43:50.167234+0000","last_clean":"2024-04-17T05:43:50.167234+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:43:50.167234+0000","last_undegraded":"2024-04-17T05:43:50.167234+0000","last_fullsized":"2024-04-17T05:43:50.167234+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:29.255105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160253+0000","last_change":"2024-04-17T04:51:39.637181+0000","last_active":"2024-04-17T05:43:50.160253+0000","last_peered":"2024-04-17T05:43:50.160253+0000","last_clean":"2024-04-17T05:43:50.160253+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:43:50.160253+0000","last_undegraded":"2024-04-17T05:43:50.160253+0000","last_fullsized":"2024-04-17T05:43:50.160253+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:38.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143904+0000","last_change":"2024-04-17T04:47:56.949538+0000","last_active":"2024-04-17T05:42:10.143904+0000","last_peered":"2024-04-17T05:42:10.143904+0000","last_clean":"2024-04-17T05:42:10.143904+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:42:10.143904+0000","last_undegraded":"2024-04-17T05:42:10.143904+0000","last_fullsized":"2024-04-17T05:42:10.143904+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:29.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143092+0000","last_change":"2024-04-17T05:07:45.726938+0000","last_active":"2024-04-17T05:42:25.143092+0000","last_peered":"2024-04-17T05:42:25.143092+0000","last_clean":"2024-04-17T05:42:25.143092+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:42:25.143092+0000","last_undegraded":"2024-04-17T05:42:25.143092+0000","last_fullsized":"2024-04-17T05:42:25.143092+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:33.108430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.166629+0000","last_change":"2024-04-17T04:51:43.648636+0000","last_active":"2024-04-17T05:43:55.166629+0000","last_peered":"2024-04-17T05:43:55.166629+0000","last_clean":"2024-04-17T05:43:55.166629+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:43:55.166629+0000","last_undegraded":"2024-04-17T05:43:55.166629+0000","last_fullsized":"2024-04-17T05:43:55.166629+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:44.200548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159094+0000","last_change":"2024-04-17T04:51:39.636922+0000","last_active":"2024-04-17T05:43:45.159094+0000","last_peered":"2024-04-17T05:43:45.159094+0000","last_clean":"2024-04-17T05:43:45.159094+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:43:45.159094+0000","last_undegraded":"2024-04-17T05:43:45.159094+0000","last_fullsized":"2024-04-17T05:43:45.159094+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:38.269210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.129678+0000","last_change":"2024-04-17T04:47:56.968894+0000","last_active":"2024-04-17T05:42:10.129678+0000","last_peered":"2024-04-17T05:42:10.129678+0000","last_clean":"2024-04-17T05:42:10.129678+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:42:10.129678+0000","last_undegraded":"2024-04-17T05:42:10.129678+0000","last_fullsized":"2024-04-17T05:42:10.129678+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:47.848243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152961+0000","last_change":"2024-04-17T05:07:45.757247+0000","last_active":"2024-04-17T05:43:10.152961+0000","last_peered":"2024-04-17T05:43:10.152961+0000","last_clean":"2024-04-17T05:43:10.152961+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:43:10.152961+0000","last_undegraded":"2024-04-17T05:43:10.152961+0000","last_fullsized":"2024-04-17T05:43:10.152961+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:26.267898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160974+0000","last_change":"2024-04-17T04:51:43.682310+0000","last_active":"2024-04-17T05:43:50.160974+0000","last_peered":"2024-04-17T05:43:50.160974+0000","last_clean":"2024-04-17T05:43:50.160974+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:43:50.160974+0000","last_undegraded":"2024-04-17T05:43:50.160974+0000","last_fullsized":"2024-04-17T05:43:50.160974+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:14.153241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'1","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165820+0000","last_change":"2024-04-17T04:51:39.617751+0000","last_active":"2024-04-17T05:43:45.165820+0000","last_peered":"2024-04-17T05:43:45.165820+0000","last_clean":"2024-04-17T05:43:45.165820+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:43:45.165820+0000","last_undegraded":"2024-04-17T05:43:45.165820+0000","last_fullsized":"2024-04-17T05:43:45.165820+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:45.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":1,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":213,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138713+0000","last_change":"2024-04-17T04:47:56.952060+0000","last_active":"2024-04-17T05:42:10.138713+0000","last_peered":"2024-04-17T05:42:10.138713+0000","last_clean":"2024-04-17T05:42:10.138713+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:42:10.138713+0000","last_undegraded":"2024-04-17T05:42:10.138713+0000","last_fullsized":"2024-04-17T05:42:10.138713+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:52.024764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.152311+0000","last_change":"2024-04-17T05:07:45.752461+0000","last_active":"2024-04-17T05:43:15.152311+0000","last_peered":"2024-04-17T05:43:15.152311+0000","last_clean":"2024-04-17T05:43:15.152311+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:43:15.152311+0000","last_undegraded":"2024-04-17T05:43:15.152311+0000","last_fullsized":"2024-04-17T05:43:15.152311+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:43.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:49.947148+0000","last_change":"2024-04-17T04:51:43.651232+0000","last_active":"2024-04-17T05:43:49.947148+0000","last_peered":"2024-04-17T05:43:49.947148+0000","last_clean":"2024-04-17T05:43:49.947148+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:43:49.947148+0000","last_undegraded":"2024-04-17T05:43:49.947148+0000","last_fullsized":"2024-04-17T05:43:49.947148+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:48.905181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159752+0000","last_change":"2024-04-17T04:51:39.630310+0000","last_active":"2024-04-17T05:43:45.159752+0000","last_peered":"2024-04-17T05:43:45.159752+0000","last_clean":"2024-04-17T05:43:45.159752+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:43:45.159752+0000","last_undegraded":"2024-04-17T05:43:45.159752+0000","last_fullsized":"2024-04-17T05:43:45.159752+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:32:14.085186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6339880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6192,"num_write":3,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":127,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.137755+0000","last_change":"2024-04-17T04:47:56.963237+0000","last_active":"2024-04-17T05:42:10.137755+0000","last_peered":"2024-04-17T05:42:10.137755+0000","last_clean":"2024-04-17T05:42:10.137755+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:42:10.137755+0000","last_undegraded":"2024-04-17T05:42:10.137755+0000","last_fullsized":"2024-04-17T05:42:10.137755+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:11.681306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'285","reported_seq":946,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150137+0000","last_change":"2024-04-17T04:36:25.377106+0000","last_active":"2024-04-17T05:43:05.150137+0000","last_peered":"2024-04-17T05:43:05.150137+0000","last_clean":"2024-04-17T05:43:05.150137+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:43:05.150137+0000","last_undegraded":"2024-04-17T05:43:05.150137+0000","last_fullsized":"2024-04-17T05:43:05.150137+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:09.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53724,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":483,"num_read_kb":725,"num_write":1250,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":675,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134422+0000","last_change":"2024-04-17T04:36:27.389744+0000","last_active":"2024-04-17T05:41:30.134422+0000","last_peered":"2024-04-17T05:41:30.134422+0000","last_clean":"2024-04-17T05:41:30.134422+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:41:30.134422+0000","last_undegraded":"2024-04-17T05:41:30.134422+0000","last_fullsized":"2024-04-17T05:41:30.134422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:14.577671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'308","reported_seq":980,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157287+0000","last_change":"2024-04-17T04:36:25.357080+0000","last_active":"2024-04-17T05:43:05.157287+0000","last_peered":"2024-04-17T05:43:05.157287+0000","last_clean":"2024-04-17T05:43:05.157287+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:43:05.157287+0000","last_undegraded":"2024-04-17T05:43:05.157287+0000","last_fullsized":"2024-04-17T05:43:05.157287+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:21.403329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10755049,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":526,"num_read_kb":10797,"num_write":1432,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":706,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:39.489661+0000","last_change":"2024-04-17T04:36:27.382828+0000","last_active":"2024-04-17T05:42:39.489661+0000","last_peered":"2024-04-17T05:42:39.489661+0000","last_clean":"2024-04-17T05:42:39.489661+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:42:39.489661+0000","last_undegraded":"2024-04-17T05:42:39.489661+0000","last_fullsized":"2024-04-17T05:42:39.489661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:30.715656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'254","reported_seq":920,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151824+0000","last_change":"2024-04-17T04:36:25.329662+0000","last_active":"2024-04-17T05:43:05.151824+0000","last_peered":"2024-04-17T05:43:05.151824+0000","last_clean":"2024-04-17T05:43:05.151824+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:43:05.151824+0000","last_undegraded":"2024-04-17T05:43:05.151824+0000","last_fullsized":"2024-04-17T05:43:05.151824+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:42.966712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528901,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":417,"num_read_kb":58401,"num_write":1131,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120297+0000","last_change":"2024-04-17T04:36:27.385236+0000","last_active":"2024-04-17T05:41:30.120297+0000","last_peered":"2024-04-17T05:41:30.120297+0000","last_clean":"2024-04-17T05:41:30.120297+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:41:30.120297+0000","last_undegraded":"2024-04-17T05:41:30.120297+0000","last_fullsized":"2024-04-17T05:41:30.120297+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:55.872226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'241","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150409+0000","last_change":"2024-04-17T04:36:25.356234+0000","last_active":"2024-04-17T05:42:40.150409+0000","last_peered":"2024-04-17T05:42:40.150409+0000","last_clean":"2024-04-17T05:42:40.150409+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:42:40.150409+0000","last_undegraded":"2024-04-17T05:42:40.150409+0000","last_fullsized":"2024-04-17T05:42:40.150409+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:14.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5765916,"num_objects":48,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":417,"num_read_kb":37884,"num_write":1085,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'304","reported_seq":897,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.129356+0000","last_change":"2024-04-17T04:36:27.385726+0000","last_active":"2024-04-17T05:41:30.129356+0000","last_peered":"2024-04-17T05:41:30.129356+0000","last_clean":"2024-04-17T05:41:30.129356+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:41:30.129356+0000","last_undegraded":"2024-04-17T05:41:30.129356+0000","last_fullsized":"2024-04-17T05:41:30.129356+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:55:33.151184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'266","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.127332+0000","last_change":"2024-04-17T04:36:25.377714+0000","last_active":"2024-04-17T05:41:25.127332+0000","last_peered":"2024-04-17T05:41:25.127332+0000","last_clean":"2024-04-17T05:41:25.127332+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:41:25.127332+0000","last_undegraded":"2024-04-17T05:41:25.127332+0000","last_fullsized":"2024-04-17T05:41:25.127332+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:34.861649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19973206,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":458,"num_read_kb":13359,"num_write":1103,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5179,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.736719+0000","last_change":"2024-04-17T04:36:27.382622+0000","last_active":"2024-04-17T05:45:44.736719+0000","last_peered":"2024-04-17T05:45:44.736719+0000","last_clean":"2024-04-17T05:45:44.736719+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:45:44.736719+0000","last_undegraded":"2024-04-17T05:45:44.736719+0000","last_fullsized":"2024-04-17T05:45:44.736719+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:23.921181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'230","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:43.796816+0000","last_change":"2024-04-17T04:36:25.352114+0000","last_active":"2024-04-17T05:41:43.796816+0000","last_peered":"2024-04-17T05:41:43.796816+0000","last_clean":"2024-04-17T05:41:43.796816+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:41:43.796816+0000","last_undegraded":"2024-04-17T05:41:43.796816+0000","last_fullsized":"2024-04-17T05:41:43.796816+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:17.459792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11021151,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":513,"num_read_kb":64592,"num_write":906,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":774,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938531+0000","last_change":"2024-04-17T04:36:27.385407+0000","last_active":"2024-04-17T05:43:09.938531+0000","last_peered":"2024-04-17T05:43:09.938531+0000","last_clean":"2024-04-17T05:43:09.938531+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:43:09.938531+0000","last_undegraded":"2024-04-17T05:43:09.938531+0000","last_fullsized":"2024-04-17T05:43:09.938531+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:03.112120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'259","reported_seq":881,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:37.900457+0000","last_change":"2024-04-17T04:36:25.376858+0000","last_active":"2024-04-17T05:42:37.900457+0000","last_peered":"2024-04-17T05:42:37.900457+0000","last_clean":"2024-04-17T05:42:37.900457+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:42:37.900457+0000","last_undegraded":"2024-04-17T05:42:37.900457+0000","last_fullsized":"2024-04-17T05:42:37.900457+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:24.809459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3318739,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":393,"num_read_kb":9422,"num_write":1064,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157138+0000","last_change":"2024-04-17T04:36:27.376772+0000","last_active":"2024-04-17T05:43:10.157138+0000","last_peered":"2024-04-17T05:43:10.157138+0000","last_clean":"2024-04-17T05:43:10.157138+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:43:10.157138+0000","last_undegraded":"2024-04-17T05:43:10.157138+0000","last_fullsized":"2024-04-17T05:43:10.157138+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:07.055230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'260","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.718287+0000","last_change":"2024-04-17T04:36:25.343980+0000","last_active":"2024-04-17T05:43:39.718287+0000","last_peered":"2024-04-17T05:43:39.718287+0000","last_clean":"2024-04-17T05:43:39.718287+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:43:39.718287+0000","last_undegraded":"2024-04-17T05:43:39.718287+0000","last_fullsized":"2024-04-17T05:43:39.718287+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:21.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20922664,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":369,"num_read_kb":14992,"num_write":1106,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.156665+0000","last_change":"2024-04-17T04:36:27.379088+0000","last_active":"2024-04-17T05:43:10.156665+0000","last_peered":"2024-04-17T05:43:10.156665+0000","last_clean":"2024-04-17T05:43:10.156665+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:43:10.156665+0000","last_undegraded":"2024-04-17T05:43:10.156665+0000","last_fullsized":"2024-04-17T05:43:10.156665+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:05.867623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125191+0000","last_change":"2024-04-17T04:36:27.385695+0000","last_active":"2024-04-17T05:41:50.125191+0000","last_peered":"2024-04-17T05:41:50.125191+0000","last_clean":"2024-04-17T05:41:50.125191+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:41:50.125191+0000","last_undegraded":"2024-04-17T05:41:50.125191+0000","last_fullsized":"2024-04-17T05:41:50.125191+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:33.234173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'255","reported_seq":862,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.148032+0000","last_change":"2024-04-17T04:37:03.648715+0000","last_active":"2024-04-17T05:42:35.148032+0000","last_peered":"2024-04-17T05:42:35.148032+0000","last_clean":"2024-04-17T05:42:35.148032+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:42:35.148032+0000","last_undegraded":"2024-04-17T05:42:35.148032+0000","last_fullsized":"2024-04-17T05:42:35.148032+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:12.036980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8728170,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":386,"num_read_kb":14103,"num_write":1112,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'255","reported_seq":889,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.407437+0000","last_change":"2024-04-17T04:36:27.383282+0000","last_active":"2024-04-17T05:45:46.407437+0000","last_peered":"2024-04-17T05:45:46.407437+0000","last_clean":"2024-04-17T05:45:46.407437+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:45:46.407437+0000","last_undegraded":"2024-04-17T05:45:46.407437+0000","last_fullsized":"2024-04-17T05:45:46.407437+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:29.116974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'216","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:47.860045+0000","last_change":"2024-04-17T04:36:25.385368+0000","last_active":"2024-04-17T05:42:47.860045+0000","last_peered":"2024-04-17T05:42:47.860045+0000","last_clean":"2024-04-17T05:42:47.860045+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:42:47.860045+0000","last_undegraded":"2024-04-17T05:42:47.860045+0000","last_fullsized":"2024-04-17T05:42:47.860045+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:35.267010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761635,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":351,"num_read_kb":10549,"num_write":915,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":723,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142870+0000","last_change":"2024-04-17T04:36:27.376262+0000","last_active":"2024-04-17T05:42:05.142870+0000","last_peered":"2024-04-17T05:42:05.142870+0000","last_clean":"2024-04-17T05:42:05.142870+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:42:05.142870+0000","last_undegraded":"2024-04-17T05:42:05.142870+0000","last_fullsized":"2024-04-17T05:42:05.142870+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:02.524761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'248","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.390159+0000","last_change":"2024-04-17T04:36:25.385723+0000","last_active":"2024-04-17T05:43:39.390159+0000","last_peered":"2024-04-17T05:43:39.390159+0000","last_clean":"2024-04-17T05:43:39.390159+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:43:39.390159+0000","last_undegraded":"2024-04-17T05:43:39.390159+0000","last_fullsized":"2024-04-17T05:43:39.390159+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:51.669081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498677,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":402,"num_read_kb":160096,"num_write":1072,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136478+0000","last_change":"2024-04-17T04:36:27.383859+0000","last_active":"2024-04-17T05:41:45.136478+0000","last_peered":"2024-04-17T05:41:45.136478+0000","last_clean":"2024-04-17T05:41:45.136478+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:41:45.136478+0000","last_undegraded":"2024-04-17T05:41:45.136478+0000","last_fullsized":"2024-04-17T05:41:45.136478+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:14.617020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'242","reported_seq":947,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:52.574040+0000","last_change":"2024-04-17T04:36:25.361757+0000","last_active":"2024-04-17T05:41:52.574040+0000","last_peered":"2024-04-17T05:41:52.574040+0000","last_clean":"2024-04-17T05:41:52.574040+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:41:52.574040+0000","last_undegraded":"2024-04-17T05:41:52.574040+0000","last_fullsized":"2024-04-17T05:41:52.574040+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:21.146448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19295786,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":522,"num_read_kb":64725,"num_write":1051,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136549+0000","last_change":"2024-04-17T04:36:27.379593+0000","last_active":"2024-04-17T05:41:45.136549+0000","last_peered":"2024-04-17T05:41:45.136549+0000","last_clean":"2024-04-17T05:41:45.136549+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:41:45.136549+0000","last_undegraded":"2024-04-17T05:41:45.136549+0000","last_fullsized":"2024-04-17T05:41:45.136549+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:37.024479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'256","reported_seq":1026,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.874797+0000","last_change":"2024-04-17T04:36:25.374400+0000","last_active":"2024-04-17T05:42:57.874797+0000","last_peered":"2024-04-17T05:42:57.874797+0000","last_clean":"2024-04-17T05:42:57.874797+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:42:57.874797+0000","last_undegraded":"2024-04-17T05:42:57.874797+0000","last_fullsized":"2024-04-17T05:42:57.874797+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:25:10.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26759829,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":748,"num_read_kb":57779,"num_write":1204,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.501738+0000","last_change":"2024-04-17T04:36:27.385279+0000","last_active":"2024-04-17T05:41:55.501738+0000","last_peered":"2024-04-17T05:41:55.501738+0000","last_clean":"2024-04-17T05:41:55.501738+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:41:55.501738+0000","last_undegraded":"2024-04-17T05:41:55.501738+0000","last_fullsized":"2024-04-17T05:41:55.501738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:17.292970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'280","reported_seq":914,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135979+0000","last_change":"2024-04-17T04:36:25.355404+0000","last_active":"2024-04-17T05:41:55.135979+0000","last_peered":"2024-04-17T05:41:55.135979+0000","last_clean":"2024-04-17T05:41:55.135979+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:41:55.135979+0000","last_undegraded":"2024-04-17T05:41:55.135979+0000","last_fullsized":"2024-04-17T05:41:55.135979+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:47.270461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10613748,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":397,"num_read_kb":836,"num_write":1108,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'152","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.671918+0000","last_change":"2024-04-17T04:36:35.418351+0000","last_active":"2024-04-17T05:45:46.671918+0000","last_peered":"2024-04-17T05:45:46.671918+0000","last_clean":"2024-04-17T05:45:46.671918+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:45:46.671918+0000","last_undegraded":"2024-04-17T05:45:46.671918+0000","last_fullsized":"2024-04-17T05:45:46.671918+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:02.595419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:51.868791+0000","last_change":"2024-04-17T04:36:27.384993+0000","last_active":"2024-04-17T05:42:51.868791+0000","last_peered":"2024-04-17T05:42:51.868791+0000","last_clean":"2024-04-17T05:42:51.868791+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:42:51.868791+0000","last_undegraded":"2024-04-17T05:42:51.868791+0000","last_fullsized":"2024-04-17T05:42:51.868791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:24.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'258","reported_seq":882,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.719639+0000","last_change":"2024-04-17T04:36:25.368467+0000","last_active":"2024-04-17T05:42:50.719639+0000","last_peered":"2024-04-17T05:42:50.719639+0000","last_clean":"2024-04-17T05:42:50.719639+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:42:50.719639+0000","last_undegraded":"2024-04-17T05:42:50.719639+0000","last_fullsized":"2024-04-17T05:42:50.719639+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:23.161029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12579502,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":373,"num_read_kb":2527,"num_write":1113,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138168+0000","last_change":"2024-04-17T04:36:29.388609+0000","last_active":"2024-04-17T05:41:45.138168+0000","last_peered":"2024-04-17T05:41:45.138168+0000","last_clean":"2024-04-17T05:41:45.138168+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:41:45.138168+0000","last_undegraded":"2024-04-17T05:41:45.138168+0000","last_fullsized":"2024-04-17T05:41:45.138168+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:53.809254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'362","reported_seq":982,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.169232+0000","last_change":"2024-04-17T04:36:31.410990+0000","last_active":"2024-04-17T05:44:15.169232+0000","last_peered":"2024-04-17T05:44:15.169232+0000","last_clean":"2024-04-17T05:44:15.169232+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:44:15.169232+0000","last_undegraded":"2024-04-17T05:44:15.169232+0000","last_fullsized":"2024-04-17T05:44:15.169232+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":300,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:24.430923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58423,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":1138,"num_write":466,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133607+0000","last_change":"2024-04-17T04:36:33.425089+0000","last_active":"2024-04-17T05:41:50.133607+0000","last_peered":"2024-04-17T05:41:50.133607+0000","last_clean":"2024-04-17T05:41:50.133607+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:41:50.133607+0000","last_undegraded":"2024-04-17T05:41:50.133607+0000","last_fullsized":"2024-04-17T05:41:50.133607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:53.730080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'169","reported_seq":722,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.761057+0000","last_change":"2024-04-17T04:36:35.416351+0000","last_active":"2024-04-17T05:45:44.761057+0000","last_peered":"2024-04-17T05:45:44.761057+0000","last_clean":"2024-04-17T05:45:44.761057+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:45:44.761057+0000","last_undegraded":"2024-04-17T05:45:44.761057+0000","last_fullsized":"2024-04-17T05:45:44.761057+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:34.873231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":673,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138074+0000","last_change":"2024-04-17T04:36:27.389710+0000","last_active":"2024-04-17T05:41:45.138074+0000","last_peered":"2024-04-17T05:41:45.138074+0000","last_clean":"2024-04-17T05:41:45.138074+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:41:45.138074+0000","last_undegraded":"2024-04-17T05:41:45.138074+0000","last_fullsized":"2024-04-17T05:41:45.138074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:11:08.849990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'278","reported_seq":1049,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133284+0000","last_change":"2024-04-17T04:36:25.328153+0000","last_active":"2024-04-17T05:41:45.133284+0000","last_peered":"2024-04-17T05:41:45.133284+0000","last_clean":"2024-04-17T05:41:45.133284+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:41:45.133284+0000","last_undegraded":"2024-04-17T05:41:45.133284+0000","last_fullsized":"2024-04-17T05:41:45.133284+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:07.072313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21193393,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":693,"num_read_kb":61435,"num_write":1302,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.170670+0000","last_change":"2024-04-17T04:36:29.373876+0000","last_active":"2024-04-17T05:42:05.170670+0000","last_peered":"2024-04-17T05:42:05.170670+0000","last_clean":"2024-04-17T05:42:05.170670+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:42:05.170670+0000","last_undegraded":"2024-04-17T05:42:05.170670+0000","last_fullsized":"2024-04-17T05:42:05.170670+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:06.355032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'99","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125438+0000","last_change":"2024-04-17T04:36:31.395007+0000","last_active":"2024-04-17T05:41:50.125438+0000","last_peered":"2024-04-17T05:41:50.125438+0000","last_clean":"2024-04-17T05:41:50.125438+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:41:50.125438+0000","last_undegraded":"2024-04-17T05:41:50.125438+0000","last_fullsized":"2024-04-17T05:41:50.125438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:47.124749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125417+0000","last_change":"2024-04-17T04:36:33.414084+0000","last_active":"2024-04-17T05:41:50.125417+0000","last_peered":"2024-04-17T05:41:50.125417+0000","last_clean":"2024-04-17T05:41:50.125417+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:41:50.125417+0000","last_undegraded":"2024-04-17T05:41:50.125417+0000","last_fullsized":"2024-04-17T05:41:50.125417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:27.803230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'123","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.222678+0000","last_change":"2024-04-17T04:36:35.427530+0000","last_active":"2024-04-17T05:45:44.222678+0000","last_peered":"2024-04-17T05:45:44.222678+0000","last_clean":"2024-04-17T05:45:44.222678+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:45:44.222678+0000","last_undegraded":"2024-04-17T05:45:44.222678+0000","last_fullsized":"2024-04-17T05:45:44.222678+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:45.651840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":659,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:09.959182+0000","last_change":"2024-04-17T04:36:27.376107+0000","last_active":"2024-04-17T05:42:09.959182+0000","last_peered":"2024-04-17T05:42:09.959182+0000","last_clean":"2024-04-17T05:42:09.959182+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:42:09.959182+0000","last_undegraded":"2024-04-17T05:42:09.959182+0000","last_fullsized":"2024-04-17T05:42:09.959182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:13.603194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'276","reported_seq":1055,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.131908+0000","last_change":"2024-04-17T04:36:25.381456+0000","last_active":"2024-04-17T05:41:45.131908+0000","last_peered":"2024-04-17T05:41:45.131908+0000","last_clean":"2024-04-17T05:41:45.131908+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:41:45.131908+0000","last_undegraded":"2024-04-17T05:41:45.131908+0000","last_fullsized":"2024-04-17T05:41:45.131908+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:08.488003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22475342,"num_objects":61,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":615,"num_read_kb":75395,"num_write":1261,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165842+0000","last_change":"2024-04-17T04:36:29.359822+0000","last_active":"2024-04-17T05:41:45.165842+0000","last_peered":"2024-04-17T05:41:45.165842+0000","last_clean":"2024-04-17T05:41:45.165842+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:41:45.165842+0000","last_undegraded":"2024-04-17T05:41:45.165842+0000","last_fullsized":"2024-04-17T05:41:45.165842+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:41.948364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'137","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.141895+0000","last_change":"2024-04-17T04:36:31.391956+0000","last_active":"2024-04-17T05:42:10.141895+0000","last_peered":"2024-04-17T05:42:10.141895+0000","last_clean":"2024-04-17T05:42:10.141895+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:42:10.141895+0000","last_undegraded":"2024-04-17T05:42:10.141895+0000","last_fullsized":"2024-04-17T05:42:10.141895+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:12.846752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'512","reported_seq":2409,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.670706+0000","last_change":"2024-04-17T04:36:33.420806+0000","last_active":"2024-04-17T05:45:57.670706+0000","last_peered":"2024-04-17T05:45:57.670706+0000","last_clean":"2024-04-17T05:45:57.670706+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:45:57.670706+0000","last_undegraded":"2024-04-17T05:45:57.670706+0000","last_fullsized":"2024-04-17T05:45:57.670706+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:52.301226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'195","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.415632+0000","last_change":"2024-04-17T04:36:35.403573+0000","last_active":"2024-04-17T05:45:46.415632+0000","last_peered":"2024-04-17T05:45:46.415632+0000","last_clean":"2024-04-17T05:45:46.415632+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:45:46.415632+0000","last_undegraded":"2024-04-17T05:45:46.415632+0000","last_fullsized":"2024-04-17T05:45:46.415632+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:09.033539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:07.139896+0000","last_change":"2024-04-17T04:36:27.395613+0000","last_active":"2024-04-17T05:43:07.139896+0000","last_peered":"2024-04-17T05:43:07.139896+0000","last_clean":"2024-04-17T05:43:07.139896+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:43:07.139896+0000","last_undegraded":"2024-04-17T05:43:07.139896+0000","last_fullsized":"2024-04-17T05:43:07.139896+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:32:19.325576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'235","reported_seq":881,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133330+0000","last_change":"2024-04-17T04:36:25.384743+0000","last_active":"2024-04-17T05:41:45.133330+0000","last_peered":"2024-04-17T05:41:45.133330+0000","last_clean":"2024-04-17T05:41:45.133330+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:41:45.133330+0000","last_undegraded":"2024-04-17T05:41:45.133330+0000","last_fullsized":"2024-04-17T05:41:45.133330+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:05.451652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727402,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":395,"num_read_kb":48921,"num_write":1085,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138271+0000","last_change":"2024-04-17T04:36:29.391920+0000","last_active":"2024-04-17T05:41:45.138271+0000","last_peered":"2024-04-17T05:41:45.138271+0000","last_clean":"2024-04-17T05:41:45.138271+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:41:45.138271+0000","last_undegraded":"2024-04-17T05:41:45.138271+0000","last_fullsized":"2024-04-17T05:41:45.138271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:00:30.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'274","reported_seq":854,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166406+0000","last_change":"2024-04-17T04:36:31.422728+0000","last_active":"2024-04-17T05:44:15.166406+0000","last_peered":"2024-04-17T05:44:15.166406+0000","last_clean":"2024-04-17T05:44:15.166406+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:44:15.166406+0000","last_undegraded":"2024-04-17T05:44:15.166406+0000","last_fullsized":"2024-04-17T05:44:15.166406+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:05.940948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62885,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":973,"num_write":370,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'529","reported_seq":2297,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.662523+0000","last_change":"2024-04-17T04:36:33.422550+0000","last_active":"2024-04-17T05:45:57.662523+0000","last_peered":"2024-04-17T05:45:57.662523+0000","last_clean":"2024-04-17T05:45:57.662523+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:45:57.662523+0000","last_undegraded":"2024-04-17T05:45:57.662523+0000","last_fullsized":"2024-04-17T05:45:57.662523+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":500,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:46.514333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'121","reported_seq":720,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163455+0000","last_change":"2024-04-17T04:36:27.398309+0000","last_active":"2024-04-17T05:43:40.163455+0000","last_peered":"2024-04-17T05:43:40.163455+0000","last_clean":"2024-04-17T05:43:40.163455+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:43:40.163455+0000","last_undegraded":"2024-04-17T05:43:40.163455+0000","last_fullsized":"2024-04-17T05:43:40.163455+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:23.743688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'230","reported_seq":863,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133579+0000","last_change":"2024-04-17T04:36:25.370799+0000","last_active":"2024-04-17T05:41:50.133579+0000","last_peered":"2024-04-17T05:41:50.133579+0000","last_clean":"2024-04-17T05:41:50.133579+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:41:50.133579+0000","last_undegraded":"2024-04-17T05:41:50.133579+0000","last_fullsized":"2024-04-17T05:41:50.133579+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:55.146915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15464766,"num_objects":51,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":386,"num_read_kb":24078,"num_write":1075,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.157843+0000","last_change":"2024-04-17T04:36:29.389738+0000","last_active":"2024-04-17T05:43:40.157843+0000","last_peered":"2024-04-17T05:43:40.157843+0000","last_clean":"2024-04-17T05:43:40.157843+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:43:40.157843+0000","last_undegraded":"2024-04-17T05:43:40.157843+0000","last_fullsized":"2024-04-17T05:43:40.157843+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:52.759093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'377","reported_seq":1009,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.167892+0000","last_change":"2024-04-17T04:36:31.422910+0000","last_active":"2024-04-17T05:44:00.167892+0000","last_peered":"2024-04-17T05:44:00.167892+0000","last_clean":"2024-04-17T05:44:00.167892+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:44:00.167892+0000","last_undegraded":"2024-04-17T05:44:00.167892+0000","last_fullsized":"2024-04-17T05:44:00.167892+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:47.096298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54931,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":606,"num_read_kb":1124,"num_write":470,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139387+0000","last_change":"2024-04-17T04:36:33.408109+0000","last_active":"2024-04-17T05:41:50.139387+0000","last_peered":"2024-04-17T05:41:50.139387+0000","last_clean":"2024-04-17T05:41:50.139387+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:41:50.139387+0000","last_undegraded":"2024-04-17T05:41:50.139387+0000","last_fullsized":"2024-04-17T05:41:50.139387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:19.914865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'560","reported_seq":1384,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.656195+0000","last_change":"2024-04-17T04:36:35.409093+0000","last_active":"2024-04-17T05:45:54.656195+0000","last_peered":"2024-04-17T05:45:54.656195+0000","last_clean":"2024-04-17T05:45:54.656195+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:45:54.656195+0000","last_undegraded":"2024-04-17T05:45:54.656195+0000","last_fullsized":"2024-04-17T05:45:54.656195+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":500,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:27.394663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":865,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.944851+0000","last_change":"2024-04-17T04:36:27.379851+0000","last_active":"2024-04-17T05:43:39.944851+0000","last_peered":"2024-04-17T05:43:39.944851+0000","last_clean":"2024-04-17T05:43:39.944851+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:43:39.944851+0000","last_undegraded":"2024-04-17T05:43:39.944851+0000","last_fullsized":"2024-04-17T05:43:39.944851+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:27.808945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'312","reported_seq":972,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:12.983824+0000","last_change":"2024-04-17T04:36:25.371549+0000","last_active":"2024-04-17T05:42:12.983824+0000","last_peered":"2024-04-17T05:42:12.983824+0000","last_clean":"2024-04-17T05:42:12.983824+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:42:12.983824+0000","last_undegraded":"2024-04-17T05:42:12.983824+0000","last_fullsized":"2024-04-17T05:42:12.983824+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:35.252546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7755476,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":488,"num_read_kb":14712,"num_write":1238,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162258+0000","last_change":"2024-04-17T04:36:29.371794+0000","last_active":"2024-04-17T05:43:40.162258+0000","last_peered":"2024-04-17T05:43:40.162258+0000","last_clean":"2024-04-17T05:43:40.162258+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:43:40.162258+0000","last_undegraded":"2024-04-17T05:43:40.162258+0000","last_fullsized":"2024-04-17T05:43:40.162258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:11.383775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'264","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198231+0000","last_change":"2024-04-17T04:36:31.392065+0000","last_active":"2024-04-17T05:44:05.198231+0000","last_peered":"2024-04-17T05:44:05.198231+0000","last_clean":"2024-04-17T05:44:05.198231+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:44:05.198231+0000","last_undegraded":"2024-04-17T05:44:05.198231+0000","last_fullsized":"2024-04-17T05:44:05.198231+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:36.246942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45337,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":414,"num_read_kb":838,"num_write":330,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166729+0000","last_change":"2024-04-17T04:36:33.416143+0000","last_active":"2024-04-17T05:41:50.166729+0000","last_peered":"2024-04-17T05:41:50.166729+0000","last_clean":"2024-04-17T05:41:50.166729+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:41:50.166729+0000","last_undegraded":"2024-04-17T05:41:50.166729+0000","last_fullsized":"2024-04-17T05:41:50.166729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:51.569882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2371","reported_seq":4173,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.067191+0000","last_change":"2024-04-17T04:36:35.418228+0000","last_active":"2024-04-17T05:45:40.067191+0000","last_peered":"2024-04-17T05:45:40.067191+0000","last_clean":"2024-04-17T05:45:40.067191+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:45:40.067191+0000","last_undegraded":"2024-04-17T05:45:40.067191+0000","last_fullsized":"2024-04-17T05:45:40.067191+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":2300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.125017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160924+0000","last_change":"2024-04-17T04:36:27.376914+0000","last_active":"2024-04-17T05:43:50.160924+0000","last_peered":"2024-04-17T05:43:50.160924+0000","last_clean":"2024-04-17T05:43:50.160924+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:43:50.160924+0000","last_undegraded":"2024-04-17T05:43:50.160924+0000","last_fullsized":"2024-04-17T05:43:50.160924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:06.533166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'288","reported_seq":962,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:12.960895+0000","last_change":"2024-04-17T04:36:25.386734+0000","last_active":"2024-04-17T05:42:12.960895+0000","last_peered":"2024-04-17T05:42:12.960895+0000","last_clean":"2024-04-17T05:42:12.960895+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:42:12.960895+0000","last_undegraded":"2024-04-17T05:42:12.960895+0000","last_fullsized":"2024-04-17T05:42:12.960895+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:49.654467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7146143,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":536,"num_read_kb":65373,"num_write":1251,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136503+0000","last_change":"2024-04-17T04:36:29.372163+0000","last_active":"2024-04-17T05:41:45.136503+0000","last_peered":"2024-04-17T05:41:45.136503+0000","last_clean":"2024-04-17T05:41:45.136503+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:41:45.136503+0000","last_undegraded":"2024-04-17T05:41:45.136503+0000","last_fullsized":"2024-04-17T05:41:45.136503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:29.815813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'330","reported_seq":941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.632269+0000","last_change":"2024-04-17T04:36:31.380567+0000","last_active":"2024-04-17T05:45:54.632269+0000","last_peered":"2024-04-17T05:45:54.632269+0000","last_clean":"2024-04-17T05:45:54.632269+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:45:54.632269+0000","last_undegraded":"2024-04-17T05:45:54.632269+0000","last_fullsized":"2024-04-17T05:45:54.632269+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:55.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":530,"num_read_kb":1046,"num_write":422,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'534","reported_seq":4610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.650496+0000","last_change":"2024-04-17T04:36:33.411578+0000","last_active":"2024-04-17T05:45:57.650496+0000","last_peered":"2024-04-17T05:45:57.650496+0000","last_clean":"2024-04-17T05:45:57.650496+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:45:57.650496+0000","last_undegraded":"2024-04-17T05:45:57.650496+0000","last_fullsized":"2024-04-17T05:45:57.650496+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.576073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161111+0000","last_change":"2024-04-17T04:36:35.427581+0000","last_active":"2024-04-17T05:43:45.161111+0000","last_peered":"2024-04-17T05:43:45.161111+0000","last_clean":"2024-04-17T05:43:45.161111+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:43:45.161111+0000","last_undegraded":"2024-04-17T05:43:45.161111+0000","last_fullsized":"2024-04-17T05:43:45.161111+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:13.700005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'95","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.081384+0000","last_change":"2024-04-17T04:36:27.370100+0000","last_active":"2024-04-17T05:42:57.081384+0000","last_peered":"2024-04-17T05:42:57.081384+0000","last_clean":"2024-04-17T05:42:57.081384+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:42:57.081384+0000","last_undegraded":"2024-04-17T05:42:57.081384+0000","last_fullsized":"2024-04-17T05:42:57.081384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:31.666063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'275","reported_seq":974,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144551+0000","last_change":"2024-04-17T04:37:03.648700+0000","last_active":"2024-04-17T05:42:20.144551+0000","last_peered":"2024-04-17T05:42:20.144551+0000","last_clean":"2024-04-17T05:42:20.144551+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:42:20.144551+0000","last_undegraded":"2024-04-17T05:42:20.144551+0000","last_fullsized":"2024-04-17T05:42:20.144551+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:49:08.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6559602,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":598,"num_read_kb":24373,"num_write":1106,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136436+0000","last_change":"2024-04-17T04:36:29.375596+0000","last_active":"2024-04-17T05:41:45.136436+0000","last_peered":"2024-04-17T05:41:45.136436+0000","last_clean":"2024-04-17T05:41:45.136436+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:41:45.136436+0000","last_undegraded":"2024-04-17T05:41:45.136436+0000","last_fullsized":"2024-04-17T05:41:45.136436+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:20.886949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'462","reported_seq":1109,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.633698+0000","last_change":"2024-04-17T04:36:31.409877+0000","last_active":"2024-04-17T05:45:54.633698+0000","last_peered":"2024-04-17T05:45:54.633698+0000","last_clean":"2024-04-17T05:45:54.633698+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:45:54.633698+0000","last_undegraded":"2024-04-17T05:45:54.633698+0000","last_fullsized":"2024-04-17T05:45:54.633698+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:19.065697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101390,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":805,"num_read_kb":1767,"num_write":654,"num_write_kb":532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.163452+0000","last_change":"2024-04-17T04:36:33.420378+0000","last_active":"2024-04-17T05:43:45.163452+0000","last_peered":"2024-04-17T05:43:45.163452+0000","last_clean":"2024-04-17T05:43:45.163452+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:43:45.163452+0000","last_undegraded":"2024-04-17T05:43:45.163452+0000","last_fullsized":"2024-04-17T05:43:45.163452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:59.519269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'148","reported_seq":689,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165985+0000","last_change":"2024-04-17T04:36:35.417260+0000","last_active":"2024-04-17T05:43:45.165985+0000","last_peered":"2024-04-17T05:43:45.165985+0000","last_clean":"2024-04-17T05:43:45.165985+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:43:45.165985+0000","last_undegraded":"2024-04-17T05:43:45.165985+0000","last_fullsized":"2024-04-17T05:43:45.165985+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:41.967458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124135+0000","last_change":"2024-04-17T04:36:27.393949+0000","last_active":"2024-04-17T05:41:45.124135+0000","last_peered":"2024-04-17T05:41:45.124135+0000","last_clean":"2024-04-17T05:41:45.124135+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:41:45.124135+0000","last_undegraded":"2024-04-17T05:41:45.124135+0000","last_fullsized":"2024-04-17T05:41:45.124135+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:12.169777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'287","reported_seq":955,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164840+0000","last_change":"2024-04-17T04:36:25.379472+0000","last_active":"2024-04-17T05:41:40.164840+0000","last_peered":"2024-04-17T05:41:40.164840+0000","last_clean":"2024-04-17T05:41:40.164840+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:41:40.164840+0000","last_undegraded":"2024-04-17T05:41:40.164840+0000","last_fullsized":"2024-04-17T05:41:40.164840+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:42:53.335830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2019600,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":501,"num_read_kb":9569,"num_write":1164,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136613+0000","last_change":"2024-04-17T04:36:29.374168+0000","last_active":"2024-04-17T05:41:45.136613+0000","last_peered":"2024-04-17T05:41:45.136613+0000","last_clean":"2024-04-17T05:41:45.136613+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:41:45.136613+0000","last_undegraded":"2024-04-17T05:41:45.136613+0000","last_fullsized":"2024-04-17T05:41:45.136613+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:22.063423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'308","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.158285+0000","last_change":"2024-04-17T04:36:31.392925+0000","last_active":"2024-04-17T05:44:05.158285+0000","last_peered":"2024-04-17T05:44:05.158285+0000","last_clean":"2024-04-17T05:44:05.158285+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:44:05.158285+0000","last_undegraded":"2024-04-17T05:44:05.158285+0000","last_fullsized":"2024-04-17T05:44:05.158285+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:08.285743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56427,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":522,"num_read_kb":1053,"num_write":466,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":494,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137350+0000","last_change":"2024-04-17T04:36:33.430909+0000","last_active":"2024-04-17T05:42:00.137350+0000","last_peered":"2024-04-17T05:42:00.137350+0000","last_clean":"2024-04-17T05:42:00.137350+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:42:00.137350+0000","last_undegraded":"2024-04-17T05:42:00.137350+0000","last_fullsized":"2024-04-17T05:42:00.137350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:54.144240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'129","reported_seq":636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.818089+0000","last_change":"2024-04-17T04:36:35.423084+0000","last_active":"2024-04-17T05:45:40.818089+0000","last_peered":"2024-04-17T05:45:40.818089+0000","last_clean":"2024-04-17T05:45:40.818089+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:45:40.818089+0000","last_undegraded":"2024-04-17T05:45:40.818089+0000","last_fullsized":"2024-04-17T05:45:40.818089+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:20:19.232177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:53.923395+0000","last_change":"2024-04-17T04:36:27.383527+0000","last_active":"2024-04-17T05:41:53.923395+0000","last_peered":"2024-04-17T05:41:53.923395+0000","last_clean":"2024-04-17T05:41:53.923395+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:41:53.923395+0000","last_undegraded":"2024-04-17T05:41:53.923395+0000","last_fullsized":"2024-04-17T05:41:53.923395+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:29.969137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'301","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159689+0000","last_change":"2024-04-17T04:36:25.371674+0000","last_active":"2024-04-17T05:43:45.159689+0000","last_peered":"2024-04-17T05:43:45.159689+0000","last_clean":"2024-04-17T05:43:45.159689+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:43:45.159689+0000","last_undegraded":"2024-04-17T05:43:45.159689+0000","last_fullsized":"2024-04-17T05:43:45.159689+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:54.700946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6845506,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":545,"num_read_kb":15072,"num_write":1194,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:29.359143+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:37.440920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'287","reported_seq":891,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.171451+0000","last_change":"2024-04-17T04:36:31.397866+0000","last_active":"2024-04-17T05:44:20.171451+0000","last_peered":"2024-04-17T05:44:20.171451+0000","last_clean":"2024-04-17T05:44:20.171451+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:44:20.171451+0000","last_undegraded":"2024-04-17T05:44:20.171451+0000","last_fullsized":"2024-04-17T05:44:20.171451+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:42.568179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51070,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":477,"num_read_kb":941,"num_write":374,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'508","reported_seq":2439,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.663877+0000","last_change":"2024-04-17T04:36:33.408560+0000","last_active":"2024-04-17T05:45:57.663877+0000","last_peered":"2024-04-17T05:45:57.663877+0000","last_clean":"2024-04-17T05:45:57.663877+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:45:57.663877+0000","last_undegraded":"2024-04-17T05:45:57.663877+0000","last_fullsized":"2024-04-17T05:45:57.663877+0000","mapping_epoch":31,"log_start":"261'400","ondisk_log_start":"261'400","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":400,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:41.906428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'96","reported_seq":602,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:42.569922+0000","last_change":"2024-04-17T04:36:35.428863+0000","last_active":"2024-04-17T05:45:42.569922+0000","last_peered":"2024-04-17T05:45:42.569922+0000","last_clean":"2024-04-17T05:45:42.569922+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:45:42.569922+0000","last_undegraded":"2024-04-17T05:45:42.569922+0000","last_fullsized":"2024-04-17T05:45:42.569922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:09.522109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.169748+0000","last_change":"2024-04-17T04:36:27.393058+0000","last_active":"2024-04-17T05:42:30.169748+0000","last_peered":"2024-04-17T05:42:30.169748+0000","last_clean":"2024-04-17T05:42:30.169748+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:42:30.169748+0000","last_undegraded":"2024-04-17T05:42:30.169748+0000","last_fullsized":"2024-04-17T05:42:30.169748+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:22.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'245","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:34.943708+0000","last_change":"2024-04-17T04:36:25.356100+0000","last_active":"2024-04-17T05:43:34.943708+0000","last_peered":"2024-04-17T05:43:34.943708+0000","last_clean":"2024-04-17T05:43:34.943708+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:43:34.943708+0000","last_undegraded":"2024-04-17T05:43:34.943708+0000","last_fullsized":"2024-04-17T05:43:34.943708+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:10.485745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10486600,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":525,"num_read_kb":5671,"num_write":1142,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"28'2","reported_seq":1150,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.527318+0000","last_change":"2024-04-17T04:36:29.389132+0000","last_active":"2024-04-17T05:45:54.527318+0000","last_peered":"2024-04-17T05:45:54.527318+0000","last_clean":"2024-04-17T05:45:54.527318+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:45:54.527318+0000","last_undegraded":"2024-04-17T05:45:54.527318+0000","last_fullsized":"2024-04-17T05:45:54.527318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:09:43.654615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'97","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166974+0000","last_change":"2024-04-17T04:36:31.421130+0000","last_active":"2024-04-17T05:41:50.166974+0000","last_peered":"2024-04-17T05:41:50.166974+0000","last_clean":"2024-04-17T05:41:50.166974+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:41:50.166974+0000","last_undegraded":"2024-04-17T05:41:50.166974+0000","last_fullsized":"2024-04-17T05:41:50.166974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:38.963408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'535","reported_seq":2243,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.671105+0000","last_change":"2024-04-17T04:36:33.421354+0000","last_active":"2024-04-17T05:45:57.671105+0000","last_peered":"2024-04-17T05:45:57.671105+0000","last_clean":"2024-04-17T05:45:57.671105+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:45:57.671105+0000","last_undegraded":"2024-04-17T05:45:57.671105+0000","last_fullsized":"2024-04-17T05:45:57.671105+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":500,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:27.382198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'97","reported_seq":600,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939559+0000","last_change":"2024-04-17T04:36:35.427123+0000","last_active":"2024-04-17T05:43:14.939559+0000","last_peered":"2024-04-17T05:43:14.939559+0000","last_clean":"2024-04-17T05:43:14.939559+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:43:14.939559+0000","last_undegraded":"2024-04-17T05:43:14.939559+0000","last_fullsized":"2024-04-17T05:43:14.939559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:26.157064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":756,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141817+0000","last_change":"2024-04-17T04:36:27.377947+0000","last_active":"2024-04-17T05:42:00.141817+0000","last_peered":"2024-04-17T05:42:00.141817+0000","last_clean":"2024-04-17T05:42:00.141817+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:42:00.141817+0000","last_undegraded":"2024-04-17T05:42:00.141817+0000","last_fullsized":"2024-04-17T05:42:00.141817+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:47.608940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'228","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.190870+0000","last_change":"2024-04-17T04:36:25.353720+0000","last_active":"2024-04-17T05:43:35.190870+0000","last_peered":"2024-04-17T05:43:35.190870+0000","last_clean":"2024-04-17T05:43:35.190870+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:43:35.190870+0000","last_undegraded":"2024-04-17T05:43:35.190870+0000","last_fullsized":"2024-04-17T05:43:35.190870+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:24.629387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15899766,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":342,"num_read_kb":36760,"num_write":925,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124238+0000","last_change":"2024-04-17T04:36:29.389735+0000","last_active":"2024-04-17T05:41:45.124238+0000","last_peered":"2024-04-17T05:41:45.124238+0000","last_clean":"2024-04-17T05:41:45.124238+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:41:45.124238+0000","last_undegraded":"2024-04-17T05:41:45.124238+0000","last_fullsized":"2024-04-17T05:41:45.124238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:19.695221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'347","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.169175+0000","last_change":"2024-04-17T04:36:31.398102+0000","last_active":"2024-04-17T05:44:05.169175+0000","last_peered":"2024-04-17T05:44:05.169175+0000","last_clean":"2024-04-17T05:44:05.169175+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:44:05.169175+0000","last_undegraded":"2024-04-17T05:44:05.169175+0000","last_fullsized":"2024-04-17T05:44:05.169175+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:41.468361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":568,"num_read_kb":1054,"num_write":442,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134628+0000","last_change":"2024-04-17T04:36:33.419643+0000","last_active":"2024-04-17T05:41:50.134628+0000","last_peered":"2024-04-17T05:41:50.134628+0000","last_clean":"2024-04-17T05:41:50.134628+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:41:50.134628+0000","last_undegraded":"2024-04-17T05:41:50.134628+0000","last_fullsized":"2024-04-17T05:41:50.134628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:51.100488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'157","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.190550+0000","last_change":"2024-04-17T04:36:35.408247+0000","last_active":"2024-04-17T05:45:20.190550+0000","last_peered":"2024-04-17T05:45:20.190550+0000","last_clean":"2024-04-17T05:45:20.190550+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:45:20.190550+0000","last_undegraded":"2024-04-17T05:45:20.190550+0000","last_fullsized":"2024-04-17T05:45:20.190550+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:26.932136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":695,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.154250+0000","last_change":"2024-04-17T04:36:27.387349+0000","last_active":"2024-04-17T05:43:20.154250+0000","last_peered":"2024-04-17T05:43:20.154250+0000","last_clean":"2024-04-17T05:43:20.154250+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:43:20.154250+0000","last_undegraded":"2024-04-17T05:43:20.154250+0000","last_fullsized":"2024-04-17T05:43:20.154250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:39.509316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":680,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141161+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T05:42:25.141161+0000","last_peered":"2024-04-17T05:42:25.141161+0000","last_clean":"2024-04-17T05:42:25.141161+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:42:25.141161+0000","last_undegraded":"2024-04-17T05:42:25.141161+0000","last_fullsized":"2024-04-17T05:42:25.141161+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:24:00.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154818+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T05:43:00.154818+0000","last_peered":"2024-04-17T05:43:00.154818+0000","last_clean":"2024-04-17T05:43:00.154818+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:43:00.154818+0000","last_undegraded":"2024-04-17T05:43:00.154818+0000","last_fullsized":"2024-04-17T05:43:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:13.926220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'276","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.184874+0000","last_change":"2024-04-17T04:36:25.364067+0000","last_active":"2024-04-17T05:43:05.184874+0000","last_peered":"2024-04-17T05:43:05.184874+0000","last_clean":"2024-04-17T05:43:05.184874+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:43:05.184874+0000","last_undegraded":"2024-04-17T05:43:05.184874+0000","last_fullsized":"2024-04-17T05:43:05.184874+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:41.272537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346866,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":496,"num_read_kb":2736,"num_write":1164,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151134+0000","last_change":"2024-04-17T04:36:29.381453+0000","last_active":"2024-04-17T05:43:10.151134+0000","last_peered":"2024-04-17T05:43:10.151134+0000","last_clean":"2024-04-17T05:43:10.151134+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:43:10.151134+0000","last_undegraded":"2024-04-17T05:43:10.151134+0000","last_fullsized":"2024-04-17T05:43:10.151134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:32.455502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'709","reported_seq":1415,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.648327+0000","last_change":"2024-04-17T04:36:31.400231+0000","last_active":"2024-04-17T05:45:54.648327+0000","last_peered":"2024-04-17T05:45:54.648327+0000","last_clean":"2024-04-17T05:45:54.648327+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:45:54.648327+0000","last_undegraded":"2024-04-17T05:45:54.648327+0000","last_fullsized":"2024-04-17T05:45:54.648327+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":600,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:02.459768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155640,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1372,"num_read_kb":2849,"num_write":1330,"num_write_kb":1169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161079+0000","last_change":"2024-04-17T04:36:33.430394+0000","last_active":"2024-04-17T05:43:45.161079+0000","last_peered":"2024-04-17T05:43:45.161079+0000","last_clean":"2024-04-17T05:43:45.161079+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:43:45.161079+0000","last_undegraded":"2024-04-17T05:43:45.161079+0000","last_fullsized":"2024-04-17T05:43:45.161079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:45.115863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'220","reported_seq":913,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.519743+0000","last_change":"2024-04-17T04:36:35.421183+0000","last_active":"2024-04-17T05:45:46.519743+0000","last_peered":"2024-04-17T05:45:46.519743+0000","last_clean":"2024-04-17T05:45:46.519743+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:45:46.519743+0000","last_undegraded":"2024-04-17T05:45:46.519743+0000","last_fullsized":"2024-04-17T05:45:46.519743+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:30.565325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159648+0000","last_change":"2024-04-17T04:36:27.365454+0000","last_active":"2024-04-17T05:43:40.159648+0000","last_peered":"2024-04-17T05:43:40.159648+0000","last_clean":"2024-04-17T05:43:40.159648+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:43:40.159648+0000","last_undegraded":"2024-04-17T05:43:40.159648+0000","last_fullsized":"2024-04-17T05:43:40.159648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:16.200473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":524,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142294+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T05:42:30.142294+0000","last_peered":"2024-04-17T05:42:30.142294+0000","last_clean":"2024-04-17T05:42:30.142294+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:42:30.142294+0000","last_undegraded":"2024-04-17T05:42:30.142294+0000","last_fullsized":"2024-04-17T05:42:30.142294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:06.659093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'280","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151677+0000","last_change":"2024-04-17T04:36:25.355561+0000","last_active":"2024-04-17T05:43:05.151677+0000","last_peered":"2024-04-17T05:43:05.151677+0000","last_clean":"2024-04-17T05:43:05.151677+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:43:05.151677+0000","last_undegraded":"2024-04-17T05:43:05.151677+0000","last_fullsized":"2024-04-17T05:43:05.151677+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:21.131078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12803559,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":507,"num_read_kb":3714,"num_write":1174,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163722+0000","last_change":"2024-04-17T04:36:29.393231+0000","last_active":"2024-04-17T05:43:40.163722+0000","last_peered":"2024-04-17T05:43:40.163722+0000","last_clean":"2024-04-17T05:43:40.163722+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:43:40.163722+0000","last_undegraded":"2024-04-17T05:43:40.163722+0000","last_fullsized":"2024-04-17T05:43:40.163722+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:30.045764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1084","reported_seq":2786,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.664430+0000","last_change":"2024-04-17T04:36:31.398322+0000","last_active":"2024-04-17T05:45:57.664430+0000","last_peered":"2024-04-17T05:45:57.664430+0000","last_clean":"2024-04-17T05:45:57.664430+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:45:57.664430+0000","last_undegraded":"2024-04-17T05:45:57.664430+0000","last_fullsized":"2024-04-17T05:45:57.664430+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":1000,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.046142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":178612,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1777,"num_read_kb":3331,"num_write":1104,"num_write_kb":931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154560+0000","last_change":"2024-04-17T04:36:33.422428+0000","last_active":"2024-04-17T05:43:25.154560+0000","last_peered":"2024-04-17T05:43:25.154560+0000","last_clean":"2024-04-17T05:43:25.154560+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:43:25.154560+0000","last_undegraded":"2024-04-17T05:43:25.154560+0000","last_fullsized":"2024-04-17T05:43:25.154560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:00.479531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'112","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.389082+0000","last_change":"2024-04-17T04:36:35.402669+0000","last_active":"2024-04-17T05:45:46.389082+0000","last_peered":"2024-04-17T05:45:46.389082+0000","last_clean":"2024-04-17T05:45:46.389082+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:45:46.389082+0000","last_undegraded":"2024-04-17T05:45:46.389082+0000","last_fullsized":"2024-04-17T05:45:46.389082+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:05.019981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164737+0000","last_change":"2024-04-17T04:36:27.376154+0000","last_active":"2024-04-17T05:43:40.164737+0000","last_peered":"2024-04-17T05:43:40.164737+0000","last_clean":"2024-04-17T05:43:40.164737+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:43:40.164737+0000","last_undegraded":"2024-04-17T05:43:40.164737+0000","last_fullsized":"2024-04-17T05:43:40.164737+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:53.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":530,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.151409+0000","last_change":"2024-04-17T04:36:17.752374+0000","last_active":"2024-04-17T05:42:55.151409+0000","last_peered":"2024-04-17T05:42:55.151409+0000","last_clean":"2024-04-17T05:42:55.151409+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:42:55.151409+0000","last_undegraded":"2024-04-17T05:42:55.151409+0000","last_fullsized":"2024-04-17T05:42:55.151409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:41.044005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'243","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150111+0000","last_change":"2024-04-17T04:36:25.343663+0000","last_active":"2024-04-17T05:42:40.150111+0000","last_peered":"2024-04-17T05:42:40.150111+0000","last_clean":"2024-04-17T05:42:40.150111+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:42:40.150111+0000","last_undegraded":"2024-04-17T05:42:40.150111+0000","last_fullsized":"2024-04-17T05:42:40.150111+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:49.114178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5703040,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":424,"num_read_kb":21971,"num_write":1062,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"28'1","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162452+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:43:40.162452+0000","last_peered":"2024-04-17T05:43:40.162452+0000","last_clean":"2024-04-17T05:43:40.162452+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:43:40.162452+0000","last_undegraded":"2024-04-17T05:43:40.162452+0000","last_fullsized":"2024-04-17T05:43:40.162452+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:01.144890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'234","reported_seq":1618,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:52.541681+0000","last_change":"2024-04-17T04:36:31.427048+0000","last_active":"2024-04-17T05:45:52.541681+0000","last_peered":"2024-04-17T05:45:52.541681+0000","last_clean":"2024-04-17T05:45:52.541681+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:45:52.541681+0000","last_undegraded":"2024-04-17T05:45:52.541681+0000","last_fullsized":"2024-04-17T05:45:52.541681+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:49:05.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153957+0000","last_change":"2024-04-17T04:36:33.426587+0000","last_active":"2024-04-17T05:43:15.153957+0000","last_peered":"2024-04-17T05:43:15.153957+0000","last_clean":"2024-04-17T05:43:15.153957+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:43:15.153957+0000","last_undegraded":"2024-04-17T05:43:15.153957+0000","last_fullsized":"2024-04-17T05:43:15.153957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:52:34.456097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'262","reported_seq":866,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.184123+0000","last_change":"2024-04-17T04:36:35.408138+0000","last_active":"2024-04-17T05:45:45.184123+0000","last_peered":"2024-04-17T05:45:45.184123+0000","last_clean":"2024-04-17T05:45:45.184123+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:45:45.184123+0000","last_undegraded":"2024-04-17T05:45:45.184123+0000","last_fullsized":"2024-04-17T05:45:45.184123+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:09.321611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158227+0000","last_change":"2024-04-17T04:36:27.361706+0000","last_active":"2024-04-17T05:43:10.158227+0000","last_peered":"2024-04-17T05:43:10.158227+0000","last_clean":"2024-04-17T05:43:10.158227+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:43:10.158227+0000","last_undegraded":"2024-04-17T05:43:10.158227+0000","last_fullsized":"2024-04-17T05:43:10.158227+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:25.186757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154761+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T05:43:00.154761+0000","last_peered":"2024-04-17T05:43:00.154761+0000","last_clean":"2024-04-17T05:43:00.154761+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:43:00.154761+0000","last_undegraded":"2024-04-17T05:43:00.154761+0000","last_fullsized":"2024-04-17T05:43:00.154761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:44:32.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'278","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:04.937303+0000","last_change":"2024-04-17T04:36:25.356426+0000","last_active":"2024-04-17T05:43:04.937303+0000","last_peered":"2024-04-17T05:43:04.937303+0000","last_clean":"2024-04-17T05:43:04.937303+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:43:04.937303+0000","last_undegraded":"2024-04-17T05:43:04.937303+0000","last_fullsized":"2024-04-17T05:43:04.937303+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:48.430125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11586207,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":563,"num_read_kb":36553,"num_write":1174,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.155027+0000","last_change":"2024-04-17T04:36:29.371781+0000","last_active":"2024-04-17T05:43:20.155027+0000","last_peered":"2024-04-17T05:43:20.155027+0000","last_clean":"2024-04-17T05:43:20.155027+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:43:20.155027+0000","last_undegraded":"2024-04-17T05:43:20.155027+0000","last_fullsized":"2024-04-17T05:43:20.155027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:40:17.404012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'139","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193023+0000","last_change":"2024-04-17T04:36:31.400012+0000","last_active":"2024-04-17T05:43:45.193023+0000","last_peered":"2024-04-17T05:43:45.193023+0000","last_clean":"2024-04-17T05:43:45.193023+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:43:45.193023+0000","last_undegraded":"2024-04-17T05:43:45.193023+0000","last_fullsized":"2024-04-17T05:43:45.193023+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:30:23.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159837+0000","last_change":"2024-04-17T04:36:33.416438+0000","last_active":"2024-04-17T05:43:45.159837+0000","last_peered":"2024-04-17T05:43:45.159837+0000","last_clean":"2024-04-17T05:43:45.159837+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:43:45.159837+0000","last_undegraded":"2024-04-17T05:43:45.159837+0000","last_fullsized":"2024-04-17T05:43:45.159837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:15:47.410813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'130","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.164559+0000","last_change":"2024-04-17T04:36:35.411210+0000","last_active":"2024-04-17T05:43:45.164559+0000","last_peered":"2024-04-17T05:43:45.164559+0000","last_clean":"2024-04-17T05:43:45.164559+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:43:45.164559+0000","last_undegraded":"2024-04-17T05:43:45.164559+0000","last_fullsized":"2024-04-17T05:43:45.164559+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:26.503048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158134+0000","last_change":"2024-04-17T04:36:27.376368+0000","last_active":"2024-04-17T05:43:10.158134+0000","last_peered":"2024-04-17T05:43:10.158134+0000","last_clean":"2024-04-17T05:43:10.158134+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:43:10.158134+0000","last_undegraded":"2024-04-17T05:43:10.158134+0000","last_fullsized":"2024-04-17T05:43:10.158134+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:23:39.170608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'231","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.159243+0000","last_change":"2024-04-17T04:36:25.368624+0000","last_active":"2024-04-17T05:43:20.159243+0000","last_peered":"2024-04-17T05:43:20.159243+0000","last_clean":"2024-04-17T05:43:20.159243+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:43:20.159243+0000","last_undegraded":"2024-04-17T05:43:20.159243+0000","last_fullsized":"2024-04-17T05:43:20.159243+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:51.630737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84605,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":406,"num_read_kb":1448,"num_write":1062,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.156128+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T05:43:00.156128+0000","last_peered":"2024-04-17T05:43:00.156128+0000","last_clean":"2024-04-17T05:43:00.156128+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:43:00.156128+0000","last_undegraded":"2024-04-17T05:43:00.156128+0000","last_fullsized":"2024-04-17T05:43:00.156128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:16.202650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152928+0000","last_change":"2024-04-17T04:36:29.384820+0000","last_active":"2024-04-17T05:43:10.152928+0000","last_peered":"2024-04-17T05:43:10.152928+0000","last_clean":"2024-04-17T05:43:10.152928+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:43:10.152928+0000","last_undegraded":"2024-04-17T05:43:10.152928+0000","last_fullsized":"2024-04-17T05:43:10.152928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:27.497471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'610","reported_seq":1296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.603615+0000","last_change":"2024-04-17T04:36:31.411434+0000","last_active":"2024-04-17T05:45:54.603615+0000","last_peered":"2024-04-17T05:45:54.603615+0000","last_clean":"2024-04-17T05:45:54.603615+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:45:54.603615+0000","last_undegraded":"2024-04-17T05:45:54.603615+0000","last_fullsized":"2024-04-17T05:45:54.603615+0000","mapping_epoch":29,"log_start":"264'500","ondisk_log_start":"264'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:30.011121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101724,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1633,"num_read_kb":2600,"num_write":2162,"num_write_kb":1969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939512+0000","last_change":"2024-04-17T04:36:33.407911+0000","last_active":"2024-04-17T05:43:14.939512+0000","last_peered":"2024-04-17T05:43:14.939512+0000","last_clean":"2024-04-17T05:43:14.939512+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:43:14.939512+0000","last_undegraded":"2024-04-17T05:43:14.939512+0000","last_fullsized":"2024-04-17T05:43:14.939512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:49:35.539455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'82","reported_seq":584,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.166476+0000","last_change":"2024-04-17T04:36:35.429331+0000","last_active":"2024-04-17T05:44:05.166476+0000","last_peered":"2024-04-17T05:44:05.166476+0000","last_clean":"2024-04-17T05:44:05.166476+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:44:05.166476+0000","last_undegraded":"2024-04-17T05:44:05.166476+0000","last_fullsized":"2024-04-17T05:44:05.166476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:03.344843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":592,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152888+0000","last_change":"2024-04-17T04:36:27.376396+0000","last_active":"2024-04-17T05:43:10.152888+0000","last_peered":"2024-04-17T05:43:10.152888+0000","last_clean":"2024-04-17T05:43:10.152888+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:43:10.152888+0000","last_undegraded":"2024-04-17T05:43:10.152888+0000","last_fullsized":"2024-04-17T05:43:10.152888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:36.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'203","reported_seq":824,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157227+0000","last_change":"2024-04-17T04:36:25.345048+0000","last_active":"2024-04-17T05:43:05.157227+0000","last_peered":"2024-04-17T05:43:05.157227+0000","last_clean":"2024-04-17T05:43:05.157227+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:43:05.157227+0000","last_undegraded":"2024-04-17T05:43:05.157227+0000","last_fullsized":"2024-04-17T05:43:05.157227+0000","mapping_epoch":23,"log_start":"264'100","ondisk_log_start":"264'100","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:40:34.714988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9471022,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":330,"num_read_kb":35211,"num_write":775,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":523,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151170+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T05:43:10.151170+0000","last_peered":"2024-04-17T05:43:10.151170+0000","last_clean":"2024-04-17T05:43:10.151170+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:43:10.151170+0000","last_undegraded":"2024-04-17T05:43:10.151170+0000","last_fullsized":"2024-04-17T05:43:10.151170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:33.083623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151795+0000","last_change":"2024-04-17T04:36:29.375400+0000","last_active":"2024-04-17T05:43:10.151795+0000","last_peered":"2024-04-17T05:43:10.151795+0000","last_clean":"2024-04-17T05:43:10.151795+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:43:10.151795+0000","last_undegraded":"2024-04-17T05:43:10.151795+0000","last_fullsized":"2024-04-17T05:43:10.151795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:16.669890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'199","reported_seq":792,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.159277+0000","last_change":"2024-04-17T04:36:31.420876+0000","last_active":"2024-04-17T05:43:15.159277+0000","last_peered":"2024-04-17T05:43:15.159277+0000","last_clean":"2024-04-17T05:43:15.159277+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:43:15.159277+0000","last_undegraded":"2024-04-17T05:43:15.159277+0000","last_fullsized":"2024-04-17T05:43:15.159277+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:23:18.629160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'536","reported_seq":4354,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.668605+0000","last_change":"2024-04-17T04:36:33.407900+0000","last_active":"2024-04-17T05:45:57.668605+0000","last_peered":"2024-04-17T05:45:57.668605+0000","last_clean":"2024-04-17T05:45:57.668605+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:45:57.668605+0000","last_undegraded":"2024-04-17T05:45:57.668605+0000","last_fullsized":"2024-04-17T05:45:57.668605+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:36:50.014322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'156","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.202363+0000","last_change":"2024-04-17T04:36:35.418551+0000","last_active":"2024-04-17T05:44:20.202363+0000","last_peered":"2024-04-17T05:44:20.202363+0000","last_clean":"2024-04-17T05:44:20.202363+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:44:20.202363+0000","last_undegraded":"2024-04-17T05:44:20.202363+0000","last_fullsized":"2024-04-17T05:44:20.202363+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:08.177362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":799,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180216+0000","last_change":"2024-04-17T04:36:27.395791+0000","last_active":"2024-04-17T05:42:45.180216+0000","last_peered":"2024-04-17T05:42:45.180216+0000","last_clean":"2024-04-17T05:42:45.180216+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:42:45.180216+0000","last_undegraded":"2024-04-17T05:42:45.180216+0000","last_fullsized":"2024-04-17T05:42:45.180216+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:29:33.119512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'309","reported_seq":966,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191121+0000","last_change":"2024-04-17T04:36:25.347483+0000","last_active":"2024-04-17T05:43:35.191121+0000","last_peered":"2024-04-17T05:43:35.191121+0000","last_clean":"2024-04-17T05:43:35.191121+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:43:35.191121+0000","last_undegraded":"2024-04-17T05:43:35.191121+0000","last_fullsized":"2024-04-17T05:43:35.191121+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:52:07.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290300,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":521,"num_read_kb":5842,"num_write":1316,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":527,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.161303+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T05:43:25.161303+0000","last_peered":"2024-04-17T05:43:25.161303+0000","last_clean":"2024-04-17T05:43:25.161303+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:43:25.161303+0000","last_undegraded":"2024-04-17T05:43:25.161303+0000","last_fullsized":"2024-04-17T05:43:25.161303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:38.757105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938398+0000","last_change":"2024-04-17T04:36:29.375601+0000","last_active":"2024-04-17T05:43:09.938398+0000","last_peered":"2024-04-17T05:43:09.938398+0000","last_clean":"2024-04-17T05:43:09.938398+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:43:09.938398+0000","last_undegraded":"2024-04-17T05:43:09.938398+0000","last_fullsized":"2024-04-17T05:43:09.938398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:57.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'136","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939607+0000","last_change":"2024-04-17T04:36:31.400243+0000","last_active":"2024-04-17T05:43:14.939607+0000","last_peered":"2024-04-17T05:43:14.939607+0000","last_clean":"2024-04-17T05:43:14.939607+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:43:14.939607+0000","last_undegraded":"2024-04-17T05:43:14.939607+0000","last_fullsized":"2024-04-17T05:43:14.939607+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:41.986999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'528","reported_seq":2299,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.662141+0000","last_change":"2024-04-17T04:36:33.421012+0000","last_active":"2024-04-17T05:45:57.662141+0000","last_peered":"2024-04-17T05:45:57.662141+0000","last_clean":"2024-04-17T05:45:57.662141+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:45:57.662141+0000","last_undegraded":"2024-04-17T05:45:57.662141+0000","last_fullsized":"2024-04-17T05:45:57.662141+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":500,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:52.921022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'218","reported_seq":734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.976785+0000","last_change":"2024-04-17T04:36:35.431849+0000","last_active":"2024-04-17T05:45:45.976785+0000","last_peered":"2024-04-17T05:45:45.976785+0000","last_clean":"2024-04-17T05:45:45.976785+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:45:45.976785+0000","last_undegraded":"2024-04-17T05:45:45.976785+0000","last_fullsized":"2024-04-17T05:45:45.976785+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:18.340086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4093","reported_seq":4724,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938365+0000","last_change":"2024-04-17T04:36:27.378432+0000","last_active":"2024-04-17T05:43:09.938365+0000","last_peered":"2024-04-17T05:43:09.938365+0000","last_clean":"2024-04-17T05:43:09.938365+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:43:09.938365+0000","last_undegraded":"2024-04-17T05:43:09.938365+0000","last_fullsized":"2024-04-17T05:43:09.938365+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":3000,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:55.591614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'291","reported_seq":908,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158359+0000","last_change":"2024-04-17T04:36:25.354707+0000","last_active":"2024-04-17T05:43:35.158359+0000","last_peered":"2024-04-17T05:43:35.158359+0000","last_clean":"2024-04-17T05:43:35.158359+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:43:35.158359+0000","last_undegraded":"2024-04-17T05:43:35.158359+0000","last_fullsized":"2024-04-17T05:43:35.158359+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:46.297295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4228470,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":401,"num_read_kb":7519,"num_write":1219,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.156524+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T05:43:30.156524+0000","last_peered":"2024-04-17T05:43:30.156524+0000","last_clean":"2024-04-17T05:43:30.156524+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:43:30.156524+0000","last_undegraded":"2024-04-17T05:43:30.156524+0000","last_fullsized":"2024-04-17T05:43:30.156524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:24.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180151+0000","last_change":"2024-04-17T04:36:29.363086+0000","last_active":"2024-04-17T05:42:45.180151+0000","last_peered":"2024-04-17T05:42:45.180151+0000","last_clean":"2024-04-17T05:42:45.180151+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:42:45.180151+0000","last_undegraded":"2024-04-17T05:42:45.180151+0000","last_fullsized":"2024-04-17T05:42:45.180151+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:48.241347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'715","reported_seq":1444,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.600048+0000","last_change":"2024-04-17T04:36:31.422637+0000","last_active":"2024-04-17T05:45:54.600048+0000","last_peered":"2024-04-17T05:45:54.600048+0000","last_clean":"2024-04-17T05:45:54.600048+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:45:54.600048+0000","last_undegraded":"2024-04-17T05:45:54.600048+0000","last_fullsized":"2024-04-17T05:45:54.600048+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":700,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:37.277476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156924,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1257,"num_read_kb":2730,"num_write":1020,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187164+0000","last_change":"2024-04-17T04:36:33.415987+0000","last_active":"2024-04-17T05:43:15.187164+0000","last_peered":"2024-04-17T05:43:15.187164+0000","last_clean":"2024-04-17T05:43:15.187164+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:43:15.187164+0000","last_undegraded":"2024-04-17T05:43:15.187164+0000","last_fullsized":"2024-04-17T05:43:15.187164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:25.307133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'102","reported_seq":591,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.397345+0000","last_change":"2024-04-17T04:36:35.424792+0000","last_active":"2024-04-17T05:42:25.397345+0000","last_peered":"2024-04-17T05:42:25.397345+0000","last_clean":"2024-04-17T05:42:25.397345+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:42:25.397345+0000","last_undegraded":"2024-04-17T05:42:25.397345+0000","last_fullsized":"2024-04-17T05:42:25.397345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:27.783198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'149","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.086388+0000","last_change":"2024-04-17T04:36:35.425561+0000","last_active":"2024-04-17T05:45:45.086388+0000","last_peered":"2024-04-17T05:45:45.086388+0000","last_clean":"2024-04-17T05:45:45.086388+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:45:45.086388+0000","last_undegraded":"2024-04-17T05:45:45.086388+0000","last_fullsized":"2024-04-17T05:45:45.086388+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:48.063997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'73","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125289+0000","last_change":"2024-04-17T04:36:27.400403+0000","last_active":"2024-04-17T05:41:50.125289+0000","last_peered":"2024-04-17T05:41:50.125289+0000","last_clean":"2024-04-17T05:41:50.125289+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:41:50.125289+0000","last_undegraded":"2024-04-17T05:41:50.125289+0000","last_fullsized":"2024-04-17T05:41:50.125289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:03.426737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'220","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.706350+0000","last_change":"2024-04-17T04:36:25.357052+0000","last_active":"2024-04-17T05:42:50.706350+0000","last_peered":"2024-04-17T05:42:50.706350+0000","last_clean":"2024-04-17T05:42:50.706350+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:42:50.706350+0000","last_undegraded":"2024-04-17T05:42:50.706350+0000","last_fullsized":"2024-04-17T05:42:50.706350+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:07:39.562019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502965,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":370,"num_read_kb":11105,"num_write":918,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139292+0000","last_change":"2024-04-17T04:36:29.391641+0000","last_active":"2024-04-17T05:41:50.139292+0000","last_peered":"2024-04-17T05:41:50.139292+0000","last_clean":"2024-04-17T05:41:50.139292+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:41:50.139292+0000","last_undegraded":"2024-04-17T05:41:50.139292+0000","last_fullsized":"2024-04-17T05:41:50.139292+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:29.991489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'296","reported_seq":886,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.200972+0000","last_change":"2024-04-17T04:36:31.406561+0000","last_active":"2024-04-17T05:44:15.200972+0000","last_peered":"2024-04-17T05:44:15.200972+0000","last_clean":"2024-04-17T05:44:15.200972+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:44:15.200972+0000","last_undegraded":"2024-04-17T05:44:15.200972+0000","last_fullsized":"2024-04-17T05:44:15.200972+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:50:50.884998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62692,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":511,"num_read_kb":1084,"num_write":418,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168016+0000","last_change":"2024-04-17T04:36:33.420539+0000","last_active":"2024-04-17T05:41:55.168016+0000","last_peered":"2024-04-17T05:41:55.168016+0000","last_clean":"2024-04-17T05:41:55.168016+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:41:55.168016+0000","last_undegraded":"2024-04-17T05:41:55.168016+0000","last_fullsized":"2024-04-17T05:41:55.168016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:42.375895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'98","reported_seq":603,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.713280+0000","last_change":"2024-04-17T04:36:35.430479+0000","last_active":"2024-04-17T05:45:44.713280+0000","last_peered":"2024-04-17T05:45:44.713280+0000","last_clean":"2024-04-17T05:45:44.713280+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:45:44.713280+0000","last_undegraded":"2024-04-17T05:45:44.713280+0000","last_fullsized":"2024-04-17T05:45:44.713280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:21:02.348756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'126","reported_seq":737,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:17.594052+0000","last_change":"2024-04-17T04:36:27.389126+0000","last_active":"2024-04-17T05:42:17.594052+0000","last_peered":"2024-04-17T05:42:17.594052+0000","last_clean":"2024-04-17T05:42:17.594052+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:42:17.594052+0000","last_undegraded":"2024-04-17T05:42:17.594052+0000","last_fullsized":"2024-04-17T05:42:17.594052+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:40.137919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'309","reported_seq":961,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144594+0000","last_change":"2024-04-17T04:37:03.635652+0000","last_active":"2024-04-17T05:42:20.144594+0000","last_peered":"2024-04-17T05:42:20.144594+0000","last_clean":"2024-04-17T05:42:20.144594+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:42:20.144594+0000","last_undegraded":"2024-04-17T05:42:20.144594+0000","last_fullsized":"2024-04-17T05:42:20.144594+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:33.722634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3797882,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":561,"num_read_kb":15946,"num_write":1479,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166897+0000","last_change":"2024-04-17T04:36:29.371357+0000","last_active":"2024-04-17T05:41:50.166897+0000","last_peered":"2024-04-17T05:41:50.166897+0000","last_clean":"2024-04-17T05:41:50.166897+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:41:50.166897+0000","last_undegraded":"2024-04-17T05:41:50.166897+0000","last_fullsized":"2024-04-17T05:41:50.166897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:50.443422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'56","reported_seq":567,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168798+0000","last_change":"2024-04-17T04:36:31.404202+0000","last_active":"2024-04-17T05:44:15.168798+0000","last_peered":"2024-04-17T05:44:15.168798+0000","last_clean":"2024-04-17T05:44:15.168798+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:44:15.168798+0000","last_undegraded":"2024-04-17T05:44:15.168798+0000","last_fullsized":"2024-04-17T05:44:15.168798+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:41.388160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134552+0000","last_change":"2024-04-17T04:36:33.416154+0000","last_active":"2024-04-17T05:41:55.134552+0000","last_peered":"2024-04-17T05:41:55.134552+0000","last_clean":"2024-04-17T05:41:55.134552+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:41:55.134552+0000","last_undegraded":"2024-04-17T05:41:55.134552+0000","last_fullsized":"2024-04-17T05:41:55.134552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:21.721127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'217","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.789782+0000","last_change":"2024-04-17T04:36:35.428428+0000","last_active":"2024-04-17T05:45:44.789782+0000","last_peered":"2024-04-17T05:45:44.789782+0000","last_clean":"2024-04-17T05:45:44.789782+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:45:44.789782+0000","last_undegraded":"2024-04-17T05:45:44.789782+0000","last_fullsized":"2024-04-17T05:45:44.789782+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:55.027046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166680+0000","last_change":"2024-04-17T04:36:27.395464+0000","last_active":"2024-04-17T05:41:50.166680+0000","last_peered":"2024-04-17T05:41:50.166680+0000","last_clean":"2024-04-17T05:41:50.166680+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:41:50.166680+0000","last_undegraded":"2024-04-17T05:41:50.166680+0000","last_fullsized":"2024-04-17T05:41:50.166680+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:23.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'286","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:43.075764+0000","last_change":"2024-04-17T04:36:25.338779+0000","last_active":"2024-04-17T05:42:43.075764+0000","last_peered":"2024-04-17T05:42:43.075764+0000","last_clean":"2024-04-17T05:42:43.075764+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:42:43.075764+0000","last_undegraded":"2024-04-17T05:42:43.075764+0000","last_fullsized":"2024-04-17T05:42:43.075764+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:59.502139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56924,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":518,"num_read_kb":11343,"num_write":1233,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133249+0000","last_change":"2024-04-17T04:36:29.381180+0000","last_active":"2024-04-17T05:41:50.133249+0000","last_peered":"2024-04-17T05:41:50.133249+0000","last_clean":"2024-04-17T05:41:50.133249+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:41:50.133249+0000","last_undegraded":"2024-04-17T05:41:50.133249+0000","last_fullsized":"2024-04-17T05:41:50.133249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:01.625306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'335","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.201110+0000","last_change":"2024-04-17T04:36:31.406900+0000","last_active":"2024-04-17T05:44:15.201110+0000","last_peered":"2024-04-17T05:44:15.201110+0000","last_clean":"2024-04-17T05:44:15.201110+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:44:15.201110+0000","last_undegraded":"2024-04-17T05:44:15.201110+0000","last_fullsized":"2024-04-17T05:44:15.201110+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:47.400810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62421,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":539,"num_read_kb":1090,"num_write":434,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":490,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135689+0000","last_change":"2024-04-17T04:36:33.412913+0000","last_active":"2024-04-17T05:41:55.135689+0000","last_peered":"2024-04-17T05:41:55.135689+0000","last_clean":"2024-04-17T05:41:55.135689+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:41:55.135689+0000","last_undegraded":"2024-04-17T05:41:55.135689+0000","last_fullsized":"2024-04-17T05:41:55.135689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:56.825814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'112","reported_seq":599,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.542927+0000","last_change":"2024-04-17T04:36:35.424722+0000","last_active":"2024-04-17T05:41:51.542927+0000","last_peered":"2024-04-17T05:41:51.542927+0000","last_clean":"2024-04-17T05:41:51.542927+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:41:51.542927+0000","last_undegraded":"2024-04-17T05:41:51.542927+0000","last_fullsized":"2024-04-17T05:41:51.542927+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:15:26.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":759,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166609+0000","last_change":"2024-04-17T04:36:27.378417+0000","last_active":"2024-04-17T05:41:50.166609+0000","last_peered":"2024-04-17T05:41:50.166609+0000","last_clean":"2024-04-17T05:41:50.166609+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:41:50.166609+0000","last_undegraded":"2024-04-17T05:41:50.166609+0000","last_fullsized":"2024-04-17T05:41:50.166609+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:17.606187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'245","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.397517+0000","last_change":"2024-04-17T04:36:25.344700+0000","last_active":"2024-04-17T05:43:39.397517+0000","last_peered":"2024-04-17T05:43:39.397517+0000","last_clean":"2024-04-17T05:43:39.397517+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:43:39.397517+0000","last_undegraded":"2024-04-17T05:43:39.397517+0000","last_fullsized":"2024-04-17T05:43:39.397517+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:28:58.108460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18113654,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":380,"num_read_kb":84514,"num_write":925,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134534+0000","last_change":"2024-04-17T04:36:29.384475+0000","last_active":"2024-04-17T05:41:50.134534+0000","last_peered":"2024-04-17T05:41:50.134534+0000","last_clean":"2024-04-17T05:41:50.134534+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:41:50.134534+0000","last_undegraded":"2024-04-17T05:41:50.134534+0000","last_fullsized":"2024-04-17T05:41:50.134534+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:57.583947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'253","reported_seq":828,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166282+0000","last_change":"2024-04-17T04:36:31.380383+0000","last_active":"2024-04-17T05:44:15.166282+0000","last_peered":"2024-04-17T05:44:15.166282+0000","last_clean":"2024-04-17T05:44:15.166282+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:44:15.166282+0000","last_undegraded":"2024-04-17T05:44:15.166282+0000","last_fullsized":"2024-04-17T05:44:15.166282+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:45.175550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51759,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":419,"num_read_kb":889,"num_write":336,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136027+0000","last_change":"2024-04-17T04:36:33.429804+0000","last_active":"2024-04-17T05:41:55.136027+0000","last_peered":"2024-04-17T05:41:55.136027+0000","last_clean":"2024-04-17T05:41:55.136027+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:41:55.136027+0000","last_undegraded":"2024-04-17T05:41:55.136027+0000","last_fullsized":"2024-04-17T05:41:55.136027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:10.357384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'289","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151938+0000","last_change":"2024-04-17T04:36:27.387575+0000","last_active":"2024-04-17T05:43:10.151938+0000","last_peered":"2024-04-17T05:43:10.151938+0000","last_clean":"2024-04-17T05:43:10.151938+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:43:10.151938+0000","last_undegraded":"2024-04-17T05:43:10.151938+0000","last_fullsized":"2024-04-17T05:43:10.151938+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:05.627702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'242","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164601+0000","last_change":"2024-04-17T04:36:25.326760+0000","last_active":"2024-04-17T05:41:40.164601+0000","last_peered":"2024-04-17T05:41:40.164601+0000","last_clean":"2024-04-17T05:41:40.164601+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:41:40.164601+0000","last_undegraded":"2024-04-17T05:41:40.164601+0000","last_fullsized":"2024-04-17T05:41:40.164601+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:25.101546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7321484,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":471,"num_read_kb":33047,"num_write":1108,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146569+0000","last_change":"2024-04-17T04:36:29.373251+0000","last_active":"2024-04-17T05:42:40.146569+0000","last_peered":"2024-04-17T05:42:40.146569+0000","last_clean":"2024-04-17T05:42:40.146569+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:42:40.146569+0000","last_undegraded":"2024-04-17T05:42:40.146569+0000","last_fullsized":"2024-04-17T05:42:40.146569+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:18:53.682774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'466","reported_seq":1107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.641533+0000","last_change":"2024-04-17T04:36:31.397750+0000","last_active":"2024-04-17T05:45:54.641533+0000","last_peered":"2024-04-17T05:45:54.641533+0000","last_clean":"2024-04-17T05:45:54.641533+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:45:54.641533+0000","last_undegraded":"2024-04-17T05:45:54.641533+0000","last_fullsized":"2024-04-17T05:45:54.641533+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":400,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:14.506372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103130,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":820,"num_read_kb":1787,"num_write":708,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.135785+0000","last_change":"2024-04-17T04:36:33.418783+0000","last_active":"2024-04-17T05:41:35.135785+0000","last_peered":"2024-04-17T05:41:35.135785+0000","last_clean":"2024-04-17T05:41:35.135785+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:41:35.135785+0000","last_undegraded":"2024-04-17T05:41:35.135785+0000","last_fullsized":"2024-04-17T05:41:35.135785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:38:46.129082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'171","reported_seq":678,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168149+0000","last_change":"2024-04-17T04:36:35.419329+0000","last_active":"2024-04-17T05:41:55.168149+0000","last_peered":"2024-04-17T05:41:55.168149+0000","last_clean":"2024-04-17T05:41:55.168149+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:41:55.168149+0000","last_undegraded":"2024-04-17T05:41:55.168149+0000","last_fullsized":"2024-04-17T05:41:55.168149+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:14.062860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":709,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146514+0000","last_change":"2024-04-17T04:36:27.386903+0000","last_active":"2024-04-17T05:42:40.146514+0000","last_peered":"2024-04-17T05:42:40.146514+0000","last_clean":"2024-04-17T05:42:40.146514+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:42:40.146514+0000","last_undegraded":"2024-04-17T05:42:40.146514+0000","last_fullsized":"2024-04-17T05:42:40.146514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:46:53.605066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'252","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.140440+0000","last_change":"2024-04-17T04:36:25.357464+0000","last_active":"2024-04-17T05:41:55.140440+0000","last_peered":"2024-04-17T05:41:55.140440+0000","last_clean":"2024-04-17T05:41:55.140440+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:41:55.140440+0000","last_undegraded":"2024-04-17T05:41:55.140440+0000","last_fullsized":"2024-04-17T05:41:55.140440+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:12.420278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6367399,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":447,"num_read_kb":112378,"num_write":1170,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151910+0000","last_change":"2024-04-17T04:36:29.376006+0000","last_active":"2024-04-17T05:43:10.151910+0000","last_peered":"2024-04-17T05:43:10.151910+0000","last_clean":"2024-04-17T05:43:10.151910+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:43:10.151910+0000","last_undegraded":"2024-04-17T05:43:10.151910+0000","last_fullsized":"2024-04-17T05:43:10.151910+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.144787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'310","reported_seq":904,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.163993+0000","last_change":"2024-04-17T04:36:31.402737+0000","last_active":"2024-04-17T05:44:05.163993+0000","last_peered":"2024-04-17T05:44:05.163993+0000","last_clean":"2024-04-17T05:44:05.163993+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:44:05.163993+0000","last_undegraded":"2024-04-17T05:44:05.163993+0000","last_fullsized":"2024-04-17T05:44:05.163993+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:20.700900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52833,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":523,"num_read_kb":1031,"num_write":436,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.136274+0000","last_change":"2024-04-17T04:36:33.415254+0000","last_active":"2024-04-17T05:41:35.136274+0000","last_peered":"2024-04-17T05:41:35.136274+0000","last_clean":"2024-04-17T05:41:35.136274+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:41:35.136274+0000","last_undegraded":"2024-04-17T05:41:35.136274+0000","last_fullsized":"2024-04-17T05:41:35.136274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:37.629549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'175","reported_seq":711,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166706+0000","last_change":"2024-04-17T04:36:35.424191+0000","last_active":"2024-04-17T05:44:15.166706+0000","last_peered":"2024-04-17T05:44:15.166706+0000","last_clean":"2024-04-17T05:44:15.166706+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:44:15.166706+0000","last_undegraded":"2024-04-17T05:44:15.166706+0000","last_fullsized":"2024-04-17T05:44:15.166706+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:48.892746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'114","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.162530+0000","last_change":"2024-04-17T04:36:27.378054+0000","last_active":"2024-04-17T05:41:30.162530+0000","last_peered":"2024-04-17T05:41:30.162530+0000","last_clean":"2024-04-17T05:41:30.162530+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:41:30.162530+0000","last_undegraded":"2024-04-17T05:41:30.162530+0000","last_fullsized":"2024-04-17T05:41:30.162530+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:41.230833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'297","reported_seq":981,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:38.607179+0000","last_change":"2024-04-17T04:36:25.356583+0000","last_active":"2024-04-17T05:41:38.607179+0000","last_peered":"2024-04-17T05:41:38.607179+0000","last_clean":"2024-04-17T05:41:38.607179+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:41:38.607179+0000","last_undegraded":"2024-04-17T05:41:38.607179+0000","last_fullsized":"2024-04-17T05:41:38.607179+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:38.408649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5881433,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":515,"num_read_kb":10795,"num_write":1289,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":831,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.514388+0000","last_change":"2024-04-17T04:36:29.370143+0000","last_active":"2024-04-17T05:45:54.514388+0000","last_peered":"2024-04-17T05:45:54.514388+0000","last_clean":"2024-04-17T05:45:54.514388+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:45:54.514388+0000","last_undegraded":"2024-04-17T05:45:54.514388+0000","last_fullsized":"2024-04-17T05:45:54.514388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:33.209495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'293","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.216972+0000","last_change":"2024-04-17T04:36:31.421343+0000","last_active":"2024-04-17T05:45:20.216972+0000","last_peered":"2024-04-17T05:45:20.216972+0000","last_clean":"2024-04-17T05:45:20.216972+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:45:20.216972+0000","last_undegraded":"2024-04-17T05:45:20.216972+0000","last_fullsized":"2024-04-17T05:45:20.216972+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:50.024878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58684,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":490,"num_read_kb":1030,"num_write":398,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.152311+0000","last_change":"2024-04-17T04:36:33.415113+0000","last_active":"2024-04-17T05:42:50.152311+0000","last_peered":"2024-04-17T05:42:50.152311+0000","last_clean":"2024-04-17T05:42:50.152311+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:42:50.152311+0000","last_undegraded":"2024-04-17T05:42:50.152311+0000","last_fullsized":"2024-04-17T05:42:50.152311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:41.644384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'102","reported_seq":601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.654720+0000","last_change":"2024-04-17T04:36:35.428333+0000","last_active":"2024-04-17T05:45:54.654720+0000","last_peered":"2024-04-17T05:45:54.654720+0000","last_clean":"2024-04-17T05:45:54.654720+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:45:54.654720+0000","last_undegraded":"2024-04-17T05:45:54.654720+0000","last_fullsized":"2024-04-17T05:45:54.654720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:39.420180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.133409+0000","last_change":"2024-04-17T04:36:27.356010+0000","last_active":"2024-04-17T05:41:30.133409+0000","last_peered":"2024-04-17T05:41:30.133409+0000","last_clean":"2024-04-17T05:41:30.133409+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:41:30.133409+0000","last_undegraded":"2024-04-17T05:41:30.133409+0000","last_fullsized":"2024-04-17T05:41:30.133409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:42.969449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'255","reported_seq":1080,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.128114+0000","last_change":"2024-04-17T04:36:25.354574+0000","last_active":"2024-04-17T05:41:25.128114+0000","last_peered":"2024-04-17T05:41:25.128114+0000","last_clean":"2024-04-17T05:41:25.128114+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:41:25.128114+0000","last_undegraded":"2024-04-17T05:41:25.128114+0000","last_fullsized":"2024-04-17T05:41:25.128114+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:55.589797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21838700,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":777,"num_read_kb":11279,"num_write":1138,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.135309+0000","last_change":"2024-04-17T04:36:29.365578+0000","last_active":"2024-04-17T05:41:30.135309+0000","last_peered":"2024-04-17T05:41:30.135309+0000","last_clean":"2024-04-17T05:41:30.135309+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:41:30.135309+0000","last_undegraded":"2024-04-17T05:41:30.135309+0000","last_fullsized":"2024-04-17T05:41:30.135309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:29.189344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'119","reported_seq":666,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.137220+0000","last_change":"2024-04-17T04:36:31.422491+0000","last_active":"2024-04-17T05:42:45.137220+0000","last_peered":"2024-04-17T05:42:45.137220+0000","last_clean":"2024-04-17T05:42:45.137220+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:42:45.137220+0000","last_undegraded":"2024-04-17T05:42:45.137220+0000","last_fullsized":"2024-04-17T05:42:45.137220+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:06.145178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'535","reported_seq":2310,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.671073+0000","last_change":"2024-04-17T04:36:33.404252+0000","last_active":"2024-04-17T05:45:57.671073+0000","last_peered":"2024-04-17T05:45:57.671073+0000","last_clean":"2024-04-17T05:45:57.671073+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:45:57.671073+0000","last_undegraded":"2024-04-17T05:45:57.671073+0000","last_fullsized":"2024-04-17T05:45:57.671073+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":500,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:11.625831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'114","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134497+0000","last_change":"2024-04-17T04:36:35.413901+0000","last_active":"2024-04-17T05:41:55.134497+0000","last_peered":"2024-04-17T05:41:55.134497+0000","last_clean":"2024-04-17T05:41:55.134497+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:41:55.134497+0000","last_undegraded":"2024-04-17T05:41:55.134497+0000","last_fullsized":"2024-04-17T05:41:55.134497+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:00:54.978078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'229","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120332+0000","last_change":"2024-04-17T04:36:27.394864+0000","last_active":"2024-04-17T05:41:30.120332+0000","last_peered":"2024-04-17T05:41:30.120332+0000","last_clean":"2024-04-17T05:41:30.120332+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:41:30.120332+0000","last_undegraded":"2024-04-17T05:41:30.120332+0000","last_fullsized":"2024-04-17T05:41:30.120332+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:50:03.593045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'268","reported_seq":867,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.133458+0000","last_change":"2024-04-17T04:36:25.369133+0000","last_active":"2024-04-17T05:41:25.133458+0000","last_peered":"2024-04-17T05:41:25.133458+0000","last_clean":"2024-04-17T05:41:25.133458+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:41:25.133458+0000","last_undegraded":"2024-04-17T05:41:25.133458+0000","last_fullsized":"2024-04-17T05:41:25.133458+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:22.406331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10495834,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":376,"num_read_kb":414,"num_write":1132,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.128372+0000","last_change":"2024-04-17T04:36:29.379152+0000","last_active":"2024-04-17T05:41:30.128372+0000","last_peered":"2024-04-17T05:41:30.128372+0000","last_clean":"2024-04-17T05:41:30.128372+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:41:30.128372+0000","last_undegraded":"2024-04-17T05:41:30.128372+0000","last_fullsized":"2024-04-17T05:41:30.128372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:52.689910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'336","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.458700+0000","last_change":"2024-04-17T04:36:31.396219+0000","last_active":"2024-04-17T05:45:46.458700+0000","last_peered":"2024-04-17T05:45:46.458700+0000","last_clean":"2024-04-17T05:45:46.458700+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:45:46.458700+0000","last_undegraded":"2024-04-17T05:45:46.458700+0000","last_fullsized":"2024-04-17T05:45:46.458700+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:39:05.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":534,"num_read_kb":1021,"num_write":420,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134718+0000","last_change":"2024-04-17T04:36:33.430639+0000","last_active":"2024-04-17T05:41:50.134718+0000","last_peered":"2024-04-17T05:41:50.134718+0000","last_clean":"2024-04-17T05:41:50.134718+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:41:50.134718+0000","last_undegraded":"2024-04-17T05:41:50.134718+0000","last_fullsized":"2024-04-17T05:41:50.134718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.795912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'127","reported_seq":629,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166471+0000","last_change":"2024-04-17T04:36:35.436213+0000","last_active":"2024-04-17T05:44:15.166471+0000","last_peered":"2024-04-17T05:44:15.166471+0000","last_clean":"2024-04-17T05:44:15.166471+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:44:15.166471+0000","last_undegraded":"2024-04-17T05:44:15.166471+0000","last_fullsized":"2024-04-17T05:44:15.166471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:44.113556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.536900+0000","last_change":"2024-04-17T04:36:27.378160+0000","last_active":"2024-04-17T05:41:51.536900+0000","last_peered":"2024-04-17T05:41:51.536900+0000","last_clean":"2024-04-17T05:41:51.536900+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:41:51.536900+0000","last_undegraded":"2024-04-17T05:41:51.536900+0000","last_fullsized":"2024-04-17T05:41:51.536900+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:53.663348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'275","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.133518+0000","last_change":"2024-04-17T04:36:25.365976+0000","last_active":"2024-04-17T05:41:25.133518+0000","last_peered":"2024-04-17T05:41:25.133518+0000","last_clean":"2024-04-17T05:41:25.133518+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:41:25.133518+0000","last_undegraded":"2024-04-17T05:41:25.133518+0000","last_fullsized":"2024-04-17T05:41:25.133518+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:05.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267001,"num_objects":79,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":484,"num_read_kb":31239,"num_write":1200,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134570+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:41:30.134570+0000","last_peered":"2024-04-17T05:41:30.134570+0000","last_clean":"2024-04-17T05:41:30.134570+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:41:30.134570+0000","last_undegraded":"2024-04-17T05:41:30.134570+0000","last_fullsized":"2024-04-17T05:41:30.134570+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:39.416117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'268","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168845+0000","last_change":"2024-04-17T04:36:31.403334+0000","last_active":"2024-04-17T05:44:15.168845+0000","last_peered":"2024-04-17T05:44:15.168845+0000","last_clean":"2024-04-17T05:44:15.168845+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:44:15.168845+0000","last_undegraded":"2024-04-17T05:44:15.168845+0000","last_fullsized":"2024-04-17T05:44:15.168845+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:25.865066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":448,"num_read_kb":958,"num_write":362,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136680+0000","last_change":"2024-04-17T04:36:33.420313+0000","last_active":"2024-04-17T05:42:00.136680+0000","last_peered":"2024-04-17T05:42:00.136680+0000","last_clean":"2024-04-17T05:42:00.136680+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:42:00.136680+0000","last_undegraded":"2024-04-17T05:42:00.136680+0000","last_fullsized":"2024-04-17T05:42:00.136680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:55.964612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'134","reported_seq":655,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.071549+0000","last_change":"2024-04-17T04:36:35.422747+0000","last_active":"2024-04-17T05:45:46.071549+0000","last_peered":"2024-04-17T05:45:46.071549+0000","last_clean":"2024-04-17T05:45:46.071549+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:45:46.071549+0000","last_undegraded":"2024-04-17T05:45:46.071549+0000","last_fullsized":"2024-04-17T05:45:46.071549+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:32.269106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'203","reported_seq":810,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:14.025767+0000","last_change":"2024-04-17T04:36:27.381423+0000","last_active":"2024-04-17T05:42:14.025767+0000","last_peered":"2024-04-17T05:42:14.025767+0000","last_clean":"2024-04-17T05:42:14.025767+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:42:14.025767+0000","last_undegraded":"2024-04-17T05:42:14.025767+0000","last_fullsized":"2024-04-17T05:42:14.025767+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:04.091357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'307","reported_seq":970,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.179182+0000","last_change":"2024-04-17T04:36:25.374623+0000","last_active":"2024-04-17T05:42:40.179182+0000","last_peered":"2024-04-17T05:42:40.179182+0000","last_clean":"2024-04-17T05:42:40.179182+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:42:40.179182+0000","last_undegraded":"2024-04-17T05:42:40.179182+0000","last_fullsized":"2024-04-17T05:42:40.179182+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:52.802493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13343203,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":445,"num_read_kb":19835,"num_write":1327,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123902+0000","last_change":"2024-04-17T04:36:29.377026+0000","last_active":"2024-04-17T05:41:45.123902+0000","last_peered":"2024-04-17T05:41:45.123902+0000","last_clean":"2024-04-17T05:41:45.123902+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:41:45.123902+0000","last_undegraded":"2024-04-17T05:41:45.123902+0000","last_fullsized":"2024-04-17T05:41:45.123902+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:26:46.743881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'255","reported_seq":822,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198320+0000","last_change":"2024-04-17T04:36:31.398159+0000","last_active":"2024-04-17T05:44:05.198320+0000","last_peered":"2024-04-17T05:44:05.198320+0000","last_clean":"2024-04-17T05:44:05.198320+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:44:05.198320+0000","last_undegraded":"2024-04-17T05:44:05.198320+0000","last_fullsized":"2024-04-17T05:44:05.198320+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:04.571296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56291,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":432,"num_read_kb":970,"num_write":360,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.121505+0000","last_change":"2024-04-17T04:36:33.401472+0000","last_active":"2024-04-17T05:41:35.121505+0000","last_peered":"2024-04-17T05:41:35.121505+0000","last_clean":"2024-04-17T05:41:35.121505+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:41:35.121505+0000","last_undegraded":"2024-04-17T05:41:35.121505+0000","last_fullsized":"2024-04-17T05:41:35.121505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:49.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'141","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.700107+0000","last_change":"2024-04-17T04:36:35.423654+0000","last_active":"2024-04-17T05:45:40.700107+0000","last_peered":"2024-04-17T05:45:40.700107+0000","last_clean":"2024-04-17T05:45:40.700107+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:45:40.700107+0000","last_undegraded":"2024-04-17T05:45:40.700107+0000","last_fullsized":"2024-04-17T05:45:40.700107+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:17:36.321043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'297","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123940+0000","last_change":"2024-04-17T04:36:27.400669+0000","last_active":"2024-04-17T05:41:45.123940+0000","last_peered":"2024-04-17T05:41:45.123940+0000","last_clean":"2024-04-17T05:41:45.123940+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:41:45.123940+0000","last_undegraded":"2024-04-17T05:41:45.123940+0000","last_fullsized":"2024-04-17T05:41:45.123940+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:59.138022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'362","reported_seq":1068,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151944+0000","last_change":"2024-04-17T04:36:25.356509+0000","last_active":"2024-04-17T05:43:05.151944+0000","last_peered":"2024-04-17T05:43:05.151944+0000","last_clean":"2024-04-17T05:43:05.151944+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:43:05.151944+0000","last_undegraded":"2024-04-17T05:43:05.151944+0000","last_fullsized":"2024-04-17T05:43:05.151944+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":300,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:14.238916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16573947,"num_objects":81,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":588,"num_read_kb":71297,"num_write":1611,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"35'6","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.508087+0000","last_change":"2024-04-17T04:36:29.376775+0000","last_active":"2024-04-17T05:45:54.508087+0000","last_peered":"2024-04-17T05:45:54.508087+0000","last_clean":"2024-04-17T05:45:54.508087+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:45:54.508087+0000","last_undegraded":"2024-04-17T05:45:54.508087+0000","last_fullsized":"2024-04-17T05:45:54.508087+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:09.589509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":498,"num_read_kb":809,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'314","reported_seq":919,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.158024+0000","last_change":"2024-04-17T04:36:31.391158+0000","last_active":"2024-04-17T05:44:05.158024+0000","last_peered":"2024-04-17T05:44:05.158024+0000","last_clean":"2024-04-17T05:44:05.158024+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:44:05.158024+0000","last_undegraded":"2024-04-17T05:44:05.158024+0000","last_fullsized":"2024-04-17T05:44:05.158024+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:43.696307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58738,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":517,"num_read_kb":1039,"num_write":414,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130642+0000","last_change":"2024-04-17T04:36:33.430195+0000","last_active":"2024-04-17T05:41:35.130642+0000","last_peered":"2024-04-17T05:41:35.130642+0000","last_clean":"2024-04-17T05:41:35.130642+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:41:35.130642+0000","last_undegraded":"2024-04-17T05:41:35.130642+0000","last_fullsized":"2024-04-17T05:41:35.130642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:37.038297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'138","reported_seq":637,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.163062+0000","last_change":"2024-04-17T04:36:35.427734+0000","last_active":"2024-04-17T05:44:25.163062+0000","last_peered":"2024-04-17T05:44:25.163062+0000","last_clean":"2024-04-17T05:44:25.163062+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:44:25.163062+0000","last_undegraded":"2024-04-17T05:44:25.163062+0000","last_fullsized":"2024-04-17T05:44:25.163062+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:48:29.121374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'154","reported_seq":788,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164804+0000","last_change":"2024-04-17T04:36:27.397669+0000","last_active":"2024-04-17T05:43:40.164804+0000","last_peered":"2024-04-17T05:43:40.164804+0000","last_clean":"2024-04-17T05:43:40.164804+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:43:40.164804+0000","last_undegraded":"2024-04-17T05:43:40.164804+0000","last_fullsized":"2024-04-17T05:43:40.164804+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:46:34.375349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'355","reported_seq":1073,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150519+0000","last_change":"2024-04-17T04:36:25.383690+0000","last_active":"2024-04-17T05:43:05.150519+0000","last_peered":"2024-04-17T05:43:05.150519+0000","last_clean":"2024-04-17T05:43:05.150519+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:43:05.150519+0000","last_undegraded":"2024-04-17T05:43:05.150519+0000","last_fullsized":"2024-04-17T05:43:05.150519+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:14.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993920,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":770,"num_read_kb":6303,"num_write":1311,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":648,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142976+0000","last_change":"2024-04-17T04:37:03.630121+0000","last_active":"2024-04-17T05:42:05.142976+0000","last_peered":"2024-04-17T05:42:05.142976+0000","last_clean":"2024-04-17T05:42:05.142976+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:42:05.142976+0000","last_undegraded":"2024-04-17T05:42:05.142976+0000","last_fullsized":"2024-04-17T05:42:05.142976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:09:56.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'293","reported_seq":959,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.153334+0000","last_change":"2024-04-17T04:36:25.355361+0000","last_active":"2024-04-17T05:43:05.153334+0000","last_peered":"2024-04-17T05:43:05.153334+0000","last_clean":"2024-04-17T05:43:05.153334+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:43:05.153334+0000","last_undegraded":"2024-04-17T05:43:05.153334+0000","last_fullsized":"2024-04-17T05:43:05.153334+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:15.756630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11689993,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":496,"num_read_kb":39832,"num_write":1126,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'148","reported_seq":762,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163630+0000","last_change":"2024-04-17T04:36:27.388520+0000","last_active":"2024-04-17T05:43:40.163630+0000","last_peered":"2024-04-17T05:43:40.163630+0000","last_clean":"2024-04-17T05:43:40.163630+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:43:40.163630+0000","last_undegraded":"2024-04-17T05:43:40.163630+0000","last_fullsized":"2024-04-17T05:43:40.163630+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:31.868037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'250","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.155879+0000","last_change":"2024-04-17T04:36:25.365759+0000","last_active":"2024-04-17T05:43:05.155879+0000","last_peered":"2024-04-17T05:43:05.155879+0000","last_clean":"2024-04-17T05:43:05.155879+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:43:05.155879+0000","last_undegraded":"2024-04-17T05:43:05.155879+0000","last_fullsized":"2024-04-17T05:43:05.155879+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:25:48.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4343917,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":379,"num_read_kb":10784,"num_write":1108,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'94","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164612+0000","last_change":"2024-04-17T04:36:27.362737+0000","last_active":"2024-04-17T05:43:40.164612+0000","last_peered":"2024-04-17T05:43:40.164612+0000","last_clean":"2024-04-17T05:43:40.164612+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:43:40.164612+0000","last_undegraded":"2024-04-17T05:43:40.164612+0000","last_fullsized":"2024-04-17T05:43:40.164612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:24:50.502747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'241","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153889+0000","last_change":"2024-04-17T04:36:25.355263+0000","last_active":"2024-04-17T05:43:15.153889+0000","last_peered":"2024-04-17T05:43:15.153889+0000","last_clean":"2024-04-17T05:43:15.153889+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:43:15.153889+0000","last_undegraded":"2024-04-17T05:43:15.153889+0000","last_fullsized":"2024-04-17T05:43:15.153889+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:43.114752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11863036,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":642,"num_read_kb":32422,"num_write":1055,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":635,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.189253+0000","last_change":"2024-04-17T04:36:27.395680+0000","last_active":"2024-04-17T05:43:25.189253+0000","last_peered":"2024-04-17T05:43:25.189253+0000","last_clean":"2024-04-17T05:43:25.189253+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:43:25.189253+0000","last_undegraded":"2024-04-17T05:43:25.189253+0000","last_fullsized":"2024-04-17T05:43:25.189253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:04.849323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'223","reported_seq":835,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191011+0000","last_change":"2024-04-17T04:37:03.637190+0000","last_active":"2024-04-17T05:43:35.191011+0000","last_peered":"2024-04-17T05:43:35.191011+0000","last_clean":"2024-04-17T05:43:35.191011+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:43:35.191011+0000","last_undegraded":"2024-04-17T05:43:35.191011+0000","last_fullsized":"2024-04-17T05:43:35.191011+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:24:25.534759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5542945,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":301,"num_read_kb":5919,"num_write":935,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938441+0000","last_change":"2024-04-17T04:36:27.395504+0000","last_active":"2024-04-17T05:43:09.938441+0000","last_peered":"2024-04-17T05:43:09.938441+0000","last_clean":"2024-04-17T05:43:09.938441+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:43:09.938441+0000","last_undegraded":"2024-04-17T05:43:09.938441+0000","last_fullsized":"2024-04-17T05:43:09.938441+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:27.120391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'257","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163317+0000","last_change":"2024-04-17T04:36:25.342748+0000","last_active":"2024-04-17T05:43:35.163317+0000","last_peered":"2024-04-17T05:43:35.163317+0000","last_clean":"2024-04-17T05:43:35.163317+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:43:35.163317+0000","last_undegraded":"2024-04-17T05:43:35.163317+0000","last_fullsized":"2024-04-17T05:43:35.163317+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:25.415287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173927,"num_objects":65,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":430,"num_read_kb":70994,"num_write":1129,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'215","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:36.452977+0000","last_change":"2024-04-17T04:36:27.384524+0000","last_active":"2024-04-17T05:43:36.452977+0000","last_peered":"2024-04-17T05:43:36.452977+0000","last_clean":"2024-04-17T05:43:36.452977+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:43:36.452977+0000","last_undegraded":"2024-04-17T05:43:36.452977+0000","last_fullsized":"2024-04-17T05:43:36.452977+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:35.098376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'314","reported_seq":995,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.157600+0000","last_change":"2024-04-17T04:36:25.386085+0000","last_active":"2024-04-17T05:43:35.157600+0000","last_peered":"2024-04-17T05:43:35.157600+0000","last_clean":"2024-04-17T05:43:35.157600+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:43:35.157600+0000","last_undegraded":"2024-04-17T05:43:35.157600+0000","last_fullsized":"2024-04-17T05:43:35.157600+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:09.994649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12903063,"num_objects":57,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":597,"num_read_kb":34079,"num_write":1419,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151711+0000","last_change":"2024-04-17T04:36:27.387044+0000","last_active":"2024-04-17T05:43:10.151711+0000","last_peered":"2024-04-17T05:43:10.151711+0000","last_clean":"2024-04-17T05:43:10.151711+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:43:10.151711+0000","last_undegraded":"2024-04-17T05:43:10.151711+0000","last_fullsized":"2024-04-17T05:43:10.151711+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:47.002935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'266","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191154+0000","last_change":"2024-04-17T04:36:25.379835+0000","last_active":"2024-04-17T05:43:35.191154+0000","last_peered":"2024-04-17T05:43:35.191154+0000","last_clean":"2024-04-17T05:43:35.191154+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:43:35.191154+0000","last_undegraded":"2024-04-17T05:43:35.191154+0000","last_fullsized":"2024-04-17T05:43:35.191154+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:54:10.099587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16832221,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":477,"num_read_kb":42709,"num_write":1222,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":738,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:49.022874+0000","last_change":"2024-04-17T04:36:27.373717+0000","last_active":"2024-04-17T05:42:49.022874+0000","last_peered":"2024-04-17T05:42:49.022874+0000","last_clean":"2024-04-17T05:42:49.022874+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:42:49.022874+0000","last_undegraded":"2024-04-17T05:42:49.022874+0000","last_fullsized":"2024-04-17T05:42:49.022874+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:38:49.102110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'237","reported_seq":871,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.165564+0000","last_change":"2024-04-17T04:36:25.361686+0000","last_active":"2024-04-17T05:43:50.165564+0000","last_peered":"2024-04-17T05:43:50.165564+0000","last_clean":"2024-04-17T05:43:50.165564+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:43:50.165564+0000","last_undegraded":"2024-04-17T05:43:50.165564+0000","last_fullsized":"2024-04-17T05:43:50.165564+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:40.663817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5566546,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":393,"num_read_kb":545,"num_write":1054,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":754,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125394+0000","last_change":"2024-04-17T04:36:27.378398+0000","last_active":"2024-04-17T05:41:50.125394+0000","last_peered":"2024-04-17T05:41:50.125394+0000","last_clean":"2024-04-17T05:41:50.125394+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:41:50.125394+0000","last_undegraded":"2024-04-17T05:41:50.125394+0000","last_fullsized":"2024-04-17T05:41:50.125394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:49.831818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'237","reported_seq":886,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.350982+0000","last_change":"2024-04-17T04:36:25.353469+0000","last_active":"2024-04-17T05:43:39.350982+0000","last_peered":"2024-04-17T05:43:39.350982+0000","last_clean":"2024-04-17T05:43:39.350982+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:43:39.350982+0000","last_undegraded":"2024-04-17T05:43:39.350982+0000","last_fullsized":"2024-04-17T05:43:39.350982+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2315115,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":391,"num_read_kb":4352,"num_write":1033,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":837,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139934+0000","last_change":"2024-04-17T04:36:27.398568+0000","last_active":"2024-04-17T05:42:00.139934+0000","last_peered":"2024-04-17T05:42:00.139934+0000","last_clean":"2024-04-17T05:42:00.139934+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:42:00.139934+0000","last_undegraded":"2024-04-17T05:42:00.139934+0000","last_fullsized":"2024-04-17T05:42:00.139934+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:48.128020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'247","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144425+0000","last_change":"2024-04-17T04:37:03.632788+0000","last_active":"2024-04-17T05:42:20.144425+0000","last_peered":"2024-04-17T05:42:20.144425+0000","last_clean":"2024-04-17T05:42:20.144425+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:42:20.144425+0000","last_undegraded":"2024-04-17T05:42:20.144425+0000","last_fullsized":"2024-04-17T05:42:20.144425+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:33.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5338870,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":373,"num_read_kb":16015,"num_write":997,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'95","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124087+0000","last_change":"2024-04-17T04:36:27.402182+0000","last_active":"2024-04-17T05:41:45.124087+0000","last_peered":"2024-04-17T05:41:45.124087+0000","last_clean":"2024-04-17T05:41:45.124087+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:41:45.124087+0000","last_undegraded":"2024-04-17T05:41:45.124087+0000","last_fullsized":"2024-04-17T05:41:45.124087+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:59.716958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'300","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132046+0000","last_change":"2024-04-17T04:36:25.333011+0000","last_active":"2024-04-17T05:41:40.132046+0000","last_peered":"2024-04-17T05:41:40.132046+0000","last_clean":"2024-04-17T05:41:40.132046+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:41:40.132046+0000","last_undegraded":"2024-04-17T05:41:40.132046+0000","last_fullsized":"2024-04-17T05:41:40.132046+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:22.239383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071966,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":686,"num_read_kb":12134,"num_write":1196,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":686,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.132122+0000","last_change":"2024-04-17T04:36:27.388791+0000","last_active":"2024-04-17T05:41:45.132122+0000","last_peered":"2024-04-17T05:41:45.132122+0000","last_clean":"2024-04-17T05:41:45.132122+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:41:45.132122+0000","last_undegraded":"2024-04-17T05:41:45.132122+0000","last_fullsized":"2024-04-17T05:41:45.132122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:21.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'281","reported_seq":917,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:54.078754+0000","last_change":"2024-04-17T04:36:25.373417+0000","last_active":"2024-04-17T05:42:54.078754+0000","last_peered":"2024-04-17T05:42:54.078754+0000","last_clean":"2024-04-17T05:42:54.078754+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:42:54.078754+0000","last_undegraded":"2024-04-17T05:42:54.078754+0000","last_fullsized":"2024-04-17T05:42:54.078754+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:11.729276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30815,"num_objects":79,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":425,"num_read_kb":657,"num_write":1176,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'221","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133347+0000","last_change":"2024-04-17T04:36:27.372140+0000","last_active":"2024-04-17T05:41:45.133347+0000","last_peered":"2024-04-17T05:41:45.133347+0000","last_clean":"2024-04-17T05:41:45.133347+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:41:45.133347+0000","last_undegraded":"2024-04-17T05:41:45.133347+0000","last_fullsized":"2024-04-17T05:41:45.133347+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.338570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'355","reported_seq":1093,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133443+0000","last_change":"2024-04-17T04:36:25.386307+0000","last_active":"2024-04-17T05:41:50.133443+0000","last_peered":"2024-04-17T05:41:50.133443+0000","last_clean":"2024-04-17T05:41:50.133443+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:41:50.133443+0000","last_undegraded":"2024-04-17T05:41:50.133443+0000","last_fullsized":"2024-04-17T05:41:50.133443+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:43.522371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18382422,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":829,"num_read_kb":32478,"num_write":1352,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165668+0000","last_change":"2024-04-17T04:36:27.385349+0000","last_active":"2024-04-17T05:41:45.165668+0000","last_peered":"2024-04-17T05:41:45.165668+0000","last_clean":"2024-04-17T05:41:45.165668+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:41:45.165668+0000","last_undegraded":"2024-04-17T05:41:45.165668+0000","last_fullsized":"2024-04-17T05:41:45.165668+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:27:44.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'264","reported_seq":940,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136676+0000","last_change":"2024-04-17T04:36:25.355339+0000","last_active":"2024-04-17T05:41:45.136676+0000","last_peered":"2024-04-17T05:41:45.136676+0000","last_clean":"2024-04-17T05:41:45.136676+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:41:45.136676+0000","last_undegraded":"2024-04-17T05:41:45.136676+0000","last_fullsized":"2024-04-17T05:41:45.136676+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:47:52.068393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7660588,"num_objects":55,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":476,"num_read_kb":46303,"num_write":1246,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'321","reported_seq":1043,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160308+0000","last_change":"2024-04-17T04:37:03.628711+0000","last_active":"2024-04-17T05:43:50.160308+0000","last_peered":"2024-04-17T05:43:50.160308+0000","last_clean":"2024-04-17T05:43:50.160308+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:43:50.160308+0000","last_undegraded":"2024-04-17T05:43:50.160308+0000","last_fullsized":"2024-04-17T05:43:50.160308+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:45.773023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'262","reported_seq":895,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:25.361750+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:44.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23096680,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":400,"num_read_kb":49597,"num_write":1165,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'238","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.137869+0000","last_change":"2024-04-17T04:36:25.345113+0000","last_active":"2024-04-17T05:41:45.137869+0000","last_peered":"2024-04-17T05:41:45.137869+0000","last_clean":"2024-04-17T05:41:45.137869+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:41:45.137869+0000","last_undegraded":"2024-04-17T05:41:45.137869+0000","last_fullsized":"2024-04-17T05:41:45.137869+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:36.604225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584415,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":354,"num_read_kb":7661,"num_write":1028,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'243","reported_seq":842,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136202+0000","last_change":"2024-04-17T04:36:27.379068+0000","last_active":"2024-04-17T05:41:55.136202+0000","last_peered":"2024-04-17T05:41:55.136202+0000","last_clean":"2024-04-17T05:41:55.136202+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:41:55.136202+0000","last_undegraded":"2024-04-17T05:41:55.136202+0000","last_fullsized":"2024-04-17T05:41:55.136202+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:27.386167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'227","reported_seq":870,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:24.492781+0000","last_change":"2024-04-17T04:36:25.357445+0000","last_active":"2024-04-17T05:42:24.492781+0000","last_peered":"2024-04-17T05:42:24.492781+0000","last_clean":"2024-04-17T05:42:24.492781+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:42:24.492781+0000","last_undegraded":"2024-04-17T05:42:24.492781+0000","last_fullsized":"2024-04-17T05:42:24.492781+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:12.816116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10515588,"num_objects":59,"num_object_clones":0,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":397,"num_read_kb":20933,"num_write":1073,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":819,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138230+0000","last_change":"2024-04-17T04:36:27.378695+0000","last_active":"2024-04-17T05:41:45.138230+0000","last_peered":"2024-04-17T05:41:45.138230+0000","last_clean":"2024-04-17T05:41:45.138230+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:41:45.138230+0000","last_undegraded":"2024-04-17T05:41:45.138230+0000","last_fullsized":"2024-04-17T05:41:45.138230+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:37.636571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1280,"ondisk_log_size":1280,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1676,"ondisk_log_size":1676,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1746118,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":17591,"num_read_kb":33647,"num_write":14851,"num_write_kb":11154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3284992,"data_stored":2735018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2071,"ondisk_log_size":2071,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1642,"num_read_kb":1951,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3948,"ondisk_log_size":3948,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":648130815,"num_objects":4413,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":30775,"num_read_kb":1864049,"num_write":73496,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930250752,"data_stored":930250752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":100760153,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":130,"num_read_kb":221363,"num_write":141,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201523200,"data_stored":201520306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":100760153,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":125,"num_read_kb":210051,"num_write":141,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201523200,"data_stored":201520306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575690,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451628,"kb_used_data":167780,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93920212,"statfs":{"total":96636764160,"available":96174297088,"internally_reserved":0,"allocated":171806720,"data_stored":169745941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":290650570},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575691,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474680,"kb_used_data":168624,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93897160,"statfs":{"total":96636764160,"available":96150691840,"internally_reserved":0,"allocated":172670976,"data_stored":170683181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":313391559},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575691,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314860,"kb_used_data":175460,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94056980,"statfs":{"total":96636764160,"available":96314347520,"internally_reserved":0,"allocated":179671040,"data_stored":177571130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575693,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":262200,"kb_used_data":158832,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94109640,"statfs":{"total":96636764160,"available":96368271360,"internally_reserved":0,"allocated":162643968,"data_stored":160677607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575691,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445376,"kb_used_data":201848,"kb_used_omap":1,"kb_used_meta":243518,"kb_avail":93926464,"statfs":{"total":96636764160,"available":96180699136,"internally_reserved":0,"allocated":206692352,"data_stored":204603759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":249362887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575695,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":277500,"kb_used_data":165876,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94094340,"statfs":{"total":96636764160,"available":96352604160,"internally_reserved":0,"allocated":169857024,"data_stored":167899459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114293189},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575693,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288536,"kb_used_data":149136,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94083304,"statfs":{"total":96636764160,"available":96341303296,"internally_reserved":0,"allocated":152715264,"data_stored":150702994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575691,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425908,"kb_used_data":151852,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93945932,"statfs":{"total":96636764160,"available":96200634368,"internally_reserved":0,"allocated":155496448,"data_stored":153432781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":126107648,"data_stored":126107648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111304704,"data_stored":111304704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93118464,"data_stored":93118464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":122077184,"data_stored":122077184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":120524800,"data_stored":120524800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":121225216,"data_stored":121225216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":118833152,"data_stored":118833152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":117059584,"data_stored":117059584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":352781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":385826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":117807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":472766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":92189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":555856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":256028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":501765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14680064,"data_stored":14680064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25214976,"data_stored":25214257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":40894464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28360704,"data_stored":28359976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21020672,"data_stored":21019944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15777792,"data_stored":15777073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31506432,"data_stored":31505713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46235648,"data_stored":46234201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15777792,"data_stored":15777064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:46:04.746 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T05:46:04.746 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:46:05.101 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:05.116 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:46:05.123 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2353,"stamp":"2024-04-17T05:46:03.280712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":852009202,"num_objects":4832,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":156306,"num_read_kb":2419174,"num_write":142134,"num_write_kb":2463295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13386,"ondisk_log_size":13386,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2940688,"kb_used_data":1339408,"kb_used_omap":12,"kb_used_meta":1601203,"kb_avail":752034032,"statfs":{"total":773094113280,"available":770082848768,"internally_reserved":0,"allocated":1371553792,"data_stored":1355316852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1639632433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":19,"num_read_kb":24,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008357"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151550+0000","last_change":"2024-04-17T05:07:45.742645+0000","last_active":"2024-04-17T05:42:45.151550+0000","last_peered":"2024-04-17T05:42:45.151550+0000","last_clean":"2024-04-17T05:42:45.151550+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:42:45.151550+0000","last_undegraded":"2024-04-17T05:42:45.151550+0000","last_fullsized":"2024-04-17T05:42:45.151550+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:06.072659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138115+0000","last_change":"2024-04-17T04:51:43.682048+0000","last_active":"2024-04-17T05:41:45.138115+0000","last_peered":"2024-04-17T05:41:45.138115+0000","last_clean":"2024-04-17T05:41:45.138115+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:41:45.138115+0000","last_undegraded":"2024-04-17T05:41:45.138115+0000","last_fullsized":"2024-04-17T05:41:45.138115+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:06.493069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164913+0000","last_change":"2024-04-17T04:51:39.638879+0000","last_active":"2024-04-17T05:41:40.164913+0000","last_peered":"2024-04-17T05:41:40.164913+0000","last_clean":"2024-04-17T05:41:40.164913+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:41:40.164913+0000","last_undegraded":"2024-04-17T05:41:40.164913+0000","last_fullsized":"2024-04-17T05:41:40.164913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:57:53.046398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":148,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139601+0000","last_change":"2024-04-17T04:47:56.968120+0000","last_active":"2024-04-17T05:42:10.139601+0000","last_peered":"2024-04-17T05:42:10.139601+0000","last_clean":"2024-04-17T05:42:10.139601+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:42:10.139601+0000","last_undegraded":"2024-04-17T05:42:10.139601+0000","last_fullsized":"2024-04-17T05:42:10.139601+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:39.089673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149910+0000","last_change":"2024-04-17T05:07:45.746586+0000","last_active":"2024-04-17T05:42:45.149910+0000","last_peered":"2024-04-17T05:42:45.149910+0000","last_clean":"2024-04-17T05:42:45.149910+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:42:45.149910+0000","last_undegraded":"2024-04-17T05:42:45.149910+0000","last_fullsized":"2024-04-17T05:42:45.149910+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:02.041624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'2","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124114+0000","last_change":"2024-04-17T04:51:43.661871+0000","last_active":"2024-04-17T05:41:45.124114+0000","last_peered":"2024-04-17T05:41:45.124114+0000","last_clean":"2024-04-17T05:41:45.124114+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:41:45.124114+0000","last_undegraded":"2024-04-17T05:41:45.124114+0000","last_fullsized":"2024-04-17T05:41:45.124114+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:46.906315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.137511+0000","last_change":"2024-04-17T04:51:39.640822+0000","last_active":"2024-04-17T05:41:40.137511+0000","last_peered":"2024-04-17T05:41:40.137511+0000","last_clean":"2024-04-17T05:41:40.137511+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:41:40.137511+0000","last_undegraded":"2024-04-17T05:41:40.137511+0000","last_fullsized":"2024-04-17T05:41:40.137511+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:51.213156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'96","reported_seq":350,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.143271+0000","last_change":"2024-04-17T04:47:56.937898+0000","last_active":"2024-04-17T05:42:15.143271+0000","last_peered":"2024-04-17T05:42:15.143271+0000","last_clean":"2024-04-17T05:42:15.143271+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:42:15.143271+0000","last_undegraded":"2024-04-17T05:42:15.143271+0000","last_fullsized":"2024-04-17T05:42:15.143271+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:02:44.307495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130858+0000","last_change":"2024-04-17T05:07:45.739412+0000","last_active":"2024-04-17T05:41:35.130858+0000","last_peered":"2024-04-17T05:41:35.130858+0000","last_clean":"2024-04-17T05:41:35.130858+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:41:35.130858+0000","last_undegraded":"2024-04-17T05:41:35.130858+0000","last_fullsized":"2024-04-17T05:41:35.130858+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:26.784407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162336+0000","last_change":"2024-04-17T04:51:43.682620+0000","last_active":"2024-04-17T05:43:50.162336+0000","last_peered":"2024-04-17T05:43:50.162336+0000","last_clean":"2024-04-17T05:43:50.162336+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:43:50.162336+0000","last_undegraded":"2024-04-17T05:43:50.162336+0000","last_fullsized":"2024-04-17T05:43:50.162336+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:29.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'2","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.164674+0000","last_change":"2024-04-17T04:51:39.605291+0000","last_active":"2024-04-17T05:43:50.164674+0000","last_peered":"2024-04-17T05:43:50.164674+0000","last_clean":"2024-04-17T05:43:50.164674+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:43:50.164674+0000","last_undegraded":"2024-04-17T05:43:50.164674+0000","last_fullsized":"2024-04-17T05:43:50.164674+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:40.774979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":121,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.150706+0000","last_change":"2024-04-17T04:47:56.953281+0000","last_active":"2024-04-17T05:43:00.150706+0000","last_peered":"2024-04-17T05:43:00.150706+0000","last_clean":"2024-04-17T05:43:00.150706+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:43:00.150706+0000","last_undegraded":"2024-04-17T05:43:00.150706+0000","last_fullsized":"2024-04-17T05:43:00.150706+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:02.682710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164774+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:41:40.164774+0000","last_peered":"2024-04-17T05:41:40.164774+0000","last_clean":"2024-04-17T05:41:40.164774+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:41:40.164774+0000","last_undegraded":"2024-04-17T05:41:40.164774+0000","last_fullsized":"2024-04-17T05:41:40.164774+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:05.623921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.195511+0000","last_change":"2024-04-17T04:51:43.676613+0000","last_active":"2024-04-17T05:43:55.195511+0000","last_peered":"2024-04-17T05:43:55.195511+0000","last_clean":"2024-04-17T05:43:55.195511+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:43:55.195511+0000","last_undegraded":"2024-04-17T05:43:55.195511+0000","last_fullsized":"2024-04-17T05:43:55.195511+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:26.269622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946142+0000","last_change":"2024-04-17T04:51:39.615233+0000","last_active":"2024-04-17T05:43:44.946142+0000","last_peered":"2024-04-17T05:43:44.946142+0000","last_clean":"2024-04-17T05:43:44.946142+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:43:44.946142+0000","last_undegraded":"2024-04-17T05:43:44.946142+0000","last_fullsized":"2024-04-17T05:43:44.946142+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:02.139603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":160,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183795+0000","last_change":"2024-04-17T04:47:56.969273+0000","last_active":"2024-04-17T05:43:00.183795+0000","last_peered":"2024-04-17T05:43:00.183795+0000","last_clean":"2024-04-17T05:43:00.183795+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:43:00.183795+0000","last_undegraded":"2024-04-17T05:43:00.183795+0000","last_fullsized":"2024-04-17T05:43:00.183795+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:24.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.194209+0000","last_change":"2024-04-17T05:07:45.745164+0000","last_active":"2024-04-17T05:43:50.194209+0000","last_peered":"2024-04-17T05:43:50.194209+0000","last_clean":"2024-04-17T05:43:50.194209+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:43:50.194209+0000","last_undegraded":"2024-04-17T05:43:50.194209+0000","last_fullsized":"2024-04-17T05:43:50.194209+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:44.184077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'1","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150275+0000","last_change":"2024-04-17T04:51:43.666389+0000","last_active":"2024-04-17T05:42:40.150275+0000","last_peered":"2024-04-17T05:42:40.150275+0000","last_clean":"2024-04-17T05:42:40.150275+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:42:40.150275+0000","last_undegraded":"2024-04-17T05:42:40.150275+0000","last_fullsized":"2024-04-17T05:42:40.150275+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:10.966058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.142876+0000","last_change":"2024-04-17T04:51:39.625539+0000","last_active":"2024-04-17T05:42:25.142876+0000","last_peered":"2024-04-17T05:42:25.142876+0000","last_clean":"2024-04-17T05:42:25.142876+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:42:25.142876+0000","last_undegraded":"2024-04-17T05:42:25.142876+0000","last_fullsized":"2024-04-17T05:42:25.142876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:55.423515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":200,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183684+0000","last_change":"2024-04-17T04:47:56.967026+0000","last_active":"2024-04-17T05:43:00.183684+0000","last_peered":"2024-04-17T05:43:00.183684+0000","last_clean":"2024-04-17T05:43:00.183684+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:43:00.183684+0000","last_undegraded":"2024-04-17T05:43:00.183684+0000","last_fullsized":"2024-04-17T05:43:00.183684+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:26.424709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158284+0000","last_change":"2024-04-17T05:07:45.754178+0000","last_active":"2024-04-17T05:43:10.158284+0000","last_peered":"2024-04-17T05:43:10.158284+0000","last_clean":"2024-04-17T05:43:10.158284+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:43:10.158284+0000","last_undegraded":"2024-04-17T05:43:10.158284+0000","last_fullsized":"2024-04-17T05:43:10.158284+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:49.281995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'4","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142841+0000","last_change":"2024-04-17T04:51:43.664767+0000","last_active":"2024-04-17T05:42:30.142841+0000","last_peered":"2024-04-17T05:42:30.142841+0000","last_clean":"2024-04-17T05:42:30.142841+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:42:30.142841+0000","last_undegraded":"2024-04-17T05:42:30.142841+0000","last_fullsized":"2024-04-17T05:42:30.142841+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:36.530590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":22528,"num_write":9,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"261'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.134896+0000","last_change":"2024-04-17T04:51:39.611306+0000","last_active":"2024-04-17T05:42:35.134896+0000","last_peered":"2024-04-17T05:42:35.134896+0000","last_clean":"2024-04-17T05:42:35.134896+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:42:35.134896+0000","last_undegraded":"2024-04-17T05:42:35.134896+0000","last_fullsized":"2024-04-17T05:42:35.134896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:51.035427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'74","reported_seq":283,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149482+0000","last_change":"2024-04-17T04:47:56.952081+0000","last_active":"2024-04-17T05:43:00.149482+0000","last_peered":"2024-04-17T05:43:00.149482+0000","last_clean":"2024-04-17T05:43:00.149482+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:43:00.149482+0000","last_undegraded":"2024-04-17T05:43:00.149482+0000","last_fullsized":"2024-04-17T05:43:00.149482+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:08:06.903329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145686+0000","last_change":"2024-04-17T05:07:45.754885+0000","last_active":"2024-04-17T05:42:45.145686+0000","last_peered":"2024-04-17T05:42:45.145686+0000","last_clean":"2024-04-17T05:42:45.145686+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:42:45.145686+0000","last_undegraded":"2024-04-17T05:42:45.145686+0000","last_fullsized":"2024-04-17T05:42:45.145686+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:02.913191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133462+0000","last_change":"2024-04-17T04:51:43.664884+0000","last_active":"2024-04-17T05:41:45.133462+0000","last_peered":"2024-04-17T05:41:45.133462+0000","last_clean":"2024-04-17T05:41:45.133462+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:41:45.133462+0000","last_undegraded":"2024-04-17T05:41:45.133462+0000","last_fullsized":"2024-04-17T05:41:45.133462+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:04.163089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'8","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122767+0000","last_change":"2024-04-17T04:51:39.621614+0000","last_active":"2024-04-17T05:41:40.122767+0000","last_peered":"2024-04-17T05:41:40.122767+0000","last_clean":"2024-04-17T05:41:40.122767+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:41:40.122767+0000","last_undegraded":"2024-04-17T05:41:40.122767+0000","last_fullsized":"2024-04-17T05:41:40.122767+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:01.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":16384,"num_write":23,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":74,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:20.127064+0000","last_change":"2024-04-17T04:47:56.955568+0000","last_active":"2024-04-17T05:41:20.127064+0000","last_peered":"2024-04-17T05:41:20.127064+0000","last_clean":"2024-04-17T05:41:20.127064+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:41:20.127064+0000","last_undegraded":"2024-04-17T05:41:20.127064+0000","last_fullsized":"2024-04-17T05:41:20.127064+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:09:24.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145754+0000","last_change":"2024-04-17T05:07:45.757519+0000","last_active":"2024-04-17T05:42:45.145754+0000","last_peered":"2024-04-17T05:42:45.145754+0000","last_clean":"2024-04-17T05:42:45.145754+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:42:45.145754+0000","last_undegraded":"2024-04-17T05:42:45.145754+0000","last_fullsized":"2024-04-17T05:42:45.145754+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:27.671557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165700+0000","last_change":"2024-04-17T04:51:43.686621+0000","last_active":"2024-04-17T05:41:45.165700+0000","last_peered":"2024-04-17T05:41:45.165700+0000","last_clean":"2024-04-17T05:41:45.165700+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:41:45.165700+0000","last_undegraded":"2024-04-17T05:41:45.165700+0000","last_fullsized":"2024-04-17T05:41:45.165700+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:36.829860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132003+0000","last_change":"2024-04-17T04:51:39.643524+0000","last_active":"2024-04-17T05:41:40.132003+0000","last_peered":"2024-04-17T05:41:40.132003+0000","last_clean":"2024-04-17T05:41:40.132003+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:41:40.132003+0000","last_undegraded":"2024-04-17T05:41:40.132003+0000","last_fullsized":"2024-04-17T05:41:40.132003+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:01.434777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":90,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163169+0000","last_change":"2024-04-17T04:47:56.932780+0000","last_active":"2024-04-17T05:43:35.163169+0000","last_peered":"2024-04-17T05:43:35.163169+0000","last_clean":"2024-04-17T05:43:35.163169+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:43:35.163169+0000","last_undegraded":"2024-04-17T05:43:35.163169+0000","last_fullsized":"2024-04-17T05:43:35.163169+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:07.644614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151106+0000","last_change":"2024-04-17T05:07:45.743105+0000","last_active":"2024-04-17T05:42:45.151106+0000","last_peered":"2024-04-17T05:42:45.151106+0000","last_clean":"2024-04-17T05:42:45.151106+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:42:45.151106+0000","last_undegraded":"2024-04-17T05:42:45.151106+0000","last_fullsized":"2024-04-17T05:42:45.151106+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:30.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176828+0000","last_change":"2024-04-17T04:51:43.664814+0000","last_active":"2024-04-17T05:42:30.176828+0000","last_peered":"2024-04-17T05:42:30.176828+0000","last_clean":"2024-04-17T05:42:30.176828+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:42:30.176828+0000","last_undegraded":"2024-04-17T05:42:30.176828+0000","last_fullsized":"2024-04-17T05:42:30.176828+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:39.897527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157053+0000","last_change":"2024-04-17T04:51:39.626132+0000","last_active":"2024-04-17T05:43:10.157053+0000","last_peered":"2024-04-17T05:43:10.157053+0000","last_clean":"2024-04-17T05:43:10.157053+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:43:10.157053+0000","last_undegraded":"2024-04-17T05:43:10.157053+0000","last_fullsized":"2024-04-17T05:43:10.157053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:23.450299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":80,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158447+0000","last_change":"2024-04-17T04:47:56.956719+0000","last_active":"2024-04-17T05:43:35.158447+0000","last_peered":"2024-04-17T05:43:35.158447+0000","last_clean":"2024-04-17T05:43:35.158447+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:43:35.158447+0000","last_undegraded":"2024-04-17T05:43:35.158447+0000","last_fullsized":"2024-04-17T05:43:35.158447+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:10.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149849+0000","last_change":"2024-04-17T05:07:45.738001+0000","last_active":"2024-04-17T05:42:45.149849+0000","last_peered":"2024-04-17T05:42:45.149849+0000","last_clean":"2024-04-17T05:42:45.149849+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:42:45.149849+0000","last_undegraded":"2024-04-17T05:42:45.149849+0000","last_fullsized":"2024-04-17T05:42:45.149849+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.048242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187309+0000","last_change":"2024-04-17T04:51:43.682839+0000","last_active":"2024-04-17T05:43:15.187309+0000","last_peered":"2024-04-17T05:43:15.187309+0000","last_clean":"2024-04-17T05:43:15.187309+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:43:15.187309+0000","last_undegraded":"2024-04-17T05:43:15.187309+0000","last_fullsized":"2024-04-17T05:43:15.187309+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:32.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'1","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141354+0000","last_change":"2024-04-17T04:51:39.639685+0000","last_active":"2024-04-17T05:42:25.141354+0000","last_peered":"2024-04-17T05:42:25.141354+0000","last_clean":"2024-04-17T05:42:25.141354+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:42:25.141354+0000","last_undegraded":"2024-04-17T05:42:25.141354+0000","last_fullsized":"2024-04-17T05:42:25.141354+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:41.954023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191081+0000","last_change":"2024-04-17T04:47:56.966803+0000","last_active":"2024-04-17T05:43:35.191081+0000","last_peered":"2024-04-17T05:43:35.191081+0000","last_clean":"2024-04-17T05:43:35.191081+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:43:35.191081+0000","last_undegraded":"2024-04-17T05:43:35.191081+0000","last_fullsized":"2024-04-17T05:43:35.191081+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:27.144075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145538+0000","last_change":"2024-04-17T05:07:45.747162+0000","last_active":"2024-04-17T05:42:45.145538+0000","last_peered":"2024-04-17T05:42:45.145538+0000","last_clean":"2024-04-17T05:42:45.145538+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:42:45.145538+0000","last_undegraded":"2024-04-17T05:42:45.145538+0000","last_fullsized":"2024-04-17T05:42:45.145538+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:25.853530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'3","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176921+0000","last_change":"2024-04-17T04:51:43.671647+0000","last_active":"2024-04-17T05:42:30.176921+0000","last_peered":"2024-04-17T05:42:30.176921+0000","last_clean":"2024-04-17T05:42:30.176921+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:42:30.176921+0000","last_undegraded":"2024-04-17T05:42:30.176921+0000","last_fullsized":"2024-04-17T05:42:30.176921+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:09:37.852227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5291313,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":8,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'10","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145913+0000","last_change":"2024-04-17T04:51:39.612849+0000","last_active":"2024-04-17T05:42:25.145913+0000","last_peered":"2024-04-17T05:42:25.145913+0000","last_clean":"2024-04-17T05:42:25.145913+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:42:25.145913+0000","last_undegraded":"2024-04-17T05:42:25.145913+0000","last_fullsized":"2024-04-17T05:42:25.145913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:41.680996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":46082,"num_write":14,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.161351+0000","last_change":"2024-04-17T04:47:56.937214+0000","last_active":"2024-04-17T05:43:35.161351+0000","last_peered":"2024-04-17T05:43:35.161351+0000","last_clean":"2024-04-17T05:43:35.161351+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:43:35.161351+0000","last_undegraded":"2024-04-17T05:43:35.161351+0000","last_fullsized":"2024-04-17T05:43:35.161351+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:02.809448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151449+0000","last_change":"2024-04-17T05:07:45.744163+0000","last_active":"2024-04-17T05:42:45.151449+0000","last_peered":"2024-04-17T05:42:45.151449+0000","last_clean":"2024-04-17T05:42:45.151449+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:42:45.151449+0000","last_undegraded":"2024-04-17T05:42:45.151449+0000","last_fullsized":"2024-04-17T05:42:45.151449+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:42.169974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'10","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.133958+0000","last_change":"2024-04-17T04:51:43.667351+0000","last_active":"2024-04-17T05:42:30.133958+0000","last_peered":"2024-04-17T05:42:30.133958+0000","last_clean":"2024-04-17T05:42:30.133958+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:42:30.133958+0000","last_undegraded":"2024-04-17T05:42:30.133958+0000","last_fullsized":"2024-04-17T05:42:30.133958+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:27.804721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":37890,"num_write":19,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143121+0000","last_change":"2024-04-17T04:51:39.639400+0000","last_active":"2024-04-17T05:42:25.143121+0000","last_peered":"2024-04-17T05:42:25.143121+0000","last_clean":"2024-04-17T05:42:25.143121+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:42:25.143121+0000","last_undegraded":"2024-04-17T05:42:25.143121+0000","last_fullsized":"2024-04-17T05:42:25.143121+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:39.092327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":211,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133386+0000","last_change":"2024-04-17T04:47:56.964248+0000","last_active":"2024-04-17T05:41:50.133386+0000","last_peered":"2024-04-17T05:41:50.133386+0000","last_clean":"2024-04-17T05:41:50.133386+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:41:50.133386+0000","last_undegraded":"2024-04-17T05:41:50.133386+0000","last_fullsized":"2024-04-17T05:41:50.133386+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:19.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163617+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:41:35.163617+0000","last_peered":"2024-04-17T05:41:35.163617+0000","last_clean":"2024-04-17T05:41:35.163617+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:41:35.163617+0000","last_undegraded":"2024-04-17T05:41:35.163617+0000","last_fullsized":"2024-04-17T05:41:35.163617+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:45.333124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'5","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157158+0000","last_change":"2024-04-17T04:51:43.670407+0000","last_active":"2024-04-17T05:43:05.157158+0000","last_peered":"2024-04-17T05:43:05.157158+0000","last_clean":"2024-04-17T05:43:05.157158+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:43:05.157158+0000","last_undegraded":"2024-04-17T05:43:05.157158+0000","last_fullsized":"2024-04-17T05:43:05.157158+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:31.623818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:59.936150+0000","last_change":"2024-04-17T04:51:39.613038+0000","last_active":"2024-04-17T05:42:59.936150+0000","last_peered":"2024-04-17T05:42:59.936150+0000","last_clean":"2024-04-17T05:42:59.936150+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:42:59.936150+0000","last_undegraded":"2024-04-17T05:42:59.936150+0000","last_fullsized":"2024-04-17T05:42:59.936150+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:51.929471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.130770+0000","last_change":"2024-04-17T04:47:56.962330+0000","last_active":"2024-04-17T05:41:40.130770+0000","last_peered":"2024-04-17T05:41:40.130770+0000","last_clean":"2024-04-17T05:41:40.130770+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:41:40.130770+0000","last_undegraded":"2024-04-17T05:41:40.130770+0000","last_fullsized":"2024-04-17T05:41:40.130770+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:38.049482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130958+0000","last_change":"2024-04-17T05:07:45.749757+0000","last_active":"2024-04-17T05:41:35.130958+0000","last_peered":"2024-04-17T05:41:35.130958+0000","last_clean":"2024-04-17T05:41:35.130958+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:41:35.130958+0000","last_undegraded":"2024-04-17T05:41:35.130958+0000","last_fullsized":"2024-04-17T05:41:35.130958+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:50:18.794745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'1","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157104+0000","last_change":"2024-04-17T04:51:43.650369+0000","last_active":"2024-04-17T05:43:05.157104+0000","last_peered":"2024-04-17T05:43:05.157104+0000","last_clean":"2024-04-17T05:43:05.157104+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:43:05.157104+0000","last_undegraded":"2024-04-17T05:43:05.157104+0000","last_fullsized":"2024-04-17T05:43:05.157104+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:16:08.661428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183604+0000","last_change":"2024-04-17T04:51:39.631783+0000","last_active":"2024-04-17T05:43:00.183604+0000","last_peered":"2024-04-17T05:43:00.183604+0000","last_clean":"2024-04-17T05:43:00.183604+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:43:00.183604+0000","last_undegraded":"2024-04-17T05:43:00.183604+0000","last_fullsized":"2024-04-17T05:43:00.183604+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:32.873589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":132,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.131963+0000","last_change":"2024-04-17T04:47:56.967539+0000","last_active":"2024-04-17T05:41:40.131963+0000","last_peered":"2024-04-17T05:41:40.131963+0000","last_clean":"2024-04-17T05:41:40.131963+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:41:40.131963+0000","last_undegraded":"2024-04-17T05:41:40.131963+0000","last_fullsized":"2024-04-17T05:41:40.131963+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:50.899642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"263'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.149607+0000","last_change":"2024-04-17T05:07:45.748765+0000","last_active":"2024-04-17T05:42:55.149607+0000","last_peered":"2024-04-17T05:42:55.149607+0000","last_clean":"2024-04-17T05:42:55.149607+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:42:55.149607+0000","last_undegraded":"2024-04-17T05:42:55.149607+0000","last_fullsized":"2024-04-17T05:42:55.149607+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:21.732472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'2","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.184978+0000","last_change":"2024-04-17T04:51:43.684076+0000","last_active":"2024-04-17T05:43:05.184978+0000","last_peered":"2024-04-17T05:43:05.184978+0000","last_clean":"2024-04-17T05:43:05.184978+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:43:05.184978+0000","last_undegraded":"2024-04-17T05:43:05.184978+0000","last_fullsized":"2024-04-17T05:43:05.184978+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.261686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":10240,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149546+0000","last_change":"2024-04-17T04:51:39.626930+0000","last_active":"2024-04-17T05:43:00.149546+0000","last_peered":"2024-04-17T05:43:00.149546+0000","last_clean":"2024-04-17T05:43:00.149546+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:43:00.149546+0000","last_undegraded":"2024-04-17T05:43:00.149546+0000","last_fullsized":"2024-04-17T05:43:00.149546+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:24:11.581902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122667+0000","last_change":"2024-04-17T04:47:56.946016+0000","last_active":"2024-04-17T05:41:40.122667+0000","last_peered":"2024-04-17T05:41:40.122667+0000","last_clean":"2024-04-17T05:41:40.122667+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:41:40.122667+0000","last_undegraded":"2024-04-17T05:41:40.122667+0000","last_fullsized":"2024-04-17T05:41:40.122667+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:39.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163578+0000","last_change":"2024-04-17T05:07:45.741199+0000","last_active":"2024-04-17T05:41:35.163578+0000","last_peered":"2024-04-17T05:41:35.163578+0000","last_clean":"2024-04-17T05:41:35.163578+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:41:35.163578+0000","last_undegraded":"2024-04-17T05:41:35.163578+0000","last_fullsized":"2024-04-17T05:41:35.163578+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:14:05.831006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150592+0000","last_change":"2024-04-17T04:51:43.673701+0000","last_active":"2024-04-17T05:43:05.150592+0000","last_peered":"2024-04-17T05:43:05.150592+0000","last_clean":"2024-04-17T05:43:05.150592+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:43:05.150592+0000","last_undegraded":"2024-04-17T05:43:05.150592+0000","last_fullsized":"2024-04-17T05:43:05.150592+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:00.395599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154845+0000","last_change":"2024-04-17T04:51:39.625916+0000","last_active":"2024-04-17T05:43:00.154845+0000","last_peered":"2024-04-17T05:43:00.154845+0000","last_clean":"2024-04-17T05:43:00.154845+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:43:00.154845+0000","last_undegraded":"2024-04-17T05:43:00.154845+0000","last_fullsized":"2024-04-17T05:43:00.154845+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:35:16.145280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":195,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164555+0000","last_change":"2024-04-17T04:47:56.960822+0000","last_active":"2024-04-17T05:41:40.164555+0000","last_peered":"2024-04-17T05:41:40.164555+0000","last_clean":"2024-04-17T05:41:40.164555+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:41:40.164555+0000","last_undegraded":"2024-04-17T05:41:40.164555+0000","last_fullsized":"2024-04-17T05:41:40.164555+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:26.089711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.147879+0000","last_change":"2024-04-17T05:07:45.745113+0000","last_active":"2024-04-17T05:42:35.147879+0000","last_peered":"2024-04-17T05:42:35.147879+0000","last_clean":"2024-04-17T05:42:35.147879+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:42:35.147879+0000","last_undegraded":"2024-04-17T05:42:35.147879+0000","last_fullsized":"2024-04-17T05:42:35.147879+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:03.124310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'4","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.190008+0000","last_change":"2024-04-17T04:51:43.682734+0000","last_active":"2024-04-17T05:43:30.190008+0000","last_peered":"2024-04-17T05:43:30.190008+0000","last_clean":"2024-04-17T05:43:30.190008+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:43:30.190008+0000","last_undegraded":"2024-04-17T05:43:30.190008+0000","last_fullsized":"2024-04-17T05:43:30.190008+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:32.531568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":9,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154462+0000","last_change":"2024-04-17T04:51:39.645602+0000","last_active":"2024-04-17T05:43:25.154462+0000","last_peered":"2024-04-17T05:43:25.154462+0000","last_clean":"2024-04-17T05:43:25.154462+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:43:25.154462+0000","last_undegraded":"2024-04-17T05:43:25.154462+0000","last_fullsized":"2024-04-17T05:43:25.154462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:02.515138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.173510+0000","last_change":"2024-04-17T04:47:56.962048+0000","last_active":"2024-04-17T05:42:15.173510+0000","last_peered":"2024-04-17T05:42:15.173510+0000","last_clean":"2024-04-17T05:42:15.173510+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:42:15.173510+0000","last_undegraded":"2024-04-17T05:42:15.173510+0000","last_fullsized":"2024-04-17T05:42:15.173510+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:15.157938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159170+0000","last_change":"2024-04-17T05:07:45.751549+0000","last_active":"2024-04-17T05:43:45.159170+0000","last_peered":"2024-04-17T05:43:45.159170+0000","last_clean":"2024-04-17T05:43:45.159170+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:43:45.159170+0000","last_undegraded":"2024-04-17T05:43:45.159170+0000","last_fullsized":"2024-04-17T05:43:45.159170+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:47.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.189964+0000","last_change":"2024-04-17T04:51:43.688891+0000","last_active":"2024-04-17T05:43:30.189964+0000","last_peered":"2024-04-17T05:43:30.189964+0000","last_clean":"2024-04-17T05:43:30.189964+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:43:30.189964+0000","last_undegraded":"2024-04-17T05:43:30.189964+0000","last_fullsized":"2024-04-17T05:43:30.189964+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:35.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":9216,"num_write":3,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'4","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.159519+0000","last_change":"2024-04-17T04:51:39.634707+0000","last_active":"2024-04-17T05:43:25.159519+0000","last_peered":"2024-04-17T05:43:25.159519+0000","last_clean":"2024-04-17T05:43:25.159519+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:43:25.159519+0000","last_undegraded":"2024-04-17T05:43:25.159519+0000","last_fullsized":"2024-04-17T05:43:25.159519+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:09.043206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":11264,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141762+0000","last_change":"2024-04-17T04:47:56.954332+0000","last_active":"2024-04-17T05:42:00.141762+0000","last_peered":"2024-04-17T05:42:00.141762+0000","last_clean":"2024-04-17T05:42:00.141762+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:42:00.141762+0000","last_undegraded":"2024-04-17T05:42:00.141762+0000","last_fullsized":"2024-04-17T05:42:00.141762+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:07.662187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141842+0000","last_change":"2024-04-17T05:07:45.741143+0000","last_active":"2024-04-17T05:42:25.141842+0000","last_peered":"2024-04-17T05:42:25.141842+0000","last_clean":"2024-04-17T05:42:25.141842+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:42:25.141842+0000","last_undegraded":"2024-04-17T05:42:25.141842+0000","last_fullsized":"2024-04-17T05:42:25.141842+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:50.193462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164848+0000","last_change":"2024-04-17T04:51:43.657675+0000","last_active":"2024-04-17T05:43:40.164848+0000","last_peered":"2024-04-17T05:43:40.164848+0000","last_clean":"2024-04-17T05:43:40.164848+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:43:40.164848+0000","last_undegraded":"2024-04-17T05:43:40.164848+0000","last_fullsized":"2024-04-17T05:43:40.164848+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:59:28.152355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'2","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.156097+0000","last_change":"2024-04-17T04:51:39.634454+0000","last_active":"2024-04-17T05:43:25.156097+0000","last_peered":"2024-04-17T05:43:25.156097+0000","last_clean":"2024-04-17T05:43:25.156097+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:43:25.156097+0000","last_undegraded":"2024-04-17T05:43:25.156097+0000","last_fullsized":"2024-04-17T05:43:25.156097+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:48.747740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":123,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136769+0000","last_change":"2024-04-17T04:47:56.963915+0000","last_active":"2024-04-17T05:42:00.136769+0000","last_peered":"2024-04-17T05:42:00.136769+0000","last_clean":"2024-04-17T05:42:00.136769+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:42:00.136769+0000","last_undegraded":"2024-04-17T05:42:00.136769+0000","last_fullsized":"2024-04-17T05:42:00.136769+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:58.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141220+0000","last_change":"2024-04-17T05:07:45.757127+0000","last_active":"2024-04-17T05:42:25.141220+0000","last_peered":"2024-04-17T05:42:25.141220+0000","last_clean":"2024-04-17T05:42:25.141220+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:42:25.141220+0000","last_undegraded":"2024-04-17T05:42:25.141220+0000","last_fullsized":"2024-04-17T05:42:25.141220+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:42.585132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130813+0000","last_change":"2024-04-17T04:52:04.726201+0000","last_active":"2024-04-17T05:41:35.130813+0000","last_peered":"2024-04-17T05:41:35.130813+0000","last_clean":"2024-04-17T05:41:35.130813+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:41:35.130813+0000","last_undegraded":"2024-04-17T05:41:35.130813+0000","last_fullsized":"2024-04-17T05:41:35.130813+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:14:03.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163394+0000","last_change":"2024-04-17T04:51:39.618531+0000","last_active":"2024-04-17T05:43:35.163394+0000","last_peered":"2024-04-17T05:43:35.163394+0000","last_clean":"2024-04-17T05:43:35.163394+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:43:35.163394+0000","last_undegraded":"2024-04-17T05:43:35.163394+0000","last_fullsized":"2024-04-17T05:43:35.163394+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:52.911539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":128,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139900+0000","last_change":"2024-04-17T04:47:56.962022+0000","last_active":"2024-04-17T05:42:00.139900+0000","last_peered":"2024-04-17T05:42:00.139900+0000","last_clean":"2024-04-17T05:42:00.139900+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:42:00.139900+0000","last_undegraded":"2024-04-17T05:42:00.139900+0000","last_fullsized":"2024-04-17T05:42:00.139900+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:04.015930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193091+0000","last_change":"2024-04-17T05:07:45.744264+0000","last_active":"2024-04-17T05:43:45.193091+0000","last_peered":"2024-04-17T05:43:45.193091+0000","last_clean":"2024-04-17T05:43:45.193091+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:43:45.193091+0000","last_undegraded":"2024-04-17T05:43:45.193091+0000","last_fullsized":"2024-04-17T05:43:45.193091+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:41.404295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'6","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.945897+0000","last_change":"2024-04-17T04:51:43.661076+0000","last_active":"2024-04-17T05:43:44.945897+0000","last_peered":"2024-04-17T05:43:44.945897+0000","last_clean":"2024-04-17T05:43:44.945897+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:43:44.945897+0000","last_undegraded":"2024-04-17T05:43:44.945897+0000","last_fullsized":"2024-04-17T05:43:44.945897+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:54.620750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48424,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":5216,"num_write":11,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159409+0000","last_change":"2024-04-17T04:51:39.633118+0000","last_active":"2024-04-17T05:43:40.159409+0000","last_peered":"2024-04-17T05:43:40.159409+0000","last_clean":"2024-04-17T05:43:40.159409+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:43:40.159409+0000","last_undegraded":"2024-04-17T05:43:40.159409+0000","last_fullsized":"2024-04-17T05:43:40.159409+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:25:34.341448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":102,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137165+0000","last_change":"2024-04-17T04:47:56.963262+0000","last_active":"2024-04-17T05:42:00.137165+0000","last_peered":"2024-04-17T05:42:00.137165+0000","last_clean":"2024-04-17T05:42:00.137165+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:42:00.137165+0000","last_undegraded":"2024-04-17T05:42:00.137165+0000","last_fullsized":"2024-04-17T05:42:00.137165+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:09.975510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.129755+0000","last_change":"2024-04-17T05:07:45.757192+0000","last_active":"2024-04-17T05:41:35.129755+0000","last_peered":"2024-04-17T05:41:35.129755+0000","last_clean":"2024-04-17T05:41:35.129755+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:41:35.129755+0000","last_undegraded":"2024-04-17T05:41:35.129755+0000","last_fullsized":"2024-04-17T05:41:35.129755+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:30:19.313475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'4","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.158953+0000","last_change":"2024-04-17T04:51:43.675078+0000","last_active":"2024-04-17T05:43:45.158953+0000","last_peered":"2024-04-17T05:43:45.158953+0000","last_clean":"2024-04-17T05:43:45.158953+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:43:45.158953+0000","last_undegraded":"2024-04-17T05:43:45.158953+0000","last_fullsized":"2024-04-17T05:43:45.158953+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:00.172403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164451+0000","last_change":"2024-04-17T04:51:39.618066+0000","last_active":"2024-04-17T05:43:40.164451+0000","last_peered":"2024-04-17T05:43:40.164451+0000","last_clean":"2024-04-17T05:43:40.164451+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:43:40.164451+0000","last_undegraded":"2024-04-17T05:43:40.164451+0000","last_fullsized":"2024-04-17T05:43:40.164451+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:51.504887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":286,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140125+0000","last_change":"2024-04-17T04:47:56.963049+0000","last_active":"2024-04-17T05:42:20.140125+0000","last_peered":"2024-04-17T05:42:20.140125+0000","last_clean":"2024-04-17T05:42:20.140125+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:42:20.140125+0000","last_undegraded":"2024-04-17T05:42:20.140125+0000","last_fullsized":"2024-04-17T05:42:20.140125+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:08.912141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159213+0000","last_change":"2024-04-17T05:07:45.754827+0000","last_active":"2024-04-17T05:43:45.159213+0000","last_peered":"2024-04-17T05:43:45.159213+0000","last_clean":"2024-04-17T05:43:45.159213+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:43:45.159213+0000","last_undegraded":"2024-04-17T05:43:45.159213+0000","last_fullsized":"2024-04-17T05:43:45.159213+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:12.463675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.162241+0000","last_change":"2024-04-17T04:51:43.655115+0000","last_active":"2024-04-17T05:43:30.162241+0000","last_peered":"2024-04-17T05:43:30.162241+0000","last_clean":"2024-04-17T05:43:30.162241+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:43:30.162241+0000","last_undegraded":"2024-04-17T05:43:30.162241+0000","last_fullsized":"2024-04-17T05:43:30.162241+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:20.469117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162125+0000","last_change":"2024-04-17T04:51:39.633266+0000","last_active":"2024-04-17T05:43:50.162125+0000","last_peered":"2024-04-17T05:43:50.162125+0000","last_clean":"2024-04-17T05:43:50.162125+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:43:50.162125+0000","last_undegraded":"2024-04-17T05:43:50.162125+0000","last_fullsized":"2024-04-17T05:43:50.162125+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:40.414020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.171752+0000","last_change":"2024-04-17T04:47:56.952507+0000","last_active":"2024-04-17T05:42:10.171752+0000","last_peered":"2024-04-17T05:42:10.171752+0000","last_clean":"2024-04-17T05:42:10.171752+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:42:10.171752+0000","last_undegraded":"2024-04-17T05:42:10.171752+0000","last_fullsized":"2024-04-17T05:42:10.171752+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:18.535221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.160987+0000","last_change":"2024-04-17T05:07:45.739696+0000","last_active":"2024-04-17T05:43:45.160987+0000","last_peered":"2024-04-17T05:43:45.160987+0000","last_clean":"2024-04-17T05:43:45.160987+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:43:45.160987+0000","last_undegraded":"2024-04-17T05:43:45.160987+0000","last_fullsized":"2024-04-17T05:43:45.160987+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:02:31.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.131990+0000","last_change":"2024-04-17T04:51:43.675773+0000","last_active":"2024-04-17T05:41:45.131990+0000","last_peered":"2024-04-17T05:41:45.131990+0000","last_clean":"2024-04-17T05:41:45.131990+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:41:45.131990+0000","last_undegraded":"2024-04-17T05:41:45.131990+0000","last_fullsized":"2024-04-17T05:41:45.131990+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:07.015940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:24.941710+0000","last_change":"2024-04-17T04:51:39.621153+0000","last_active":"2024-04-17T05:43:24.941710+0000","last_peered":"2024-04-17T05:43:24.941710+0000","last_clean":"2024-04-17T05:43:24.941710+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:43:24.941710+0000","last_undegraded":"2024-04-17T05:43:24.941710+0000","last_fullsized":"2024-04-17T05:43:24.941710+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:36.053793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":188,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138665+0000","last_change":"2024-04-17T04:47:56.966581+0000","last_active":"2024-04-17T05:42:10.138665+0000","last_peered":"2024-04-17T05:42:10.138665+0000","last_clean":"2024-04-17T05:42:10.138665+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:42:10.138665+0000","last_undegraded":"2024-04-17T05:42:10.138665+0000","last_fullsized":"2024-04-17T05:42:10.138665+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:35.900737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151057+0000","last_change":"2024-04-17T05:07:45.747152+0000","last_active":"2024-04-17T05:43:10.151057+0000","last_peered":"2024-04-17T05:43:10.151057+0000","last_clean":"2024-04-17T05:43:10.151057+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:43:10.151057+0000","last_undegraded":"2024-04-17T05:43:10.151057+0000","last_fullsized":"2024-04-17T05:43:10.151057+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:50:06.059672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136102+0000","last_change":"2024-04-17T04:51:43.682823+0000","last_active":"2024-04-17T05:41:55.136102+0000","last_peered":"2024-04-17T05:41:55.136102+0000","last_clean":"2024-04-17T05:41:55.136102+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:41:55.136102+0000","last_undegraded":"2024-04-17T05:41:55.136102+0000","last_fullsized":"2024-04-17T05:41:55.136102+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:56:21.798951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'7","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159240+0000","last_change":"2024-04-17T04:51:39.636037+0000","last_active":"2024-04-17T05:43:45.159240+0000","last_peered":"2024-04-17T05:43:45.159240+0000","last_clean":"2024-04-17T05:43:45.159240+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:43:45.159240+0000","last_undegraded":"2024-04-17T05:43:45.159240+0000","last_fullsized":"2024-04-17T05:43:45.159240+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:03.304591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":11291,"num_write":12,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":149,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139651+0000","last_change":"2024-04-17T04:47:56.956029+0000","last_active":"2024-04-17T05:42:10.139651+0000","last_peered":"2024-04-17T05:42:10.139651+0000","last_clean":"2024-04-17T05:42:10.139651+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:42:10.139651+0000","last_undegraded":"2024-04-17T05:42:10.139651+0000","last_fullsized":"2024-04-17T05:42:10.139651+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:21.889729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157001+0000","last_change":"2024-04-17T05:07:45.753495+0000","last_active":"2024-04-17T05:43:10.157001+0000","last_peered":"2024-04-17T05:43:10.157001+0000","last_clean":"2024-04-17T05:43:10.157001+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:43:10.157001+0000","last_undegraded":"2024-04-17T05:43:10.157001+0000","last_fullsized":"2024-04-17T05:43:10.157001+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:29:25.567114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162221+0000","last_change":"2024-04-17T04:51:43.681581+0000","last_active":"2024-04-17T05:43:50.162221+0000","last_peered":"2024-04-17T05:43:50.162221+0000","last_clean":"2024-04-17T05:43:50.162221+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:43:50.162221+0000","last_undegraded":"2024-04-17T05:43:50.162221+0000","last_fullsized":"2024-04-17T05:43:50.162221+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:53.170686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'2","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.165685+0000","last_change":"2024-04-17T04:51:39.634757+0000","last_active":"2024-04-17T05:43:55.165685+0000","last_peered":"2024-04-17T05:43:55.165685+0000","last_clean":"2024-04-17T05:43:55.165685+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:43:55.165685+0000","last_undegraded":"2024-04-17T05:43:55.165685+0000","last_fullsized":"2024-04-17T05:43:55.165685+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:38.980103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":196,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138819+0000","last_change":"2024-04-17T04:47:56.946324+0000","last_active":"2024-04-17T05:42:10.138819+0000","last_peered":"2024-04-17T05:42:10.138819+0000","last_clean":"2024-04-17T05:42:10.138819+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:42:10.138819+0000","last_undegraded":"2024-04-17T05:42:10.138819+0000","last_fullsized":"2024-04-17T05:42:10.138819+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:09.179720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167123+0000","last_change":"2024-04-17T05:07:45.736217+0000","last_active":"2024-04-17T05:43:50.167123+0000","last_peered":"2024-04-17T05:43:50.167123+0000","last_clean":"2024-04-17T05:43:50.167123+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:43:50.167123+0000","last_undegraded":"2024-04-17T05:43:50.167123+0000","last_fullsized":"2024-04-17T05:43:50.167123+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:04.504706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'17","reported_seq":88,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159324+0000","last_change":"2024-04-17T04:51:43.675854+0000","last_active":"2024-04-17T05:43:45.159324+0000","last_peered":"2024-04-17T05:43:45.159324+0000","last_clean":"2024-04-17T05:43:45.159324+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:43:45.159324+0000","last_undegraded":"2024-04-17T05:43:45.159324+0000","last_fullsized":"2024-04-17T05:43:45.159324+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:54.193750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23068672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":54274,"num_write":16,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'10","reported_seq":79,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163666+0000","last_change":"2024-04-17T04:51:39.635143+0000","last_active":"2024-04-17T05:43:40.163666+0000","last_peered":"2024-04-17T05:43:40.163666+0000","last_clean":"2024-04-17T05:43:40.163666+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:43:40.163666+0000","last_undegraded":"2024-04-17T05:43:40.163666+0000","last_fullsized":"2024-04-17T05:43:40.163666+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:53:08.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140249+0000","last_change":"2024-04-17T04:47:56.969411+0000","last_active":"2024-04-17T05:42:20.140249+0000","last_peered":"2024-04-17T05:42:20.140249+0000","last_clean":"2024-04-17T05:42:20.140249+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:42:20.140249+0000","last_undegraded":"2024-04-17T05:42:20.140249+0000","last_fullsized":"2024-04-17T05:42:20.140249+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:56.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938315+0000","last_change":"2024-04-17T05:07:45.749227+0000","last_active":"2024-04-17T05:43:09.938315+0000","last_peered":"2024-04-17T05:43:09.938315+0000","last_clean":"2024-04-17T05:43:09.938315+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:43:09.938315+0000","last_undegraded":"2024-04-17T05:43:09.938315+0000","last_fullsized":"2024-04-17T05:43:09.938315+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:15.680393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946093+0000","last_change":"2024-04-17T04:51:43.648648+0000","last_active":"2024-04-17T05:43:44.946093+0000","last_peered":"2024-04-17T05:43:44.946093+0000","last_clean":"2024-04-17T05:43:44.946093+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:43:44.946093+0000","last_undegraded":"2024-04-17T05:43:44.946093+0000","last_fullsized":"2024-04-17T05:43:44.946093+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:57:25.905644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.158015+0000","last_change":"2024-04-17T04:51:39.646180+0000","last_active":"2024-04-17T05:43:40.158015+0000","last_peered":"2024-04-17T05:43:40.158015+0000","last_clean":"2024-04-17T05:43:40.158015+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:43:40.158015+0000","last_undegraded":"2024-04-17T05:43:40.158015+0000","last_fullsized":"2024-04-17T05:43:40.158015+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:55.361466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":110,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143901+0000","last_change":"2024-04-17T04:47:56.947428+0000","last_active":"2024-04-17T05:42:10.143901+0000","last_peered":"2024-04-17T05:42:10.143901+0000","last_clean":"2024-04-17T05:42:10.143901+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:42:10.143901+0000","last_undegraded":"2024-04-17T05:42:10.143901+0000","last_fullsized":"2024-04-17T05:42:10.143901+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:57.355596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"263'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145978+0000","last_change":"2024-04-17T05:07:45.746243+0000","last_active":"2024-04-17T05:42:25.145978+0000","last_peered":"2024-04-17T05:42:25.145978+0000","last_clean":"2024-04-17T05:42:25.145978+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:42:25.145978+0000","last_undegraded":"2024-04-17T05:42:25.145978+0000","last_fullsized":"2024-04-17T05:42:25.145978+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:08:11.491152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'7","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167234+0000","last_change":"2024-04-17T04:51:43.675539+0000","last_active":"2024-04-17T05:43:50.167234+0000","last_peered":"2024-04-17T05:43:50.167234+0000","last_clean":"2024-04-17T05:43:50.167234+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:43:50.167234+0000","last_undegraded":"2024-04-17T05:43:50.167234+0000","last_fullsized":"2024-04-17T05:43:50.167234+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:29.255105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160253+0000","last_change":"2024-04-17T04:51:39.637181+0000","last_active":"2024-04-17T05:43:50.160253+0000","last_peered":"2024-04-17T05:43:50.160253+0000","last_clean":"2024-04-17T05:43:50.160253+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:43:50.160253+0000","last_undegraded":"2024-04-17T05:43:50.160253+0000","last_fullsized":"2024-04-17T05:43:50.160253+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:38.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143904+0000","last_change":"2024-04-17T04:47:56.949538+0000","last_active":"2024-04-17T05:42:10.143904+0000","last_peered":"2024-04-17T05:42:10.143904+0000","last_clean":"2024-04-17T05:42:10.143904+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:42:10.143904+0000","last_undegraded":"2024-04-17T05:42:10.143904+0000","last_fullsized":"2024-04-17T05:42:10.143904+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:29.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143092+0000","last_change":"2024-04-17T05:07:45.726938+0000","last_active":"2024-04-17T05:42:25.143092+0000","last_peered":"2024-04-17T05:42:25.143092+0000","last_clean":"2024-04-17T05:42:25.143092+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:42:25.143092+0000","last_undegraded":"2024-04-17T05:42:25.143092+0000","last_fullsized":"2024-04-17T05:42:25.143092+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:33.108430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'2","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.166629+0000","last_change":"2024-04-17T04:51:43.648636+0000","last_active":"2024-04-17T05:43:55.166629+0000","last_peered":"2024-04-17T05:43:55.166629+0000","last_clean":"2024-04-17T05:43:55.166629+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:43:55.166629+0000","last_undegraded":"2024-04-17T05:43:55.166629+0000","last_fullsized":"2024-04-17T05:43:55.166629+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:44.200548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159094+0000","last_change":"2024-04-17T04:51:39.636922+0000","last_active":"2024-04-17T05:43:45.159094+0000","last_peered":"2024-04-17T05:43:45.159094+0000","last_clean":"2024-04-17T05:43:45.159094+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:43:45.159094+0000","last_undegraded":"2024-04-17T05:43:45.159094+0000","last_fullsized":"2024-04-17T05:43:45.159094+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:38.269210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.129678+0000","last_change":"2024-04-17T04:47:56.968894+0000","last_active":"2024-04-17T05:42:10.129678+0000","last_peered":"2024-04-17T05:42:10.129678+0000","last_clean":"2024-04-17T05:42:10.129678+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:42:10.129678+0000","last_undegraded":"2024-04-17T05:42:10.129678+0000","last_fullsized":"2024-04-17T05:42:10.129678+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:47.848243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152961+0000","last_change":"2024-04-17T05:07:45.757247+0000","last_active":"2024-04-17T05:43:10.152961+0000","last_peered":"2024-04-17T05:43:10.152961+0000","last_clean":"2024-04-17T05:43:10.152961+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:43:10.152961+0000","last_undegraded":"2024-04-17T05:43:10.152961+0000","last_fullsized":"2024-04-17T05:43:10.152961+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:26.267898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160974+0000","last_change":"2024-04-17T04:51:43.682310+0000","last_active":"2024-04-17T05:43:50.160974+0000","last_peered":"2024-04-17T05:43:50.160974+0000","last_clean":"2024-04-17T05:43:50.160974+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:43:50.160974+0000","last_undegraded":"2024-04-17T05:43:50.160974+0000","last_fullsized":"2024-04-17T05:43:50.160974+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:14.153241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'1","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165820+0000","last_change":"2024-04-17T04:51:39.617751+0000","last_active":"2024-04-17T05:43:45.165820+0000","last_peered":"2024-04-17T05:43:45.165820+0000","last_clean":"2024-04-17T05:43:45.165820+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:43:45.165820+0000","last_undegraded":"2024-04-17T05:43:45.165820+0000","last_fullsized":"2024-04-17T05:43:45.165820+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:45.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":1,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":213,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138713+0000","last_change":"2024-04-17T04:47:56.952060+0000","last_active":"2024-04-17T05:42:10.138713+0000","last_peered":"2024-04-17T05:42:10.138713+0000","last_clean":"2024-04-17T05:42:10.138713+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:42:10.138713+0000","last_undegraded":"2024-04-17T05:42:10.138713+0000","last_fullsized":"2024-04-17T05:42:10.138713+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:52.024764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.152311+0000","last_change":"2024-04-17T05:07:45.752461+0000","last_active":"2024-04-17T05:43:15.152311+0000","last_peered":"2024-04-17T05:43:15.152311+0000","last_clean":"2024-04-17T05:43:15.152311+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:43:15.152311+0000","last_undegraded":"2024-04-17T05:43:15.152311+0000","last_fullsized":"2024-04-17T05:43:15.152311+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:43.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:49.947148+0000","last_change":"2024-04-17T04:51:43.651232+0000","last_active":"2024-04-17T05:43:49.947148+0000","last_peered":"2024-04-17T05:43:49.947148+0000","last_clean":"2024-04-17T05:43:49.947148+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:43:49.947148+0000","last_undegraded":"2024-04-17T05:43:49.947148+0000","last_fullsized":"2024-04-17T05:43:49.947148+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:48.905181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159752+0000","last_change":"2024-04-17T04:51:39.630310+0000","last_active":"2024-04-17T05:43:45.159752+0000","last_peered":"2024-04-17T05:43:45.159752+0000","last_clean":"2024-04-17T05:43:45.159752+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:43:45.159752+0000","last_undegraded":"2024-04-17T05:43:45.159752+0000","last_fullsized":"2024-04-17T05:43:45.159752+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:32:14.085186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6339880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6192,"num_write":3,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":127,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.137755+0000","last_change":"2024-04-17T04:47:56.963237+0000","last_active":"2024-04-17T05:42:10.137755+0000","last_peered":"2024-04-17T05:42:10.137755+0000","last_clean":"2024-04-17T05:42:10.137755+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:42:10.137755+0000","last_undegraded":"2024-04-17T05:42:10.137755+0000","last_fullsized":"2024-04-17T05:42:10.137755+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:11.681306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'285","reported_seq":946,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150137+0000","last_change":"2024-04-17T04:36:25.377106+0000","last_active":"2024-04-17T05:43:05.150137+0000","last_peered":"2024-04-17T05:43:05.150137+0000","last_clean":"2024-04-17T05:43:05.150137+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:43:05.150137+0000","last_undegraded":"2024-04-17T05:43:05.150137+0000","last_fullsized":"2024-04-17T05:43:05.150137+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:09.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53724,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":483,"num_read_kb":725,"num_write":1250,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":675,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134422+0000","last_change":"2024-04-17T04:36:27.389744+0000","last_active":"2024-04-17T05:41:30.134422+0000","last_peered":"2024-04-17T05:41:30.134422+0000","last_clean":"2024-04-17T05:41:30.134422+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:41:30.134422+0000","last_undegraded":"2024-04-17T05:41:30.134422+0000","last_fullsized":"2024-04-17T05:41:30.134422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:14.577671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'308","reported_seq":980,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157287+0000","last_change":"2024-04-17T04:36:25.357080+0000","last_active":"2024-04-17T05:43:05.157287+0000","last_peered":"2024-04-17T05:43:05.157287+0000","last_clean":"2024-04-17T05:43:05.157287+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:43:05.157287+0000","last_undegraded":"2024-04-17T05:43:05.157287+0000","last_fullsized":"2024-04-17T05:43:05.157287+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:21.403329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10755049,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":526,"num_read_kb":10797,"num_write":1432,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":706,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:39.489661+0000","last_change":"2024-04-17T04:36:27.382828+0000","last_active":"2024-04-17T05:42:39.489661+0000","last_peered":"2024-04-17T05:42:39.489661+0000","last_clean":"2024-04-17T05:42:39.489661+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:42:39.489661+0000","last_undegraded":"2024-04-17T05:42:39.489661+0000","last_fullsized":"2024-04-17T05:42:39.489661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:30.715656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'254","reported_seq":920,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151824+0000","last_change":"2024-04-17T04:36:25.329662+0000","last_active":"2024-04-17T05:43:05.151824+0000","last_peered":"2024-04-17T05:43:05.151824+0000","last_clean":"2024-04-17T05:43:05.151824+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:43:05.151824+0000","last_undegraded":"2024-04-17T05:43:05.151824+0000","last_fullsized":"2024-04-17T05:43:05.151824+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:42.966712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528901,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":417,"num_read_kb":58401,"num_write":1131,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120297+0000","last_change":"2024-04-17T04:36:27.385236+0000","last_active":"2024-04-17T05:41:30.120297+0000","last_peered":"2024-04-17T05:41:30.120297+0000","last_clean":"2024-04-17T05:41:30.120297+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:41:30.120297+0000","last_undegraded":"2024-04-17T05:41:30.120297+0000","last_fullsized":"2024-04-17T05:41:30.120297+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:55.872226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'241","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150409+0000","last_change":"2024-04-17T04:36:25.356234+0000","last_active":"2024-04-17T05:42:40.150409+0000","last_peered":"2024-04-17T05:42:40.150409+0000","last_clean":"2024-04-17T05:42:40.150409+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:42:40.150409+0000","last_undegraded":"2024-04-17T05:42:40.150409+0000","last_fullsized":"2024-04-17T05:42:40.150409+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:14.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5765916,"num_objects":48,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":417,"num_read_kb":37884,"num_write":1085,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'304","reported_seq":897,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.129356+0000","last_change":"2024-04-17T04:36:27.385726+0000","last_active":"2024-04-17T05:41:30.129356+0000","last_peered":"2024-04-17T05:41:30.129356+0000","last_clean":"2024-04-17T05:41:30.129356+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:41:30.129356+0000","last_undegraded":"2024-04-17T05:41:30.129356+0000","last_fullsized":"2024-04-17T05:41:30.129356+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:55:33.151184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'266","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.127332+0000","last_change":"2024-04-17T04:36:25.377714+0000","last_active":"2024-04-17T05:41:25.127332+0000","last_peered":"2024-04-17T05:41:25.127332+0000","last_clean":"2024-04-17T05:41:25.127332+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:41:25.127332+0000","last_undegraded":"2024-04-17T05:41:25.127332+0000","last_fullsized":"2024-04-17T05:41:25.127332+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:34.861649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19973206,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":458,"num_read_kb":13359,"num_write":1103,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5179,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.736719+0000","last_change":"2024-04-17T04:36:27.382622+0000","last_active":"2024-04-17T05:45:44.736719+0000","last_peered":"2024-04-17T05:45:44.736719+0000","last_clean":"2024-04-17T05:45:44.736719+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:45:44.736719+0000","last_undegraded":"2024-04-17T05:45:44.736719+0000","last_fullsized":"2024-04-17T05:45:44.736719+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:23.921181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'230","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:43.796816+0000","last_change":"2024-04-17T04:36:25.352114+0000","last_active":"2024-04-17T05:41:43.796816+0000","last_peered":"2024-04-17T05:41:43.796816+0000","last_clean":"2024-04-17T05:41:43.796816+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:41:43.796816+0000","last_undegraded":"2024-04-17T05:41:43.796816+0000","last_fullsized":"2024-04-17T05:41:43.796816+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:17.459792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11021151,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":513,"num_read_kb":64592,"num_write":906,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":774,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938531+0000","last_change":"2024-04-17T04:36:27.385407+0000","last_active":"2024-04-17T05:43:09.938531+0000","last_peered":"2024-04-17T05:43:09.938531+0000","last_clean":"2024-04-17T05:43:09.938531+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:43:09.938531+0000","last_undegraded":"2024-04-17T05:43:09.938531+0000","last_fullsized":"2024-04-17T05:43:09.938531+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:03.112120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'259","reported_seq":881,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:37.900457+0000","last_change":"2024-04-17T04:36:25.376858+0000","last_active":"2024-04-17T05:42:37.900457+0000","last_peered":"2024-04-17T05:42:37.900457+0000","last_clean":"2024-04-17T05:42:37.900457+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:42:37.900457+0000","last_undegraded":"2024-04-17T05:42:37.900457+0000","last_fullsized":"2024-04-17T05:42:37.900457+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:24.809459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3318739,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":393,"num_read_kb":9422,"num_write":1064,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157138+0000","last_change":"2024-04-17T04:36:27.376772+0000","last_active":"2024-04-17T05:43:10.157138+0000","last_peered":"2024-04-17T05:43:10.157138+0000","last_clean":"2024-04-17T05:43:10.157138+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:43:10.157138+0000","last_undegraded":"2024-04-17T05:43:10.157138+0000","last_fullsized":"2024-04-17T05:43:10.157138+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:07.055230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'260","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.718287+0000","last_change":"2024-04-17T04:36:25.343980+0000","last_active":"2024-04-17T05:43:39.718287+0000","last_peered":"2024-04-17T05:43:39.718287+0000","last_clean":"2024-04-17T05:43:39.718287+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:43:39.718287+0000","last_undegraded":"2024-04-17T05:43:39.718287+0000","last_fullsized":"2024-04-17T05:43:39.718287+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:21.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20922664,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":369,"num_read_kb":14992,"num_write":1106,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.156665+0000","last_change":"2024-04-17T04:36:27.379088+0000","last_active":"2024-04-17T05:43:10.156665+0000","last_peered":"2024-04-17T05:43:10.156665+0000","last_clean":"2024-04-17T05:43:10.156665+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:43:10.156665+0000","last_undegraded":"2024-04-17T05:43:10.156665+0000","last_fullsized":"2024-04-17T05:43:10.156665+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:05.867623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125191+0000","last_change":"2024-04-17T04:36:27.385695+0000","last_active":"2024-04-17T05:41:50.125191+0000","last_peered":"2024-04-17T05:41:50.125191+0000","last_clean":"2024-04-17T05:41:50.125191+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:41:50.125191+0000","last_undegraded":"2024-04-17T05:41:50.125191+0000","last_fullsized":"2024-04-17T05:41:50.125191+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:33.234173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'255","reported_seq":862,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.148032+0000","last_change":"2024-04-17T04:37:03.648715+0000","last_active":"2024-04-17T05:42:35.148032+0000","last_peered":"2024-04-17T05:42:35.148032+0000","last_clean":"2024-04-17T05:42:35.148032+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:42:35.148032+0000","last_undegraded":"2024-04-17T05:42:35.148032+0000","last_fullsized":"2024-04-17T05:42:35.148032+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:12.036980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8728170,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":386,"num_read_kb":14103,"num_write":1112,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'255","reported_seq":889,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.407437+0000","last_change":"2024-04-17T04:36:27.383282+0000","last_active":"2024-04-17T05:45:46.407437+0000","last_peered":"2024-04-17T05:45:46.407437+0000","last_clean":"2024-04-17T05:45:46.407437+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:45:46.407437+0000","last_undegraded":"2024-04-17T05:45:46.407437+0000","last_fullsized":"2024-04-17T05:45:46.407437+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:29.116974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'216","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:47.860045+0000","last_change":"2024-04-17T04:36:25.385368+0000","last_active":"2024-04-17T05:42:47.860045+0000","last_peered":"2024-04-17T05:42:47.860045+0000","last_clean":"2024-04-17T05:42:47.860045+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:42:47.860045+0000","last_undegraded":"2024-04-17T05:42:47.860045+0000","last_fullsized":"2024-04-17T05:42:47.860045+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:35.267010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761635,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":351,"num_read_kb":10549,"num_write":915,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":723,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142870+0000","last_change":"2024-04-17T04:36:27.376262+0000","last_active":"2024-04-17T05:42:05.142870+0000","last_peered":"2024-04-17T05:42:05.142870+0000","last_clean":"2024-04-17T05:42:05.142870+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:42:05.142870+0000","last_undegraded":"2024-04-17T05:42:05.142870+0000","last_fullsized":"2024-04-17T05:42:05.142870+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:02.524761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'248","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.390159+0000","last_change":"2024-04-17T04:36:25.385723+0000","last_active":"2024-04-17T05:43:39.390159+0000","last_peered":"2024-04-17T05:43:39.390159+0000","last_clean":"2024-04-17T05:43:39.390159+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:43:39.390159+0000","last_undegraded":"2024-04-17T05:43:39.390159+0000","last_fullsized":"2024-04-17T05:43:39.390159+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:51.669081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498677,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":402,"num_read_kb":160096,"num_write":1072,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136478+0000","last_change":"2024-04-17T04:36:27.383859+0000","last_active":"2024-04-17T05:41:45.136478+0000","last_peered":"2024-04-17T05:41:45.136478+0000","last_clean":"2024-04-17T05:41:45.136478+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:41:45.136478+0000","last_undegraded":"2024-04-17T05:41:45.136478+0000","last_fullsized":"2024-04-17T05:41:45.136478+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:14.617020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'242","reported_seq":947,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:52.574040+0000","last_change":"2024-04-17T04:36:25.361757+0000","last_active":"2024-04-17T05:41:52.574040+0000","last_peered":"2024-04-17T05:41:52.574040+0000","last_clean":"2024-04-17T05:41:52.574040+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:41:52.574040+0000","last_undegraded":"2024-04-17T05:41:52.574040+0000","last_fullsized":"2024-04-17T05:41:52.574040+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:21.146448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19295786,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":522,"num_read_kb":64725,"num_write":1051,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136549+0000","last_change":"2024-04-17T04:36:27.379593+0000","last_active":"2024-04-17T05:41:45.136549+0000","last_peered":"2024-04-17T05:41:45.136549+0000","last_clean":"2024-04-17T05:41:45.136549+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:41:45.136549+0000","last_undegraded":"2024-04-17T05:41:45.136549+0000","last_fullsized":"2024-04-17T05:41:45.136549+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:37.024479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'256","reported_seq":1026,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.874797+0000","last_change":"2024-04-17T04:36:25.374400+0000","last_active":"2024-04-17T05:42:57.874797+0000","last_peered":"2024-04-17T05:42:57.874797+0000","last_clean":"2024-04-17T05:42:57.874797+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:42:57.874797+0000","last_undegraded":"2024-04-17T05:42:57.874797+0000","last_fullsized":"2024-04-17T05:42:57.874797+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:25:10.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26759829,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":748,"num_read_kb":57779,"num_write":1204,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.501738+0000","last_change":"2024-04-17T04:36:27.385279+0000","last_active":"2024-04-17T05:41:55.501738+0000","last_peered":"2024-04-17T05:41:55.501738+0000","last_clean":"2024-04-17T05:41:55.501738+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:41:55.501738+0000","last_undegraded":"2024-04-17T05:41:55.501738+0000","last_fullsized":"2024-04-17T05:41:55.501738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:17.292970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'280","reported_seq":914,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135979+0000","last_change":"2024-04-17T04:36:25.355404+0000","last_active":"2024-04-17T05:41:55.135979+0000","last_peered":"2024-04-17T05:41:55.135979+0000","last_clean":"2024-04-17T05:41:55.135979+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:41:55.135979+0000","last_undegraded":"2024-04-17T05:41:55.135979+0000","last_fullsized":"2024-04-17T05:41:55.135979+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:47.270461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10613748,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":397,"num_read_kb":836,"num_write":1108,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'152","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.671918+0000","last_change":"2024-04-17T04:36:35.418351+0000","last_active":"2024-04-17T05:45:46.671918+0000","last_peered":"2024-04-17T05:45:46.671918+0000","last_clean":"2024-04-17T05:45:46.671918+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:45:46.671918+0000","last_undegraded":"2024-04-17T05:45:46.671918+0000","last_fullsized":"2024-04-17T05:45:46.671918+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:02.595419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:51.868791+0000","last_change":"2024-04-17T04:36:27.384993+0000","last_active":"2024-04-17T05:42:51.868791+0000","last_peered":"2024-04-17T05:42:51.868791+0000","last_clean":"2024-04-17T05:42:51.868791+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:42:51.868791+0000","last_undegraded":"2024-04-17T05:42:51.868791+0000","last_fullsized":"2024-04-17T05:42:51.868791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:24.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'258","reported_seq":882,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.719639+0000","last_change":"2024-04-17T04:36:25.368467+0000","last_active":"2024-04-17T05:42:50.719639+0000","last_peered":"2024-04-17T05:42:50.719639+0000","last_clean":"2024-04-17T05:42:50.719639+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:42:50.719639+0000","last_undegraded":"2024-04-17T05:42:50.719639+0000","last_fullsized":"2024-04-17T05:42:50.719639+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:23.161029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12579502,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":373,"num_read_kb":2527,"num_write":1113,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138168+0000","last_change":"2024-04-17T04:36:29.388609+0000","last_active":"2024-04-17T05:41:45.138168+0000","last_peered":"2024-04-17T05:41:45.138168+0000","last_clean":"2024-04-17T05:41:45.138168+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:41:45.138168+0000","last_undegraded":"2024-04-17T05:41:45.138168+0000","last_fullsized":"2024-04-17T05:41:45.138168+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:53.809254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'362","reported_seq":982,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.169232+0000","last_change":"2024-04-17T04:36:31.410990+0000","last_active":"2024-04-17T05:44:15.169232+0000","last_peered":"2024-04-17T05:44:15.169232+0000","last_clean":"2024-04-17T05:44:15.169232+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:44:15.169232+0000","last_undegraded":"2024-04-17T05:44:15.169232+0000","last_fullsized":"2024-04-17T05:44:15.169232+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":300,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:24.430923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58423,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":1138,"num_write":466,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133607+0000","last_change":"2024-04-17T04:36:33.425089+0000","last_active":"2024-04-17T05:41:50.133607+0000","last_peered":"2024-04-17T05:41:50.133607+0000","last_clean":"2024-04-17T05:41:50.133607+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:41:50.133607+0000","last_undegraded":"2024-04-17T05:41:50.133607+0000","last_fullsized":"2024-04-17T05:41:50.133607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:53.730080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'169","reported_seq":722,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.761057+0000","last_change":"2024-04-17T04:36:35.416351+0000","last_active":"2024-04-17T05:45:44.761057+0000","last_peered":"2024-04-17T05:45:44.761057+0000","last_clean":"2024-04-17T05:45:44.761057+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:45:44.761057+0000","last_undegraded":"2024-04-17T05:45:44.761057+0000","last_fullsized":"2024-04-17T05:45:44.761057+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:34.873231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":673,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138074+0000","last_change":"2024-04-17T04:36:27.389710+0000","last_active":"2024-04-17T05:41:45.138074+0000","last_peered":"2024-04-17T05:41:45.138074+0000","last_clean":"2024-04-17T05:41:45.138074+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:41:45.138074+0000","last_undegraded":"2024-04-17T05:41:45.138074+0000","last_fullsized":"2024-04-17T05:41:45.138074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:11:08.849990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'278","reported_seq":1049,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133284+0000","last_change":"2024-04-17T04:36:25.328153+0000","last_active":"2024-04-17T05:41:45.133284+0000","last_peered":"2024-04-17T05:41:45.133284+0000","last_clean":"2024-04-17T05:41:45.133284+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:41:45.133284+0000","last_undegraded":"2024-04-17T05:41:45.133284+0000","last_fullsized":"2024-04-17T05:41:45.133284+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:07.072313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21193393,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":693,"num_read_kb":61435,"num_write":1302,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.170670+0000","last_change":"2024-04-17T04:36:29.373876+0000","last_active":"2024-04-17T05:42:05.170670+0000","last_peered":"2024-04-17T05:42:05.170670+0000","last_clean":"2024-04-17T05:42:05.170670+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:42:05.170670+0000","last_undegraded":"2024-04-17T05:42:05.170670+0000","last_fullsized":"2024-04-17T05:42:05.170670+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:06.355032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'99","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125438+0000","last_change":"2024-04-17T04:36:31.395007+0000","last_active":"2024-04-17T05:41:50.125438+0000","last_peered":"2024-04-17T05:41:50.125438+0000","last_clean":"2024-04-17T05:41:50.125438+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:41:50.125438+0000","last_undegraded":"2024-04-17T05:41:50.125438+0000","last_fullsized":"2024-04-17T05:41:50.125438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:47.124749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125417+0000","last_change":"2024-04-17T04:36:33.414084+0000","last_active":"2024-04-17T05:41:50.125417+0000","last_peered":"2024-04-17T05:41:50.125417+0000","last_clean":"2024-04-17T05:41:50.125417+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:41:50.125417+0000","last_undegraded":"2024-04-17T05:41:50.125417+0000","last_fullsized":"2024-04-17T05:41:50.125417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:27.803230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'123","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.222678+0000","last_change":"2024-04-17T04:36:35.427530+0000","last_active":"2024-04-17T05:45:44.222678+0000","last_peered":"2024-04-17T05:45:44.222678+0000","last_clean":"2024-04-17T05:45:44.222678+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:45:44.222678+0000","last_undegraded":"2024-04-17T05:45:44.222678+0000","last_fullsized":"2024-04-17T05:45:44.222678+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:45.651840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":659,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:09.959182+0000","last_change":"2024-04-17T04:36:27.376107+0000","last_active":"2024-04-17T05:42:09.959182+0000","last_peered":"2024-04-17T05:42:09.959182+0000","last_clean":"2024-04-17T05:42:09.959182+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:42:09.959182+0000","last_undegraded":"2024-04-17T05:42:09.959182+0000","last_fullsized":"2024-04-17T05:42:09.959182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:13.603194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'276","reported_seq":1055,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.131908+0000","last_change":"2024-04-17T04:36:25.381456+0000","last_active":"2024-04-17T05:41:45.131908+0000","last_peered":"2024-04-17T05:41:45.131908+0000","last_clean":"2024-04-17T05:41:45.131908+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:41:45.131908+0000","last_undegraded":"2024-04-17T05:41:45.131908+0000","last_fullsized":"2024-04-17T05:41:45.131908+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:08.488003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22475342,"num_objects":61,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":615,"num_read_kb":75395,"num_write":1261,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165842+0000","last_change":"2024-04-17T04:36:29.359822+0000","last_active":"2024-04-17T05:41:45.165842+0000","last_peered":"2024-04-17T05:41:45.165842+0000","last_clean":"2024-04-17T05:41:45.165842+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:41:45.165842+0000","last_undegraded":"2024-04-17T05:41:45.165842+0000","last_fullsized":"2024-04-17T05:41:45.165842+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:41.948364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'137","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.141895+0000","last_change":"2024-04-17T04:36:31.391956+0000","last_active":"2024-04-17T05:42:10.141895+0000","last_peered":"2024-04-17T05:42:10.141895+0000","last_clean":"2024-04-17T05:42:10.141895+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:42:10.141895+0000","last_undegraded":"2024-04-17T05:42:10.141895+0000","last_fullsized":"2024-04-17T05:42:10.141895+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:12.846752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'512","reported_seq":2409,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.670706+0000","last_change":"2024-04-17T04:36:33.420806+0000","last_active":"2024-04-17T05:45:57.670706+0000","last_peered":"2024-04-17T05:45:57.670706+0000","last_clean":"2024-04-17T05:45:57.670706+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:45:57.670706+0000","last_undegraded":"2024-04-17T05:45:57.670706+0000","last_fullsized":"2024-04-17T05:45:57.670706+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:52.301226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'195","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.415632+0000","last_change":"2024-04-17T04:36:35.403573+0000","last_active":"2024-04-17T05:45:46.415632+0000","last_peered":"2024-04-17T05:45:46.415632+0000","last_clean":"2024-04-17T05:45:46.415632+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:45:46.415632+0000","last_undegraded":"2024-04-17T05:45:46.415632+0000","last_fullsized":"2024-04-17T05:45:46.415632+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:09.033539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:07.139896+0000","last_change":"2024-04-17T04:36:27.395613+0000","last_active":"2024-04-17T05:43:07.139896+0000","last_peered":"2024-04-17T05:43:07.139896+0000","last_clean":"2024-04-17T05:43:07.139896+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:43:07.139896+0000","last_undegraded":"2024-04-17T05:43:07.139896+0000","last_fullsized":"2024-04-17T05:43:07.139896+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:32:19.325576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'235","reported_seq":881,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133330+0000","last_change":"2024-04-17T04:36:25.384743+0000","last_active":"2024-04-17T05:41:45.133330+0000","last_peered":"2024-04-17T05:41:45.133330+0000","last_clean":"2024-04-17T05:41:45.133330+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:41:45.133330+0000","last_undegraded":"2024-04-17T05:41:45.133330+0000","last_fullsized":"2024-04-17T05:41:45.133330+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:05.451652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727402,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":395,"num_read_kb":48921,"num_write":1085,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138271+0000","last_change":"2024-04-17T04:36:29.391920+0000","last_active":"2024-04-17T05:41:45.138271+0000","last_peered":"2024-04-17T05:41:45.138271+0000","last_clean":"2024-04-17T05:41:45.138271+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:41:45.138271+0000","last_undegraded":"2024-04-17T05:41:45.138271+0000","last_fullsized":"2024-04-17T05:41:45.138271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:00:30.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'274","reported_seq":854,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166406+0000","last_change":"2024-04-17T04:36:31.422728+0000","last_active":"2024-04-17T05:44:15.166406+0000","last_peered":"2024-04-17T05:44:15.166406+0000","last_clean":"2024-04-17T05:44:15.166406+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:44:15.166406+0000","last_undegraded":"2024-04-17T05:44:15.166406+0000","last_fullsized":"2024-04-17T05:44:15.166406+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:05.940948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62885,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":973,"num_write":370,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'529","reported_seq":2297,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.662523+0000","last_change":"2024-04-17T04:36:33.422550+0000","last_active":"2024-04-17T05:45:57.662523+0000","last_peered":"2024-04-17T05:45:57.662523+0000","last_clean":"2024-04-17T05:45:57.662523+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:45:57.662523+0000","last_undegraded":"2024-04-17T05:45:57.662523+0000","last_fullsized":"2024-04-17T05:45:57.662523+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":500,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:46.514333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'121","reported_seq":720,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163455+0000","last_change":"2024-04-17T04:36:27.398309+0000","last_active":"2024-04-17T05:43:40.163455+0000","last_peered":"2024-04-17T05:43:40.163455+0000","last_clean":"2024-04-17T05:43:40.163455+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:43:40.163455+0000","last_undegraded":"2024-04-17T05:43:40.163455+0000","last_fullsized":"2024-04-17T05:43:40.163455+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:23.743688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'230","reported_seq":863,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133579+0000","last_change":"2024-04-17T04:36:25.370799+0000","last_active":"2024-04-17T05:41:50.133579+0000","last_peered":"2024-04-17T05:41:50.133579+0000","last_clean":"2024-04-17T05:41:50.133579+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:41:50.133579+0000","last_undegraded":"2024-04-17T05:41:50.133579+0000","last_fullsized":"2024-04-17T05:41:50.133579+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:55.146915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15464766,"num_objects":51,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":386,"num_read_kb":24078,"num_write":1075,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.157843+0000","last_change":"2024-04-17T04:36:29.389738+0000","last_active":"2024-04-17T05:43:40.157843+0000","last_peered":"2024-04-17T05:43:40.157843+0000","last_clean":"2024-04-17T05:43:40.157843+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:43:40.157843+0000","last_undegraded":"2024-04-17T05:43:40.157843+0000","last_fullsized":"2024-04-17T05:43:40.157843+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:52.759093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'377","reported_seq":1009,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.167892+0000","last_change":"2024-04-17T04:36:31.422910+0000","last_active":"2024-04-17T05:44:00.167892+0000","last_peered":"2024-04-17T05:44:00.167892+0000","last_clean":"2024-04-17T05:44:00.167892+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:44:00.167892+0000","last_undegraded":"2024-04-17T05:44:00.167892+0000","last_fullsized":"2024-04-17T05:44:00.167892+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:47.096298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54931,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":606,"num_read_kb":1124,"num_write":470,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139387+0000","last_change":"2024-04-17T04:36:33.408109+0000","last_active":"2024-04-17T05:41:50.139387+0000","last_peered":"2024-04-17T05:41:50.139387+0000","last_clean":"2024-04-17T05:41:50.139387+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:41:50.139387+0000","last_undegraded":"2024-04-17T05:41:50.139387+0000","last_fullsized":"2024-04-17T05:41:50.139387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:19.914865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'560","reported_seq":1384,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.656195+0000","last_change":"2024-04-17T04:36:35.409093+0000","last_active":"2024-04-17T05:45:54.656195+0000","last_peered":"2024-04-17T05:45:54.656195+0000","last_clean":"2024-04-17T05:45:54.656195+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:45:54.656195+0000","last_undegraded":"2024-04-17T05:45:54.656195+0000","last_fullsized":"2024-04-17T05:45:54.656195+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":500,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:27.394663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":865,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.944851+0000","last_change":"2024-04-17T04:36:27.379851+0000","last_active":"2024-04-17T05:43:39.944851+0000","last_peered":"2024-04-17T05:43:39.944851+0000","last_clean":"2024-04-17T05:43:39.944851+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:43:39.944851+0000","last_undegraded":"2024-04-17T05:43:39.944851+0000","last_fullsized":"2024-04-17T05:43:39.944851+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:27.808945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'312","reported_seq":972,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:12.983824+0000","last_change":"2024-04-17T04:36:25.371549+0000","last_active":"2024-04-17T05:42:12.983824+0000","last_peered":"2024-04-17T05:42:12.983824+0000","last_clean":"2024-04-17T05:42:12.983824+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:42:12.983824+0000","last_undegraded":"2024-04-17T05:42:12.983824+0000","last_fullsized":"2024-04-17T05:42:12.983824+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:35.252546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7755476,"num_objects":85,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":488,"num_read_kb":14712,"num_write":1238,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162258+0000","last_change":"2024-04-17T04:36:29.371794+0000","last_active":"2024-04-17T05:43:40.162258+0000","last_peered":"2024-04-17T05:43:40.162258+0000","last_clean":"2024-04-17T05:43:40.162258+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:43:40.162258+0000","last_undegraded":"2024-04-17T05:43:40.162258+0000","last_fullsized":"2024-04-17T05:43:40.162258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:11.383775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'264","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198231+0000","last_change":"2024-04-17T04:36:31.392065+0000","last_active":"2024-04-17T05:44:05.198231+0000","last_peered":"2024-04-17T05:44:05.198231+0000","last_clean":"2024-04-17T05:44:05.198231+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:44:05.198231+0000","last_undegraded":"2024-04-17T05:44:05.198231+0000","last_fullsized":"2024-04-17T05:44:05.198231+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:36.246942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45337,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":414,"num_read_kb":838,"num_write":330,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166729+0000","last_change":"2024-04-17T04:36:33.416143+0000","last_active":"2024-04-17T05:41:50.166729+0000","last_peered":"2024-04-17T05:41:50.166729+0000","last_clean":"2024-04-17T05:41:50.166729+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:41:50.166729+0000","last_undegraded":"2024-04-17T05:41:50.166729+0000","last_fullsized":"2024-04-17T05:41:50.166729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:51.569882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2371","reported_seq":4173,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.067191+0000","last_change":"2024-04-17T04:36:35.418228+0000","last_active":"2024-04-17T05:45:40.067191+0000","last_peered":"2024-04-17T05:45:40.067191+0000","last_clean":"2024-04-17T05:45:40.067191+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:45:40.067191+0000","last_undegraded":"2024-04-17T05:45:40.067191+0000","last_fullsized":"2024-04-17T05:45:40.067191+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":2300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.125017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160924+0000","last_change":"2024-04-17T04:36:27.376914+0000","last_active":"2024-04-17T05:43:50.160924+0000","last_peered":"2024-04-17T05:43:50.160924+0000","last_clean":"2024-04-17T05:43:50.160924+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:43:50.160924+0000","last_undegraded":"2024-04-17T05:43:50.160924+0000","last_fullsized":"2024-04-17T05:43:50.160924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:06.533166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'288","reported_seq":962,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:12.960895+0000","last_change":"2024-04-17T04:36:25.386734+0000","last_active":"2024-04-17T05:42:12.960895+0000","last_peered":"2024-04-17T05:42:12.960895+0000","last_clean":"2024-04-17T05:42:12.960895+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:42:12.960895+0000","last_undegraded":"2024-04-17T05:42:12.960895+0000","last_fullsized":"2024-04-17T05:42:12.960895+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:49.654467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7146143,"num_objects":73,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":536,"num_read_kb":65373,"num_write":1251,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136503+0000","last_change":"2024-04-17T04:36:29.372163+0000","last_active":"2024-04-17T05:41:45.136503+0000","last_peered":"2024-04-17T05:41:45.136503+0000","last_clean":"2024-04-17T05:41:45.136503+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:41:45.136503+0000","last_undegraded":"2024-04-17T05:41:45.136503+0000","last_fullsized":"2024-04-17T05:41:45.136503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:29.815813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'330","reported_seq":941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.632269+0000","last_change":"2024-04-17T04:36:31.380567+0000","last_active":"2024-04-17T05:45:54.632269+0000","last_peered":"2024-04-17T05:45:54.632269+0000","last_clean":"2024-04-17T05:45:54.632269+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:45:54.632269+0000","last_undegraded":"2024-04-17T05:45:54.632269+0000","last_fullsized":"2024-04-17T05:45:54.632269+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:55.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":530,"num_read_kb":1046,"num_write":422,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'534","reported_seq":4610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.650496+0000","last_change":"2024-04-17T04:36:33.411578+0000","last_active":"2024-04-17T05:45:57.650496+0000","last_peered":"2024-04-17T05:45:57.650496+0000","last_clean":"2024-04-17T05:45:57.650496+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:45:57.650496+0000","last_undegraded":"2024-04-17T05:45:57.650496+0000","last_fullsized":"2024-04-17T05:45:57.650496+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.576073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161111+0000","last_change":"2024-04-17T04:36:35.427581+0000","last_active":"2024-04-17T05:43:45.161111+0000","last_peered":"2024-04-17T05:43:45.161111+0000","last_clean":"2024-04-17T05:43:45.161111+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:43:45.161111+0000","last_undegraded":"2024-04-17T05:43:45.161111+0000","last_fullsized":"2024-04-17T05:43:45.161111+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:13.700005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'95","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.081384+0000","last_change":"2024-04-17T04:36:27.370100+0000","last_active":"2024-04-17T05:42:57.081384+0000","last_peered":"2024-04-17T05:42:57.081384+0000","last_clean":"2024-04-17T05:42:57.081384+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:42:57.081384+0000","last_undegraded":"2024-04-17T05:42:57.081384+0000","last_fullsized":"2024-04-17T05:42:57.081384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:31.666063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'275","reported_seq":974,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144551+0000","last_change":"2024-04-17T04:37:03.648700+0000","last_active":"2024-04-17T05:42:20.144551+0000","last_peered":"2024-04-17T05:42:20.144551+0000","last_clean":"2024-04-17T05:42:20.144551+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:42:20.144551+0000","last_undegraded":"2024-04-17T05:42:20.144551+0000","last_fullsized":"2024-04-17T05:42:20.144551+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:49:08.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6559602,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":598,"num_read_kb":24373,"num_write":1106,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136436+0000","last_change":"2024-04-17T04:36:29.375596+0000","last_active":"2024-04-17T05:41:45.136436+0000","last_peered":"2024-04-17T05:41:45.136436+0000","last_clean":"2024-04-17T05:41:45.136436+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:41:45.136436+0000","last_undegraded":"2024-04-17T05:41:45.136436+0000","last_fullsized":"2024-04-17T05:41:45.136436+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:20.886949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'462","reported_seq":1109,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.633698+0000","last_change":"2024-04-17T04:36:31.409877+0000","last_active":"2024-04-17T05:45:54.633698+0000","last_peered":"2024-04-17T05:45:54.633698+0000","last_clean":"2024-04-17T05:45:54.633698+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:45:54.633698+0000","last_undegraded":"2024-04-17T05:45:54.633698+0000","last_fullsized":"2024-04-17T05:45:54.633698+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:19.065697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101390,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":805,"num_read_kb":1767,"num_write":654,"num_write_kb":532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.163452+0000","last_change":"2024-04-17T04:36:33.420378+0000","last_active":"2024-04-17T05:43:45.163452+0000","last_peered":"2024-04-17T05:43:45.163452+0000","last_clean":"2024-04-17T05:43:45.163452+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:43:45.163452+0000","last_undegraded":"2024-04-17T05:43:45.163452+0000","last_fullsized":"2024-04-17T05:43:45.163452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:59.519269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'148","reported_seq":689,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165985+0000","last_change":"2024-04-17T04:36:35.417260+0000","last_active":"2024-04-17T05:43:45.165985+0000","last_peered":"2024-04-17T05:43:45.165985+0000","last_clean":"2024-04-17T05:43:45.165985+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:43:45.165985+0000","last_undegraded":"2024-04-17T05:43:45.165985+0000","last_fullsized":"2024-04-17T05:43:45.165985+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:41.967458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124135+0000","last_change":"2024-04-17T04:36:27.393949+0000","last_active":"2024-04-17T05:41:45.124135+0000","last_peered":"2024-04-17T05:41:45.124135+0000","last_clean":"2024-04-17T05:41:45.124135+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:41:45.124135+0000","last_undegraded":"2024-04-17T05:41:45.124135+0000","last_fullsized":"2024-04-17T05:41:45.124135+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:12.169777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'287","reported_seq":955,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164840+0000","last_change":"2024-04-17T04:36:25.379472+0000","last_active":"2024-04-17T05:41:40.164840+0000","last_peered":"2024-04-17T05:41:40.164840+0000","last_clean":"2024-04-17T05:41:40.164840+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:41:40.164840+0000","last_undegraded":"2024-04-17T05:41:40.164840+0000","last_fullsized":"2024-04-17T05:41:40.164840+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:42:53.335830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2019600,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":501,"num_read_kb":9569,"num_write":1164,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136613+0000","last_change":"2024-04-17T04:36:29.374168+0000","last_active":"2024-04-17T05:41:45.136613+0000","last_peered":"2024-04-17T05:41:45.136613+0000","last_clean":"2024-04-17T05:41:45.136613+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:41:45.136613+0000","last_undegraded":"2024-04-17T05:41:45.136613+0000","last_fullsized":"2024-04-17T05:41:45.136613+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:22.063423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'308","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.158285+0000","last_change":"2024-04-17T04:36:31.392925+0000","last_active":"2024-04-17T05:44:05.158285+0000","last_peered":"2024-04-17T05:44:05.158285+0000","last_clean":"2024-04-17T05:44:05.158285+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:44:05.158285+0000","last_undegraded":"2024-04-17T05:44:05.158285+0000","last_fullsized":"2024-04-17T05:44:05.158285+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:08.285743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56427,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":522,"num_read_kb":1053,"num_write":466,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":494,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137350+0000","last_change":"2024-04-17T04:36:33.430909+0000","last_active":"2024-04-17T05:42:00.137350+0000","last_peered":"2024-04-17T05:42:00.137350+0000","last_clean":"2024-04-17T05:42:00.137350+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:42:00.137350+0000","last_undegraded":"2024-04-17T05:42:00.137350+0000","last_fullsized":"2024-04-17T05:42:00.137350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:54.144240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'129","reported_seq":636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.818089+0000","last_change":"2024-04-17T04:36:35.423084+0000","last_active":"2024-04-17T05:45:40.818089+0000","last_peered":"2024-04-17T05:45:40.818089+0000","last_clean":"2024-04-17T05:45:40.818089+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:45:40.818089+0000","last_undegraded":"2024-04-17T05:45:40.818089+0000","last_fullsized":"2024-04-17T05:45:40.818089+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:20:19.232177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:53.923395+0000","last_change":"2024-04-17T04:36:27.383527+0000","last_active":"2024-04-17T05:41:53.923395+0000","last_peered":"2024-04-17T05:41:53.923395+0000","last_clean":"2024-04-17T05:41:53.923395+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:41:53.923395+0000","last_undegraded":"2024-04-17T05:41:53.923395+0000","last_fullsized":"2024-04-17T05:41:53.923395+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:29.969137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'301","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159689+0000","last_change":"2024-04-17T04:36:25.371674+0000","last_active":"2024-04-17T05:43:45.159689+0000","last_peered":"2024-04-17T05:43:45.159689+0000","last_clean":"2024-04-17T05:43:45.159689+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:43:45.159689+0000","last_undegraded":"2024-04-17T05:43:45.159689+0000","last_fullsized":"2024-04-17T05:43:45.159689+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:54.700946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6845506,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":545,"num_read_kb":15072,"num_write":1194,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:29.359143+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:37.440920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'287","reported_seq":891,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.171451+0000","last_change":"2024-04-17T04:36:31.397866+0000","last_active":"2024-04-17T05:44:20.171451+0000","last_peered":"2024-04-17T05:44:20.171451+0000","last_clean":"2024-04-17T05:44:20.171451+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:44:20.171451+0000","last_undegraded":"2024-04-17T05:44:20.171451+0000","last_fullsized":"2024-04-17T05:44:20.171451+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:42.568179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51070,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":477,"num_read_kb":941,"num_write":374,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'508","reported_seq":2439,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.663877+0000","last_change":"2024-04-17T04:36:33.408560+0000","last_active":"2024-04-17T05:45:57.663877+0000","last_peered":"2024-04-17T05:45:57.663877+0000","last_clean":"2024-04-17T05:45:57.663877+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:45:57.663877+0000","last_undegraded":"2024-04-17T05:45:57.663877+0000","last_fullsized":"2024-04-17T05:45:57.663877+0000","mapping_epoch":31,"log_start":"261'400","ondisk_log_start":"261'400","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":400,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:41.906428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'96","reported_seq":602,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:42.569922+0000","last_change":"2024-04-17T04:36:35.428863+0000","last_active":"2024-04-17T05:45:42.569922+0000","last_peered":"2024-04-17T05:45:42.569922+0000","last_clean":"2024-04-17T05:45:42.569922+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:45:42.569922+0000","last_undegraded":"2024-04-17T05:45:42.569922+0000","last_fullsized":"2024-04-17T05:45:42.569922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:09.522109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.169748+0000","last_change":"2024-04-17T04:36:27.393058+0000","last_active":"2024-04-17T05:42:30.169748+0000","last_peered":"2024-04-17T05:42:30.169748+0000","last_clean":"2024-04-17T05:42:30.169748+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:42:30.169748+0000","last_undegraded":"2024-04-17T05:42:30.169748+0000","last_fullsized":"2024-04-17T05:42:30.169748+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:22.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'245","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:34.943708+0000","last_change":"2024-04-17T04:36:25.356100+0000","last_active":"2024-04-17T05:43:34.943708+0000","last_peered":"2024-04-17T05:43:34.943708+0000","last_clean":"2024-04-17T05:43:34.943708+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:43:34.943708+0000","last_undegraded":"2024-04-17T05:43:34.943708+0000","last_fullsized":"2024-04-17T05:43:34.943708+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:10.485745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10486600,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":525,"num_read_kb":5671,"num_write":1142,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"28'2","reported_seq":1150,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.527318+0000","last_change":"2024-04-17T04:36:29.389132+0000","last_active":"2024-04-17T05:45:54.527318+0000","last_peered":"2024-04-17T05:45:54.527318+0000","last_clean":"2024-04-17T05:45:54.527318+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:45:54.527318+0000","last_undegraded":"2024-04-17T05:45:54.527318+0000","last_fullsized":"2024-04-17T05:45:54.527318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:09:43.654615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'97","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166974+0000","last_change":"2024-04-17T04:36:31.421130+0000","last_active":"2024-04-17T05:41:50.166974+0000","last_peered":"2024-04-17T05:41:50.166974+0000","last_clean":"2024-04-17T05:41:50.166974+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:41:50.166974+0000","last_undegraded":"2024-04-17T05:41:50.166974+0000","last_fullsized":"2024-04-17T05:41:50.166974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:38.963408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'535","reported_seq":2243,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.671105+0000","last_change":"2024-04-17T04:36:33.421354+0000","last_active":"2024-04-17T05:45:57.671105+0000","last_peered":"2024-04-17T05:45:57.671105+0000","last_clean":"2024-04-17T05:45:57.671105+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:45:57.671105+0000","last_undegraded":"2024-04-17T05:45:57.671105+0000","last_fullsized":"2024-04-17T05:45:57.671105+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":500,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:27.382198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'97","reported_seq":600,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939559+0000","last_change":"2024-04-17T04:36:35.427123+0000","last_active":"2024-04-17T05:43:14.939559+0000","last_peered":"2024-04-17T05:43:14.939559+0000","last_clean":"2024-04-17T05:43:14.939559+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:43:14.939559+0000","last_undegraded":"2024-04-17T05:43:14.939559+0000","last_fullsized":"2024-04-17T05:43:14.939559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:26.157064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":756,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141817+0000","last_change":"2024-04-17T04:36:27.377947+0000","last_active":"2024-04-17T05:42:00.141817+0000","last_peered":"2024-04-17T05:42:00.141817+0000","last_clean":"2024-04-17T05:42:00.141817+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:42:00.141817+0000","last_undegraded":"2024-04-17T05:42:00.141817+0000","last_fullsized":"2024-04-17T05:42:00.141817+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:47.608940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'228","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.190870+0000","last_change":"2024-04-17T04:36:25.353720+0000","last_active":"2024-04-17T05:43:35.190870+0000","last_peered":"2024-04-17T05:43:35.190870+0000","last_clean":"2024-04-17T05:43:35.190870+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:43:35.190870+0000","last_undegraded":"2024-04-17T05:43:35.190870+0000","last_fullsized":"2024-04-17T05:43:35.190870+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:24.629387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15899766,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":342,"num_read_kb":36760,"num_write":925,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124238+0000","last_change":"2024-04-17T04:36:29.389735+0000","last_active":"2024-04-17T05:41:45.124238+0000","last_peered":"2024-04-17T05:41:45.124238+0000","last_clean":"2024-04-17T05:41:45.124238+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:41:45.124238+0000","last_undegraded":"2024-04-17T05:41:45.124238+0000","last_fullsized":"2024-04-17T05:41:45.124238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:19.695221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'347","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.169175+0000","last_change":"2024-04-17T04:36:31.398102+0000","last_active":"2024-04-17T05:44:05.169175+0000","last_peered":"2024-04-17T05:44:05.169175+0000","last_clean":"2024-04-17T05:44:05.169175+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:44:05.169175+0000","last_undegraded":"2024-04-17T05:44:05.169175+0000","last_fullsized":"2024-04-17T05:44:05.169175+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:41.468361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52190,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":568,"num_read_kb":1054,"num_write":442,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134628+0000","last_change":"2024-04-17T04:36:33.419643+0000","last_active":"2024-04-17T05:41:50.134628+0000","last_peered":"2024-04-17T05:41:50.134628+0000","last_clean":"2024-04-17T05:41:50.134628+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:41:50.134628+0000","last_undegraded":"2024-04-17T05:41:50.134628+0000","last_fullsized":"2024-04-17T05:41:50.134628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:51.100488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'157","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.190550+0000","last_change":"2024-04-17T04:36:35.408247+0000","last_active":"2024-04-17T05:45:20.190550+0000","last_peered":"2024-04-17T05:45:20.190550+0000","last_clean":"2024-04-17T05:45:20.190550+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:45:20.190550+0000","last_undegraded":"2024-04-17T05:45:20.190550+0000","last_fullsized":"2024-04-17T05:45:20.190550+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:26.932136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":695,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.154250+0000","last_change":"2024-04-17T04:36:27.387349+0000","last_active":"2024-04-17T05:43:20.154250+0000","last_peered":"2024-04-17T05:43:20.154250+0000","last_clean":"2024-04-17T05:43:20.154250+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:43:20.154250+0000","last_undegraded":"2024-04-17T05:43:20.154250+0000","last_fullsized":"2024-04-17T05:43:20.154250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:39.509316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":680,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141161+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T05:42:25.141161+0000","last_peered":"2024-04-17T05:42:25.141161+0000","last_clean":"2024-04-17T05:42:25.141161+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:42:25.141161+0000","last_undegraded":"2024-04-17T05:42:25.141161+0000","last_fullsized":"2024-04-17T05:42:25.141161+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:24:00.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154818+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T05:43:00.154818+0000","last_peered":"2024-04-17T05:43:00.154818+0000","last_clean":"2024-04-17T05:43:00.154818+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:43:00.154818+0000","last_undegraded":"2024-04-17T05:43:00.154818+0000","last_fullsized":"2024-04-17T05:43:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:13.926220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'276","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.184874+0000","last_change":"2024-04-17T04:36:25.364067+0000","last_active":"2024-04-17T05:43:05.184874+0000","last_peered":"2024-04-17T05:43:05.184874+0000","last_clean":"2024-04-17T05:43:05.184874+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:43:05.184874+0000","last_undegraded":"2024-04-17T05:43:05.184874+0000","last_fullsized":"2024-04-17T05:43:05.184874+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:41.272537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346866,"num_objects":58,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":496,"num_read_kb":2736,"num_write":1164,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151134+0000","last_change":"2024-04-17T04:36:29.381453+0000","last_active":"2024-04-17T05:43:10.151134+0000","last_peered":"2024-04-17T05:43:10.151134+0000","last_clean":"2024-04-17T05:43:10.151134+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:43:10.151134+0000","last_undegraded":"2024-04-17T05:43:10.151134+0000","last_fullsized":"2024-04-17T05:43:10.151134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:32.455502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'709","reported_seq":1415,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.648327+0000","last_change":"2024-04-17T04:36:31.400231+0000","last_active":"2024-04-17T05:45:54.648327+0000","last_peered":"2024-04-17T05:45:54.648327+0000","last_clean":"2024-04-17T05:45:54.648327+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:45:54.648327+0000","last_undegraded":"2024-04-17T05:45:54.648327+0000","last_fullsized":"2024-04-17T05:45:54.648327+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":600,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:02.459768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155640,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1372,"num_read_kb":2849,"num_write":1330,"num_write_kb":1169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161079+0000","last_change":"2024-04-17T04:36:33.430394+0000","last_active":"2024-04-17T05:43:45.161079+0000","last_peered":"2024-04-17T05:43:45.161079+0000","last_clean":"2024-04-17T05:43:45.161079+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:43:45.161079+0000","last_undegraded":"2024-04-17T05:43:45.161079+0000","last_fullsized":"2024-04-17T05:43:45.161079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:45.115863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'220","reported_seq":913,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.519743+0000","last_change":"2024-04-17T04:36:35.421183+0000","last_active":"2024-04-17T05:45:46.519743+0000","last_peered":"2024-04-17T05:45:46.519743+0000","last_clean":"2024-04-17T05:45:46.519743+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:45:46.519743+0000","last_undegraded":"2024-04-17T05:45:46.519743+0000","last_fullsized":"2024-04-17T05:45:46.519743+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:30.565325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159648+0000","last_change":"2024-04-17T04:36:27.365454+0000","last_active":"2024-04-17T05:43:40.159648+0000","last_peered":"2024-04-17T05:43:40.159648+0000","last_clean":"2024-04-17T05:43:40.159648+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:43:40.159648+0000","last_undegraded":"2024-04-17T05:43:40.159648+0000","last_fullsized":"2024-04-17T05:43:40.159648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:16.200473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":524,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142294+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T05:42:30.142294+0000","last_peered":"2024-04-17T05:42:30.142294+0000","last_clean":"2024-04-17T05:42:30.142294+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:42:30.142294+0000","last_undegraded":"2024-04-17T05:42:30.142294+0000","last_fullsized":"2024-04-17T05:42:30.142294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:06.659093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'280","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151677+0000","last_change":"2024-04-17T04:36:25.355561+0000","last_active":"2024-04-17T05:43:05.151677+0000","last_peered":"2024-04-17T05:43:05.151677+0000","last_clean":"2024-04-17T05:43:05.151677+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:43:05.151677+0000","last_undegraded":"2024-04-17T05:43:05.151677+0000","last_fullsized":"2024-04-17T05:43:05.151677+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:21.131078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12803559,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":507,"num_read_kb":3714,"num_write":1174,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163722+0000","last_change":"2024-04-17T04:36:29.393231+0000","last_active":"2024-04-17T05:43:40.163722+0000","last_peered":"2024-04-17T05:43:40.163722+0000","last_clean":"2024-04-17T05:43:40.163722+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:43:40.163722+0000","last_undegraded":"2024-04-17T05:43:40.163722+0000","last_fullsized":"2024-04-17T05:43:40.163722+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:30.045764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1084","reported_seq":2786,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.664430+0000","last_change":"2024-04-17T04:36:31.398322+0000","last_active":"2024-04-17T05:45:57.664430+0000","last_peered":"2024-04-17T05:45:57.664430+0000","last_clean":"2024-04-17T05:45:57.664430+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:45:57.664430+0000","last_undegraded":"2024-04-17T05:45:57.664430+0000","last_fullsized":"2024-04-17T05:45:57.664430+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":1000,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.046142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":178612,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1777,"num_read_kb":3331,"num_write":1104,"num_write_kb":931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154560+0000","last_change":"2024-04-17T04:36:33.422428+0000","last_active":"2024-04-17T05:43:25.154560+0000","last_peered":"2024-04-17T05:43:25.154560+0000","last_clean":"2024-04-17T05:43:25.154560+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:43:25.154560+0000","last_undegraded":"2024-04-17T05:43:25.154560+0000","last_fullsized":"2024-04-17T05:43:25.154560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:00.479531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'112","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.389082+0000","last_change":"2024-04-17T04:36:35.402669+0000","last_active":"2024-04-17T05:45:46.389082+0000","last_peered":"2024-04-17T05:45:46.389082+0000","last_clean":"2024-04-17T05:45:46.389082+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:45:46.389082+0000","last_undegraded":"2024-04-17T05:45:46.389082+0000","last_fullsized":"2024-04-17T05:45:46.389082+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:05.019981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164737+0000","last_change":"2024-04-17T04:36:27.376154+0000","last_active":"2024-04-17T05:43:40.164737+0000","last_peered":"2024-04-17T05:43:40.164737+0000","last_clean":"2024-04-17T05:43:40.164737+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:43:40.164737+0000","last_undegraded":"2024-04-17T05:43:40.164737+0000","last_fullsized":"2024-04-17T05:43:40.164737+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:53.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":530,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.151409+0000","last_change":"2024-04-17T04:36:17.752374+0000","last_active":"2024-04-17T05:42:55.151409+0000","last_peered":"2024-04-17T05:42:55.151409+0000","last_clean":"2024-04-17T05:42:55.151409+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:42:55.151409+0000","last_undegraded":"2024-04-17T05:42:55.151409+0000","last_fullsized":"2024-04-17T05:42:55.151409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:41.044005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'243","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.150111+0000","last_change":"2024-04-17T04:36:25.343663+0000","last_active":"2024-04-17T05:42:40.150111+0000","last_peered":"2024-04-17T05:42:40.150111+0000","last_clean":"2024-04-17T05:42:40.150111+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:42:40.150111+0000","last_undegraded":"2024-04-17T05:42:40.150111+0000","last_fullsized":"2024-04-17T05:42:40.150111+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:49.114178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5703040,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":424,"num_read_kb":21971,"num_write":1062,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"28'1","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162452+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:43:40.162452+0000","last_peered":"2024-04-17T05:43:40.162452+0000","last_clean":"2024-04-17T05:43:40.162452+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:43:40.162452+0000","last_undegraded":"2024-04-17T05:43:40.162452+0000","last_fullsized":"2024-04-17T05:43:40.162452+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:01.144890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'234","reported_seq":1618,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:52.541681+0000","last_change":"2024-04-17T04:36:31.427048+0000","last_active":"2024-04-17T05:45:52.541681+0000","last_peered":"2024-04-17T05:45:52.541681+0000","last_clean":"2024-04-17T05:45:52.541681+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:45:52.541681+0000","last_undegraded":"2024-04-17T05:45:52.541681+0000","last_fullsized":"2024-04-17T05:45:52.541681+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:49:05.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153957+0000","last_change":"2024-04-17T04:36:33.426587+0000","last_active":"2024-04-17T05:43:15.153957+0000","last_peered":"2024-04-17T05:43:15.153957+0000","last_clean":"2024-04-17T05:43:15.153957+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:43:15.153957+0000","last_undegraded":"2024-04-17T05:43:15.153957+0000","last_fullsized":"2024-04-17T05:43:15.153957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:52:34.456097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'262","reported_seq":866,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.184123+0000","last_change":"2024-04-17T04:36:35.408138+0000","last_active":"2024-04-17T05:45:45.184123+0000","last_peered":"2024-04-17T05:45:45.184123+0000","last_clean":"2024-04-17T05:45:45.184123+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:45:45.184123+0000","last_undegraded":"2024-04-17T05:45:45.184123+0000","last_fullsized":"2024-04-17T05:45:45.184123+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:09.321611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158227+0000","last_change":"2024-04-17T04:36:27.361706+0000","last_active":"2024-04-17T05:43:10.158227+0000","last_peered":"2024-04-17T05:43:10.158227+0000","last_clean":"2024-04-17T05:43:10.158227+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:43:10.158227+0000","last_undegraded":"2024-04-17T05:43:10.158227+0000","last_fullsized":"2024-04-17T05:43:10.158227+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:25.186757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154761+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T05:43:00.154761+0000","last_peered":"2024-04-17T05:43:00.154761+0000","last_clean":"2024-04-17T05:43:00.154761+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:43:00.154761+0000","last_undegraded":"2024-04-17T05:43:00.154761+0000","last_fullsized":"2024-04-17T05:43:00.154761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:44:32.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'278","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:04.937303+0000","last_change":"2024-04-17T04:36:25.356426+0000","last_active":"2024-04-17T05:43:04.937303+0000","last_peered":"2024-04-17T05:43:04.937303+0000","last_clean":"2024-04-17T05:43:04.937303+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:43:04.937303+0000","last_undegraded":"2024-04-17T05:43:04.937303+0000","last_fullsized":"2024-04-17T05:43:04.937303+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:48.430125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11586207,"num_objects":75,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":563,"num_read_kb":36553,"num_write":1174,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.155027+0000","last_change":"2024-04-17T04:36:29.371781+0000","last_active":"2024-04-17T05:43:20.155027+0000","last_peered":"2024-04-17T05:43:20.155027+0000","last_clean":"2024-04-17T05:43:20.155027+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:43:20.155027+0000","last_undegraded":"2024-04-17T05:43:20.155027+0000","last_fullsized":"2024-04-17T05:43:20.155027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:40:17.404012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'139","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193023+0000","last_change":"2024-04-17T04:36:31.400012+0000","last_active":"2024-04-17T05:43:45.193023+0000","last_peered":"2024-04-17T05:43:45.193023+0000","last_clean":"2024-04-17T05:43:45.193023+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:43:45.193023+0000","last_undegraded":"2024-04-17T05:43:45.193023+0000","last_fullsized":"2024-04-17T05:43:45.193023+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:30:23.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159837+0000","last_change":"2024-04-17T04:36:33.416438+0000","last_active":"2024-04-17T05:43:45.159837+0000","last_peered":"2024-04-17T05:43:45.159837+0000","last_clean":"2024-04-17T05:43:45.159837+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:43:45.159837+0000","last_undegraded":"2024-04-17T05:43:45.159837+0000","last_fullsized":"2024-04-17T05:43:45.159837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:15:47.410813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'130","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.164559+0000","last_change":"2024-04-17T04:36:35.411210+0000","last_active":"2024-04-17T05:43:45.164559+0000","last_peered":"2024-04-17T05:43:45.164559+0000","last_clean":"2024-04-17T05:43:45.164559+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:43:45.164559+0000","last_undegraded":"2024-04-17T05:43:45.164559+0000","last_fullsized":"2024-04-17T05:43:45.164559+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:26.503048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158134+0000","last_change":"2024-04-17T04:36:27.376368+0000","last_active":"2024-04-17T05:43:10.158134+0000","last_peered":"2024-04-17T05:43:10.158134+0000","last_clean":"2024-04-17T05:43:10.158134+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:43:10.158134+0000","last_undegraded":"2024-04-17T05:43:10.158134+0000","last_fullsized":"2024-04-17T05:43:10.158134+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:23:39.170608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'231","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.159243+0000","last_change":"2024-04-17T04:36:25.368624+0000","last_active":"2024-04-17T05:43:20.159243+0000","last_peered":"2024-04-17T05:43:20.159243+0000","last_clean":"2024-04-17T05:43:20.159243+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:43:20.159243+0000","last_undegraded":"2024-04-17T05:43:20.159243+0000","last_fullsized":"2024-04-17T05:43:20.159243+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:51.630737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84605,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":406,"num_read_kb":1448,"num_write":1062,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.156128+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T05:43:00.156128+0000","last_peered":"2024-04-17T05:43:00.156128+0000","last_clean":"2024-04-17T05:43:00.156128+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:43:00.156128+0000","last_undegraded":"2024-04-17T05:43:00.156128+0000","last_fullsized":"2024-04-17T05:43:00.156128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:16.202650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152928+0000","last_change":"2024-04-17T04:36:29.384820+0000","last_active":"2024-04-17T05:43:10.152928+0000","last_peered":"2024-04-17T05:43:10.152928+0000","last_clean":"2024-04-17T05:43:10.152928+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:43:10.152928+0000","last_undegraded":"2024-04-17T05:43:10.152928+0000","last_fullsized":"2024-04-17T05:43:10.152928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:27.497471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'610","reported_seq":1296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.603615+0000","last_change":"2024-04-17T04:36:31.411434+0000","last_active":"2024-04-17T05:45:54.603615+0000","last_peered":"2024-04-17T05:45:54.603615+0000","last_clean":"2024-04-17T05:45:54.603615+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:45:54.603615+0000","last_undegraded":"2024-04-17T05:45:54.603615+0000","last_fullsized":"2024-04-17T05:45:54.603615+0000","mapping_epoch":29,"log_start":"264'500","ondisk_log_start":"264'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:30.011121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101724,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1633,"num_read_kb":2600,"num_write":2162,"num_write_kb":1969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939512+0000","last_change":"2024-04-17T04:36:33.407911+0000","last_active":"2024-04-17T05:43:14.939512+0000","last_peered":"2024-04-17T05:43:14.939512+0000","last_clean":"2024-04-17T05:43:14.939512+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:43:14.939512+0000","last_undegraded":"2024-04-17T05:43:14.939512+0000","last_fullsized":"2024-04-17T05:43:14.939512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:49:35.539455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'82","reported_seq":584,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.166476+0000","last_change":"2024-04-17T04:36:35.429331+0000","last_active":"2024-04-17T05:44:05.166476+0000","last_peered":"2024-04-17T05:44:05.166476+0000","last_clean":"2024-04-17T05:44:05.166476+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:44:05.166476+0000","last_undegraded":"2024-04-17T05:44:05.166476+0000","last_fullsized":"2024-04-17T05:44:05.166476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:03.344843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":592,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152888+0000","last_change":"2024-04-17T04:36:27.376396+0000","last_active":"2024-04-17T05:43:10.152888+0000","last_peered":"2024-04-17T05:43:10.152888+0000","last_clean":"2024-04-17T05:43:10.152888+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:43:10.152888+0000","last_undegraded":"2024-04-17T05:43:10.152888+0000","last_fullsized":"2024-04-17T05:43:10.152888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:36.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'203","reported_seq":824,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157227+0000","last_change":"2024-04-17T04:36:25.345048+0000","last_active":"2024-04-17T05:43:05.157227+0000","last_peered":"2024-04-17T05:43:05.157227+0000","last_clean":"2024-04-17T05:43:05.157227+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:43:05.157227+0000","last_undegraded":"2024-04-17T05:43:05.157227+0000","last_fullsized":"2024-04-17T05:43:05.157227+0000","mapping_epoch":23,"log_start":"264'100","ondisk_log_start":"264'100","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:40:34.714988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9471022,"num_objects":68,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":330,"num_read_kb":35211,"num_write":775,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":523,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151170+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T05:43:10.151170+0000","last_peered":"2024-04-17T05:43:10.151170+0000","last_clean":"2024-04-17T05:43:10.151170+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:43:10.151170+0000","last_undegraded":"2024-04-17T05:43:10.151170+0000","last_fullsized":"2024-04-17T05:43:10.151170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:33.083623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151795+0000","last_change":"2024-04-17T04:36:29.375400+0000","last_active":"2024-04-17T05:43:10.151795+0000","last_peered":"2024-04-17T05:43:10.151795+0000","last_clean":"2024-04-17T05:43:10.151795+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:43:10.151795+0000","last_undegraded":"2024-04-17T05:43:10.151795+0000","last_fullsized":"2024-04-17T05:43:10.151795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:16.669890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'199","reported_seq":792,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.159277+0000","last_change":"2024-04-17T04:36:31.420876+0000","last_active":"2024-04-17T05:43:15.159277+0000","last_peered":"2024-04-17T05:43:15.159277+0000","last_clean":"2024-04-17T05:43:15.159277+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:43:15.159277+0000","last_undegraded":"2024-04-17T05:43:15.159277+0000","last_fullsized":"2024-04-17T05:43:15.159277+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:23:18.629160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'536","reported_seq":4354,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.668605+0000","last_change":"2024-04-17T04:36:33.407900+0000","last_active":"2024-04-17T05:45:57.668605+0000","last_peered":"2024-04-17T05:45:57.668605+0000","last_clean":"2024-04-17T05:45:57.668605+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:45:57.668605+0000","last_undegraded":"2024-04-17T05:45:57.668605+0000","last_fullsized":"2024-04-17T05:45:57.668605+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:36:50.014322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'156","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.202363+0000","last_change":"2024-04-17T04:36:35.418551+0000","last_active":"2024-04-17T05:44:20.202363+0000","last_peered":"2024-04-17T05:44:20.202363+0000","last_clean":"2024-04-17T05:44:20.202363+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:44:20.202363+0000","last_undegraded":"2024-04-17T05:44:20.202363+0000","last_fullsized":"2024-04-17T05:44:20.202363+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:08.177362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":799,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180216+0000","last_change":"2024-04-17T04:36:27.395791+0000","last_active":"2024-04-17T05:42:45.180216+0000","last_peered":"2024-04-17T05:42:45.180216+0000","last_clean":"2024-04-17T05:42:45.180216+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:42:45.180216+0000","last_undegraded":"2024-04-17T05:42:45.180216+0000","last_fullsized":"2024-04-17T05:42:45.180216+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:29:33.119512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'309","reported_seq":966,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191121+0000","last_change":"2024-04-17T04:36:25.347483+0000","last_active":"2024-04-17T05:43:35.191121+0000","last_peered":"2024-04-17T05:43:35.191121+0000","last_clean":"2024-04-17T05:43:35.191121+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:43:35.191121+0000","last_undegraded":"2024-04-17T05:43:35.191121+0000","last_fullsized":"2024-04-17T05:43:35.191121+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:52:07.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290300,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":521,"num_read_kb":5842,"num_write":1316,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":527,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.161303+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T05:43:25.161303+0000","last_peered":"2024-04-17T05:43:25.161303+0000","last_clean":"2024-04-17T05:43:25.161303+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:43:25.161303+0000","last_undegraded":"2024-04-17T05:43:25.161303+0000","last_fullsized":"2024-04-17T05:43:25.161303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:38.757105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938398+0000","last_change":"2024-04-17T04:36:29.375601+0000","last_active":"2024-04-17T05:43:09.938398+0000","last_peered":"2024-04-17T05:43:09.938398+0000","last_clean":"2024-04-17T05:43:09.938398+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:43:09.938398+0000","last_undegraded":"2024-04-17T05:43:09.938398+0000","last_fullsized":"2024-04-17T05:43:09.938398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:57.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'136","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939607+0000","last_change":"2024-04-17T04:36:31.400243+0000","last_active":"2024-04-17T05:43:14.939607+0000","last_peered":"2024-04-17T05:43:14.939607+0000","last_clean":"2024-04-17T05:43:14.939607+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:43:14.939607+0000","last_undegraded":"2024-04-17T05:43:14.939607+0000","last_fullsized":"2024-04-17T05:43:14.939607+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:41.986999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'528","reported_seq":2299,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.662141+0000","last_change":"2024-04-17T04:36:33.421012+0000","last_active":"2024-04-17T05:45:57.662141+0000","last_peered":"2024-04-17T05:45:57.662141+0000","last_clean":"2024-04-17T05:45:57.662141+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:45:57.662141+0000","last_undegraded":"2024-04-17T05:45:57.662141+0000","last_fullsized":"2024-04-17T05:45:57.662141+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":500,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:52.921022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'218","reported_seq":734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.976785+0000","last_change":"2024-04-17T04:36:35.431849+0000","last_active":"2024-04-17T05:45:45.976785+0000","last_peered":"2024-04-17T05:45:45.976785+0000","last_clean":"2024-04-17T05:45:45.976785+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:45:45.976785+0000","last_undegraded":"2024-04-17T05:45:45.976785+0000","last_fullsized":"2024-04-17T05:45:45.976785+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:18.340086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4093","reported_seq":4724,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938365+0000","last_change":"2024-04-17T04:36:27.378432+0000","last_active":"2024-04-17T05:43:09.938365+0000","last_peered":"2024-04-17T05:43:09.938365+0000","last_clean":"2024-04-17T05:43:09.938365+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:43:09.938365+0000","last_undegraded":"2024-04-17T05:43:09.938365+0000","last_fullsized":"2024-04-17T05:43:09.938365+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":3000,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:55.591614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'291","reported_seq":908,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158359+0000","last_change":"2024-04-17T04:36:25.354707+0000","last_active":"2024-04-17T05:43:35.158359+0000","last_peered":"2024-04-17T05:43:35.158359+0000","last_clean":"2024-04-17T05:43:35.158359+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:43:35.158359+0000","last_undegraded":"2024-04-17T05:43:35.158359+0000","last_fullsized":"2024-04-17T05:43:35.158359+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:46.297295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4228470,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":401,"num_read_kb":7519,"num_write":1219,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.156524+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T05:43:30.156524+0000","last_peered":"2024-04-17T05:43:30.156524+0000","last_clean":"2024-04-17T05:43:30.156524+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:43:30.156524+0000","last_undegraded":"2024-04-17T05:43:30.156524+0000","last_fullsized":"2024-04-17T05:43:30.156524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:24.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180151+0000","last_change":"2024-04-17T04:36:29.363086+0000","last_active":"2024-04-17T05:42:45.180151+0000","last_peered":"2024-04-17T05:42:45.180151+0000","last_clean":"2024-04-17T05:42:45.180151+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:42:45.180151+0000","last_undegraded":"2024-04-17T05:42:45.180151+0000","last_fullsized":"2024-04-17T05:42:45.180151+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:48.241347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'715","reported_seq":1444,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.600048+0000","last_change":"2024-04-17T04:36:31.422637+0000","last_active":"2024-04-17T05:45:54.600048+0000","last_peered":"2024-04-17T05:45:54.600048+0000","last_clean":"2024-04-17T05:45:54.600048+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:45:54.600048+0000","last_undegraded":"2024-04-17T05:45:54.600048+0000","last_fullsized":"2024-04-17T05:45:54.600048+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":700,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:37.277476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":156924,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1257,"num_read_kb":2730,"num_write":1020,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187164+0000","last_change":"2024-04-17T04:36:33.415987+0000","last_active":"2024-04-17T05:43:15.187164+0000","last_peered":"2024-04-17T05:43:15.187164+0000","last_clean":"2024-04-17T05:43:15.187164+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:43:15.187164+0000","last_undegraded":"2024-04-17T05:43:15.187164+0000","last_fullsized":"2024-04-17T05:43:15.187164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:25.307133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'102","reported_seq":591,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.397345+0000","last_change":"2024-04-17T04:36:35.424792+0000","last_active":"2024-04-17T05:42:25.397345+0000","last_peered":"2024-04-17T05:42:25.397345+0000","last_clean":"2024-04-17T05:42:25.397345+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:42:25.397345+0000","last_undegraded":"2024-04-17T05:42:25.397345+0000","last_fullsized":"2024-04-17T05:42:25.397345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:27.783198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'149","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.086388+0000","last_change":"2024-04-17T04:36:35.425561+0000","last_active":"2024-04-17T05:45:45.086388+0000","last_peered":"2024-04-17T05:45:45.086388+0000","last_clean":"2024-04-17T05:45:45.086388+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:45:45.086388+0000","last_undegraded":"2024-04-17T05:45:45.086388+0000","last_fullsized":"2024-04-17T05:45:45.086388+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:48.063997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'73","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125289+0000","last_change":"2024-04-17T04:36:27.400403+0000","last_active":"2024-04-17T05:41:50.125289+0000","last_peered":"2024-04-17T05:41:50.125289+0000","last_clean":"2024-04-17T05:41:50.125289+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:41:50.125289+0000","last_undegraded":"2024-04-17T05:41:50.125289+0000","last_fullsized":"2024-04-17T05:41:50.125289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:03.426737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'220","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.706350+0000","last_change":"2024-04-17T04:36:25.357052+0000","last_active":"2024-04-17T05:42:50.706350+0000","last_peered":"2024-04-17T05:42:50.706350+0000","last_clean":"2024-04-17T05:42:50.706350+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:42:50.706350+0000","last_undegraded":"2024-04-17T05:42:50.706350+0000","last_fullsized":"2024-04-17T05:42:50.706350+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:07:39.562019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1502965,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":370,"num_read_kb":11105,"num_write":918,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139292+0000","last_change":"2024-04-17T04:36:29.391641+0000","last_active":"2024-04-17T05:41:50.139292+0000","last_peered":"2024-04-17T05:41:50.139292+0000","last_clean":"2024-04-17T05:41:50.139292+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:41:50.139292+0000","last_undegraded":"2024-04-17T05:41:50.139292+0000","last_fullsized":"2024-04-17T05:41:50.139292+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:29.991489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'296","reported_seq":886,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.200972+0000","last_change":"2024-04-17T04:36:31.406561+0000","last_active":"2024-04-17T05:44:15.200972+0000","last_peered":"2024-04-17T05:44:15.200972+0000","last_clean":"2024-04-17T05:44:15.200972+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:44:15.200972+0000","last_undegraded":"2024-04-17T05:44:15.200972+0000","last_fullsized":"2024-04-17T05:44:15.200972+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:50:50.884998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62692,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":511,"num_read_kb":1084,"num_write":418,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168016+0000","last_change":"2024-04-17T04:36:33.420539+0000","last_active":"2024-04-17T05:41:55.168016+0000","last_peered":"2024-04-17T05:41:55.168016+0000","last_clean":"2024-04-17T05:41:55.168016+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:41:55.168016+0000","last_undegraded":"2024-04-17T05:41:55.168016+0000","last_fullsized":"2024-04-17T05:41:55.168016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:42.375895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'98","reported_seq":603,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.713280+0000","last_change":"2024-04-17T04:36:35.430479+0000","last_active":"2024-04-17T05:45:44.713280+0000","last_peered":"2024-04-17T05:45:44.713280+0000","last_clean":"2024-04-17T05:45:44.713280+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:45:44.713280+0000","last_undegraded":"2024-04-17T05:45:44.713280+0000","last_fullsized":"2024-04-17T05:45:44.713280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:21:02.348756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'126","reported_seq":737,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:17.594052+0000","last_change":"2024-04-17T04:36:27.389126+0000","last_active":"2024-04-17T05:42:17.594052+0000","last_peered":"2024-04-17T05:42:17.594052+0000","last_clean":"2024-04-17T05:42:17.594052+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:42:17.594052+0000","last_undegraded":"2024-04-17T05:42:17.594052+0000","last_fullsized":"2024-04-17T05:42:17.594052+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:40.137919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'309","reported_seq":961,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144594+0000","last_change":"2024-04-17T04:37:03.635652+0000","last_active":"2024-04-17T05:42:20.144594+0000","last_peered":"2024-04-17T05:42:20.144594+0000","last_clean":"2024-04-17T05:42:20.144594+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:42:20.144594+0000","last_undegraded":"2024-04-17T05:42:20.144594+0000","last_fullsized":"2024-04-17T05:42:20.144594+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:33.722634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3797882,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":561,"num_read_kb":15946,"num_write":1479,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166897+0000","last_change":"2024-04-17T04:36:29.371357+0000","last_active":"2024-04-17T05:41:50.166897+0000","last_peered":"2024-04-17T05:41:50.166897+0000","last_clean":"2024-04-17T05:41:50.166897+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:41:50.166897+0000","last_undegraded":"2024-04-17T05:41:50.166897+0000","last_fullsized":"2024-04-17T05:41:50.166897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:50.443422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'56","reported_seq":567,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168798+0000","last_change":"2024-04-17T04:36:31.404202+0000","last_active":"2024-04-17T05:44:15.168798+0000","last_peered":"2024-04-17T05:44:15.168798+0000","last_clean":"2024-04-17T05:44:15.168798+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:44:15.168798+0000","last_undegraded":"2024-04-17T05:44:15.168798+0000","last_fullsized":"2024-04-17T05:44:15.168798+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:41.388160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134552+0000","last_change":"2024-04-17T04:36:33.416154+0000","last_active":"2024-04-17T05:41:55.134552+0000","last_peered":"2024-04-17T05:41:55.134552+0000","last_clean":"2024-04-17T05:41:55.134552+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:41:55.134552+0000","last_undegraded":"2024-04-17T05:41:55.134552+0000","last_fullsized":"2024-04-17T05:41:55.134552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:21.721127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'217","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.789782+0000","last_change":"2024-04-17T04:36:35.428428+0000","last_active":"2024-04-17T05:45:44.789782+0000","last_peered":"2024-04-17T05:45:44.789782+0000","last_clean":"2024-04-17T05:45:44.789782+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:45:44.789782+0000","last_undegraded":"2024-04-17T05:45:44.789782+0000","last_fullsized":"2024-04-17T05:45:44.789782+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:55.027046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166680+0000","last_change":"2024-04-17T04:36:27.395464+0000","last_active":"2024-04-17T05:41:50.166680+0000","last_peered":"2024-04-17T05:41:50.166680+0000","last_clean":"2024-04-17T05:41:50.166680+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:41:50.166680+0000","last_undegraded":"2024-04-17T05:41:50.166680+0000","last_fullsized":"2024-04-17T05:41:50.166680+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:23.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'286","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:43.075764+0000","last_change":"2024-04-17T04:36:25.338779+0000","last_active":"2024-04-17T05:42:43.075764+0000","last_peered":"2024-04-17T05:42:43.075764+0000","last_clean":"2024-04-17T05:42:43.075764+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:42:43.075764+0000","last_undegraded":"2024-04-17T05:42:43.075764+0000","last_fullsized":"2024-04-17T05:42:43.075764+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:59.502139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56924,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":518,"num_read_kb":11343,"num_write":1233,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133249+0000","last_change":"2024-04-17T04:36:29.381180+0000","last_active":"2024-04-17T05:41:50.133249+0000","last_peered":"2024-04-17T05:41:50.133249+0000","last_clean":"2024-04-17T05:41:50.133249+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:41:50.133249+0000","last_undegraded":"2024-04-17T05:41:50.133249+0000","last_fullsized":"2024-04-17T05:41:50.133249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:01.625306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'335","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.201110+0000","last_change":"2024-04-17T04:36:31.406900+0000","last_active":"2024-04-17T05:44:15.201110+0000","last_peered":"2024-04-17T05:44:15.201110+0000","last_clean":"2024-04-17T05:44:15.201110+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:44:15.201110+0000","last_undegraded":"2024-04-17T05:44:15.201110+0000","last_fullsized":"2024-04-17T05:44:15.201110+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:47.400810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62421,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":539,"num_read_kb":1090,"num_write":434,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":490,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135689+0000","last_change":"2024-04-17T04:36:33.412913+0000","last_active":"2024-04-17T05:41:55.135689+0000","last_peered":"2024-04-17T05:41:55.135689+0000","last_clean":"2024-04-17T05:41:55.135689+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:41:55.135689+0000","last_undegraded":"2024-04-17T05:41:55.135689+0000","last_fullsized":"2024-04-17T05:41:55.135689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:56.825814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'112","reported_seq":599,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.542927+0000","last_change":"2024-04-17T04:36:35.424722+0000","last_active":"2024-04-17T05:41:51.542927+0000","last_peered":"2024-04-17T05:41:51.542927+0000","last_clean":"2024-04-17T05:41:51.542927+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:41:51.542927+0000","last_undegraded":"2024-04-17T05:41:51.542927+0000","last_fullsized":"2024-04-17T05:41:51.542927+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:15:26.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":759,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166609+0000","last_change":"2024-04-17T04:36:27.378417+0000","last_active":"2024-04-17T05:41:50.166609+0000","last_peered":"2024-04-17T05:41:50.166609+0000","last_clean":"2024-04-17T05:41:50.166609+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:41:50.166609+0000","last_undegraded":"2024-04-17T05:41:50.166609+0000","last_fullsized":"2024-04-17T05:41:50.166609+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:17.606187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'245","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.397517+0000","last_change":"2024-04-17T04:36:25.344700+0000","last_active":"2024-04-17T05:43:39.397517+0000","last_peered":"2024-04-17T05:43:39.397517+0000","last_clean":"2024-04-17T05:43:39.397517+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:43:39.397517+0000","last_undegraded":"2024-04-17T05:43:39.397517+0000","last_fullsized":"2024-04-17T05:43:39.397517+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:28:58.108460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18113654,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":380,"num_read_kb":84514,"num_write":925,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134534+0000","last_change":"2024-04-17T04:36:29.384475+0000","last_active":"2024-04-17T05:41:50.134534+0000","last_peered":"2024-04-17T05:41:50.134534+0000","last_clean":"2024-04-17T05:41:50.134534+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:41:50.134534+0000","last_undegraded":"2024-04-17T05:41:50.134534+0000","last_fullsized":"2024-04-17T05:41:50.134534+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:57.583947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'253","reported_seq":828,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166282+0000","last_change":"2024-04-17T04:36:31.380383+0000","last_active":"2024-04-17T05:44:15.166282+0000","last_peered":"2024-04-17T05:44:15.166282+0000","last_clean":"2024-04-17T05:44:15.166282+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:44:15.166282+0000","last_undegraded":"2024-04-17T05:44:15.166282+0000","last_fullsized":"2024-04-17T05:44:15.166282+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:45.175550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51759,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":419,"num_read_kb":889,"num_write":336,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136027+0000","last_change":"2024-04-17T04:36:33.429804+0000","last_active":"2024-04-17T05:41:55.136027+0000","last_peered":"2024-04-17T05:41:55.136027+0000","last_clean":"2024-04-17T05:41:55.136027+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:41:55.136027+0000","last_undegraded":"2024-04-17T05:41:55.136027+0000","last_fullsized":"2024-04-17T05:41:55.136027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:10.357384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'289","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151938+0000","last_change":"2024-04-17T04:36:27.387575+0000","last_active":"2024-04-17T05:43:10.151938+0000","last_peered":"2024-04-17T05:43:10.151938+0000","last_clean":"2024-04-17T05:43:10.151938+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:43:10.151938+0000","last_undegraded":"2024-04-17T05:43:10.151938+0000","last_fullsized":"2024-04-17T05:43:10.151938+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:05.627702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'242","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164601+0000","last_change":"2024-04-17T04:36:25.326760+0000","last_active":"2024-04-17T05:41:40.164601+0000","last_peered":"2024-04-17T05:41:40.164601+0000","last_clean":"2024-04-17T05:41:40.164601+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:41:40.164601+0000","last_undegraded":"2024-04-17T05:41:40.164601+0000","last_fullsized":"2024-04-17T05:41:40.164601+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:25.101546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7321484,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":471,"num_read_kb":33047,"num_write":1108,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146569+0000","last_change":"2024-04-17T04:36:29.373251+0000","last_active":"2024-04-17T05:42:40.146569+0000","last_peered":"2024-04-17T05:42:40.146569+0000","last_clean":"2024-04-17T05:42:40.146569+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:42:40.146569+0000","last_undegraded":"2024-04-17T05:42:40.146569+0000","last_fullsized":"2024-04-17T05:42:40.146569+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:18:53.682774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'466","reported_seq":1107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.641533+0000","last_change":"2024-04-17T04:36:31.397750+0000","last_active":"2024-04-17T05:45:54.641533+0000","last_peered":"2024-04-17T05:45:54.641533+0000","last_clean":"2024-04-17T05:45:54.641533+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:45:54.641533+0000","last_undegraded":"2024-04-17T05:45:54.641533+0000","last_fullsized":"2024-04-17T05:45:54.641533+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":400,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:14.506372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103130,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":820,"num_read_kb":1787,"num_write":708,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.135785+0000","last_change":"2024-04-17T04:36:33.418783+0000","last_active":"2024-04-17T05:41:35.135785+0000","last_peered":"2024-04-17T05:41:35.135785+0000","last_clean":"2024-04-17T05:41:35.135785+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:41:35.135785+0000","last_undegraded":"2024-04-17T05:41:35.135785+0000","last_fullsized":"2024-04-17T05:41:35.135785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:38:46.129082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'171","reported_seq":678,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168149+0000","last_change":"2024-04-17T04:36:35.419329+0000","last_active":"2024-04-17T05:41:55.168149+0000","last_peered":"2024-04-17T05:41:55.168149+0000","last_clean":"2024-04-17T05:41:55.168149+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:41:55.168149+0000","last_undegraded":"2024-04-17T05:41:55.168149+0000","last_fullsized":"2024-04-17T05:41:55.168149+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:14.062860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":709,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146514+0000","last_change":"2024-04-17T04:36:27.386903+0000","last_active":"2024-04-17T05:42:40.146514+0000","last_peered":"2024-04-17T05:42:40.146514+0000","last_clean":"2024-04-17T05:42:40.146514+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:42:40.146514+0000","last_undegraded":"2024-04-17T05:42:40.146514+0000","last_fullsized":"2024-04-17T05:42:40.146514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:46:53.605066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'252","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.140440+0000","last_change":"2024-04-17T04:36:25.357464+0000","last_active":"2024-04-17T05:41:55.140440+0000","last_peered":"2024-04-17T05:41:55.140440+0000","last_clean":"2024-04-17T05:41:55.140440+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:41:55.140440+0000","last_undegraded":"2024-04-17T05:41:55.140440+0000","last_fullsized":"2024-04-17T05:41:55.140440+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:12.420278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6367399,"num_objects":60,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":447,"num_read_kb":112378,"num_write":1170,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151910+0000","last_change":"2024-04-17T04:36:29.376006+0000","last_active":"2024-04-17T05:43:10.151910+0000","last_peered":"2024-04-17T05:43:10.151910+0000","last_clean":"2024-04-17T05:43:10.151910+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:43:10.151910+0000","last_undegraded":"2024-04-17T05:43:10.151910+0000","last_fullsized":"2024-04-17T05:43:10.151910+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.144787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'310","reported_seq":904,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.163993+0000","last_change":"2024-04-17T04:36:31.402737+0000","last_active":"2024-04-17T05:44:05.163993+0000","last_peered":"2024-04-17T05:44:05.163993+0000","last_clean":"2024-04-17T05:44:05.163993+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:44:05.163993+0000","last_undegraded":"2024-04-17T05:44:05.163993+0000","last_fullsized":"2024-04-17T05:44:05.163993+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:20.700900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52833,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":523,"num_read_kb":1031,"num_write":436,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.136274+0000","last_change":"2024-04-17T04:36:33.415254+0000","last_active":"2024-04-17T05:41:35.136274+0000","last_peered":"2024-04-17T05:41:35.136274+0000","last_clean":"2024-04-17T05:41:35.136274+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:41:35.136274+0000","last_undegraded":"2024-04-17T05:41:35.136274+0000","last_fullsized":"2024-04-17T05:41:35.136274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:37.629549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'175","reported_seq":711,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166706+0000","last_change":"2024-04-17T04:36:35.424191+0000","last_active":"2024-04-17T05:44:15.166706+0000","last_peered":"2024-04-17T05:44:15.166706+0000","last_clean":"2024-04-17T05:44:15.166706+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:44:15.166706+0000","last_undegraded":"2024-04-17T05:44:15.166706+0000","last_fullsized":"2024-04-17T05:44:15.166706+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:48.892746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'114","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.162530+0000","last_change":"2024-04-17T04:36:27.378054+0000","last_active":"2024-04-17T05:41:30.162530+0000","last_peered":"2024-04-17T05:41:30.162530+0000","last_clean":"2024-04-17T05:41:30.162530+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:41:30.162530+0000","last_undegraded":"2024-04-17T05:41:30.162530+0000","last_fullsized":"2024-04-17T05:41:30.162530+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:41.230833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'297","reported_seq":981,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:38.607179+0000","last_change":"2024-04-17T04:36:25.356583+0000","last_active":"2024-04-17T05:41:38.607179+0000","last_peered":"2024-04-17T05:41:38.607179+0000","last_clean":"2024-04-17T05:41:38.607179+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:41:38.607179+0000","last_undegraded":"2024-04-17T05:41:38.607179+0000","last_fullsized":"2024-04-17T05:41:38.607179+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:38.408649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5881433,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":515,"num_read_kb":10795,"num_write":1289,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":831,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.514388+0000","last_change":"2024-04-17T04:36:29.370143+0000","last_active":"2024-04-17T05:45:54.514388+0000","last_peered":"2024-04-17T05:45:54.514388+0000","last_clean":"2024-04-17T05:45:54.514388+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:45:54.514388+0000","last_undegraded":"2024-04-17T05:45:54.514388+0000","last_fullsized":"2024-04-17T05:45:54.514388+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:33.209495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'293","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.216972+0000","last_change":"2024-04-17T04:36:31.421343+0000","last_active":"2024-04-17T05:45:20.216972+0000","last_peered":"2024-04-17T05:45:20.216972+0000","last_clean":"2024-04-17T05:45:20.216972+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:45:20.216972+0000","last_undegraded":"2024-04-17T05:45:20.216972+0000","last_fullsized":"2024-04-17T05:45:20.216972+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:50.024878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58684,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":490,"num_read_kb":1030,"num_write":398,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.152311+0000","last_change":"2024-04-17T04:36:33.415113+0000","last_active":"2024-04-17T05:42:50.152311+0000","last_peered":"2024-04-17T05:42:50.152311+0000","last_clean":"2024-04-17T05:42:50.152311+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:42:50.152311+0000","last_undegraded":"2024-04-17T05:42:50.152311+0000","last_fullsized":"2024-04-17T05:42:50.152311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:41.644384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'102","reported_seq":601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.654720+0000","last_change":"2024-04-17T04:36:35.428333+0000","last_active":"2024-04-17T05:45:54.654720+0000","last_peered":"2024-04-17T05:45:54.654720+0000","last_clean":"2024-04-17T05:45:54.654720+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:45:54.654720+0000","last_undegraded":"2024-04-17T05:45:54.654720+0000","last_fullsized":"2024-04-17T05:45:54.654720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:39.420180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.133409+0000","last_change":"2024-04-17T04:36:27.356010+0000","last_active":"2024-04-17T05:41:30.133409+0000","last_peered":"2024-04-17T05:41:30.133409+0000","last_clean":"2024-04-17T05:41:30.133409+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:41:30.133409+0000","last_undegraded":"2024-04-17T05:41:30.133409+0000","last_fullsized":"2024-04-17T05:41:30.133409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:42.969449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'255","reported_seq":1080,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.128114+0000","last_change":"2024-04-17T04:36:25.354574+0000","last_active":"2024-04-17T05:41:25.128114+0000","last_peered":"2024-04-17T05:41:25.128114+0000","last_clean":"2024-04-17T05:41:25.128114+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:41:25.128114+0000","last_undegraded":"2024-04-17T05:41:25.128114+0000","last_fullsized":"2024-04-17T05:41:25.128114+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:55.589797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21838700,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":777,"num_read_kb":11279,"num_write":1138,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.135309+0000","last_change":"2024-04-17T04:36:29.365578+0000","last_active":"2024-04-17T05:41:30.135309+0000","last_peered":"2024-04-17T05:41:30.135309+0000","last_clean":"2024-04-17T05:41:30.135309+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:41:30.135309+0000","last_undegraded":"2024-04-17T05:41:30.135309+0000","last_fullsized":"2024-04-17T05:41:30.135309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:29.189344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'119","reported_seq":666,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.137220+0000","last_change":"2024-04-17T04:36:31.422491+0000","last_active":"2024-04-17T05:42:45.137220+0000","last_peered":"2024-04-17T05:42:45.137220+0000","last_clean":"2024-04-17T05:42:45.137220+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:42:45.137220+0000","last_undegraded":"2024-04-17T05:42:45.137220+0000","last_fullsized":"2024-04-17T05:42:45.137220+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:06.145178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'535","reported_seq":2310,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:57.671073+0000","last_change":"2024-04-17T04:36:33.404252+0000","last_active":"2024-04-17T05:45:57.671073+0000","last_peered":"2024-04-17T05:45:57.671073+0000","last_clean":"2024-04-17T05:45:57.671073+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:45:57.671073+0000","last_undegraded":"2024-04-17T05:45:57.671073+0000","last_fullsized":"2024-04-17T05:45:57.671073+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":500,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:11.625831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'114","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134497+0000","last_change":"2024-04-17T04:36:35.413901+0000","last_active":"2024-04-17T05:41:55.134497+0000","last_peered":"2024-04-17T05:41:55.134497+0000","last_clean":"2024-04-17T05:41:55.134497+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:41:55.134497+0000","last_undegraded":"2024-04-17T05:41:55.134497+0000","last_fullsized":"2024-04-17T05:41:55.134497+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:00:54.978078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'229","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120332+0000","last_change":"2024-04-17T04:36:27.394864+0000","last_active":"2024-04-17T05:41:30.120332+0000","last_peered":"2024-04-17T05:41:30.120332+0000","last_clean":"2024-04-17T05:41:30.120332+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:41:30.120332+0000","last_undegraded":"2024-04-17T05:41:30.120332+0000","last_fullsized":"2024-04-17T05:41:30.120332+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:50:03.593045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'268","reported_seq":867,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.133458+0000","last_change":"2024-04-17T04:36:25.369133+0000","last_active":"2024-04-17T05:41:25.133458+0000","last_peered":"2024-04-17T05:41:25.133458+0000","last_clean":"2024-04-17T05:41:25.133458+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:41:25.133458+0000","last_undegraded":"2024-04-17T05:41:25.133458+0000","last_fullsized":"2024-04-17T05:41:25.133458+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:22.406331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10495834,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":376,"num_read_kb":414,"num_write":1132,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.128372+0000","last_change":"2024-04-17T04:36:29.379152+0000","last_active":"2024-04-17T05:41:30.128372+0000","last_peered":"2024-04-17T05:41:30.128372+0000","last_clean":"2024-04-17T05:41:30.128372+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:41:30.128372+0000","last_undegraded":"2024-04-17T05:41:30.128372+0000","last_fullsized":"2024-04-17T05:41:30.128372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:52.689910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'336","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.458700+0000","last_change":"2024-04-17T04:36:31.396219+0000","last_active":"2024-04-17T05:45:46.458700+0000","last_peered":"2024-04-17T05:45:46.458700+0000","last_clean":"2024-04-17T05:45:46.458700+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:45:46.458700+0000","last_undegraded":"2024-04-17T05:45:46.458700+0000","last_fullsized":"2024-04-17T05:45:46.458700+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:39:05.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":534,"num_read_kb":1021,"num_write":420,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134718+0000","last_change":"2024-04-17T04:36:33.430639+0000","last_active":"2024-04-17T05:41:50.134718+0000","last_peered":"2024-04-17T05:41:50.134718+0000","last_clean":"2024-04-17T05:41:50.134718+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:41:50.134718+0000","last_undegraded":"2024-04-17T05:41:50.134718+0000","last_fullsized":"2024-04-17T05:41:50.134718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.795912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'127","reported_seq":629,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166471+0000","last_change":"2024-04-17T04:36:35.436213+0000","last_active":"2024-04-17T05:44:15.166471+0000","last_peered":"2024-04-17T05:44:15.166471+0000","last_clean":"2024-04-17T05:44:15.166471+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:44:15.166471+0000","last_undegraded":"2024-04-17T05:44:15.166471+0000","last_fullsized":"2024-04-17T05:44:15.166471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:44.113556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.536900+0000","last_change":"2024-04-17T04:36:27.378160+0000","last_active":"2024-04-17T05:41:51.536900+0000","last_peered":"2024-04-17T05:41:51.536900+0000","last_clean":"2024-04-17T05:41:51.536900+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:41:51.536900+0000","last_undegraded":"2024-04-17T05:41:51.536900+0000","last_fullsized":"2024-04-17T05:41:51.536900+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:53.663348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'275","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:25.133518+0000","last_change":"2024-04-17T04:36:25.365976+0000","last_active":"2024-04-17T05:41:25.133518+0000","last_peered":"2024-04-17T05:41:25.133518+0000","last_clean":"2024-04-17T05:41:25.133518+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:41:25.133518+0000","last_undegraded":"2024-04-17T05:41:25.133518+0000","last_fullsized":"2024-04-17T05:41:25.133518+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:05.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267001,"num_objects":79,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":484,"num_read_kb":31239,"num_write":1200,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134570+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:41:30.134570+0000","last_peered":"2024-04-17T05:41:30.134570+0000","last_clean":"2024-04-17T05:41:30.134570+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:41:30.134570+0000","last_undegraded":"2024-04-17T05:41:30.134570+0000","last_fullsized":"2024-04-17T05:41:30.134570+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:39.416117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'268","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168845+0000","last_change":"2024-04-17T04:36:31.403334+0000","last_active":"2024-04-17T05:44:15.168845+0000","last_peered":"2024-04-17T05:44:15.168845+0000","last_clean":"2024-04-17T05:44:15.168845+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:44:15.168845+0000","last_undegraded":"2024-04-17T05:44:15.168845+0000","last_fullsized":"2024-04-17T05:44:15.168845+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:25.865066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":448,"num_read_kb":958,"num_write":362,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136680+0000","last_change":"2024-04-17T04:36:33.420313+0000","last_active":"2024-04-17T05:42:00.136680+0000","last_peered":"2024-04-17T05:42:00.136680+0000","last_clean":"2024-04-17T05:42:00.136680+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:42:00.136680+0000","last_undegraded":"2024-04-17T05:42:00.136680+0000","last_fullsized":"2024-04-17T05:42:00.136680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:55.964612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'134","reported_seq":655,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.071549+0000","last_change":"2024-04-17T04:36:35.422747+0000","last_active":"2024-04-17T05:45:46.071549+0000","last_peered":"2024-04-17T05:45:46.071549+0000","last_clean":"2024-04-17T05:45:46.071549+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:45:46.071549+0000","last_undegraded":"2024-04-17T05:45:46.071549+0000","last_fullsized":"2024-04-17T05:45:46.071549+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:32.269106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'203","reported_seq":810,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:14.025767+0000","last_change":"2024-04-17T04:36:27.381423+0000","last_active":"2024-04-17T05:42:14.025767+0000","last_peered":"2024-04-17T05:42:14.025767+0000","last_clean":"2024-04-17T05:42:14.025767+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:42:14.025767+0000","last_undegraded":"2024-04-17T05:42:14.025767+0000","last_fullsized":"2024-04-17T05:42:14.025767+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:04.091357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'307","reported_seq":970,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.179182+0000","last_change":"2024-04-17T04:36:25.374623+0000","last_active":"2024-04-17T05:42:40.179182+0000","last_peered":"2024-04-17T05:42:40.179182+0000","last_clean":"2024-04-17T05:42:40.179182+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:42:40.179182+0000","last_undegraded":"2024-04-17T05:42:40.179182+0000","last_fullsized":"2024-04-17T05:42:40.179182+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:52.802493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13343203,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":445,"num_read_kb":19835,"num_write":1327,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123902+0000","last_change":"2024-04-17T04:36:29.377026+0000","last_active":"2024-04-17T05:41:45.123902+0000","last_peered":"2024-04-17T05:41:45.123902+0000","last_clean":"2024-04-17T05:41:45.123902+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:41:45.123902+0000","last_undegraded":"2024-04-17T05:41:45.123902+0000","last_fullsized":"2024-04-17T05:41:45.123902+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:26:46.743881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'255","reported_seq":822,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198320+0000","last_change":"2024-04-17T04:36:31.398159+0000","last_active":"2024-04-17T05:44:05.198320+0000","last_peered":"2024-04-17T05:44:05.198320+0000","last_clean":"2024-04-17T05:44:05.198320+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:44:05.198320+0000","last_undegraded":"2024-04-17T05:44:05.198320+0000","last_fullsized":"2024-04-17T05:44:05.198320+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:04.571296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56291,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":432,"num_read_kb":970,"num_write":360,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.121505+0000","last_change":"2024-04-17T04:36:33.401472+0000","last_active":"2024-04-17T05:41:35.121505+0000","last_peered":"2024-04-17T05:41:35.121505+0000","last_clean":"2024-04-17T05:41:35.121505+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:41:35.121505+0000","last_undegraded":"2024-04-17T05:41:35.121505+0000","last_fullsized":"2024-04-17T05:41:35.121505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:49.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'141","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.700107+0000","last_change":"2024-04-17T04:36:35.423654+0000","last_active":"2024-04-17T05:45:40.700107+0000","last_peered":"2024-04-17T05:45:40.700107+0000","last_clean":"2024-04-17T05:45:40.700107+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:45:40.700107+0000","last_undegraded":"2024-04-17T05:45:40.700107+0000","last_fullsized":"2024-04-17T05:45:40.700107+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:17:36.321043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'297","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123940+0000","last_change":"2024-04-17T04:36:27.400669+0000","last_active":"2024-04-17T05:41:45.123940+0000","last_peered":"2024-04-17T05:41:45.123940+0000","last_clean":"2024-04-17T05:41:45.123940+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:41:45.123940+0000","last_undegraded":"2024-04-17T05:41:45.123940+0000","last_fullsized":"2024-04-17T05:41:45.123940+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:59.138022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'362","reported_seq":1068,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.151944+0000","last_change":"2024-04-17T04:36:25.356509+0000","last_active":"2024-04-17T05:43:05.151944+0000","last_peered":"2024-04-17T05:43:05.151944+0000","last_clean":"2024-04-17T05:43:05.151944+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:43:05.151944+0000","last_undegraded":"2024-04-17T05:43:05.151944+0000","last_fullsized":"2024-04-17T05:43:05.151944+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":300,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:14.238916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16573947,"num_objects":81,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":588,"num_read_kb":71297,"num_write":1611,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"35'6","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.508087+0000","last_change":"2024-04-17T04:36:29.376775+0000","last_active":"2024-04-17T05:45:54.508087+0000","last_peered":"2024-04-17T05:45:54.508087+0000","last_clean":"2024-04-17T05:45:54.508087+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:45:54.508087+0000","last_undegraded":"2024-04-17T05:45:54.508087+0000","last_fullsized":"2024-04-17T05:45:54.508087+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:09.589509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":498,"num_read_kb":809,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'314","reported_seq":919,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.158024+0000","last_change":"2024-04-17T04:36:31.391158+0000","last_active":"2024-04-17T05:44:05.158024+0000","last_peered":"2024-04-17T05:44:05.158024+0000","last_clean":"2024-04-17T05:44:05.158024+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:44:05.158024+0000","last_undegraded":"2024-04-17T05:44:05.158024+0000","last_fullsized":"2024-04-17T05:44:05.158024+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:43.696307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58738,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":517,"num_read_kb":1039,"num_write":414,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130642+0000","last_change":"2024-04-17T04:36:33.430195+0000","last_active":"2024-04-17T05:41:35.130642+0000","last_peered":"2024-04-17T05:41:35.130642+0000","last_clean":"2024-04-17T05:41:35.130642+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:41:35.130642+0000","last_undegraded":"2024-04-17T05:41:35.130642+0000","last_fullsized":"2024-04-17T05:41:35.130642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:37.038297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'138","reported_seq":637,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.163062+0000","last_change":"2024-04-17T04:36:35.427734+0000","last_active":"2024-04-17T05:44:25.163062+0000","last_peered":"2024-04-17T05:44:25.163062+0000","last_clean":"2024-04-17T05:44:25.163062+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:44:25.163062+0000","last_undegraded":"2024-04-17T05:44:25.163062+0000","last_fullsized":"2024-04-17T05:44:25.163062+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:48:29.121374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'154","reported_seq":788,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164804+0000","last_change":"2024-04-17T04:36:27.397669+0000","last_active":"2024-04-17T05:43:40.164804+0000","last_peered":"2024-04-17T05:43:40.164804+0000","last_clean":"2024-04-17T05:43:40.164804+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:43:40.164804+0000","last_undegraded":"2024-04-17T05:43:40.164804+0000","last_fullsized":"2024-04-17T05:43:40.164804+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:46:34.375349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'355","reported_seq":1073,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150519+0000","last_change":"2024-04-17T04:36:25.383690+0000","last_active":"2024-04-17T05:43:05.150519+0000","last_peered":"2024-04-17T05:43:05.150519+0000","last_clean":"2024-04-17T05:43:05.150519+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:43:05.150519+0000","last_undegraded":"2024-04-17T05:43:05.150519+0000","last_fullsized":"2024-04-17T05:43:05.150519+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:14.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993920,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":770,"num_read_kb":6303,"num_write":1311,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":648,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142976+0000","last_change":"2024-04-17T04:37:03.630121+0000","last_active":"2024-04-17T05:42:05.142976+0000","last_peered":"2024-04-17T05:42:05.142976+0000","last_clean":"2024-04-17T05:42:05.142976+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:42:05.142976+0000","last_undegraded":"2024-04-17T05:42:05.142976+0000","last_fullsized":"2024-04-17T05:42:05.142976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:09:56.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'293","reported_seq":959,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.153334+0000","last_change":"2024-04-17T04:36:25.355361+0000","last_active":"2024-04-17T05:43:05.153334+0000","last_peered":"2024-04-17T05:43:05.153334+0000","last_clean":"2024-04-17T05:43:05.153334+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:43:05.153334+0000","last_undegraded":"2024-04-17T05:43:05.153334+0000","last_fullsized":"2024-04-17T05:43:05.153334+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:15.756630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11689993,"num_objects":82,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":496,"num_read_kb":39832,"num_write":1126,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'148","reported_seq":762,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163630+0000","last_change":"2024-04-17T04:36:27.388520+0000","last_active":"2024-04-17T05:43:40.163630+0000","last_peered":"2024-04-17T05:43:40.163630+0000","last_clean":"2024-04-17T05:43:40.163630+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:43:40.163630+0000","last_undegraded":"2024-04-17T05:43:40.163630+0000","last_fullsized":"2024-04-17T05:43:40.163630+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:31.868037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'250","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.155879+0000","last_change":"2024-04-17T04:36:25.365759+0000","last_active":"2024-04-17T05:43:05.155879+0000","last_peered":"2024-04-17T05:43:05.155879+0000","last_clean":"2024-04-17T05:43:05.155879+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:43:05.155879+0000","last_undegraded":"2024-04-17T05:43:05.155879+0000","last_fullsized":"2024-04-17T05:43:05.155879+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:25:48.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4343917,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":379,"num_read_kb":10784,"num_write":1108,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'94","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164612+0000","last_change":"2024-04-17T04:36:27.362737+0000","last_active":"2024-04-17T05:43:40.164612+0000","last_peered":"2024-04-17T05:43:40.164612+0000","last_clean":"2024-04-17T05:43:40.164612+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:43:40.164612+0000","last_undegraded":"2024-04-17T05:43:40.164612+0000","last_fullsized":"2024-04-17T05:43:40.164612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:24:50.502747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'241","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153889+0000","last_change":"2024-04-17T04:36:25.355263+0000","last_active":"2024-04-17T05:43:15.153889+0000","last_peered":"2024-04-17T05:43:15.153889+0000","last_clean":"2024-04-17T05:43:15.153889+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:43:15.153889+0000","last_undegraded":"2024-04-17T05:43:15.153889+0000","last_fullsized":"2024-04-17T05:43:15.153889+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:43.114752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11863036,"num_objects":70,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":642,"num_read_kb":32422,"num_write":1055,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":635,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.189253+0000","last_change":"2024-04-17T04:36:27.395680+0000","last_active":"2024-04-17T05:43:25.189253+0000","last_peered":"2024-04-17T05:43:25.189253+0000","last_clean":"2024-04-17T05:43:25.189253+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:43:25.189253+0000","last_undegraded":"2024-04-17T05:43:25.189253+0000","last_fullsized":"2024-04-17T05:43:25.189253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:04.849323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'223","reported_seq":835,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191011+0000","last_change":"2024-04-17T04:37:03.637190+0000","last_active":"2024-04-17T05:43:35.191011+0000","last_peered":"2024-04-17T05:43:35.191011+0000","last_clean":"2024-04-17T05:43:35.191011+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:43:35.191011+0000","last_undegraded":"2024-04-17T05:43:35.191011+0000","last_fullsized":"2024-04-17T05:43:35.191011+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:24:25.534759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5542945,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":301,"num_read_kb":5919,"num_write":935,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938441+0000","last_change":"2024-04-17T04:36:27.395504+0000","last_active":"2024-04-17T05:43:09.938441+0000","last_peered":"2024-04-17T05:43:09.938441+0000","last_clean":"2024-04-17T05:43:09.938441+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:43:09.938441+0000","last_undegraded":"2024-04-17T05:43:09.938441+0000","last_fullsized":"2024-04-17T05:43:09.938441+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:27.120391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'257","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163317+0000","last_change":"2024-04-17T04:36:25.342748+0000","last_active":"2024-04-17T05:43:35.163317+0000","last_peered":"2024-04-17T05:43:35.163317+0000","last_clean":"2024-04-17T05:43:35.163317+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:43:35.163317+0000","last_undegraded":"2024-04-17T05:43:35.163317+0000","last_fullsized":"2024-04-17T05:43:35.163317+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:25.415287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173927,"num_objects":65,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":430,"num_read_kb":70994,"num_write":1129,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'215","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:36.452977+0000","last_change":"2024-04-17T04:36:27.384524+0000","last_active":"2024-04-17T05:43:36.452977+0000","last_peered":"2024-04-17T05:43:36.452977+0000","last_clean":"2024-04-17T05:43:36.452977+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:43:36.452977+0000","last_undegraded":"2024-04-17T05:43:36.452977+0000","last_fullsized":"2024-04-17T05:43:36.452977+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:35.098376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'314","reported_seq":995,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.157600+0000","last_change":"2024-04-17T04:36:25.386085+0000","last_active":"2024-04-17T05:43:35.157600+0000","last_peered":"2024-04-17T05:43:35.157600+0000","last_clean":"2024-04-17T05:43:35.157600+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:43:35.157600+0000","last_undegraded":"2024-04-17T05:43:35.157600+0000","last_fullsized":"2024-04-17T05:43:35.157600+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:09.994649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12903063,"num_objects":57,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":597,"num_read_kb":34079,"num_write":1419,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151711+0000","last_change":"2024-04-17T04:36:27.387044+0000","last_active":"2024-04-17T05:43:10.151711+0000","last_peered":"2024-04-17T05:43:10.151711+0000","last_clean":"2024-04-17T05:43:10.151711+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:43:10.151711+0000","last_undegraded":"2024-04-17T05:43:10.151711+0000","last_fullsized":"2024-04-17T05:43:10.151711+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:47.002935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'266","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191154+0000","last_change":"2024-04-17T04:36:25.379835+0000","last_active":"2024-04-17T05:43:35.191154+0000","last_peered":"2024-04-17T05:43:35.191154+0000","last_clean":"2024-04-17T05:43:35.191154+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:43:35.191154+0000","last_undegraded":"2024-04-17T05:43:35.191154+0000","last_fullsized":"2024-04-17T05:43:35.191154+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:54:10.099587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16832221,"num_objects":76,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":477,"num_read_kb":42709,"num_write":1222,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":738,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:49.022874+0000","last_change":"2024-04-17T04:36:27.373717+0000","last_active":"2024-04-17T05:42:49.022874+0000","last_peered":"2024-04-17T05:42:49.022874+0000","last_clean":"2024-04-17T05:42:49.022874+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:42:49.022874+0000","last_undegraded":"2024-04-17T05:42:49.022874+0000","last_fullsized":"2024-04-17T05:42:49.022874+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:38:49.102110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'237","reported_seq":871,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.165564+0000","last_change":"2024-04-17T04:36:25.361686+0000","last_active":"2024-04-17T05:43:50.165564+0000","last_peered":"2024-04-17T05:43:50.165564+0000","last_clean":"2024-04-17T05:43:50.165564+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:43:50.165564+0000","last_undegraded":"2024-04-17T05:43:50.165564+0000","last_fullsized":"2024-04-17T05:43:50.165564+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:40.663817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5566546,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":393,"num_read_kb":545,"num_write":1054,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":754,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125394+0000","last_change":"2024-04-17T04:36:27.378398+0000","last_active":"2024-04-17T05:41:50.125394+0000","last_peered":"2024-04-17T05:41:50.125394+0000","last_clean":"2024-04-17T05:41:50.125394+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:41:50.125394+0000","last_undegraded":"2024-04-17T05:41:50.125394+0000","last_fullsized":"2024-04-17T05:41:50.125394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:49.831818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'237","reported_seq":886,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.350982+0000","last_change":"2024-04-17T04:36:25.353469+0000","last_active":"2024-04-17T05:43:39.350982+0000","last_peered":"2024-04-17T05:43:39.350982+0000","last_clean":"2024-04-17T05:43:39.350982+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:43:39.350982+0000","last_undegraded":"2024-04-17T05:43:39.350982+0000","last_fullsized":"2024-04-17T05:43:39.350982+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2315115,"num_objects":71,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":391,"num_read_kb":4352,"num_write":1033,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":837,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139934+0000","last_change":"2024-04-17T04:36:27.398568+0000","last_active":"2024-04-17T05:42:00.139934+0000","last_peered":"2024-04-17T05:42:00.139934+0000","last_clean":"2024-04-17T05:42:00.139934+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:42:00.139934+0000","last_undegraded":"2024-04-17T05:42:00.139934+0000","last_fullsized":"2024-04-17T05:42:00.139934+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:48.128020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'247","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.144425+0000","last_change":"2024-04-17T04:37:03.632788+0000","last_active":"2024-04-17T05:42:20.144425+0000","last_peered":"2024-04-17T05:42:20.144425+0000","last_clean":"2024-04-17T05:42:20.144425+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:42:20.144425+0000","last_undegraded":"2024-04-17T05:42:20.144425+0000","last_fullsized":"2024-04-17T05:42:20.144425+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:33.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5338870,"num_objects":74,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":373,"num_read_kb":16015,"num_write":997,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'95","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124087+0000","last_change":"2024-04-17T04:36:27.402182+0000","last_active":"2024-04-17T05:41:45.124087+0000","last_peered":"2024-04-17T05:41:45.124087+0000","last_clean":"2024-04-17T05:41:45.124087+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:41:45.124087+0000","last_undegraded":"2024-04-17T05:41:45.124087+0000","last_fullsized":"2024-04-17T05:41:45.124087+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:59.716958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'300","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132046+0000","last_change":"2024-04-17T04:36:25.333011+0000","last_active":"2024-04-17T05:41:40.132046+0000","last_peered":"2024-04-17T05:41:40.132046+0000","last_clean":"2024-04-17T05:41:40.132046+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:41:40.132046+0000","last_undegraded":"2024-04-17T05:41:40.132046+0000","last_fullsized":"2024-04-17T05:41:40.132046+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:22.239383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071966,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":686,"num_read_kb":12134,"num_write":1196,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":686,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.132122+0000","last_change":"2024-04-17T04:36:27.388791+0000","last_active":"2024-04-17T05:41:45.132122+0000","last_peered":"2024-04-17T05:41:45.132122+0000","last_clean":"2024-04-17T05:41:45.132122+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:41:45.132122+0000","last_undegraded":"2024-04-17T05:41:45.132122+0000","last_fullsized":"2024-04-17T05:41:45.132122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:21.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'281","reported_seq":917,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:54.078754+0000","last_change":"2024-04-17T04:36:25.373417+0000","last_active":"2024-04-17T05:42:54.078754+0000","last_peered":"2024-04-17T05:42:54.078754+0000","last_clean":"2024-04-17T05:42:54.078754+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:42:54.078754+0000","last_undegraded":"2024-04-17T05:42:54.078754+0000","last_fullsized":"2024-04-17T05:42:54.078754+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:11.729276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30815,"num_objects":79,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":425,"num_read_kb":657,"num_write":1176,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'221","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133347+0000","last_change":"2024-04-17T04:36:27.372140+0000","last_active":"2024-04-17T05:41:45.133347+0000","last_peered":"2024-04-17T05:41:45.133347+0000","last_clean":"2024-04-17T05:41:45.133347+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:41:45.133347+0000","last_undegraded":"2024-04-17T05:41:45.133347+0000","last_fullsized":"2024-04-17T05:41:45.133347+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.338570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'355","reported_seq":1093,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133443+0000","last_change":"2024-04-17T04:36:25.386307+0000","last_active":"2024-04-17T05:41:50.133443+0000","last_peered":"2024-04-17T05:41:50.133443+0000","last_clean":"2024-04-17T05:41:50.133443+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:41:50.133443+0000","last_undegraded":"2024-04-17T05:41:50.133443+0000","last_fullsized":"2024-04-17T05:41:50.133443+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:43.522371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18382422,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":829,"num_read_kb":32478,"num_write":1352,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165668+0000","last_change":"2024-04-17T04:36:27.385349+0000","last_active":"2024-04-17T05:41:45.165668+0000","last_peered":"2024-04-17T05:41:45.165668+0000","last_clean":"2024-04-17T05:41:45.165668+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:41:45.165668+0000","last_undegraded":"2024-04-17T05:41:45.165668+0000","last_fullsized":"2024-04-17T05:41:45.165668+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:27:44.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'264","reported_seq":940,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136676+0000","last_change":"2024-04-17T04:36:25.355339+0000","last_active":"2024-04-17T05:41:45.136676+0000","last_peered":"2024-04-17T05:41:45.136676+0000","last_clean":"2024-04-17T05:41:45.136676+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:41:45.136676+0000","last_undegraded":"2024-04-17T05:41:45.136676+0000","last_fullsized":"2024-04-17T05:41:45.136676+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:47:52.068393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7660588,"num_objects":55,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":476,"num_read_kb":46303,"num_write":1246,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'321","reported_seq":1043,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160308+0000","last_change":"2024-04-17T04:37:03.628711+0000","last_active":"2024-04-17T05:43:50.160308+0000","last_peered":"2024-04-17T05:43:50.160308+0000","last_clean":"2024-04-17T05:43:50.160308+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:43:50.160308+0000","last_undegraded":"2024-04-17T05:43:50.160308+0000","last_fullsized":"2024-04-17T05:43:50.160308+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:45.773023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'262","reported_seq":895,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:25.361750+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:44.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23096680,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":400,"num_read_kb":49597,"num_write":1165,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'238","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.137869+0000","last_change":"2024-04-17T04:36:25.345113+0000","last_active":"2024-04-17T05:41:45.137869+0000","last_peered":"2024-04-17T05:41:45.137869+0000","last_clean":"2024-04-17T05:41:45.137869+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:41:45.137869+0000","last_undegraded":"2024-04-17T05:41:45.137869+0000","last_fullsized":"2024-04-17T05:41:45.137869+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:36.604225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584415,"num_objects":67,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":354,"num_read_kb":7661,"num_write":1028,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'243","reported_seq":842,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136202+0000","last_change":"2024-04-17T04:36:27.379068+0000","last_active":"2024-04-17T05:41:55.136202+0000","last_peered":"2024-04-17T05:41:55.136202+0000","last_clean":"2024-04-17T05:41:55.136202+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:41:55.136202+0000","last_undegraded":"2024-04-17T05:41:55.136202+0000","last_fullsized":"2024-04-17T05:41:55.136202+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:27.386167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'227","reported_seq":870,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:24.492781+0000","last_change":"2024-04-17T04:36:25.357445+0000","last_active":"2024-04-17T05:42:24.492781+0000","last_peered":"2024-04-17T05:42:24.492781+0000","last_clean":"2024-04-17T05:42:24.492781+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:42:24.492781+0000","last_undegraded":"2024-04-17T05:42:24.492781+0000","last_fullsized":"2024-04-17T05:42:24.492781+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:12.816116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10515588,"num_objects":59,"num_object_clones":0,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":397,"num_read_kb":20933,"num_write":1073,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":819,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138230+0000","last_change":"2024-04-17T04:36:27.378695+0000","last_active":"2024-04-17T05:41:45.138230+0000","last_peered":"2024-04-17T05:41:45.138230+0000","last_clean":"2024-04-17T05:41:45.138230+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:41:45.138230+0000","last_undegraded":"2024-04-17T05:41:45.138230+0000","last_fullsized":"2024-04-17T05:41:45.138230+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:37.636571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1280,"ondisk_log_size":1280,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1676,"ondisk_log_size":1676,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1746118,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":17591,"num_read_kb":33647,"num_write":14851,"num_write_kb":11154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3284992,"data_stored":2735018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2071,"ondisk_log_size":2071,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1642,"num_read_kb":1951,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3948,"ondisk_log_size":3948,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":648130815,"num_objects":4413,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":30775,"num_read_kb":1864049,"num_write":73496,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930250752,"data_stored":930250752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":100760153,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":130,"num_read_kb":221363,"num_write":141,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201523200,"data_stored":201520306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":100760153,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":125,"num_read_kb":210051,"num_write":141,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":201523200,"data_stored":201520306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575690,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451628,"kb_used_data":167780,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93920212,"statfs":{"total":96636764160,"available":96174297088,"internally_reserved":0,"allocated":171806720,"data_stored":169745941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":290650570},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575691,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474680,"kb_used_data":168624,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93897160,"statfs":{"total":96636764160,"available":96150691840,"internally_reserved":0,"allocated":172670976,"data_stored":170683181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":313391559},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575691,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314860,"kb_used_data":175460,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94056980,"statfs":{"total":96636764160,"available":96314347520,"internally_reserved":0,"allocated":179671040,"data_stored":177571130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575693,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":262200,"kb_used_data":158832,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94109640,"statfs":{"total":96636764160,"available":96368271360,"internally_reserved":0,"allocated":162643968,"data_stored":160677607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575691,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445376,"kb_used_data":201848,"kb_used_omap":1,"kb_used_meta":243518,"kb_avail":93926464,"statfs":{"total":96636764160,"available":96180699136,"internally_reserved":0,"allocated":206692352,"data_stored":204603759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":249362887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575695,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":277500,"kb_used_data":165876,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94094340,"statfs":{"total":96636764160,"available":96352604160,"internally_reserved":0,"allocated":169857024,"data_stored":167899459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114293189},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575693,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288536,"kb_used_data":149136,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94083304,"statfs":{"total":96636764160,"available":96341303296,"internally_reserved":0,"allocated":152715264,"data_stored":150702994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575691,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425908,"kb_used_data":151852,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":93945932,"statfs":{"total":96636764160,"available":96200634368,"internally_reserved":0,"allocated":155496448,"data_stored":153432781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280623557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":126107648,"data_stored":126107648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":111304704,"data_stored":111304704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93118464,"data_stored":93118464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":122077184,"data_stored":122077184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":120524800,"data_stored":120524800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":121225216,"data_stored":121225216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":118833152,"data_stored":118833152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":117059584,"data_stored":117059584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":352781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":385826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":117807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":472766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":92189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":555856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":256028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":501765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14680064,"data_stored":14680064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25214976,"data_stored":25214257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":40894464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28360704,"data_stored":28359976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21020672,"data_stored":21019944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15777792,"data_stored":15777073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":31506432,"data_stored":31505713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46235648,"data_stored":46234201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15777792,"data_stored":15777064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:46:05.137 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T05:46:05.449 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:05.449 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":264,"fsid":"5b040342-9999-436c-bdde-ed9b9762bed0","created":"2024-04-17T04:35:57.785945+0000","modified":"2024-04-17T05:07:46.698872+0000","last_up_change":"2024-04-17T04:36:08.952272+0000","last_in_change":"2024-04-17T04:36:00.137416+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":68,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T04:36:10.517211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-17T04:36:15.959683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-04-17T04:36:23.274531+0000","flags":1,"flags_names":"hashpspool","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":12288,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-04-17T04:36:25.632547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-04-17T04:36:27.481649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-04-17T04:36:29.691974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-04-17T04:36:31.420102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-04-17T04:36:33.534837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":65,"pool_name":"default.rgw.buckets.non-ec","create_time":"2024-04-17T04:47:55.618279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":66,"pool_name":"default.rgw.buckets.data.frozen","create_time":"2024-04-17T04:51:38.527020+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":67,"pool_name":"default.rgw.buckets.data.lukewarm","create_time":"2024-04-17T04:51:42.056571+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"259","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":68,"pool_name":"default.rgw.otp","create_time":"2024-04-17T05:07:44.261838+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"369d3f72-44a9-4b30-aa1c-c88765270bb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6801","nonce":1670897847}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6803","nonce":1670897847}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6811","nonce":1670897847}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":1670897847},{"type":"v1","addr":"172.21.15.88:6807","nonce":1670897847}]},"public_addr":"172.21.15.88:6801/1670897847","cluster_addr":"172.21.15.88:6803/1670897847","heartbeat_back_addr":"172.21.15.88:6811/1670897847","heartbeat_front_addr":"172.21.15.88:6807/1670897847","state":["exists","up"]},{"osd":1,"uuid":"fb9c30b2-9bee-4675-b3b1-f3961b2c2ae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6817","nonce":3314042889}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6819","nonce":3314042889}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6823","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6825","nonce":3314042889}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":3314042889},{"type":"v1","addr":"172.21.15.88:6822","nonce":3314042889}]},"public_addr":"172.21.15.88:6817/3314042889","cluster_addr":"172.21.15.88:6819/3314042889","heartbeat_back_addr":"172.21.15.88:6825/3314042889","heartbeat_front_addr":"172.21.15.88:6822/3314042889","state":["exists","up"]},{"osd":2,"uuid":"3d978434-aeba-4bdf-ab28-21dfd67bed1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6806","nonce":508820735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6810","nonce":508820735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6815","nonce":508820735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":508820735},{"type":"v1","addr":"172.21.15.88:6813","nonce":508820735}]},"public_addr":"172.21.15.88:6806/508820735","cluster_addr":"172.21.15.88:6810/508820735","heartbeat_back_addr":"172.21.15.88:6815/508820735","heartbeat_front_addr":"172.21.15.88:6813/508820735","state":["exists","up"]},{"osd":3,"uuid":"158b3428-03f3-4d64-a5e1-ec30159d91c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6824","nonce":217804779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6827","nonce":217804779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6831","nonce":217804779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":217804779},{"type":"v1","addr":"172.21.15.88:6829","nonce":217804779}]},"public_addr":"172.21.15.88:6824/217804779","cluster_addr":"172.21.15.88:6827/217804779","heartbeat_back_addr":"172.21.15.88:6831/217804779","heartbeat_front_addr":"172.21.15.88:6829/217804779","state":["exists","up"]},{"osd":4,"uuid":"62360a76-7bc7-4668-ba57-cbcf6cd589d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6810","nonce":2413638230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6814","nonce":2413638230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6822","nonce":2413638230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":2413638230},{"type":"v1","addr":"172.21.15.154:6818","nonce":2413638230}]},"public_addr":"172.21.15.154:6810/2413638230","cluster_addr":"172.21.15.154:6814/2413638230","heartbeat_back_addr":"172.21.15.154:6822/2413638230","heartbeat_front_addr":"172.21.15.154:6818/2413638230","state":["exists","up"]},{"osd":5,"uuid":"fd243a29-01d5-43c0-acaf-d36ab1e3d981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6813","nonce":3628934341}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6817","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6819","nonce":3628934341}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6828","nonce":3628934341}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":3628934341},{"type":"v1","addr":"172.21.15.154:6824","nonce":3628934341}]},"public_addr":"172.21.15.154:6813/3628934341","cluster_addr":"172.21.15.154:6819/3628934341","heartbeat_back_addr":"172.21.15.154:6828/3628934341","heartbeat_front_addr":"172.21.15.154:6824/3628934341","state":["exists","up"]},{"osd":6,"uuid":"7d374000-ae06-4891-98bf-073f3737dd03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6815","nonce":1439702329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6821","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6825","nonce":1439702329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6831","nonce":1439702329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6827","nonce":1439702329},{"type":"v1","addr":"172.21.15.154:6829","nonce":1439702329}]},"public_addr":"172.21.15.154:6815/1439702329","cluster_addr":"172.21.15.154:6825/1439702329","heartbeat_back_addr":"172.21.15.154:6831/1439702329","heartbeat_front_addr":"172.21.15.154:6829/1439702329","state":["exists","up"]},{"osd":7,"uuid":"01de9b40-4992-4f7f-9408-98cd9533e7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":262,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6801","nonce":1245445760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6803","nonce":1245445760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6807","nonce":1245445760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":1245445760},{"type":"v1","addr":"172.21.15.154:6805","nonce":1245445760}]},"public_addr":"172.21.15.154:6801/1245445760","cluster_addr":"172.21.15.154:6803/1245445760","heartbeat_back_addr":"172.21.15.154:6807/1245445760","heartbeat_front_addr":"172.21.15.154:6805/1245445760","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:08.009769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.775680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.793533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.757704+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.747333+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.725796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.771745+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T04:36:05.730259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.24","mappings":[{"from":4,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2},{"from":5,"to":3}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]},{"pgid":"67.e","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"3","m":"1","name":"testprofile","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":26,"snaps":[{"begin":1,"length":2}]},{"pool":31,"snaps":[{"begin":1,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T05:46:06.468 INFO:tasks.ceph:Scrubbing osd.0 2024-04-17T05:46:06.469 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:06.631 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:06.631 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:06.631 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:06.648 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-17T05:46:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 0 to deep-scrub 2024-04-17T05:46:06.948 INFO:tasks.ceph:Scrubbing osd.1 2024-04-17T05:46:06.949 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:07.090 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:07.090 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:07.091 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:07.106 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-17T05:46:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 1 to deep-scrub 2024-04-17T05:46:07.410 INFO:tasks.ceph:Scrubbing osd.2 2024-04-17T05:46:07.410 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:07.547 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:07.547 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:07.548 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:07.562 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-17T05:46:07.839 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 2 to deep-scrub 2024-04-17T05:46:07.857 INFO:tasks.ceph:Scrubbing osd.3 2024-04-17T05:46:07.857 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:07.992 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:07.992 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:07.992 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:08.007 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-17T05:46:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 3 to deep-scrub 2024-04-17T05:46:08.289 INFO:tasks.ceph:Scrubbing osd.4 2024-04-17T05:46:08.290 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:08.429 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:08.430 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:08.430 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:08.444 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-17T05:46:08.736 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 4 to deep-scrub 2024-04-17T05:46:08.754 INFO:tasks.ceph:Scrubbing osd.5 2024-04-17T05:46:08.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:08.908 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:08.908 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:08.908 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:08.924 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-17T05:46:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 5 to deep-scrub 2024-04-17T05:46:09.221 INFO:tasks.ceph:Scrubbing osd.6 2024-04-17T05:46:09.221 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:09.368 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:09.368 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:09.369 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:09.384 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-17T05:46:09.672 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 6 to deep-scrub 2024-04-17T05:46:09.692 INFO:tasks.ceph:Scrubbing osd.7 2024-04-17T05:46:09.693 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-17T05:46:09.844 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-17T05:46:09.845 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-17T05:46:09.845 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-17T05:46:09.861 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-17T05:46:10.147 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 7 to deep-scrub 2024-04-17T05:46:10.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:46:10.530 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:10.542 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:46:10.552 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2356,"stamp":"2024-04-17T05:46:09.282969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223466608,"num_objects":822,"num_object_clones":0,"num_object_copies":2530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":822,"num_whiteouts":0,"num_read":160833,"num_read_kb":2423241,"num_write":146364,"num_write_kb":2463359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12984,"ondisk_log_size":12984,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2006744,"kb_used_data":347624,"kb_used_omap":12,"kb_used_meta":1658099,"kb_avail":752967976,"statfs":{"total":773094113280,"available":771039207424,"internally_reserved":0,"allocated":355966976,"data_stored":340014674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1697893937},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-628542594,"num_objects":-4010,"num_object_clones":0,"num_object_copies":-15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4010,"num_whiteouts":0,"num_read":4527,"num_read_kb":4067,"num_write":4230,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005189"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151550+0000","last_change":"2024-04-17T05:07:45.742645+0000","last_active":"2024-04-17T05:42:45.151550+0000","last_peered":"2024-04-17T05:42:45.151550+0000","last_clean":"2024-04-17T05:42:45.151550+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:42:45.151550+0000","last_undegraded":"2024-04-17T05:42:45.151550+0000","last_fullsized":"2024-04-17T05:42:45.151550+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:43:06.072659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138115+0000","last_change":"2024-04-17T04:51:43.682048+0000","last_active":"2024-04-17T05:41:45.138115+0000","last_peered":"2024-04-17T05:41:45.138115+0000","last_clean":"2024-04-17T05:41:45.138115+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:41:45.138115+0000","last_undegraded":"2024-04-17T05:41:45.138115+0000","last_fullsized":"2024-04-17T05:41:45.138115+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:06.493069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164913+0000","last_change":"2024-04-17T04:51:39.638879+0000","last_active":"2024-04-17T05:41:40.164913+0000","last_peered":"2024-04-17T05:41:40.164913+0000","last_clean":"2024-04-17T05:41:40.164913+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:41:40.164913+0000","last_undegraded":"2024-04-17T05:41:40.164913+0000","last_fullsized":"2024-04-17T05:41:40.164913+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:57:53.046398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":148,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139601+0000","last_change":"2024-04-17T04:47:56.968120+0000","last_active":"2024-04-17T05:42:10.139601+0000","last_peered":"2024-04-17T05:42:10.139601+0000","last_clean":"2024-04-17T05:42:10.139601+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:42:10.139601+0000","last_undegraded":"2024-04-17T05:42:10.139601+0000","last_fullsized":"2024-04-17T05:42:10.139601+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:53:39.089673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149910+0000","last_change":"2024-04-17T05:07:45.746586+0000","last_active":"2024-04-17T05:42:45.149910+0000","last_peered":"2024-04-17T05:42:45.149910+0000","last_clean":"2024-04-17T05:42:45.149910+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:42:45.149910+0000","last_undegraded":"2024-04-17T05:42:45.149910+0000","last_fullsized":"2024-04-17T05:42:45.149910+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:02.041624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'3","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.160784+0000","last_change":"2024-04-17T04:51:43.661871+0000","last_active":"2024-04-17T05:44:15.160784+0000","last_peered":"2024-04-17T05:44:15.160784+0000","last_clean":"2024-04-17T05:44:15.160784+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:44:15.160784+0000","last_undegraded":"2024-04-17T05:44:15.160784+0000","last_fullsized":"2024-04-17T05:44:15.160784+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:46.906315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.137511+0000","last_change":"2024-04-17T04:51:39.640822+0000","last_active":"2024-04-17T05:41:40.137511+0000","last_peered":"2024-04-17T05:41:40.137511+0000","last_clean":"2024-04-17T05:41:40.137511+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:41:40.137511+0000","last_undegraded":"2024-04-17T05:41:40.137511+0000","last_fullsized":"2024-04-17T05:41:40.137511+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:51.213156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'96","reported_seq":350,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.143271+0000","last_change":"2024-04-17T04:47:56.937898+0000","last_active":"2024-04-17T05:42:15.143271+0000","last_peered":"2024-04-17T05:42:15.143271+0000","last_clean":"2024-04-17T05:42:15.143271+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:42:15.143271+0000","last_undegraded":"2024-04-17T05:42:15.143271+0000","last_fullsized":"2024-04-17T05:42:15.143271+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:02:44.307495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130858+0000","last_change":"2024-04-17T05:07:45.739412+0000","last_active":"2024-04-17T05:41:35.130858+0000","last_peered":"2024-04-17T05:41:35.130858+0000","last_clean":"2024-04-17T05:41:35.130858+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:41:35.130858+0000","last_undegraded":"2024-04-17T05:41:35.130858+0000","last_fullsized":"2024-04-17T05:41:35.130858+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:26.784407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162336+0000","last_change":"2024-04-17T04:51:43.682620+0000","last_active":"2024-04-17T05:43:50.162336+0000","last_peered":"2024-04-17T05:43:50.162336+0000","last_clean":"2024-04-17T05:43:50.162336+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:43:50.162336+0000","last_undegraded":"2024-04-17T05:43:50.162336+0000","last_fullsized":"2024-04-17T05:43:50.162336+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:29.937212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'3","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.192665+0000","last_change":"2024-04-17T04:51:39.605291+0000","last_active":"2024-04-17T05:45:50.192665+0000","last_peered":"2024-04-17T05:45:50.192665+0000","last_clean":"2024-04-17T05:45:50.192665+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:45:50.192665+0000","last_undegraded":"2024-04-17T05:45:50.192665+0000","last_fullsized":"2024-04-17T05:45:50.192665+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:40.774979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":121,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.150706+0000","last_change":"2024-04-17T04:47:56.953281+0000","last_active":"2024-04-17T05:43:00.150706+0000","last_peered":"2024-04-17T05:43:00.150706+0000","last_clean":"2024-04-17T05:43:00.150706+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:43:00.150706+0000","last_undegraded":"2024-04-17T05:43:00.150706+0000","last_fullsized":"2024-04-17T05:43:00.150706+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:02.682710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164774+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:41:40.164774+0000","last_peered":"2024-04-17T05:41:40.164774+0000","last_clean":"2024-04-17T05:41:40.164774+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:41:40.164774+0000","last_undegraded":"2024-04-17T05:41:40.164774+0000","last_fullsized":"2024-04-17T05:41:40.164774+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:05.623921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.195511+0000","last_change":"2024-04-17T04:51:43.676613+0000","last_active":"2024-04-17T05:43:55.195511+0000","last_peered":"2024-04-17T05:43:55.195511+0000","last_clean":"2024-04-17T05:43:55.195511+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:43:55.195511+0000","last_undegraded":"2024-04-17T05:43:55.195511+0000","last_fullsized":"2024-04-17T05:43:55.195511+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:26.269622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946142+0000","last_change":"2024-04-17T04:51:39.615233+0000","last_active":"2024-04-17T05:43:44.946142+0000","last_peered":"2024-04-17T05:43:44.946142+0000","last_clean":"2024-04-17T05:43:44.946142+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:43:44.946142+0000","last_undegraded":"2024-04-17T05:43:44.946142+0000","last_fullsized":"2024-04-17T05:43:44.946142+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:02.139603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":160,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183795+0000","last_change":"2024-04-17T04:47:56.969273+0000","last_active":"2024-04-17T05:43:00.183795+0000","last_peered":"2024-04-17T05:43:00.183795+0000","last_clean":"2024-04-17T05:43:00.183795+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:43:00.183795+0000","last_undegraded":"2024-04-17T05:43:00.183795+0000","last_fullsized":"2024-04-17T05:43:00.183795+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:24.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.194209+0000","last_change":"2024-04-17T05:07:45.745164+0000","last_active":"2024-04-17T05:43:50.194209+0000","last_peered":"2024-04-17T05:43:50.194209+0000","last_clean":"2024-04-17T05:43:50.194209+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:43:50.194209+0000","last_undegraded":"2024-04-17T05:43:50.194209+0000","last_fullsized":"2024-04-17T05:43:50.194209+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:44.184077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'2","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.182184+0000","last_change":"2024-04-17T04:51:43.666389+0000","last_active":"2024-04-17T05:45:05.182184+0000","last_peered":"2024-04-17T05:45:05.182184+0000","last_clean":"2024-04-17T05:45:05.182184+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:45:05.182184+0000","last_undegraded":"2024-04-17T05:45:05.182184+0000","last_fullsized":"2024-04-17T05:45:05.182184+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:10.966058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.142876+0000","last_change":"2024-04-17T04:51:39.625539+0000","last_active":"2024-04-17T05:42:25.142876+0000","last_peered":"2024-04-17T05:42:25.142876+0000","last_clean":"2024-04-17T05:42:25.142876+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:42:25.142876+0000","last_undegraded":"2024-04-17T05:42:25.142876+0000","last_fullsized":"2024-04-17T05:42:25.142876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:55.423515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":200,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183684+0000","last_change":"2024-04-17T04:47:56.967026+0000","last_active":"2024-04-17T05:43:00.183684+0000","last_peered":"2024-04-17T05:43:00.183684+0000","last_clean":"2024-04-17T05:43:00.183684+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:43:00.183684+0000","last_undegraded":"2024-04-17T05:43:00.183684+0000","last_fullsized":"2024-04-17T05:43:00.183684+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:26.424709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158284+0000","last_change":"2024-04-17T05:07:45.754178+0000","last_active":"2024-04-17T05:43:10.158284+0000","last_peered":"2024-04-17T05:43:10.158284+0000","last_clean":"2024-04-17T05:43:10.158284+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:43:10.158284+0000","last_undegraded":"2024-04-17T05:43:10.158284+0000","last_fullsized":"2024-04-17T05:43:10.158284+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:49.281995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'6","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.021212+0000","last_change":"2024-04-17T04:51:43.664767+0000","last_active":"2024-04-17T05:46:02.021212+0000","last_peered":"2024-04-17T05:46:02.021212+0000","last_clean":"2024-04-17T05:46:02.021212+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:46:02.021212+0000","last_undegraded":"2024-04-17T05:46:02.021212+0000","last_fullsized":"2024-04-17T05:46:02.021212+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:36.530590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":22528,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"261'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.134896+0000","last_change":"2024-04-17T04:51:39.611306+0000","last_active":"2024-04-17T05:42:35.134896+0000","last_peered":"2024-04-17T05:42:35.134896+0000","last_clean":"2024-04-17T05:42:35.134896+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:42:35.134896+0000","last_undegraded":"2024-04-17T05:42:35.134896+0000","last_fullsized":"2024-04-17T05:42:35.134896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:51.035427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'74","reported_seq":283,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.149482+0000","last_change":"2024-04-17T04:47:56.952081+0000","last_active":"2024-04-17T05:43:00.149482+0000","last_peered":"2024-04-17T05:43:00.149482+0000","last_clean":"2024-04-17T05:43:00.149482+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:43:00.149482+0000","last_undegraded":"2024-04-17T05:43:00.149482+0000","last_fullsized":"2024-04-17T05:43:00.149482+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:08:06.903329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145686+0000","last_change":"2024-04-17T05:07:45.754885+0000","last_active":"2024-04-17T05:42:45.145686+0000","last_peered":"2024-04-17T05:42:45.145686+0000","last_clean":"2024-04-17T05:42:45.145686+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:42:45.145686+0000","last_undegraded":"2024-04-17T05:42:45.145686+0000","last_fullsized":"2024-04-17T05:42:45.145686+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:02.913191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133462+0000","last_change":"2024-04-17T04:51:43.664884+0000","last_active":"2024-04-17T05:41:45.133462+0000","last_peered":"2024-04-17T05:41:45.133462+0000","last_clean":"2024-04-17T05:41:45.133462+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:41:45.133462+0000","last_undegraded":"2024-04-17T05:41:45.133462+0000","last_fullsized":"2024-04-17T05:41:45.133462+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:51:04.163089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'9","reported_seq":67,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159448+0000","last_change":"2024-04-17T04:51:39.621614+0000","last_active":"2024-04-17T05:44:10.159448+0000","last_peered":"2024-04-17T05:44:10.159448+0000","last_clean":"2024-04-17T05:44:10.159448+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:44:10.159448+0000","last_undegraded":"2024-04-17T05:44:10.159448+0000","last_fullsized":"2024-04-17T05:44:10.159448+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:01.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":16384,"num_write":24,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":74,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:20.127064+0000","last_change":"2024-04-17T04:47:56.955568+0000","last_active":"2024-04-17T05:41:20.127064+0000","last_peered":"2024-04-17T05:41:20.127064+0000","last_clean":"2024-04-17T05:41:20.127064+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:41:20.127064+0000","last_undegraded":"2024-04-17T05:41:20.127064+0000","last_fullsized":"2024-04-17T05:41:20.127064+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:09:24.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145754+0000","last_change":"2024-04-17T05:07:45.757519+0000","last_active":"2024-04-17T05:42:45.145754+0000","last_peered":"2024-04-17T05:42:45.145754+0000","last_clean":"2024-04-17T05:42:45.145754+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:42:45.145754+0000","last_undegraded":"2024-04-17T05:42:45.145754+0000","last_fullsized":"2024-04-17T05:42:45.145754+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:27.671557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165700+0000","last_change":"2024-04-17T04:51:43.686621+0000","last_active":"2024-04-17T05:41:45.165700+0000","last_peered":"2024-04-17T05:41:45.165700+0000","last_clean":"2024-04-17T05:41:45.165700+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:41:45.165700+0000","last_undegraded":"2024-04-17T05:41:45.165700+0000","last_fullsized":"2024-04-17T05:41:45.165700+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:36.829860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.132003+0000","last_change":"2024-04-17T04:51:39.643524+0000","last_active":"2024-04-17T05:41:40.132003+0000","last_peered":"2024-04-17T05:41:40.132003+0000","last_clean":"2024-04-17T05:41:40.132003+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:41:40.132003+0000","last_undegraded":"2024-04-17T05:41:40.132003+0000","last_fullsized":"2024-04-17T05:41:40.132003+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:01.434777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":90,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163169+0000","last_change":"2024-04-17T04:47:56.932780+0000","last_active":"2024-04-17T05:43:35.163169+0000","last_peered":"2024-04-17T05:43:35.163169+0000","last_clean":"2024-04-17T05:43:35.163169+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:43:35.163169+0000","last_undegraded":"2024-04-17T05:43:35.163169+0000","last_fullsized":"2024-04-17T05:43:35.163169+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:07.644614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151106+0000","last_change":"2024-04-17T05:07:45.743105+0000","last_active":"2024-04-17T05:42:45.151106+0000","last_peered":"2024-04-17T05:42:45.151106+0000","last_clean":"2024-04-17T05:42:45.151106+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:42:45.151106+0000","last_undegraded":"2024-04-17T05:42:45.151106+0000","last_fullsized":"2024-04-17T05:42:45.151106+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:30.294560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.176828+0000","last_change":"2024-04-17T04:51:43.664814+0000","last_active":"2024-04-17T05:42:30.176828+0000","last_peered":"2024-04-17T05:42:30.176828+0000","last_clean":"2024-04-17T05:42:30.176828+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:42:30.176828+0000","last_undegraded":"2024-04-17T05:42:30.176828+0000","last_fullsized":"2024-04-17T05:42:30.176828+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:39.897527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157053+0000","last_change":"2024-04-17T04:51:39.626132+0000","last_active":"2024-04-17T05:43:10.157053+0000","last_peered":"2024-04-17T05:43:10.157053+0000","last_clean":"2024-04-17T05:43:10.157053+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:43:10.157053+0000","last_undegraded":"2024-04-17T05:43:10.157053+0000","last_fullsized":"2024-04-17T05:43:10.157053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:23.450299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":80,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.158447+0000","last_change":"2024-04-17T04:47:56.956719+0000","last_active":"2024-04-17T05:43:35.158447+0000","last_peered":"2024-04-17T05:43:35.158447+0000","last_clean":"2024-04-17T05:43:35.158447+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:43:35.158447+0000","last_undegraded":"2024-04-17T05:43:35.158447+0000","last_fullsized":"2024-04-17T05:43:35.158447+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:43:10.431237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.149849+0000","last_change":"2024-04-17T05:07:45.738001+0000","last_active":"2024-04-17T05:42:45.149849+0000","last_peered":"2024-04-17T05:42:45.149849+0000","last_clean":"2024-04-17T05:42:45.149849+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:42:45.149849+0000","last_undegraded":"2024-04-17T05:42:45.149849+0000","last_fullsized":"2024-04-17T05:42:45.149849+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:06.048242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187309+0000","last_change":"2024-04-17T04:51:43.682839+0000","last_active":"2024-04-17T05:43:15.187309+0000","last_peered":"2024-04-17T05:43:15.187309+0000","last_clean":"2024-04-17T05:43:15.187309+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:43:15.187309+0000","last_undegraded":"2024-04-17T05:43:15.187309+0000","last_fullsized":"2024-04-17T05:43:15.187309+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:32.566827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:50.173942+0000","last_change":"2024-04-17T04:51:39.639685+0000","last_active":"2024-04-17T05:44:50.173942+0000","last_peered":"2024-04-17T05:44:50.173942+0000","last_clean":"2024-04-17T05:44:50.173942+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:44:50.173942+0000","last_undegraded":"2024-04-17T05:44:50.173942+0000","last_fullsized":"2024-04-17T05:44:50.173942+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:41.954023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.191081+0000","last_change":"2024-04-17T04:47:56.966803+0000","last_active":"2024-04-17T05:43:35.191081+0000","last_peered":"2024-04-17T05:43:35.191081+0000","last_clean":"2024-04-17T05:43:35.191081+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:43:35.191081+0000","last_undegraded":"2024-04-17T05:43:35.191081+0000","last_fullsized":"2024-04-17T05:43:35.191081+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:27.144075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.145538+0000","last_change":"2024-04-17T05:07:45.747162+0000","last_active":"2024-04-17T05:42:45.145538+0000","last_peered":"2024-04-17T05:42:45.145538+0000","last_clean":"2024-04-17T05:42:45.145538+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:42:45.145538+0000","last_undegraded":"2024-04-17T05:42:45.145538+0000","last_fullsized":"2024-04-17T05:42:45.145538+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:25.853530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'5","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.212787+0000","last_change":"2024-04-17T04:51:43.671647+0000","last_active":"2024-04-17T05:46:03.212787+0000","last_peered":"2024-04-17T05:46:03.212787+0000","last_clean":"2024-04-17T05:46:03.212787+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:46:03.212787+0000","last_undegraded":"2024-04-17T05:46:03.212787+0000","last_fullsized":"2024-04-17T05:46:03.212787+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:09:37.852227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":48,"num_write":10,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'13","reported_seq":81,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.606829+0000","last_change":"2024-04-17T04:51:39.612849+0000","last_active":"2024-04-17T05:46:02.606829+0000","last_peered":"2024-04-17T05:46:02.606829+0000","last_clean":"2024-04-17T05:46:02.606829+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:46:02.606829+0000","last_undegraded":"2024-04-17T05:46:02.606829+0000","last_fullsized":"2024-04-17T05:46:02.606829+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:41.680996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":46082,"num_write":17,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":98,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.161351+0000","last_change":"2024-04-17T04:47:56.937214+0000","last_active":"2024-04-17T05:43:35.161351+0000","last_peered":"2024-04-17T05:43:35.161351+0000","last_clean":"2024-04-17T05:43:35.161351+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:43:35.161351+0000","last_undegraded":"2024-04-17T05:43:35.161351+0000","last_fullsized":"2024-04-17T05:43:35.161351+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:45:02.809448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.151449+0000","last_change":"2024-04-17T05:07:45.744163+0000","last_active":"2024-04-17T05:42:45.151449+0000","last_peered":"2024-04-17T05:42:45.151449+0000","last_clean":"2024-04-17T05:42:45.151449+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:42:45.151449+0000","last_undegraded":"2024-04-17T05:42:45.151449+0000","last_fullsized":"2024-04-17T05:42:45.151449+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:42.169974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'11","reported_seq":72,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:55.169625+0000","last_change":"2024-04-17T04:51:43.667351+0000","last_active":"2024-04-17T05:44:55.169625+0000","last_peered":"2024-04-17T05:44:55.169625+0000","last_clean":"2024-04-17T05:44:55.169625+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:44:55.169625+0000","last_undegraded":"2024-04-17T05:44:55.169625+0000","last_fullsized":"2024-04-17T05:44:55.169625+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:27.804721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":37890,"num_write":20,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143121+0000","last_change":"2024-04-17T04:51:39.639400+0000","last_active":"2024-04-17T05:42:25.143121+0000","last_peered":"2024-04-17T05:42:25.143121+0000","last_clean":"2024-04-17T05:42:25.143121+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:42:25.143121+0000","last_undegraded":"2024-04-17T05:42:25.143121+0000","last_fullsized":"2024-04-17T05:42:25.143121+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:39.092327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":211,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133386+0000","last_change":"2024-04-17T04:47:56.964248+0000","last_active":"2024-04-17T05:41:50.133386+0000","last_peered":"2024-04-17T05:41:50.133386+0000","last_clean":"2024-04-17T05:41:50.133386+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:41:50.133386+0000","last_undegraded":"2024-04-17T05:41:50.133386+0000","last_fullsized":"2024-04-17T05:41:50.133386+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:19.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163617+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:41:35.163617+0000","last_peered":"2024-04-17T05:41:35.163617+0000","last_clean":"2024-04-17T05:41:35.163617+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:41:35.163617+0000","last_undegraded":"2024-04-17T05:41:35.163617+0000","last_fullsized":"2024-04-17T05:41:35.163617+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:45.333124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'5","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.157158+0000","last_change":"2024-04-17T04:51:43.670407+0000","last_active":"2024-04-17T05:43:05.157158+0000","last_peered":"2024-04-17T05:43:05.157158+0000","last_clean":"2024-04-17T05:43:05.157158+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:43:05.157158+0000","last_undegraded":"2024-04-17T05:43:05.157158+0000","last_fullsized":"2024-04-17T05:43:05.157158+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:31.623818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.175612+0000","last_change":"2024-04-17T04:51:39.613038+0000","last_active":"2024-04-17T05:45:20.175612+0000","last_peered":"2024-04-17T05:45:20.175612+0000","last_clean":"2024-04-17T05:45:20.175612+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:45:20.175612+0000","last_undegraded":"2024-04-17T05:45:20.175612+0000","last_fullsized":"2024-04-17T05:45:20.175612+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:51.929471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.130770+0000","last_change":"2024-04-17T04:47:56.962330+0000","last_active":"2024-04-17T05:41:40.130770+0000","last_peered":"2024-04-17T05:41:40.130770+0000","last_clean":"2024-04-17T05:41:40.130770+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:41:40.130770+0000","last_undegraded":"2024-04-17T05:41:40.130770+0000","last_fullsized":"2024-04-17T05:41:40.130770+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:38.049482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130958+0000","last_change":"2024-04-17T05:07:45.749757+0000","last_active":"2024-04-17T05:41:35.130958+0000","last_peered":"2024-04-17T05:41:35.130958+0000","last_clean":"2024-04-17T05:41:35.130958+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:41:35.130958+0000","last_undegraded":"2024-04-17T05:41:35.130958+0000","last_fullsized":"2024-04-17T05:41:35.130958+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:50:18.794745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'2","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:25.191624+0000","last_change":"2024-04-17T04:51:43.650369+0000","last_active":"2024-04-17T05:45:25.191624+0000","last_peered":"2024-04-17T05:45:25.191624+0000","last_clean":"2024-04-17T05:45:25.191624+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:45:25.191624+0000","last_undegraded":"2024-04-17T05:45:25.191624+0000","last_fullsized":"2024-04-17T05:45:25.191624+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:16:08.661428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.183604+0000","last_change":"2024-04-17T04:51:39.631783+0000","last_active":"2024-04-17T05:43:00.183604+0000","last_peered":"2024-04-17T05:43:00.183604+0000","last_clean":"2024-04-17T05:43:00.183604+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:43:00.183604+0000","last_undegraded":"2024-04-17T05:43:00.183604+0000","last_fullsized":"2024-04-17T05:43:00.183604+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:32.873589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":132,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.131963+0000","last_change":"2024-04-17T04:47:56.967539+0000","last_active":"2024-04-17T05:41:40.131963+0000","last_peered":"2024-04-17T05:41:40.131963+0000","last_clean":"2024-04-17T05:41:40.131963+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:41:40.131963+0000","last_undegraded":"2024-04-17T05:41:40.131963+0000","last_fullsized":"2024-04-17T05:41:40.131963+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:50.899642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"263'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.149607+0000","last_change":"2024-04-17T05:07:45.748765+0000","last_active":"2024-04-17T05:42:55.149607+0000","last_peered":"2024-04-17T05:42:55.149607+0000","last_clean":"2024-04-17T05:42:55.149607+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:42:55.149607+0000","last_undegraded":"2024-04-17T05:42:55.149607+0000","last_fullsized":"2024-04-17T05:42:55.149607+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:21.732472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'3","reported_seq":51,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:25.217937+0000","last_change":"2024-04-17T04:51:43.684076+0000","last_active":"2024-04-17T05:45:25.217937+0000","last_peered":"2024-04-17T05:45:25.217937+0000","last_clean":"2024-04-17T05:45:25.217937+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:45:25.217937+0000","last_undegraded":"2024-04-17T05:45:25.217937+0000","last_fullsized":"2024-04-17T05:45:25.217937+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.261686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'5","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.610398+0000","last_change":"2024-04-17T04:51:39.626930+0000","last_active":"2024-04-17T05:46:02.610398+0000","last_peered":"2024-04-17T05:46:02.610398+0000","last_clean":"2024-04-17T05:46:02.610398+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:46:02.610398+0000","last_undegraded":"2024-04-17T05:46:02.610398+0000","last_fullsized":"2024-04-17T05:46:02.610398+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:24:11.581902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.122667+0000","last_change":"2024-04-17T04:47:56.946016+0000","last_active":"2024-04-17T05:41:40.122667+0000","last_peered":"2024-04-17T05:41:40.122667+0000","last_clean":"2024-04-17T05:41:40.122667+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:41:40.122667+0000","last_undegraded":"2024-04-17T05:41:40.122667+0000","last_fullsized":"2024-04-17T05:41:40.122667+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:39.518892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'1","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.163578+0000","last_change":"2024-04-17T05:07:45.741199+0000","last_active":"2024-04-17T05:41:35.163578+0000","last_peered":"2024-04-17T05:41:35.163578+0000","last_clean":"2024-04-17T05:41:35.163578+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:41:35.163578+0000","last_undegraded":"2024-04-17T05:41:35.163578+0000","last_fullsized":"2024-04-17T05:41:35.163578+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:14:05.831006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:05.150592+0000","last_change":"2024-04-17T04:51:43.673701+0000","last_active":"2024-04-17T05:43:05.150592+0000","last_peered":"2024-04-17T05:43:05.150592+0000","last_clean":"2024-04-17T05:43:05.150592+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:43:05.150592+0000","last_undegraded":"2024-04-17T05:43:05.150592+0000","last_fullsized":"2024-04-17T05:43:05.150592+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:00.395599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154845+0000","last_change":"2024-04-17T04:51:39.625916+0000","last_active":"2024-04-17T05:43:00.154845+0000","last_peered":"2024-04-17T05:43:00.154845+0000","last_clean":"2024-04-17T05:43:00.154845+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:43:00.154845+0000","last_undegraded":"2024-04-17T05:43:00.154845+0000","last_fullsized":"2024-04-17T05:43:00.154845+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:35:16.145280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":195,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:40.164555+0000","last_change":"2024-04-17T04:47:56.960822+0000","last_active":"2024-04-17T05:41:40.164555+0000","last_peered":"2024-04-17T05:41:40.164555+0000","last_clean":"2024-04-17T05:41:40.164555+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:41:40.164555+0000","last_undegraded":"2024-04-17T05:41:40.164555+0000","last_fullsized":"2024-04-17T05:41:40.164555+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:26.089711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:35.147879+0000","last_change":"2024-04-17T05:07:45.745113+0000","last_active":"2024-04-17T05:42:35.147879+0000","last_peered":"2024-04-17T05:42:35.147879+0000","last_clean":"2024-04-17T05:42:35.147879+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:42:35.147879+0000","last_undegraded":"2024-04-17T05:42:35.147879+0000","last_fullsized":"2024-04-17T05:42:35.147879+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:03.124310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'6","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.102710+0000","last_change":"2024-04-17T04:51:43.682734+0000","last_active":"2024-04-17T05:46:02.102710+0000","last_peered":"2024-04-17T05:46:02.102710+0000","last_clean":"2024-04-17T05:46:02.102710+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:46:02.102710+0000","last_undegraded":"2024-04-17T05:46:02.102710+0000","last_fullsized":"2024-04-17T05:46:02.102710+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:32.531568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":11,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.185187+0000","last_change":"2024-04-17T04:51:39.645602+0000","last_active":"2024-04-17T05:45:40.185187+0000","last_peered":"2024-04-17T05:45:40.185187+0000","last_clean":"2024-04-17T05:45:40.185187+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:45:40.185187+0000","last_undegraded":"2024-04-17T05:45:40.185187+0000","last_fullsized":"2024-04-17T05:45:40.185187+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:02.515138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:15.173510+0000","last_change":"2024-04-17T04:47:56.962048+0000","last_active":"2024-04-17T05:42:15.173510+0000","last_peered":"2024-04-17T05:42:15.173510+0000","last_clean":"2024-04-17T05:42:15.173510+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:42:15.173510+0000","last_undegraded":"2024-04-17T05:42:15.173510+0000","last_fullsized":"2024-04-17T05:42:15.173510+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:10:15.157938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159170+0000","last_change":"2024-04-17T05:07:45.751549+0000","last_active":"2024-04-17T05:43:45.159170+0000","last_peered":"2024-04-17T05:43:45.159170+0000","last_clean":"2024-04-17T05:43:45.159170+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:43:45.159170+0000","last_undegraded":"2024-04-17T05:43:45.159170+0000","last_fullsized":"2024-04-17T05:43:45.159170+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:47.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'5","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.459886+0000","last_change":"2024-04-17T04:51:43.688891+0000","last_active":"2024-04-17T05:46:01.459886+0000","last_peered":"2024-04-17T05:46:01.459886+0000","last_clean":"2024-04-17T05:46:01.459886+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:46:01.459886+0000","last_undegraded":"2024-04-17T05:46:01.459886+0000","last_fullsized":"2024-04-17T05:46:01.459886+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:35.298171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'5","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.190498+0000","last_change":"2024-04-17T04:51:39.634707+0000","last_active":"2024-04-17T05:45:40.190498+0000","last_peered":"2024-04-17T05:45:40.190498+0000","last_clean":"2024-04-17T05:45:40.190498+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:45:40.190498+0000","last_undegraded":"2024-04-17T05:45:40.190498+0000","last_fullsized":"2024-04-17T05:45:40.190498+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:09.043206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":11264,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141762+0000","last_change":"2024-04-17T04:47:56.954332+0000","last_active":"2024-04-17T05:42:00.141762+0000","last_peered":"2024-04-17T05:42:00.141762+0000","last_clean":"2024-04-17T05:42:00.141762+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:42:00.141762+0000","last_undegraded":"2024-04-17T05:42:00.141762+0000","last_fullsized":"2024-04-17T05:42:00.141762+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:07.662187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141842+0000","last_change":"2024-04-17T05:07:45.741143+0000","last_active":"2024-04-17T05:42:25.141842+0000","last_peered":"2024-04-17T05:42:25.141842+0000","last_clean":"2024-04-17T05:42:25.141842+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:42:25.141842+0000","last_undegraded":"2024-04-17T05:42:25.141842+0000","last_fullsized":"2024-04-17T05:42:25.141842+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:50.193462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164848+0000","last_change":"2024-04-17T04:51:43.657675+0000","last_active":"2024-04-17T05:43:40.164848+0000","last_peered":"2024-04-17T05:43:40.164848+0000","last_clean":"2024-04-17T05:43:40.164848+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:43:40.164848+0000","last_undegraded":"2024-04-17T05:43:40.164848+0000","last_fullsized":"2024-04-17T05:43:40.164848+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:59:28.152355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.191673+0000","last_change":"2024-04-17T04:51:39.634454+0000","last_active":"2024-04-17T05:45:40.191673+0000","last_peered":"2024-04-17T05:45:40.191673+0000","last_clean":"2024-04-17T05:45:40.191673+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:45:40.191673+0000","last_undegraded":"2024-04-17T05:45:40.191673+0000","last_fullsized":"2024-04-17T05:45:40.191673+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:48.747740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":123,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136769+0000","last_change":"2024-04-17T04:47:56.963915+0000","last_active":"2024-04-17T05:42:00.136769+0000","last_peered":"2024-04-17T05:42:00.136769+0000","last_clean":"2024-04-17T05:42:00.136769+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:42:00.136769+0000","last_undegraded":"2024-04-17T05:42:00.136769+0000","last_fullsized":"2024-04-17T05:42:00.136769+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:58.494222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141220+0000","last_change":"2024-04-17T05:07:45.757127+0000","last_active":"2024-04-17T05:42:25.141220+0000","last_peered":"2024-04-17T05:42:25.141220+0000","last_clean":"2024-04-17T05:42:25.141220+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:42:25.141220+0000","last_undegraded":"2024-04-17T05:42:25.141220+0000","last_fullsized":"2024-04-17T05:42:25.141220+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:23:42.585132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162275+0000","last_change":"2024-04-17T04:52:04.726201+0000","last_active":"2024-04-17T05:43:50.162275+0000","last_peered":"2024-04-17T05:43:50.162275+0000","last_clean":"2024-04-17T05:43:50.162275+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:43:50.162275+0000","last_undegraded":"2024-04-17T05:43:50.162275+0000","last_fullsized":"2024-04-17T05:43:50.162275+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:14:03.659722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:35.163394+0000","last_change":"2024-04-17T04:51:39.618531+0000","last_active":"2024-04-17T05:43:35.163394+0000","last_peered":"2024-04-17T05:43:35.163394+0000","last_clean":"2024-04-17T05:43:35.163394+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:43:35.163394+0000","last_undegraded":"2024-04-17T05:43:35.163394+0000","last_fullsized":"2024-04-17T05:43:35.163394+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:27:52.911539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":128,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139900+0000","last_change":"2024-04-17T04:47:56.962022+0000","last_active":"2024-04-17T05:42:00.139900+0000","last_peered":"2024-04-17T05:42:00.139900+0000","last_clean":"2024-04-17T05:42:00.139900+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:42:00.139900+0000","last_undegraded":"2024-04-17T05:42:00.139900+0000","last_fullsized":"2024-04-17T05:42:00.139900+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:04.015930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193091+0000","last_change":"2024-04-17T05:07:45.744264+0000","last_active":"2024-04-17T05:43:45.193091+0000","last_peered":"2024-04-17T05:43:45.193091+0000","last_clean":"2024-04-17T05:43:45.193091+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:43:45.193091+0000","last_undegraded":"2024-04-17T05:43:45.193091+0000","last_fullsized":"2024-04-17T05:43:45.193091+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:41.404295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'7","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.183346+0000","last_change":"2024-04-17T04:51:43.661076+0000","last_active":"2024-04-17T05:45:55.183346+0000","last_peered":"2024-04-17T05:45:55.183346+0000","last_clean":"2024-04-17T05:45:55.183346+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:45:55.183346+0000","last_undegraded":"2024-04-17T05:45:55.183346+0000","last_fullsized":"2024-04-17T05:45:55.183346+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:54.620750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":5216,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159409+0000","last_change":"2024-04-17T04:51:39.633118+0000","last_active":"2024-04-17T05:43:40.159409+0000","last_peered":"2024-04-17T05:43:40.159409+0000","last_clean":"2024-04-17T05:43:40.159409+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:43:40.159409+0000","last_undegraded":"2024-04-17T05:43:40.159409+0000","last_fullsized":"2024-04-17T05:43:40.159409+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:25:34.341448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":102,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137165+0000","last_change":"2024-04-17T04:47:56.963262+0000","last_active":"2024-04-17T05:42:00.137165+0000","last_peered":"2024-04-17T05:42:00.137165+0000","last_clean":"2024-04-17T05:42:00.137165+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:42:00.137165+0000","last_undegraded":"2024-04-17T05:42:00.137165+0000","last_fullsized":"2024-04-17T05:42:00.137165+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:09.975510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160492+0000","last_change":"2024-04-17T05:07:45.757192+0000","last_active":"2024-04-17T05:43:50.160492+0000","last_peered":"2024-04-17T05:43:50.160492+0000","last_clean":"2024-04-17T05:43:50.160492+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:43:50.160492+0000","last_undegraded":"2024-04-17T05:43:50.160492+0000","last_fullsized":"2024-04-17T05:43:50.160492+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:30:19.313475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'5","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.188378+0000","last_change":"2024-04-17T04:51:43.675078+0000","last_active":"2024-04-17T05:45:55.188378+0000","last_peered":"2024-04-17T05:45:55.188378+0000","last_clean":"2024-04-17T05:45:55.188378+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:45:55.188378+0000","last_undegraded":"2024-04-17T05:45:55.188378+0000","last_fullsized":"2024-04-17T05:45:55.188378+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:21:00.172403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.197299+0000","last_change":"2024-04-17T04:51:39.618066+0000","last_active":"2024-04-17T05:45:50.197299+0000","last_peered":"2024-04-17T05:45:50.197299+0000","last_clean":"2024-04-17T05:45:50.197299+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:45:50.197299+0000","last_undegraded":"2024-04-17T05:45:50.197299+0000","last_fullsized":"2024-04-17T05:45:50.197299+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:51.504887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":286,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140125+0000","last_change":"2024-04-17T04:47:56.963049+0000","last_active":"2024-04-17T05:42:20.140125+0000","last_peered":"2024-04-17T05:42:20.140125+0000","last_clean":"2024-04-17T05:42:20.140125+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:42:20.140125+0000","last_undegraded":"2024-04-17T05:42:20.140125+0000","last_fullsized":"2024-04-17T05:42:20.140125+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:08.912141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159213+0000","last_change":"2024-04-17T05:07:45.754827+0000","last_active":"2024-04-17T05:43:45.159213+0000","last_peered":"2024-04-17T05:43:45.159213+0000","last_clean":"2024-04-17T05:43:45.159213+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:43:45.159213+0000","last_undegraded":"2024-04-17T05:43:45.159213+0000","last_fullsized":"2024-04-17T05:43:45.159213+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:12.463675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.162241+0000","last_change":"2024-04-17T04:51:43.655115+0000","last_active":"2024-04-17T05:43:30.162241+0000","last_peered":"2024-04-17T05:43:30.162241+0000","last_clean":"2024-04-17T05:43:30.162241+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:43:30.162241+0000","last_undegraded":"2024-04-17T05:43:30.162241+0000","last_fullsized":"2024-04-17T05:43:30.162241+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:20.469117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162125+0000","last_change":"2024-04-17T04:51:39.633266+0000","last_active":"2024-04-17T05:43:50.162125+0000","last_peered":"2024-04-17T05:43:50.162125+0000","last_clean":"2024-04-17T05:43:50.162125+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:43:50.162125+0000","last_undegraded":"2024-04-17T05:43:50.162125+0000","last_fullsized":"2024-04-17T05:43:50.162125+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:40.414020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":106,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.171752+0000","last_change":"2024-04-17T04:47:56.952507+0000","last_active":"2024-04-17T05:42:10.171752+0000","last_peered":"2024-04-17T05:42:10.171752+0000","last_clean":"2024-04-17T05:42:10.171752+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:42:10.171752+0000","last_undegraded":"2024-04-17T05:42:10.171752+0000","last_fullsized":"2024-04-17T05:42:10.171752+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:18.535221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.160987+0000","last_change":"2024-04-17T05:07:45.739696+0000","last_active":"2024-04-17T05:43:45.160987+0000","last_peered":"2024-04-17T05:43:45.160987+0000","last_clean":"2024-04-17T05:43:45.160987+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:43:45.160987+0000","last_undegraded":"2024-04-17T05:43:45.160987+0000","last_fullsized":"2024-04-17T05:43:45.160987+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:02:31.906289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.161513+0000","last_change":"2024-04-17T04:51:43.675773+0000","last_active":"2024-04-17T05:43:55.161513+0000","last_peered":"2024-04-17T05:43:55.161513+0000","last_clean":"2024-04-17T05:43:55.161513+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:43:55.161513+0000","last_undegraded":"2024-04-17T05:43:55.161513+0000","last_fullsized":"2024-04-17T05:43:55.161513+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:07.015940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.180009+0000","last_change":"2024-04-17T04:51:39.621153+0000","last_active":"2024-04-17T05:45:40.180009+0000","last_peered":"2024-04-17T05:45:40.180009+0000","last_clean":"2024-04-17T05:45:40.180009+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:45:40.180009+0000","last_undegraded":"2024-04-17T05:45:40.180009+0000","last_fullsized":"2024-04-17T05:45:40.180009+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:36.053793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":188,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138665+0000","last_change":"2024-04-17T04:47:56.966581+0000","last_active":"2024-04-17T05:42:10.138665+0000","last_peered":"2024-04-17T05:42:10.138665+0000","last_clean":"2024-04-17T05:42:10.138665+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:42:10.138665+0000","last_undegraded":"2024-04-17T05:42:10.138665+0000","last_fullsized":"2024-04-17T05:42:10.138665+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:35.900737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151057+0000","last_change":"2024-04-17T05:07:45.747152+0000","last_active":"2024-04-17T05:43:10.151057+0000","last_peered":"2024-04-17T05:43:10.151057+0000","last_clean":"2024-04-17T05:43:10.151057+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:43:10.151057+0000","last_undegraded":"2024-04-17T05:43:10.151057+0000","last_fullsized":"2024-04-17T05:43:10.151057+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:50:06.059672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.165033+0000","last_change":"2024-04-17T04:51:43.682823+0000","last_active":"2024-04-17T05:44:00.165033+0000","last_peered":"2024-04-17T05:44:00.165033+0000","last_clean":"2024-04-17T05:44:00.165033+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:44:00.165033+0000","last_undegraded":"2024-04-17T05:44:00.165033+0000","last_fullsized":"2024-04-17T05:44:00.165033+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:56:21.798951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'8","reported_seq":69,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.187422+0000","last_change":"2024-04-17T04:51:39.636037+0000","last_active":"2024-04-17T05:45:50.187422+0000","last_peered":"2024-04-17T05:45:50.187422+0000","last_clean":"2024-04-17T05:45:50.187422+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:45:50.187422+0000","last_undegraded":"2024-04-17T05:45:50.187422+0000","last_fullsized":"2024-04-17T05:45:50.187422+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:03.304591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11291,"num_write":13,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":149,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.139651+0000","last_change":"2024-04-17T04:47:56.956029+0000","last_active":"2024-04-17T05:42:10.139651+0000","last_peered":"2024-04-17T05:42:10.139651+0000","last_clean":"2024-04-17T05:42:10.139651+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:42:10.139651+0000","last_undegraded":"2024-04-17T05:42:10.139651+0000","last_fullsized":"2024-04-17T05:42:10.139651+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:21.889729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157001+0000","last_change":"2024-04-17T05:07:45.753495+0000","last_active":"2024-04-17T05:43:10.157001+0000","last_peered":"2024-04-17T05:43:10.157001+0000","last_clean":"2024-04-17T05:43:10.157001+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:43:10.157001+0000","last_undegraded":"2024-04-17T05:43:10.157001+0000","last_fullsized":"2024-04-17T05:43:10.157001+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:29:25.567114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.162221+0000","last_change":"2024-04-17T04:51:43.681581+0000","last_active":"2024-04-17T05:43:50.162221+0000","last_peered":"2024-04-17T05:43:50.162221+0000","last_clean":"2024-04-17T05:43:50.162221+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:43:50.162221+0000","last_undegraded":"2024-04-17T05:43:50.162221+0000","last_fullsized":"2024-04-17T05:43:50.162221+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:53.170686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'3","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.194740+0000","last_change":"2024-04-17T04:51:39.634757+0000","last_active":"2024-04-17T05:46:00.194740+0000","last_peered":"2024-04-17T05:46:00.194740+0000","last_clean":"2024-04-17T05:46:00.194740+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:46:00.194740+0000","last_undegraded":"2024-04-17T05:46:00.194740+0000","last_fullsized":"2024-04-17T05:46:00.194740+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:38.980103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":196,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138819+0000","last_change":"2024-04-17T04:47:56.946324+0000","last_active":"2024-04-17T05:42:10.138819+0000","last_peered":"2024-04-17T05:42:10.138819+0000","last_clean":"2024-04-17T05:42:10.138819+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:42:10.138819+0000","last_undegraded":"2024-04-17T05:42:10.138819+0000","last_fullsized":"2024-04-17T05:42:10.138819+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:09.179720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.167123+0000","last_change":"2024-04-17T05:07:45.736217+0000","last_active":"2024-04-17T05:43:50.167123+0000","last_peered":"2024-04-17T05:43:50.167123+0000","last_clean":"2024-04-17T05:43:50.167123+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:43:50.167123+0000","last_undegraded":"2024-04-17T05:43:50.167123+0000","last_fullsized":"2024-04-17T05:43:50.167123+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:04.504706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'20","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.108013+0000","last_change":"2024-04-17T04:51:43.675854+0000","last_active":"2024-04-17T05:46:02.108013+0000","last_peered":"2024-04-17T05:46:02.108013+0000","last_clean":"2024-04-17T05:46:02.108013+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:46:02.108013+0000","last_undegraded":"2024-04-17T05:46:02.108013+0000","last_fullsized":"2024-04-17T05:46:02.108013+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:19:54.193750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":54274,"num_write":19,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'10","reported_seq":79,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163666+0000","last_change":"2024-04-17T04:51:39.635143+0000","last_active":"2024-04-17T05:43:40.163666+0000","last_peered":"2024-04-17T05:43:40.163666+0000","last_clean":"2024-04-17T05:43:40.163666+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:43:40.163666+0000","last_undegraded":"2024-04-17T05:43:40.163666+0000","last_fullsized":"2024-04-17T05:43:40.163666+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:53:08.025744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:20.140249+0000","last_change":"2024-04-17T04:47:56.969411+0000","last_active":"2024-04-17T05:42:20.140249+0000","last_peered":"2024-04-17T05:42:20.140249+0000","last_clean":"2024-04-17T05:42:20.140249+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:42:20.140249+0000","last_undegraded":"2024-04-17T05:42:20.140249+0000","last_fullsized":"2024-04-17T05:42:20.140249+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:56.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938315+0000","last_change":"2024-04-17T05:07:45.749227+0000","last_active":"2024-04-17T05:43:09.938315+0000","last_peered":"2024-04-17T05:43:09.938315+0000","last_clean":"2024-04-17T05:43:09.938315+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:43:09.938315+0000","last_undegraded":"2024-04-17T05:43:09.938315+0000","last_fullsized":"2024-04-17T05:43:09.938315+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:15.680393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:44.946093+0000","last_change":"2024-04-17T04:51:43.648648+0000","last_active":"2024-04-17T05:43:44.946093+0000","last_peered":"2024-04-17T05:43:44.946093+0000","last_clean":"2024-04-17T05:43:44.946093+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:43:44.946093+0000","last_undegraded":"2024-04-17T05:43:44.946093+0000","last_fullsized":"2024-04-17T05:43:44.946093+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:57:25.905644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'6","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.187527+0000","last_change":"2024-04-17T04:51:39.646180+0000","last_active":"2024-04-17T05:45:50.187527+0000","last_peered":"2024-04-17T05:45:50.187527+0000","last_clean":"2024-04-17T05:45:50.187527+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:45:50.187527+0000","last_undegraded":"2024-04-17T05:45:50.187527+0000","last_fullsized":"2024-04-17T05:45:50.187527+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:55.361466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":8192,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":110,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143901+0000","last_change":"2024-04-17T04:47:56.947428+0000","last_active":"2024-04-17T05:42:10.143901+0000","last_peered":"2024-04-17T05:42:10.143901+0000","last_clean":"2024-04-17T05:42:10.143901+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:42:10.143901+0000","last_undegraded":"2024-04-17T05:42:10.143901+0000","last_fullsized":"2024-04-17T05:42:10.143901+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:56:57.355596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"263'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.145978+0000","last_change":"2024-04-17T05:07:45.746243+0000","last_active":"2024-04-17T05:42:25.145978+0000","last_peered":"2024-04-17T05:42:25.145978+0000","last_clean":"2024-04-17T05:42:25.145978+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:42:25.145978+0000","last_undegraded":"2024-04-17T05:42:25.145978+0000","last_fullsized":"2024-04-17T05:42:25.145978+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:08:11.491152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'8","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.198468+0000","last_change":"2024-04-17T04:51:43.675539+0000","last_active":"2024-04-17T05:45:55.198468+0000","last_peered":"2024-04-17T05:45:55.198468+0000","last_clean":"2024-04-17T05:45:55.198468+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:45:55.198468+0000","last_undegraded":"2024-04-17T05:45:55.198468+0000","last_fullsized":"2024-04-17T05:45:55.198468+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:29.255105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":4098,"num_write":17,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160253+0000","last_change":"2024-04-17T04:51:39.637181+0000","last_active":"2024-04-17T05:43:50.160253+0000","last_peered":"2024-04-17T05:43:50.160253+0000","last_clean":"2024-04-17T05:43:50.160253+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:43:50.160253+0000","last_undegraded":"2024-04-17T05:43:50.160253+0000","last_fullsized":"2024-04-17T05:43:50.160253+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:34:38.011795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.143904+0000","last_change":"2024-04-17T04:47:56.949538+0000","last_active":"2024-04-17T05:42:10.143904+0000","last_peered":"2024-04-17T05:42:10.143904+0000","last_clean":"2024-04-17T05:42:10.143904+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:42:10.143904+0000","last_undegraded":"2024-04-17T05:42:10.143904+0000","last_fullsized":"2024-04-17T05:42:10.143904+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:29.512598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.143092+0000","last_change":"2024-04-17T05:07:45.726938+0000","last_active":"2024-04-17T05:42:25.143092+0000","last_peered":"2024-04-17T05:42:25.143092+0000","last_clean":"2024-04-17T05:42:25.143092+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:42:25.143092+0000","last_undegraded":"2024-04-17T05:42:25.143092+0000","last_fullsized":"2024-04-17T05:42:25.143092+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:33.108430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'3","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.192935+0000","last_change":"2024-04-17T04:51:43.648636+0000","last_active":"2024-04-17T05:45:55.192935+0000","last_peered":"2024-04-17T05:45:55.192935+0000","last_clean":"2024-04-17T05:45:55.192935+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:45:55.192935+0000","last_undegraded":"2024-04-17T05:45:55.192935+0000","last_fullsized":"2024-04-17T05:45:55.192935+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:01:44.200548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159094+0000","last_change":"2024-04-17T04:51:39.636922+0000","last_active":"2024-04-17T05:43:45.159094+0000","last_peered":"2024-04-17T05:43:45.159094+0000","last_clean":"2024-04-17T05:43:45.159094+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:43:45.159094+0000","last_undegraded":"2024-04-17T05:43:45.159094+0000","last_fullsized":"2024-04-17T05:43:45.159094+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:39:38.269210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.129678+0000","last_change":"2024-04-17T04:47:56.968894+0000","last_active":"2024-04-17T05:42:10.129678+0000","last_peered":"2024-04-17T05:42:10.129678+0000","last_clean":"2024-04-17T05:42:10.129678+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:42:10.129678+0000","last_undegraded":"2024-04-17T05:42:10.129678+0000","last_fullsized":"2024-04-17T05:42:10.129678+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:47.848243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152961+0000","last_change":"2024-04-17T05:07:45.757247+0000","last_active":"2024-04-17T05:43:10.152961+0000","last_peered":"2024-04-17T05:43:10.152961+0000","last_clean":"2024-04-17T05:43:10.152961+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:43:10.152961+0000","last_undegraded":"2024-04-17T05:43:10.152961+0000","last_fullsized":"2024-04-17T05:43:10.152961+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:13:26.267898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160974+0000","last_change":"2024-04-17T04:51:43.682310+0000","last_active":"2024-04-17T05:43:50.160974+0000","last_peered":"2024-04-17T05:43:50.160974+0000","last_clean":"2024-04-17T05:43:50.160974+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:43:50.160974+0000","last_undegraded":"2024-04-17T05:43:50.160974+0000","last_fullsized":"2024-04-17T05:43:50.160974+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:14.153241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'2","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.197366+0000","last_change":"2024-04-17T04:51:39.617751+0000","last_active":"2024-04-17T05:45:50.197366+0000","last_peered":"2024-04-17T05:45:50.197366+0000","last_clean":"2024-04-17T05:45:50.197366+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:45:50.197366+0000","last_undegraded":"2024-04-17T05:45:50.197366+0000","last_fullsized":"2024-04-17T05:45:50.197366+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:45.814751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":213,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.138713+0000","last_change":"2024-04-17T04:47:56.952060+0000","last_active":"2024-04-17T05:42:10.138713+0000","last_peered":"2024-04-17T05:42:10.138713+0000","last_clean":"2024-04-17T05:42:10.138713+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:42:10.138713+0000","last_undegraded":"2024-04-17T05:42:10.138713+0000","last_fullsized":"2024-04-17T05:42:10.138713+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:52.024764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.152311+0000","last_change":"2024-04-17T05:07:45.752461+0000","last_active":"2024-04-17T05:43:15.152311+0000","last_peered":"2024-04-17T05:43:15.152311+0000","last_clean":"2024-04-17T05:43:15.152311+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:43:15.152311+0000","last_undegraded":"2024-04-17T05:43:15.152311+0000","last_fullsized":"2024-04-17T05:43:15.152311+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:43.596172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:49.947148+0000","last_change":"2024-04-17T04:51:43.651232+0000","last_active":"2024-04-17T05:43:49.947148+0000","last_peered":"2024-04-17T05:43:49.947148+0000","last_clean":"2024-04-17T05:43:49.947148+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:43:49.947148+0000","last_undegraded":"2024-04-17T05:43:49.947148+0000","last_fullsized":"2024-04-17T05:43:49.947148+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:48.905181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'5","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.539399+0000","last_change":"2024-04-17T04:51:39.630310+0000","last_active":"2024-04-17T05:46:02.539399+0000","last_peered":"2024-04-17T05:46:02.539399+0000","last_clean":"2024-04-17T05:46:02.539399+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:46:02.539399+0000","last_undegraded":"2024-04-17T05:46:02.539399+0000","last_fullsized":"2024-04-17T05:46:02.539399+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:32:14.085186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6192,"num_write":5,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":127,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.137755+0000","last_change":"2024-04-17T04:47:56.963237+0000","last_active":"2024-04-17T05:42:10.137755+0000","last_peered":"2024-04-17T05:42:10.137755+0000","last_clean":"2024-04-17T05:42:10.137755+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:42:10.137755+0000","last_undegraded":"2024-04-17T05:42:10.137755+0000","last_fullsized":"2024-04-17T05:42:10.137755+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:47:55.905428+0000","last_clean_scrub_stamp":"2024-04-17T04:47:55.905428+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:11.681306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'349","reported_seq":1010,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.873550+0000","last_change":"2024-04-17T04:36:25.377106+0000","last_active":"2024-04-17T05:46:03.873550+0000","last_peered":"2024-04-17T05:46:03.873550+0000","last_clean":"2024-04-17T05:46:03.873550+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:46:03.873550+0000","last_undegraded":"2024-04-17T05:46:03.873550+0000","last_fullsized":"2024-04-17T05:46:03.873550+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:09.065634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":725,"num_write":1314,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":675,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134422+0000","last_change":"2024-04-17T04:36:27.389744+0000","last_active":"2024-04-17T05:41:30.134422+0000","last_peered":"2024-04-17T05:41:30.134422+0000","last_clean":"2024-04-17T05:41:30.134422+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:41:30.134422+0000","last_undegraded":"2024-04-17T05:41:30.134422+0000","last_fullsized":"2024-04-17T05:41:30.134422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:38:14.577671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'382","reported_seq":1054,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.808583+0000","last_change":"2024-04-17T04:36:25.357080+0000","last_active":"2024-04-17T05:46:03.808583+0000","last_peered":"2024-04-17T05:46:03.808583+0000","last_clean":"2024-04-17T05:46:03.808583+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:46:03.808583+0000","last_undegraded":"2024-04-17T05:46:03.808583+0000","last_fullsized":"2024-04-17T05:46:03.808583+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:21.403329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":600,"num_read_kb":10797,"num_write":1506,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":706,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:39.489661+0000","last_change":"2024-04-17T04:36:27.382828+0000","last_active":"2024-04-17T05:42:39.489661+0000","last_peered":"2024-04-17T05:42:39.489661+0000","last_clean":"2024-04-17T05:42:39.489661+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:42:39.489661+0000","last_undegraded":"2024-04-17T05:42:39.489661+0000","last_fullsized":"2024-04-17T05:42:39.489661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:30.715656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'312","reported_seq":978,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.803731+0000","last_change":"2024-04-17T04:36:25.329662+0000","last_active":"2024-04-17T05:46:03.803731+0000","last_peered":"2024-04-17T05:46:03.803731+0000","last_clean":"2024-04-17T05:46:03.803731+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:46:03.803731+0000","last_undegraded":"2024-04-17T05:46:03.803731+0000","last_fullsized":"2024-04-17T05:46:03.803731+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:42.966712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11583546,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":475,"num_read_kb":58403,"num_write":1188,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120297+0000","last_change":"2024-04-17T04:36:27.385236+0000","last_active":"2024-04-17T05:41:30.120297+0000","last_peered":"2024-04-17T05:41:30.120297+0000","last_clean":"2024-04-17T05:41:30.120297+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:41:30.120297+0000","last_undegraded":"2024-04-17T05:41:30.120297+0000","last_fullsized":"2024-04-17T05:41:30.120297+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:55.872226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'284","reported_seq":944,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.850703+0000","last_change":"2024-04-17T04:36:25.356234+0000","last_active":"2024-04-17T05:46:03.850703+0000","last_peered":"2024-04-17T05:46:03.850703+0000","last_clean":"2024-04-17T05:46:03.850703+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:46:03.850703+0000","last_undegraded":"2024-04-17T05:46:03.850703+0000","last_fullsized":"2024-04-17T05:46:03.850703+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:14.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174737,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":460,"num_read_kb":37884,"num_write":1128,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'304","reported_seq":897,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.129356+0000","last_change":"2024-04-17T04:36:27.385726+0000","last_active":"2024-04-17T05:41:30.129356+0000","last_peered":"2024-04-17T05:41:30.129356+0000","last_clean":"2024-04-17T05:41:30.129356+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:41:30.129356+0000","last_undegraded":"2024-04-17T05:41:30.129356+0000","last_fullsized":"2024-04-17T05:41:30.129356+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:55:33.151184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'336","reported_seq":996,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.772419+0000","last_change":"2024-04-17T04:36:25.377714+0000","last_active":"2024-04-17T05:46:03.772419+0000","last_peered":"2024-04-17T05:46:03.772419+0000","last_clean":"2024-04-17T05:46:03.772419+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:46:03.772419+0000","last_undegraded":"2024-04-17T05:46:03.772419+0000","last_fullsized":"2024-04-17T05:46:03.772419+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:34.861649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":528,"num_read_kb":13359,"num_write":1173,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5179,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.736719+0000","last_change":"2024-04-17T04:36:27.382622+0000","last_active":"2024-04-17T05:45:44.736719+0000","last_peered":"2024-04-17T05:45:44.736719+0000","last_clean":"2024-04-17T05:45:44.736719+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:45:44.736719+0000","last_undegraded":"2024-04-17T05:45:44.736719+0000","last_fullsized":"2024-04-17T05:45:44.736719+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:23.921181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'301","reported_seq":1014,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.842568+0000","last_change":"2024-04-17T04:36:25.352114+0000","last_active":"2024-04-17T05:46:03.842568+0000","last_peered":"2024-04-17T05:46:03.842568+0000","last_clean":"2024-04-17T05:46:03.842568+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:46:03.842568+0000","last_undegraded":"2024-04-17T05:46:03.842568+0000","last_fullsized":"2024-04-17T05:46:03.842568+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:17.459792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206995,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":584,"num_read_kb":64592,"num_write":977,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":774,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938531+0000","last_change":"2024-04-17T04:36:27.385407+0000","last_active":"2024-04-17T05:43:09.938531+0000","last_peered":"2024-04-17T05:43:09.938531+0000","last_clean":"2024-04-17T05:43:09.938531+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:43:09.938531+0000","last_undegraded":"2024-04-17T05:43:09.938531+0000","last_fullsized":"2024-04-17T05:43:09.938531+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:03.112120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'324","reported_seq":946,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.854118+0000","last_change":"2024-04-17T04:36:25.376858+0000","last_active":"2024-04-17T05:46:03.854118+0000","last_peered":"2024-04-17T05:46:03.854118+0000","last_clean":"2024-04-17T05:46:03.854118+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:46:03.854118+0000","last_undegraded":"2024-04-17T05:46:03.854118+0000","last_fullsized":"2024-04-17T05:46:03.854118+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:01:24.809459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269430,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":458,"num_read_kb":9424,"num_write":1128,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.157138+0000","last_change":"2024-04-17T04:36:27.376772+0000","last_active":"2024-04-17T05:43:10.157138+0000","last_peered":"2024-04-17T05:43:10.157138+0000","last_clean":"2024-04-17T05:43:10.157138+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:43:10.157138+0000","last_undegraded":"2024-04-17T05:43:10.157138+0000","last_fullsized":"2024-04-17T05:43:10.157138+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:07.055230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'318","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.824366+0000","last_change":"2024-04-17T04:36:25.343980+0000","last_active":"2024-04-17T05:46:03.824366+0000","last_peered":"2024-04-17T05:46:03.824366+0000","last_clean":"2024-04-17T05:46:03.824366+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:46:03.824366+0000","last_undegraded":"2024-04-17T05:46:03.824366+0000","last_fullsized":"2024-04-17T05:46:03.824366+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:21.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076711,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":427,"num_read_kb":14992,"num_write":1164,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.156665+0000","last_change":"2024-04-17T04:36:27.379088+0000","last_active":"2024-04-17T05:43:10.156665+0000","last_peered":"2024-04-17T05:43:10.156665+0000","last_clean":"2024-04-17T05:43:10.156665+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:43:10.156665+0000","last_undegraded":"2024-04-17T05:43:10.156665+0000","last_fullsized":"2024-04-17T05:43:10.156665+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:05.867623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125191+0000","last_change":"2024-04-17T04:36:27.385695+0000","last_active":"2024-04-17T05:41:50.125191+0000","last_peered":"2024-04-17T05:41:50.125191+0000","last_clean":"2024-04-17T05:41:50.125191+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:41:50.125191+0000","last_undegraded":"2024-04-17T05:41:50.125191+0000","last_fullsized":"2024-04-17T05:41:50.125191+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:33.234173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'322","reported_seq":929,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.865162+0000","last_change":"2024-04-17T04:37:03.648715+0000","last_active":"2024-04-17T05:46:03.865162+0000","last_peered":"2024-04-17T05:46:03.865162+0000","last_clean":"2024-04-17T05:46:03.865162+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:46:03.865162+0000","last_undegraded":"2024-04-17T05:46:03.865162+0000","last_fullsized":"2024-04-17T05:46:03.865162+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:12.036980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8704691,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":453,"num_read_kb":14105,"num_write":1178,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'255","reported_seq":889,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.407437+0000","last_change":"2024-04-17T04:36:27.383282+0000","last_active":"2024-04-17T05:45:46.407437+0000","last_peered":"2024-04-17T05:45:46.407437+0000","last_clean":"2024-04-17T05:45:46.407437+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:45:46.407437+0000","last_undegraded":"2024-04-17T05:45:46.407437+0000","last_fullsized":"2024-04-17T05:45:46.407437+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:29.116974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'270","reported_seq":888,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.883434+0000","last_change":"2024-04-17T04:36:25.385368+0000","last_active":"2024-04-17T05:46:03.883434+0000","last_peered":"2024-04-17T05:46:03.883434+0000","last_clean":"2024-04-17T05:46:03.883434+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:46:03.883434+0000","last_undegraded":"2024-04-17T05:46:03.883434+0000","last_fullsized":"2024-04-17T05:46:03.883434+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:35.267010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1730173,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":405,"num_read_kb":10551,"num_write":968,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":723,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142870+0000","last_change":"2024-04-17T04:36:27.376262+0000","last_active":"2024-04-17T05:42:05.142870+0000","last_peered":"2024-04-17T05:42:05.142870+0000","last_clean":"2024-04-17T05:42:05.142870+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:42:05.142870+0000","last_undegraded":"2024-04-17T05:42:05.142870+0000","last_fullsized":"2024-04-17T05:42:05.142870+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:02.524761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'302","reported_seq":984,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.794779+0000","last_change":"2024-04-17T04:36:25.385723+0000","last_active":"2024-04-17T05:46:03.794779+0000","last_peered":"2024-04-17T05:46:03.794779+0000","last_clean":"2024-04-17T05:46:03.794779+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:46:03.794779+0000","last_undegraded":"2024-04-17T05:46:03.794779+0000","last_fullsized":"2024-04-17T05:46:03.794779+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:51.669081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044776,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":456,"num_read_kb":160096,"num_write":1126,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136478+0000","last_change":"2024-04-17T04:36:27.383859+0000","last_active":"2024-04-17T05:41:45.136478+0000","last_peered":"2024-04-17T05:41:45.136478+0000","last_clean":"2024-04-17T05:41:45.136478+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:41:45.136478+0000","last_undegraded":"2024-04-17T05:41:45.136478+0000","last_fullsized":"2024-04-17T05:41:45.136478+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:20:14.617020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'306","reported_seq":1011,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.841878+0000","last_change":"2024-04-17T04:36:25.361757+0000","last_active":"2024-04-17T05:46:03.841878+0000","last_peered":"2024-04-17T05:46:03.841878+0000","last_clean":"2024-04-17T05:46:03.841878+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:46:03.841878+0000","last_undegraded":"2024-04-17T05:46:03.841878+0000","last_fullsized":"2024-04-17T05:46:03.841878+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:21.146448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399248,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":586,"num_read_kb":64730,"num_write":1112,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136549+0000","last_change":"2024-04-17T04:36:27.379593+0000","last_active":"2024-04-17T05:41:45.136549+0000","last_peered":"2024-04-17T05:41:45.136549+0000","last_clean":"2024-04-17T05:41:45.136549+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:41:45.136549+0000","last_undegraded":"2024-04-17T05:41:45.136549+0000","last_fullsized":"2024-04-17T05:41:45.136549+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:37.024479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'318","reported_seq":1088,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.829499+0000","last_change":"2024-04-17T04:36:25.374400+0000","last_active":"2024-04-17T05:46:03.829499+0000","last_peered":"2024-04-17T05:46:03.829499+0000","last_clean":"2024-04-17T05:46:03.829499+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:46:03.829499+0000","last_undegraded":"2024-04-17T05:46:03.829499+0000","last_fullsized":"2024-04-17T05:46:03.829499+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:25:10.957983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170354,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":810,"num_read_kb":57781,"num_write":1265,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.501738+0000","last_change":"2024-04-17T04:36:27.385279+0000","last_active":"2024-04-17T05:41:55.501738+0000","last_peered":"2024-04-17T05:41:55.501738+0000","last_clean":"2024-04-17T05:41:55.501738+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:41:55.501738+0000","last_undegraded":"2024-04-17T05:41:55.501738+0000","last_fullsized":"2024-04-17T05:41:55.501738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:11:17.292970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'346","reported_seq":980,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.862023+0000","last_change":"2024-04-17T04:36:25.355404+0000","last_active":"2024-04-17T05:46:03.862023+0000","last_peered":"2024-04-17T05:46:03.862023+0000","last_clean":"2024-04-17T05:46:03.862023+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:46:03.862023+0000","last_undegraded":"2024-04-17T05:46:03.862023+0000","last_fullsized":"2024-04-17T05:46:03.862023+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:12:47.270461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244513,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":463,"num_read_kb":836,"num_write":1174,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'152","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.671918+0000","last_change":"2024-04-17T04:36:35.418351+0000","last_active":"2024-04-17T05:45:46.671918+0000","last_peered":"2024-04-17T05:45:46.671918+0000","last_clean":"2024-04-17T05:45:46.671918+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:45:46.671918+0000","last_undegraded":"2024-04-17T05:45:46.671918+0000","last_fullsized":"2024-04-17T05:45:46.671918+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:08:02.595419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:51.868791+0000","last_change":"2024-04-17T04:36:27.384993+0000","last_active":"2024-04-17T05:42:51.868791+0000","last_peered":"2024-04-17T05:42:51.868791+0000","last_clean":"2024-04-17T05:42:51.868791+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:42:51.868791+0000","last_undegraded":"2024-04-17T05:42:51.868791+0000","last_fullsized":"2024-04-17T05:42:51.868791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:24.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'314","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.856828+0000","last_change":"2024-04-17T04:36:25.368467+0000","last_active":"2024-04-17T05:46:03.856828+0000","last_peered":"2024-04-17T05:46:03.856828+0000","last_clean":"2024-04-17T05:46:03.856828+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:46:03.856828+0000","last_undegraded":"2024-04-17T05:46:03.856828+0000","last_fullsized":"2024-04-17T05:46:03.856828+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:57:23.161029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7313223,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":429,"num_read_kb":2527,"num_write":1169,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138168+0000","last_change":"2024-04-17T04:36:29.388609+0000","last_active":"2024-04-17T05:41:45.138168+0000","last_peered":"2024-04-17T05:41:45.138168+0000","last_clean":"2024-04-17T05:41:45.138168+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:41:45.138168+0000","last_undegraded":"2024-04-17T05:41:45.138168+0000","last_fullsized":"2024-04-17T05:41:45.138168+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:53.809254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'367","reported_seq":992,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.759108+0000","last_change":"2024-04-17T04:36:31.410990+0000","last_active":"2024-04-17T05:46:03.759108+0000","last_peered":"2024-04-17T05:46:03.759108+0000","last_clean":"2024-04-17T05:46:03.759108+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:46:03.759108+0000","last_undegraded":"2024-04-17T05:46:03.759108+0000","last_fullsized":"2024-04-17T05:46:03.759108+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:24.430923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45299,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":596,"num_read_kb":1277,"num_write":472,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133607+0000","last_change":"2024-04-17T04:36:33.425089+0000","last_active":"2024-04-17T05:41:50.133607+0000","last_peered":"2024-04-17T05:41:50.133607+0000","last_clean":"2024-04-17T05:41:50.133607+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:41:50.133607+0000","last_undegraded":"2024-04-17T05:41:50.133607+0000","last_fullsized":"2024-04-17T05:41:50.133607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:53.730080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'169","reported_seq":722,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.761057+0000","last_change":"2024-04-17T04:36:35.416351+0000","last_active":"2024-04-17T05:45:44.761057+0000","last_peered":"2024-04-17T05:45:44.761057+0000","last_clean":"2024-04-17T05:45:44.761057+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:45:44.761057+0000","last_undegraded":"2024-04-17T05:45:44.761057+0000","last_fullsized":"2024-04-17T05:45:44.761057+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:34.873231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":673,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138074+0000","last_change":"2024-04-17T04:36:27.389710+0000","last_active":"2024-04-17T05:41:45.138074+0000","last_peered":"2024-04-17T05:41:45.138074+0000","last_clean":"2024-04-17T05:41:45.138074+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:41:45.138074+0000","last_undegraded":"2024-04-17T05:41:45.138074+0000","last_fullsized":"2024-04-17T05:41:45.138074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:11:08.849990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'339","reported_seq":1110,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.803797+0000","last_change":"2024-04-17T04:36:25.328153+0000","last_active":"2024-04-17T05:46:03.803797+0000","last_peered":"2024-04-17T05:46:03.803797+0000","last_clean":"2024-04-17T05:46:03.803797+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:46:03.803797+0000","last_undegraded":"2024-04-17T05:46:03.803797+0000","last_fullsized":"2024-04-17T05:46:03.803797+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:07.072313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1328,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":754,"num_read_kb":61435,"num_write":1363,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.170670+0000","last_change":"2024-04-17T04:36:29.373876+0000","last_active":"2024-04-17T05:42:05.170670+0000","last_peered":"2024-04-17T05:42:05.170670+0000","last_clean":"2024-04-17T05:42:05.170670+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:42:05.170670+0000","last_undegraded":"2024-04-17T05:42:05.170670+0000","last_fullsized":"2024-04-17T05:42:05.170670+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:06.355032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'99","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125438+0000","last_change":"2024-04-17T04:36:31.395007+0000","last_active":"2024-04-17T05:41:50.125438+0000","last_peered":"2024-04-17T05:41:50.125438+0000","last_clean":"2024-04-17T05:41:50.125438+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:41:50.125438+0000","last_undegraded":"2024-04-17T05:41:50.125438+0000","last_fullsized":"2024-04-17T05:41:50.125438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:20:47.124749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125417+0000","last_change":"2024-04-17T04:36:33.414084+0000","last_active":"2024-04-17T05:41:50.125417+0000","last_peered":"2024-04-17T05:41:50.125417+0000","last_clean":"2024-04-17T05:41:50.125417+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:41:50.125417+0000","last_undegraded":"2024-04-17T05:41:50.125417+0000","last_fullsized":"2024-04-17T05:41:50.125417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:27.803230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'123","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.222678+0000","last_change":"2024-04-17T04:36:35.427530+0000","last_active":"2024-04-17T05:45:44.222678+0000","last_peered":"2024-04-17T05:45:44.222678+0000","last_clean":"2024-04-17T05:45:44.222678+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:45:44.222678+0000","last_undegraded":"2024-04-17T05:45:44.222678+0000","last_fullsized":"2024-04-17T05:45:44.222678+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:37:45.651840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":659,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:09.959182+0000","last_change":"2024-04-17T04:36:27.376107+0000","last_active":"2024-04-17T05:42:09.959182+0000","last_peered":"2024-04-17T05:42:09.959182+0000","last_clean":"2024-04-17T05:42:09.959182+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:42:09.959182+0000","last_undegraded":"2024-04-17T05:42:09.959182+0000","last_fullsized":"2024-04-17T05:42:09.959182+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:14:13.603194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'333","reported_seq":1112,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.793803+0000","last_change":"2024-04-17T04:36:25.381456+0000","last_active":"2024-04-17T05:46:03.793803+0000","last_peered":"2024-04-17T05:46:03.793803+0000","last_clean":"2024-04-17T05:46:03.793803+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:46:03.793803+0000","last_undegraded":"2024-04-17T05:46:03.793803+0000","last_fullsized":"2024-04-17T05:46:03.793803+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:08.488003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":672,"num_read_kb":75397,"num_write":1317,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165842+0000","last_change":"2024-04-17T04:36:29.359822+0000","last_active":"2024-04-17T05:41:45.165842+0000","last_peered":"2024-04-17T05:41:45.165842+0000","last_clean":"2024-04-17T05:41:45.165842+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:41:45.165842+0000","last_undegraded":"2024-04-17T05:41:45.165842+0000","last_fullsized":"2024-04-17T05:41:45.165842+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:41.948364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'137","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:10.141895+0000","last_change":"2024-04-17T04:36:31.391956+0000","last_active":"2024-04-17T05:42:10.141895+0000","last_peered":"2024-04-17T05:42:10.141895+0000","last_clean":"2024-04-17T05:42:10.141895+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:42:10.141895+0000","last_undegraded":"2024-04-17T05:42:10.141895+0000","last_fullsized":"2024-04-17T05:42:10.141895+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:12.846752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'519","reported_seq":2416,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.476578+0000","last_change":"2024-04-17T04:36:33.420806+0000","last_active":"2024-04-17T05:46:03.476578+0000","last_peered":"2024-04-17T05:46:03.476578+0000","last_clean":"2024-04-17T05:46:03.476578+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:46:03.476578+0000","last_undegraded":"2024-04-17T05:46:03.476578+0000","last_fullsized":"2024-04-17T05:46:03.476578+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:52.301226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'195","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.415632+0000","last_change":"2024-04-17T04:36:35.403573+0000","last_active":"2024-04-17T05:45:46.415632+0000","last_peered":"2024-04-17T05:45:46.415632+0000","last_clean":"2024-04-17T05:45:46.415632+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:45:46.415632+0000","last_undegraded":"2024-04-17T05:45:46.415632+0000","last_fullsized":"2024-04-17T05:45:46.415632+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:09.033539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:07.139896+0000","last_change":"2024-04-17T04:36:27.395613+0000","last_active":"2024-04-17T05:43:07.139896+0000","last_peered":"2024-04-17T05:43:07.139896+0000","last_clean":"2024-04-17T05:43:07.139896+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:43:07.139896+0000","last_undegraded":"2024-04-17T05:43:07.139896+0000","last_fullsized":"2024-04-17T05:43:07.139896+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:32:19.325576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'293","reported_seq":939,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.875938+0000","last_change":"2024-04-17T04:36:25.384743+0000","last_active":"2024-04-17T05:46:03.875938+0000","last_peered":"2024-04-17T05:46:03.875938+0000","last_clean":"2024-04-17T05:46:03.875938+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:46:03.875938+0000","last_undegraded":"2024-04-17T05:46:03.875938+0000","last_fullsized":"2024-04-17T05:46:03.875938+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:05.451652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3722445,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":48921,"num_write":1143,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138271+0000","last_change":"2024-04-17T04:36:29.391920+0000","last_active":"2024-04-17T05:41:45.138271+0000","last_peered":"2024-04-17T05:41:45.138271+0000","last_clean":"2024-04-17T05:41:45.138271+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:41:45.138271+0000","last_undegraded":"2024-04-17T05:41:45.138271+0000","last_fullsized":"2024-04-17T05:41:45.138271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:00:30.032113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'279","reported_seq":864,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.370538+0000","last_change":"2024-04-17T04:36:31.422728+0000","last_active":"2024-04-17T05:46:03.370538+0000","last_peered":"2024-04-17T05:46:03.370538+0000","last_clean":"2024-04-17T05:46:03.370538+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:46:03.370538+0000","last_undegraded":"2024-04-17T05:46:03.370538+0000","last_fullsized":"2024-04-17T05:46:03.370538+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:05.940948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51171,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1108,"num_write":376,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'534","reported_seq":2302,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.208019+0000","last_change":"2024-04-17T04:36:33.422550+0000","last_active":"2024-04-17T05:46:03.208019+0000","last_peered":"2024-04-17T05:46:03.208019+0000","last_clean":"2024-04-17T05:46:03.208019+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:46:03.208019+0000","last_undegraded":"2024-04-17T05:46:03.208019+0000","last_fullsized":"2024-04-17T05:46:03.208019+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:46.514333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'121","reported_seq":720,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163455+0000","last_change":"2024-04-17T04:36:27.398309+0000","last_active":"2024-04-17T05:43:40.163455+0000","last_peered":"2024-04-17T05:43:40.163455+0000","last_clean":"2024-04-17T05:43:40.163455+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:43:40.163455+0000","last_undegraded":"2024-04-17T05:43:40.163455+0000","last_fullsized":"2024-04-17T05:43:40.163455+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:09:23.743688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'281","reported_seq":914,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.826944+0000","last_change":"2024-04-17T04:36:25.370799+0000","last_active":"2024-04-17T05:46:03.826944+0000","last_peered":"2024-04-17T05:46:03.826944+0000","last_clean":"2024-04-17T05:46:03.826944+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:46:03.826944+0000","last_undegraded":"2024-04-17T05:46:03.826944+0000","last_fullsized":"2024-04-17T05:46:03.826944+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:55.146915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6974633,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":437,"num_read_kb":24084,"num_write":1121,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.157843+0000","last_change":"2024-04-17T04:36:29.389738+0000","last_active":"2024-04-17T05:43:40.157843+0000","last_peered":"2024-04-17T05:43:40.157843+0000","last_clean":"2024-04-17T05:43:40.157843+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:43:40.157843+0000","last_undegraded":"2024-04-17T05:43:40.157843+0000","last_fullsized":"2024-04-17T05:43:40.157843+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:52.759093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'382","reported_seq":1019,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.546439+0000","last_change":"2024-04-17T04:36:31.422910+0000","last_active":"2024-04-17T05:46:01.546439+0000","last_peered":"2024-04-17T05:46:01.546439+0000","last_clean":"2024-04-17T05:46:01.546439+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:46:01.546439+0000","last_undegraded":"2024-04-17T05:46:01.546439+0000","last_fullsized":"2024-04-17T05:46:01.546439+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:13:47.096298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44506,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":619,"num_read_kb":1253,"num_write":476,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139387+0000","last_change":"2024-04-17T04:36:33.408109+0000","last_active":"2024-04-17T05:41:50.139387+0000","last_peered":"2024-04-17T05:41:50.139387+0000","last_clean":"2024-04-17T05:41:50.139387+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:41:50.139387+0000","last_undegraded":"2024-04-17T05:41:50.139387+0000","last_fullsized":"2024-04-17T05:41:50.139387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:19.914865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'560","reported_seq":1384,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.656195+0000","last_change":"2024-04-17T04:36:35.409093+0000","last_active":"2024-04-17T05:45:54.656195+0000","last_peered":"2024-04-17T05:45:54.656195+0000","last_clean":"2024-04-17T05:45:54.656195+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:45:54.656195+0000","last_undegraded":"2024-04-17T05:45:54.656195+0000","last_fullsized":"2024-04-17T05:45:54.656195+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":500,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:18:27.394663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":865,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:39.944851+0000","last_change":"2024-04-17T04:36:27.379851+0000","last_active":"2024-04-17T05:43:39.944851+0000","last_peered":"2024-04-17T05:43:39.944851+0000","last_clean":"2024-04-17T05:43:39.944851+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:43:39.944851+0000","last_undegraded":"2024-04-17T05:43:39.944851+0000","last_fullsized":"2024-04-17T05:43:39.944851+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:27.808945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'395","reported_seq":1055,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.877555+0000","last_change":"2024-04-17T04:36:25.371549+0000","last_active":"2024-04-17T05:46:03.877555+0000","last_peered":"2024-04-17T05:46:03.877555+0000","last_clean":"2024-04-17T05:46:03.877555+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:46:03.877555+0000","last_undegraded":"2024-04-17T05:46:03.877555+0000","last_fullsized":"2024-04-17T05:46:03.877555+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:35.252546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396022,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":571,"num_read_kb":14718,"num_write":1316,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162258+0000","last_change":"2024-04-17T04:36:29.371794+0000","last_active":"2024-04-17T05:43:40.162258+0000","last_peered":"2024-04-17T05:43:40.162258+0000","last_clean":"2024-04-17T05:43:40.162258+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:43:40.162258+0000","last_undegraded":"2024-04-17T05:43:40.162258+0000","last_fullsized":"2024-04-17T05:43:40.162258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:11.383775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'269","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.598098+0000","last_change":"2024-04-17T04:36:31.392065+0000","last_active":"2024-04-17T05:46:02.598098+0000","last_peered":"2024-04-17T05:46:02.598098+0000","last_clean":"2024-04-17T05:46:02.598098+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:46:02.598098+0000","last_undegraded":"2024-04-17T05:46:02.598098+0000","last_fullsized":"2024-04-17T05:46:02.598098+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:43:36.246942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42027,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":427,"num_read_kb":943,"num_write":336,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166729+0000","last_change":"2024-04-17T04:36:33.416143+0000","last_active":"2024-04-17T05:41:50.166729+0000","last_peered":"2024-04-17T05:41:50.166729+0000","last_clean":"2024-04-17T05:41:50.166729+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:41:50.166729+0000","last_undegraded":"2024-04-17T05:41:50.166729+0000","last_fullsized":"2024-04-17T05:41:50.166729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:12:51.569882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2371","reported_seq":4173,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.067191+0000","last_change":"2024-04-17T04:36:35.418228+0000","last_active":"2024-04-17T05:45:40.067191+0000","last_peered":"2024-04-17T05:45:40.067191+0000","last_clean":"2024-04-17T05:45:40.067191+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:45:40.067191+0000","last_undegraded":"2024-04-17T05:45:40.067191+0000","last_fullsized":"2024-04-17T05:45:40.067191+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":2300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.125017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160924+0000","last_change":"2024-04-17T04:36:27.376914+0000","last_active":"2024-04-17T05:43:50.160924+0000","last_peered":"2024-04-17T05:43:50.160924+0000","last_clean":"2024-04-17T05:43:50.160924+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:43:50.160924+0000","last_undegraded":"2024-04-17T05:43:50.160924+0000","last_fullsized":"2024-04-17T05:43:50.160924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:06.533166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'356","reported_seq":1030,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.675133+0000","last_change":"2024-04-17T04:36:25.386734+0000","last_active":"2024-04-17T05:46:03.675133+0000","last_peered":"2024-04-17T05:46:03.675133+0000","last_clean":"2024-04-17T05:46:03.675133+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:46:03.675133+0000","last_undegraded":"2024-04-17T05:46:03.675133+0000","last_fullsized":"2024-04-17T05:46:03.675133+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:49.654467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524369,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":604,"num_read_kb":65373,"num_write":1319,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136503+0000","last_change":"2024-04-17T04:36:29.372163+0000","last_active":"2024-04-17T05:41:45.136503+0000","last_peered":"2024-04-17T05:41:45.136503+0000","last_clean":"2024-04-17T05:41:45.136503+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:41:45.136503+0000","last_undegraded":"2024-04-17T05:41:45.136503+0000","last_fullsized":"2024-04-17T05:41:45.136503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:25:29.815813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'335","reported_seq":951,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.755239+0000","last_change":"2024-04-17T04:36:31.380567+0000","last_active":"2024-04-17T05:46:01.755239+0000","last_peered":"2024-04-17T05:46:01.755239+0000","last_clean":"2024-04-17T05:46:01.755239+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:46:01.755239+0000","last_undegraded":"2024-04-17T05:46:01.755239+0000","last_fullsized":"2024-04-17T05:46:01.755239+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:55.178709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48038,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":543,"num_read_kb":1167,"num_write":428,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'539","reported_seq":4615,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.390570+0000","last_change":"2024-04-17T04:36:33.411578+0000","last_active":"2024-04-17T05:46:03.390570+0000","last_peered":"2024-04-17T05:46:03.390570+0000","last_clean":"2024-04-17T05:46:03.390570+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:46:03.390570+0000","last_undegraded":"2024-04-17T05:46:03.390570+0000","last_fullsized":"2024-04-17T05:46:03.390570+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:50.576073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161111+0000","last_change":"2024-04-17T04:36:35.427581+0000","last_active":"2024-04-17T05:43:45.161111+0000","last_peered":"2024-04-17T05:43:45.161111+0000","last_clean":"2024-04-17T05:43:45.161111+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:43:45.161111+0000","last_undegraded":"2024-04-17T05:43:45.161111+0000","last_fullsized":"2024-04-17T05:43:45.161111+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:13.700005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'95","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:57.081384+0000","last_change":"2024-04-17T04:36:27.370100+0000","last_active":"2024-04-17T05:42:57.081384+0000","last_peered":"2024-04-17T05:42:57.081384+0000","last_clean":"2024-04-17T05:42:57.081384+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:42:57.081384+0000","last_undegraded":"2024-04-17T05:42:57.081384+0000","last_fullsized":"2024-04-17T05:42:57.081384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:31.666063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'339","reported_seq":1038,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.851278+0000","last_change":"2024-04-17T04:37:03.648700+0000","last_active":"2024-04-17T05:46:03.851278+0000","last_peered":"2024-04-17T05:46:03.851278+0000","last_clean":"2024-04-17T05:46:03.851278+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:46:03.851278+0000","last_undegraded":"2024-04-17T05:46:03.851278+0000","last_fullsized":"2024-04-17T05:46:03.851278+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:49:08.594901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":662,"num_read_kb":24375,"num_write":1169,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:55.165608+0000","last_change":"2024-04-17T04:36:29.375596+0000","last_active":"2024-04-17T05:43:55.165608+0000","last_peered":"2024-04-17T05:43:55.165608+0000","last_clean":"2024-04-17T05:43:55.165608+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:43:55.165608+0000","last_undegraded":"2024-04-17T05:43:55.165608+0000","last_fullsized":"2024-04-17T05:43:55.165608+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:20.886949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'472","reported_seq":1129,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.824673+0000","last_change":"2024-04-17T04:36:31.409877+0000","last_active":"2024-04-17T05:46:02.824673+0000","last_peered":"2024-04-17T05:46:02.824673+0000","last_clean":"2024-04-17T05:46:02.824673+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:46:02.824673+0000","last_undegraded":"2024-04-17T05:46:02.824673+0000","last_fullsized":"2024-04-17T05:46:02.824673+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:19.065697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87769,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":831,"num_read_kb":1997,"num_write":666,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.163452+0000","last_change":"2024-04-17T04:36:33.420378+0000","last_active":"2024-04-17T05:43:45.163452+0000","last_peered":"2024-04-17T05:43:45.163452+0000","last_clean":"2024-04-17T05:43:45.163452+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:43:45.163452+0000","last_undegraded":"2024-04-17T05:43:45.163452+0000","last_fullsized":"2024-04-17T05:43:45.163452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:59.519269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'148","reported_seq":689,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.165985+0000","last_change":"2024-04-17T04:36:35.417260+0000","last_active":"2024-04-17T05:43:45.165985+0000","last_peered":"2024-04-17T05:43:45.165985+0000","last_clean":"2024-04-17T05:43:45.165985+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:43:45.165985+0000","last_undegraded":"2024-04-17T05:43:45.165985+0000","last_fullsized":"2024-04-17T05:43:45.165985+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:28:41.967458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":770,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124135+0000","last_change":"2024-04-17T04:36:27.393949+0000","last_active":"2024-04-17T05:41:45.124135+0000","last_peered":"2024-04-17T05:41:45.124135+0000","last_clean":"2024-04-17T05:41:45.124135+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:41:45.124135+0000","last_undegraded":"2024-04-17T05:41:45.124135+0000","last_fullsized":"2024-04-17T05:41:45.124135+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:12.169777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'350","reported_seq":1018,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.863691+0000","last_change":"2024-04-17T04:36:25.379472+0000","last_active":"2024-04-17T05:46:03.863691+0000","last_peered":"2024-04-17T05:46:03.863691+0000","last_clean":"2024-04-17T05:46:03.863691+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:46:03.863691+0000","last_undegraded":"2024-04-17T05:46:03.863691+0000","last_fullsized":"2024-04-17T05:46:03.863691+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:42:53.335830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1985843,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":564,"num_read_kb":9569,"num_write":1227,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.136613+0000","last_change":"2024-04-17T04:36:29.374168+0000","last_active":"2024-04-17T05:41:45.136613+0000","last_peered":"2024-04-17T05:41:45.136613+0000","last_clean":"2024-04-17T05:41:45.136613+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:41:45.136613+0000","last_undegraded":"2024-04-17T05:41:45.136613+0000","last_fullsized":"2024-04-17T05:41:45.136613+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:22.063423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'313","reported_seq":911,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.906206+0000","last_change":"2024-04-17T04:36:31.392925+0000","last_active":"2024-04-17T05:46:02.906206+0000","last_peered":"2024-04-17T05:46:02.906206+0000","last_clean":"2024-04-17T05:46:02.906206+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:46:02.906206+0000","last_undegraded":"2024-04-17T05:46:02.906206+0000","last_fullsized":"2024-04-17T05:46:02.906206+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:08.285743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":535,"num_read_kb":1188,"num_write":472,"num_write_kb":375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":494,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.137350+0000","last_change":"2024-04-17T04:36:33.430909+0000","last_active":"2024-04-17T05:42:00.137350+0000","last_peered":"2024-04-17T05:42:00.137350+0000","last_clean":"2024-04-17T05:42:00.137350+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:42:00.137350+0000","last_undegraded":"2024-04-17T05:42:00.137350+0000","last_fullsized":"2024-04-17T05:42:00.137350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:54.144240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'129","reported_seq":636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.818089+0000","last_change":"2024-04-17T04:36:35.423084+0000","last_active":"2024-04-17T05:45:40.818089+0000","last_peered":"2024-04-17T05:45:40.818089+0000","last_clean":"2024-04-17T05:45:40.818089+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:45:40.818089+0000","last_undegraded":"2024-04-17T05:45:40.818089+0000","last_fullsized":"2024-04-17T05:45:40.818089+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:20:19.232177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:53.923395+0000","last_change":"2024-04-17T04:36:27.383527+0000","last_active":"2024-04-17T05:41:53.923395+0000","last_peered":"2024-04-17T05:41:53.923395+0000","last_clean":"2024-04-17T05:41:53.923395+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:41:53.923395+0000","last_undegraded":"2024-04-17T05:41:53.923395+0000","last_fullsized":"2024-04-17T05:41:53.923395+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:53:29.969137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'365","reported_seq":1031,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.777332+0000","last_change":"2024-04-17T04:36:25.371674+0000","last_active":"2024-04-17T05:46:03.777332+0000","last_peered":"2024-04-17T05:46:03.777332+0000","last_clean":"2024-04-17T05:46:03.777332+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:46:03.777332+0000","last_undegraded":"2024-04-17T05:46:03.777332+0000","last_fullsized":"2024-04-17T05:46:03.777332+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:54.700946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524382,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":609,"num_read_kb":15072,"num_write":1258,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138308+0000","last_change":"2024-04-17T04:36:29.359143+0000","last_active":"2024-04-17T05:41:45.138308+0000","last_peered":"2024-04-17T05:41:45.138308+0000","last_clean":"2024-04-17T05:41:45.138308+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:41:45.138308+0000","last_undegraded":"2024-04-17T05:41:45.138308+0000","last_fullsized":"2024-04-17T05:41:45.138308+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:37.440920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'292","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.378885+0000","last_change":"2024-04-17T04:36:31.397866+0000","last_active":"2024-04-17T05:46:02.378885+0000","last_peered":"2024-04-17T05:46:02.378885+0000","last_clean":"2024-04-17T05:46:02.378885+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:46:02.378885+0000","last_undegraded":"2024-04-17T05:46:02.378885+0000","last_fullsized":"2024-04-17T05:46:02.378885+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:42.568179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47207,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":490,"num_read_kb":1044,"num_write":380,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'512","reported_seq":2443,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.036766+0000","last_change":"2024-04-17T04:36:33.408560+0000","last_active":"2024-04-17T05:46:03.036766+0000","last_peered":"2024-04-17T05:46:03.036766+0000","last_clean":"2024-04-17T05:46:03.036766+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:46:03.036766+0000","last_undegraded":"2024-04-17T05:46:03.036766+0000","last_fullsized":"2024-04-17T05:46:03.036766+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:41.906428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'96","reported_seq":602,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:42.569922+0000","last_change":"2024-04-17T04:36:35.428863+0000","last_active":"2024-04-17T05:45:42.569922+0000","last_peered":"2024-04-17T05:45:42.569922+0000","last_clean":"2024-04-17T05:45:42.569922+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:45:42.569922+0000","last_undegraded":"2024-04-17T05:45:42.569922+0000","last_fullsized":"2024-04-17T05:45:42.569922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:09.522109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.169748+0000","last_change":"2024-04-17T04:36:27.393058+0000","last_active":"2024-04-17T05:42:30.169748+0000","last_peered":"2024-04-17T05:42:30.169748+0000","last_clean":"2024-04-17T05:42:30.169748+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:42:30.169748+0000","last_undegraded":"2024-04-17T05:42:30.169748+0000","last_fullsized":"2024-04-17T05:42:30.169748+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:22.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'295","reported_seq":985,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.889290+0000","last_change":"2024-04-17T04:36:25.356100+0000","last_active":"2024-04-17T05:46:03.889290+0000","last_peered":"2024-04-17T05:46:03.889290+0000","last_clean":"2024-04-17T05:46:03.889290+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:46:03.889290+0000","last_undegraded":"2024-04-17T05:46:03.889290+0000","last_fullsized":"2024-04-17T05:46:03.889290+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:10.485745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243143,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":575,"num_read_kb":5671,"num_write":1192,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"28'2","reported_seq":1154,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.183219+0000","last_change":"2024-04-17T04:36:29.389132+0000","last_active":"2024-04-17T05:46:01.183219+0000","last_peered":"2024-04-17T05:46:01.183219+0000","last_clean":"2024-04-17T05:46:01.183219+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:46:01.183219+0000","last_undegraded":"2024-04-17T05:46:01.183219+0000","last_fullsized":"2024-04-17T05:46:01.183219+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:09:43.654615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":652,"num_read_kb":652,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'97","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166974+0000","last_change":"2024-04-17T04:36:31.421130+0000","last_active":"2024-04-17T05:41:50.166974+0000","last_peered":"2024-04-17T05:41:50.166974+0000","last_clean":"2024-04-17T05:41:50.166974+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:41:50.166974+0000","last_undegraded":"2024-04-17T05:41:50.166974+0000","last_fullsized":"2024-04-17T05:41:50.166974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:36:38.963408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'539","reported_seq":2247,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.046568+0000","last_change":"2024-04-17T04:36:33.421354+0000","last_active":"2024-04-17T05:46:02.046568+0000","last_peered":"2024-04-17T05:46:02.046568+0000","last_clean":"2024-04-17T05:46:02.046568+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:46:02.046568+0000","last_undegraded":"2024-04-17T05:46:02.046568+0000","last_fullsized":"2024-04-17T05:46:02.046568+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:33:27.382198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'97","reported_seq":600,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939559+0000","last_change":"2024-04-17T04:36:35.427123+0000","last_active":"2024-04-17T05:43:14.939559+0000","last_peered":"2024-04-17T05:43:14.939559+0000","last_clean":"2024-04-17T05:43:14.939559+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:43:14.939559+0000","last_undegraded":"2024-04-17T05:43:14.939559+0000","last_fullsized":"2024-04-17T05:43:14.939559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:21:26.157064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":756,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.141817+0000","last_change":"2024-04-17T04:36:27.377947+0000","last_active":"2024-04-17T05:42:00.141817+0000","last_peered":"2024-04-17T05:42:00.141817+0000","last_clean":"2024-04-17T05:42:00.141817+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:42:00.141817+0000","last_undegraded":"2024-04-17T05:42:00.141817+0000","last_fullsized":"2024-04-17T05:42:00.141817+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:47.608940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'296","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.852632+0000","last_change":"2024-04-17T04:36:25.353720+0000","last_active":"2024-04-17T05:46:03.852632+0000","last_peered":"2024-04-17T05:46:03.852632+0000","last_clean":"2024-04-17T05:46:03.852632+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:46:03.852632+0000","last_undegraded":"2024-04-17T05:46:03.852632+0000","last_fullsized":"2024-04-17T05:46:03.852632+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:43:24.629387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170337,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":410,"num_read_kb":36760,"num_write":993,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124238+0000","last_change":"2024-04-17T04:36:29.389735+0000","last_active":"2024-04-17T05:41:45.124238+0000","last_peered":"2024-04-17T05:41:45.124238+0000","last_clean":"2024-04-17T05:41:45.124238+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:41:45.124238+0000","last_undegraded":"2024-04-17T05:41:45.124238+0000","last_fullsized":"2024-04-17T05:41:45.124238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:55:19.695221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'352","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.835798+0000","last_change":"2024-04-17T04:36:31.398102+0000","last_active":"2024-04-17T05:46:03.835798+0000","last_peered":"2024-04-17T05:46:03.835798+0000","last_clean":"2024-04-17T05:46:03.835798+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:46:03.835798+0000","last_undegraded":"2024-04-17T05:46:03.835798+0000","last_fullsized":"2024-04-17T05:46:03.835798+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:41.468361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45806,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1167,"num_write":448,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134628+0000","last_change":"2024-04-17T04:36:33.419643+0000","last_active":"2024-04-17T05:41:50.134628+0000","last_peered":"2024-04-17T05:41:50.134628+0000","last_clean":"2024-04-17T05:41:50.134628+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:41:50.134628+0000","last_undegraded":"2024-04-17T05:41:50.134628+0000","last_fullsized":"2024-04-17T05:41:50.134628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:51.100488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'157","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.190550+0000","last_change":"2024-04-17T04:36:35.408247+0000","last_active":"2024-04-17T05:45:20.190550+0000","last_peered":"2024-04-17T05:45:20.190550+0000","last_clean":"2024-04-17T05:45:20.190550+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:45:20.190550+0000","last_undegraded":"2024-04-17T05:45:20.190550+0000","last_fullsized":"2024-04-17T05:45:20.190550+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:00:26.932136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":695,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.154250+0000","last_change":"2024-04-17T04:36:27.387349+0000","last_active":"2024-04-17T05:43:20.154250+0000","last_peered":"2024-04-17T05:43:20.154250+0000","last_clean":"2024-04-17T05:43:20.154250+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:43:20.154250+0000","last_undegraded":"2024-04-17T05:43:20.154250+0000","last_fullsized":"2024-04-17T05:43:20.154250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:39.509316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":680,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.141161+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T05:42:25.141161+0000","last_peered":"2024-04-17T05:42:25.141161+0000","last_clean":"2024-04-17T05:42:25.141161+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:42:25.141161+0000","last_undegraded":"2024-04-17T05:42:25.141161+0000","last_fullsized":"2024-04-17T05:42:25.141161+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:24:00.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154818+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T05:43:00.154818+0000","last_peered":"2024-04-17T05:43:00.154818+0000","last_clean":"2024-04-17T05:43:00.154818+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:43:00.154818+0000","last_undegraded":"2024-04-17T05:43:00.154818+0000","last_fullsized":"2024-04-17T05:43:00.154818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:13.926220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'327","reported_seq":978,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.750604+0000","last_change":"2024-04-17T04:36:25.364067+0000","last_active":"2024-04-17T05:46:03.750604+0000","last_peered":"2024-04-17T05:46:03.750604+0000","last_clean":"2024-04-17T05:46:03.750604+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:46:03.750604+0000","last_undegraded":"2024-04-17T05:46:03.750604+0000","last_fullsized":"2024-04-17T05:46:03.750604+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:41.272537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344976,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":2736,"num_write":1215,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151134+0000","last_change":"2024-04-17T04:36:29.381453+0000","last_active":"2024-04-17T05:43:10.151134+0000","last_peered":"2024-04-17T05:43:10.151134+0000","last_clean":"2024-04-17T05:43:10.151134+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:43:10.151134+0000","last_undegraded":"2024-04-17T05:43:10.151134+0000","last_fullsized":"2024-04-17T05:43:10.151134+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:32.455502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'724","reported_seq":1445,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.622724+0000","last_change":"2024-04-17T04:36:31.400231+0000","last_active":"2024-04-17T05:46:03.622724+0000","last_peered":"2024-04-17T05:46:03.622724+0000","last_clean":"2024-04-17T05:46:03.622724+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:46:03.622724+0000","last_undegraded":"2024-04-17T05:46:03.622724+0000","last_fullsized":"2024-04-17T05:46:03.622724+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":700,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:04:02.459768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1411,"num_read_kb":3198,"num_write":1348,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.161079+0000","last_change":"2024-04-17T04:36:33.430394+0000","last_active":"2024-04-17T05:43:45.161079+0000","last_peered":"2024-04-17T05:43:45.161079+0000","last_clean":"2024-04-17T05:43:45.161079+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:43:45.161079+0000","last_undegraded":"2024-04-17T05:43:45.161079+0000","last_fullsized":"2024-04-17T05:43:45.161079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:45.115863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'220","reported_seq":913,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.519743+0000","last_change":"2024-04-17T04:36:35.421183+0000","last_active":"2024-04-17T05:45:46.519743+0000","last_peered":"2024-04-17T05:45:46.519743+0000","last_clean":"2024-04-17T05:45:46.519743+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:45:46.519743+0000","last_undegraded":"2024-04-17T05:45:46.519743+0000","last_fullsized":"2024-04-17T05:45:46.519743+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:30.565325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":668,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.159648+0000","last_change":"2024-04-17T04:36:27.365454+0000","last_active":"2024-04-17T05:43:40.159648+0000","last_peered":"2024-04-17T05:43:40.159648+0000","last_clean":"2024-04-17T05:43:40.159648+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:43:40.159648+0000","last_undegraded":"2024-04-17T05:43:40.159648+0000","last_fullsized":"2024-04-17T05:43:40.159648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:16.200473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":524,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:30.142294+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T05:42:30.142294+0000","last_peered":"2024-04-17T05:42:30.142294+0000","last_clean":"2024-04-17T05:42:30.142294+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:42:30.142294+0000","last_undegraded":"2024-04-17T05:42:30.142294+0000","last_fullsized":"2024-04-17T05:42:30.142294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:15:06.659093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'343","reported_seq":995,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.846941+0000","last_change":"2024-04-17T04:36:25.355561+0000","last_active":"2024-04-17T05:46:03.846941+0000","last_peered":"2024-04-17T05:46:03.846941+0000","last_clean":"2024-04-17T05:46:03.846941+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:46:03.846941+0000","last_undegraded":"2024-04-17T05:46:03.846941+0000","last_fullsized":"2024-04-17T05:46:03.846941+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:21.131078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2309902,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":570,"num_read_kb":3714,"num_write":1237,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163722+0000","last_change":"2024-04-17T04:36:29.393231+0000","last_active":"2024-04-17T05:43:40.163722+0000","last_peered":"2024-04-17T05:43:40.163722+0000","last_clean":"2024-04-17T05:43:40.163722+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:43:40.163722+0000","last_undegraded":"2024-04-17T05:43:40.163722+0000","last_fullsized":"2024-04-17T05:43:40.163722+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:58:30.045764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1101","reported_seq":2819,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.894512+0000","last_change":"2024-04-17T04:36:31.398322+0000","last_active":"2024-04-17T05:46:03.894512+0000","last_peered":"2024-04-17T05:46:03.894512+0000","last_clean":"2024-04-17T05:46:03.894512+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:46:03.894512+0000","last_undegraded":"2024-04-17T05:46:03.894512+0000","last_fullsized":"2024-04-17T05:46:03.894512+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:02:09.046142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1819,"num_read_kb":3719,"num_write":1122,"num_write_kb":937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.154560+0000","last_change":"2024-04-17T04:36:33.422428+0000","last_active":"2024-04-17T05:43:25.154560+0000","last_peered":"2024-04-17T05:43:25.154560+0000","last_clean":"2024-04-17T05:43:25.154560+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:43:25.154560+0000","last_undegraded":"2024-04-17T05:43:25.154560+0000","last_fullsized":"2024-04-17T05:43:25.154560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:00.479531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'112","reported_seq":621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.389082+0000","last_change":"2024-04-17T04:36:35.402669+0000","last_active":"2024-04-17T05:45:46.389082+0000","last_peered":"2024-04-17T05:45:46.389082+0000","last_clean":"2024-04-17T05:45:46.389082+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:45:46.389082+0000","last_undegraded":"2024-04-17T05:45:46.389082+0000","last_fullsized":"2024-04-17T05:45:46.389082+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:03:05.019981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1017,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164737+0000","last_change":"2024-04-17T04:36:27.376154+0000","last_active":"2024-04-17T05:43:40.164737+0000","last_peered":"2024-04-17T05:43:40.164737+0000","last_clean":"2024-04-17T05:43:40.164737+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:43:40.164737+0000","last_undegraded":"2024-04-17T05:43:40.164737+0000","last_fullsized":"2024-04-17T05:43:40.164737+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:53.161990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":530,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:55.151409+0000","last_change":"2024-04-17T04:36:17.752374+0000","last_active":"2024-04-17T05:42:55.151409+0000","last_peered":"2024-04-17T05:42:55.151409+0000","last_clean":"2024-04-17T05:42:55.151409+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:42:55.151409+0000","last_undegraded":"2024-04-17T05:42:55.151409+0000","last_fullsized":"2024-04-17T05:42:55.151409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:41.044005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'298","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.845019+0000","last_change":"2024-04-17T04:36:25.343663+0000","last_active":"2024-04-17T05:46:03.845019+0000","last_peered":"2024-04-17T05:46:03.845019+0000","last_clean":"2024-04-17T05:46:03.845019+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:46:03.845019+0000","last_undegraded":"2024-04-17T05:46:03.845019+0000","last_fullsized":"2024-04-17T05:46:03.845019+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:49.114178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5679739,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":479,"num_read_kb":21975,"num_write":1114,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"28'1","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.162452+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:43:40.162452+0000","last_peered":"2024-04-17T05:43:40.162452+0000","last_clean":"2024-04-17T05:43:40.162452+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:43:40.162452+0000","last_undegraded":"2024-04-17T05:43:40.162452+0000","last_fullsized":"2024-04-17T05:43:40.162452+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:01.144890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'234","reported_seq":1618,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:52.541681+0000","last_change":"2024-04-17T04:36:31.427048+0000","last_active":"2024-04-17T05:45:52.541681+0000","last_peered":"2024-04-17T05:45:52.541681+0000","last_clean":"2024-04-17T05:45:52.541681+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:45:52.541681+0000","last_undegraded":"2024-04-17T05:45:52.541681+0000","last_fullsized":"2024-04-17T05:45:52.541681+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:49:05.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.153957+0000","last_change":"2024-04-17T04:36:33.426587+0000","last_active":"2024-04-17T05:43:15.153957+0000","last_peered":"2024-04-17T05:43:15.153957+0000","last_clean":"2024-04-17T05:43:15.153957+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:43:15.153957+0000","last_undegraded":"2024-04-17T05:43:15.153957+0000","last_fullsized":"2024-04-17T05:43:15.153957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:52:34.456097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'262","reported_seq":866,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.184123+0000","last_change":"2024-04-17T04:36:35.408138+0000","last_active":"2024-04-17T05:45:45.184123+0000","last_peered":"2024-04-17T05:45:45.184123+0000","last_clean":"2024-04-17T05:45:45.184123+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:45:45.184123+0000","last_undegraded":"2024-04-17T05:45:45.184123+0000","last_fullsized":"2024-04-17T05:45:45.184123+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:09.321611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158227+0000","last_change":"2024-04-17T04:36:27.361706+0000","last_active":"2024-04-17T05:43:10.158227+0000","last_peered":"2024-04-17T05:43:10.158227+0000","last_clean":"2024-04-17T05:43:10.158227+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:43:10.158227+0000","last_undegraded":"2024-04-17T05:43:10.158227+0000","last_fullsized":"2024-04-17T05:43:10.158227+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:06:25.186757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":521,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.154761+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T05:43:00.154761+0000","last_peered":"2024-04-17T05:43:00.154761+0000","last_clean":"2024-04-17T05:43:00.154761+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:43:00.154761+0000","last_undegraded":"2024-04-17T05:43:00.154761+0000","last_fullsized":"2024-04-17T05:43:00.154761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:44:32.358200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'341","reported_seq":1098,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.811913+0000","last_change":"2024-04-17T04:36:25.356426+0000","last_active":"2024-04-17T05:46:03.811913+0000","last_peered":"2024-04-17T05:46:03.811913+0000","last_clean":"2024-04-17T05:46:03.811913+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:46:03.811913+0000","last_undegraded":"2024-04-17T05:46:03.811913+0000","last_fullsized":"2024-04-17T05:46:03.811913+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:48.430125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048803,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":626,"num_read_kb":36553,"num_write":1237,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:20.155027+0000","last_change":"2024-04-17T04:36:29.371781+0000","last_active":"2024-04-17T05:43:20.155027+0000","last_peered":"2024-04-17T05:43:20.155027+0000","last_clean":"2024-04-17T05:43:20.155027+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:43:20.155027+0000","last_undegraded":"2024-04-17T05:43:20.155027+0000","last_fullsized":"2024-04-17T05:43:20.155027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:40:17.404012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'139","reported_seq":699,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.193023+0000","last_change":"2024-04-17T04:36:31.400012+0000","last_active":"2024-04-17T05:43:45.193023+0000","last_peered":"2024-04-17T05:43:45.193023+0000","last_clean":"2024-04-17T05:43:45.193023+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:43:45.193023+0000","last_undegraded":"2024-04-17T05:43:45.193023+0000","last_fullsized":"2024-04-17T05:43:45.193023+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:30:23.255610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.159837+0000","last_change":"2024-04-17T04:36:33.416438+0000","last_active":"2024-04-17T05:43:45.159837+0000","last_peered":"2024-04-17T05:43:45.159837+0000","last_clean":"2024-04-17T05:43:45.159837+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:43:45.159837+0000","last_undegraded":"2024-04-17T05:43:45.159837+0000","last_fullsized":"2024-04-17T05:43:45.159837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:15:47.410813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'130","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:45.164559+0000","last_change":"2024-04-17T04:36:35.411210+0000","last_active":"2024-04-17T05:43:45.164559+0000","last_peered":"2024-04-17T05:43:45.164559+0000","last_clean":"2024-04-17T05:43:45.164559+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:43:45.164559+0000","last_undegraded":"2024-04-17T05:43:45.164559+0000","last_fullsized":"2024-04-17T05:43:45.164559+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:26.503048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.158134+0000","last_change":"2024-04-17T04:36:27.376368+0000","last_active":"2024-04-17T05:43:10.158134+0000","last_peered":"2024-04-17T05:43:10.158134+0000","last_clean":"2024-04-17T05:43:10.158134+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:43:10.158134+0000","last_undegraded":"2024-04-17T05:43:10.158134+0000","last_fullsized":"2024-04-17T05:43:10.158134+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:23:39.170608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'300","reported_seq":962,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.856288+0000","last_change":"2024-04-17T04:36:25.368624+0000","last_active":"2024-04-17T05:46:03.856288+0000","last_peered":"2024-04-17T05:46:03.856288+0000","last_clean":"2024-04-17T05:46:03.856288+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:46:03.856288+0000","last_undegraded":"2024-04-17T05:46:03.856288+0000","last_fullsized":"2024-04-17T05:46:03.856288+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:51.630737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":475,"num_read_kb":1448,"num_write":1131,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:00.156128+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T05:43:00.156128+0000","last_peered":"2024-04-17T05:43:00.156128+0000","last_clean":"2024-04-17T05:43:00.156128+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:43:00.156128+0000","last_undegraded":"2024-04-17T05:43:00.156128+0000","last_fullsized":"2024-04-17T05:43:00.156128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:16.202650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152928+0000","last_change":"2024-04-17T04:36:29.384820+0000","last_active":"2024-04-17T05:43:10.152928+0000","last_peered":"2024-04-17T05:43:10.152928+0000","last_clean":"2024-04-17T05:43:10.152928+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:43:10.152928+0000","last_undegraded":"2024-04-17T05:43:10.152928+0000","last_fullsized":"2024-04-17T05:43:10.152928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:27.497471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'620","reported_seq":1316,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.446914+0000","last_change":"2024-04-17T04:36:31.411434+0000","last_active":"2024-04-17T05:46:03.446914+0000","last_peered":"2024-04-17T05:46:03.446914+0000","last_clean":"2024-04-17T05:46:03.446914+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:46:03.446914+0000","last_undegraded":"2024-04-17T05:46:03.446914+0000","last_fullsized":"2024-04-17T05:46:03.446914+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":600,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:30.011121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86860,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1659,"num_read_kb":2834,"num_write":2174,"num_write_kb":1973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939512+0000","last_change":"2024-04-17T04:36:33.407911+0000","last_active":"2024-04-17T05:43:14.939512+0000","last_peered":"2024-04-17T05:43:14.939512+0000","last_clean":"2024-04-17T05:43:14.939512+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:43:14.939512+0000","last_undegraded":"2024-04-17T05:43:14.939512+0000","last_fullsized":"2024-04-17T05:43:14.939512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:49:35.539455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'82","reported_seq":584,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.166476+0000","last_change":"2024-04-17T04:36:35.429331+0000","last_active":"2024-04-17T05:44:05.166476+0000","last_peered":"2024-04-17T05:44:05.166476+0000","last_clean":"2024-04-17T05:44:05.166476+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:44:05.166476+0000","last_undegraded":"2024-04-17T05:44:05.166476+0000","last_fullsized":"2024-04-17T05:44:05.166476+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:03.344843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":592,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.152888+0000","last_change":"2024-04-17T04:36:27.376396+0000","last_active":"2024-04-17T05:43:10.152888+0000","last_peered":"2024-04-17T05:43:10.152888+0000","last_clean":"2024-04-17T05:43:10.152888+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:43:10.152888+0000","last_undegraded":"2024-04-17T05:43:10.152888+0000","last_fullsized":"2024-04-17T05:43:10.152888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:36.959008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'269","reported_seq":890,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.858133+0000","last_change":"2024-04-17T04:36:25.345048+0000","last_active":"2024-04-17T05:46:03.858133+0000","last_peered":"2024-04-17T05:46:03.858133+0000","last_clean":"2024-04-17T05:46:03.858133+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:46:03.858133+0000","last_undegraded":"2024-04-17T05:46:03.858133+0000","last_fullsized":"2024-04-17T05:46:03.858133+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:40:34.714988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194313,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":396,"num_read_kb":35213,"num_write":840,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":523,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151170+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T05:43:10.151170+0000","last_peered":"2024-04-17T05:43:10.151170+0000","last_clean":"2024-04-17T05:43:10.151170+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:43:10.151170+0000","last_undegraded":"2024-04-17T05:43:10.151170+0000","last_fullsized":"2024-04-17T05:43:10.151170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:33.083623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151795+0000","last_change":"2024-04-17T04:36:29.375400+0000","last_active":"2024-04-17T05:43:10.151795+0000","last_peered":"2024-04-17T05:43:10.151795+0000","last_clean":"2024-04-17T05:43:10.151795+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:43:10.151795+0000","last_undegraded":"2024-04-17T05:43:10.151795+0000","last_fullsized":"2024-04-17T05:43:10.151795+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:16.669890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'199","reported_seq":792,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.159277+0000","last_change":"2024-04-17T04:36:31.420876+0000","last_active":"2024-04-17T05:43:15.159277+0000","last_peered":"2024-04-17T05:43:15.159277+0000","last_clean":"2024-04-17T05:43:15.159277+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:43:15.159277+0000","last_undegraded":"2024-04-17T05:43:15.159277+0000","last_fullsized":"2024-04-17T05:43:15.159277+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:23:18.629160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'540","reported_seq":4358,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.564768+0000","last_change":"2024-04-17T04:36:33.407900+0000","last_active":"2024-04-17T05:46:02.564768+0000","last_peered":"2024-04-17T05:46:02.564768+0000","last_clean":"2024-04-17T05:46:02.564768+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:46:02.564768+0000","last_undegraded":"2024-04-17T05:46:02.564768+0000","last_fullsized":"2024-04-17T05:46:02.564768+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":500,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:36:50.014322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'156","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.202363+0000","last_change":"2024-04-17T04:36:35.418551+0000","last_active":"2024-04-17T05:44:20.202363+0000","last_peered":"2024-04-17T05:44:20.202363+0000","last_clean":"2024-04-17T05:44:20.202363+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:44:20.202363+0000","last_undegraded":"2024-04-17T05:44:20.202363+0000","last_fullsized":"2024-04-17T05:44:20.202363+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:08.177362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":799,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180216+0000","last_change":"2024-04-17T04:36:27.395791+0000","last_active":"2024-04-17T05:42:45.180216+0000","last_peered":"2024-04-17T05:42:45.180216+0000","last_clean":"2024-04-17T05:42:45.180216+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:42:45.180216+0000","last_undegraded":"2024-04-17T05:42:45.180216+0000","last_fullsized":"2024-04-17T05:42:45.180216+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:29:33.119512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'379","reported_seq":1036,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.823188+0000","last_change":"2024-04-17T04:36:25.347483+0000","last_active":"2024-04-17T05:46:03.823188+0000","last_peered":"2024-04-17T05:46:03.823188+0000","last_clean":"2024-04-17T05:46:03.823188+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:46:03.823188+0000","last_undegraded":"2024-04-17T05:46:03.823188+0000","last_fullsized":"2024-04-17T05:46:03.823188+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:52:07.029677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262239,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":591,"num_read_kb":5844,"num_write":1385,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":527,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.161303+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T05:43:25.161303+0000","last_peered":"2024-04-17T05:43:25.161303+0000","last_clean":"2024-04-17T05:43:25.161303+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:43:25.161303+0000","last_undegraded":"2024-04-17T05:43:25.161303+0000","last_fullsized":"2024-04-17T05:43:25.161303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:06:38.757105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938398+0000","last_change":"2024-04-17T04:36:29.375601+0000","last_active":"2024-04-17T05:43:09.938398+0000","last_peered":"2024-04-17T05:43:09.938398+0000","last_clean":"2024-04-17T05:43:09.938398+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:43:09.938398+0000","last_undegraded":"2024-04-17T05:43:09.938398+0000","last_fullsized":"2024-04-17T05:43:09.938398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:27:57.723009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'136","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:14.939607+0000","last_change":"2024-04-17T04:36:31.400243+0000","last_active":"2024-04-17T05:43:14.939607+0000","last_peered":"2024-04-17T05:43:14.939607+0000","last_clean":"2024-04-17T05:43:14.939607+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:43:14.939607+0000","last_undegraded":"2024-04-17T05:43:14.939607+0000","last_fullsized":"2024-04-17T05:43:14.939607+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:41.986999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'533","reported_seq":2304,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.208005+0000","last_change":"2024-04-17T04:36:33.421012+0000","last_active":"2024-04-17T05:46:03.208005+0000","last_peered":"2024-04-17T05:46:03.208005+0000","last_clean":"2024-04-17T05:46:03.208005+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:46:03.208005+0000","last_undegraded":"2024-04-17T05:46:03.208005+0000","last_fullsized":"2024-04-17T05:46:03.208005+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":500,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:51:52.921022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'218","reported_seq":734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.976785+0000","last_change":"2024-04-17T04:36:35.431849+0000","last_active":"2024-04-17T05:45:45.976785+0000","last_peered":"2024-04-17T05:45:45.976785+0000","last_clean":"2024-04-17T05:45:45.976785+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:45:45.976785+0000","last_undegraded":"2024-04-17T05:45:45.976785+0000","last_fullsized":"2024-04-17T05:45:45.976785+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:18.340086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4093","reported_seq":4724,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938365+0000","last_change":"2024-04-17T04:36:27.378432+0000","last_active":"2024-04-17T05:43:09.938365+0000","last_peered":"2024-04-17T05:43:09.938365+0000","last_clean":"2024-04-17T05:43:09.938365+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:43:09.938365+0000","last_undegraded":"2024-04-17T05:43:09.938365+0000","last_fullsized":"2024-04-17T05:43:09.938365+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":3000,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:55.591614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'356","reported_seq":973,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.879695+0000","last_change":"2024-04-17T04:36:25.354707+0000","last_active":"2024-04-17T05:46:03.879695+0000","last_peered":"2024-04-17T05:46:03.879695+0000","last_clean":"2024-04-17T05:46:03.879695+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:46:03.879695+0000","last_undegraded":"2024-04-17T05:46:03.879695+0000","last_fullsized":"2024-04-17T05:46:03.879695+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:31:46.297295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145969,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":466,"num_read_kb":7519,"num_write":1284,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:30.156524+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T05:43:30.156524+0000","last_peered":"2024-04-17T05:43:30.156524+0000","last_clean":"2024-04-17T05:43:30.156524+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:43:30.156524+0000","last_undegraded":"2024-04-17T05:43:30.156524+0000","last_fullsized":"2024-04-17T05:43:30.156524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:09:24.296217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.180151+0000","last_change":"2024-04-17T04:36:29.363086+0000","last_active":"2024-04-17T05:42:45.180151+0000","last_peered":"2024-04-17T05:42:45.180151+0000","last_clean":"2024-04-17T05:42:45.180151+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:42:45.180151+0000","last_undegraded":"2024-04-17T05:42:45.180151+0000","last_fullsized":"2024-04-17T05:42:45.180151+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:48.241347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'730","reported_seq":1474,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.685425+0000","last_change":"2024-04-17T04:36:31.422637+0000","last_active":"2024-04-17T05:46:03.685425+0000","last_peered":"2024-04-17T05:46:03.685425+0000","last_clean":"2024-04-17T05:46:03.685425+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:46:03.685425+0000","last_undegraded":"2024-04-17T05:46:03.685425+0000","last_fullsized":"2024-04-17T05:46:03.685425+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:37.277476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138854,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1296,"num_read_kb":3065,"num_write":1038,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:15.187164+0000","last_change":"2024-04-17T04:36:33.415987+0000","last_active":"2024-04-17T05:43:15.187164+0000","last_peered":"2024-04-17T05:43:15.187164+0000","last_clean":"2024-04-17T05:43:15.187164+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:43:15.187164+0000","last_undegraded":"2024-04-17T05:43:15.187164+0000","last_fullsized":"2024-04-17T05:43:15.187164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:31:25.307133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'102","reported_seq":591,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:25.397345+0000","last_change":"2024-04-17T04:36:35.424792+0000","last_active":"2024-04-17T05:42:25.397345+0000","last_peered":"2024-04-17T05:42:25.397345+0000","last_clean":"2024-04-17T05:42:25.397345+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:42:25.397345+0000","last_undegraded":"2024-04-17T05:42:25.397345+0000","last_fullsized":"2024-04-17T05:42:25.397345+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:27.783198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'149","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.086388+0000","last_change":"2024-04-17T04:36:35.425561+0000","last_active":"2024-04-17T05:45:45.086388+0000","last_peered":"2024-04-17T05:45:45.086388+0000","last_clean":"2024-04-17T05:45:45.086388+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:45:45.086388+0000","last_undegraded":"2024-04-17T05:45:45.086388+0000","last_fullsized":"2024-04-17T05:45:45.086388+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:48.063997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'73","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125289+0000","last_change":"2024-04-17T04:36:27.400403+0000","last_active":"2024-04-17T05:41:50.125289+0000","last_peered":"2024-04-17T05:41:50.125289+0000","last_clean":"2024-04-17T05:41:50.125289+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:41:50.125289+0000","last_undegraded":"2024-04-17T05:41:50.125289+0000","last_fullsized":"2024-04-17T05:41:50.125289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:03:03.426737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'281","reported_seq":919,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.857492+0000","last_change":"2024-04-17T04:36:25.357052+0000","last_active":"2024-04-17T05:46:03.857492+0000","last_peered":"2024-04-17T05:46:03.857492+0000","last_clean":"2024-04-17T05:46:03.857492+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:46:03.857492+0000","last_undegraded":"2024-04-17T05:46:03.857492+0000","last_fullsized":"2024-04-17T05:46:03.857492+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:07:39.562019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017984,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":431,"num_read_kb":11105,"num_write":979,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.139292+0000","last_change":"2024-04-17T04:36:29.391641+0000","last_active":"2024-04-17T05:41:50.139292+0000","last_peered":"2024-04-17T05:41:50.139292+0000","last_clean":"2024-04-17T05:41:50.139292+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:41:50.139292+0000","last_undegraded":"2024-04-17T05:41:50.139292+0000","last_fullsized":"2024-04-17T05:41:50.139292+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:29.991489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'301","reported_seq":896,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.657032+0000","last_change":"2024-04-17T04:36:31.406561+0000","last_active":"2024-04-17T05:46:01.657032+0000","last_peered":"2024-04-17T05:46:01.657032+0000","last_clean":"2024-04-17T05:46:01.657032+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:46:01.657032+0000","last_undegraded":"2024-04-17T05:46:01.657032+0000","last_fullsized":"2024-04-17T05:46:01.657032+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:50:50.884998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44026,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":524,"num_read_kb":1245,"num_write":424,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168016+0000","last_change":"2024-04-17T04:36:33.420539+0000","last_active":"2024-04-17T05:41:55.168016+0000","last_peered":"2024-04-17T05:41:55.168016+0000","last_clean":"2024-04-17T05:41:55.168016+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:41:55.168016+0000","last_undegraded":"2024-04-17T05:41:55.168016+0000","last_fullsized":"2024-04-17T05:41:55.168016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:42.375895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'98","reported_seq":603,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.713280+0000","last_change":"2024-04-17T04:36:35.430479+0000","last_active":"2024-04-17T05:45:44.713280+0000","last_peered":"2024-04-17T05:45:44.713280+0000","last_clean":"2024-04-17T05:45:44.713280+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:45:44.713280+0000","last_undegraded":"2024-04-17T05:45:44.713280+0000","last_fullsized":"2024-04-17T05:45:44.713280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:21:02.348756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'126","reported_seq":737,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:17.594052+0000","last_change":"2024-04-17T04:36:27.389126+0000","last_active":"2024-04-17T05:42:17.594052+0000","last_peered":"2024-04-17T05:42:17.594052+0000","last_clean":"2024-04-17T05:42:17.594052+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:42:17.594052+0000","last_undegraded":"2024-04-17T05:42:17.594052+0000","last_fullsized":"2024-04-17T05:42:17.594052+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:40.137919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'373","reported_seq":1025,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.860785+0000","last_change":"2024-04-17T04:37:03.635652+0000","last_active":"2024-04-17T05:46:03.860785+0000","last_peered":"2024-04-17T05:46:03.860785+0000","last_clean":"2024-04-17T05:46:03.860785+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:46:03.860785+0000","last_undegraded":"2024-04-17T05:46:03.860785+0000","last_fullsized":"2024-04-17T05:46:03.860785+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:33.722634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":625,"num_read_kb":15946,"num_write":1543,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166897+0000","last_change":"2024-04-17T04:36:29.371357+0000","last_active":"2024-04-17T05:41:50.166897+0000","last_peered":"2024-04-17T05:41:50.166897+0000","last_clean":"2024-04-17T05:41:50.166897+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:41:50.166897+0000","last_undegraded":"2024-04-17T05:41:50.166897+0000","last_fullsized":"2024-04-17T05:41:50.166897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:50.443422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'56","reported_seq":567,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168798+0000","last_change":"2024-04-17T04:36:31.404202+0000","last_active":"2024-04-17T05:44:15.168798+0000","last_peered":"2024-04-17T05:44:15.168798+0000","last_clean":"2024-04-17T05:44:15.168798+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:44:15.168798+0000","last_undegraded":"2024-04-17T05:44:15.168798+0000","last_fullsized":"2024-04-17T05:44:15.168798+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:38:41.388160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134552+0000","last_change":"2024-04-17T04:36:33.416154+0000","last_active":"2024-04-17T05:41:55.134552+0000","last_peered":"2024-04-17T05:41:55.134552+0000","last_clean":"2024-04-17T05:41:55.134552+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:41:55.134552+0000","last_undegraded":"2024-04-17T05:41:55.134552+0000","last_fullsized":"2024-04-17T05:41:55.134552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:53:21.721127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'217","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.789782+0000","last_change":"2024-04-17T04:36:35.428428+0000","last_active":"2024-04-17T05:45:44.789782+0000","last_peered":"2024-04-17T05:45:44.789782+0000","last_clean":"2024-04-17T05:45:44.789782+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:45:44.789782+0000","last_undegraded":"2024-04-17T05:45:44.789782+0000","last_fullsized":"2024-04-17T05:45:44.789782+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":200,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:01:55.027046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166680+0000","last_change":"2024-04-17T04:36:27.395464+0000","last_active":"2024-04-17T05:41:50.166680+0000","last_peered":"2024-04-17T05:41:50.166680+0000","last_clean":"2024-04-17T05:41:50.166680+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:41:50.166680+0000","last_undegraded":"2024-04-17T05:41:50.166680+0000","last_fullsized":"2024-04-17T05:41:50.166680+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:23.189462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'356","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.867830+0000","last_change":"2024-04-17T04:36:25.338779+0000","last_active":"2024-04-17T05:46:03.867830+0000","last_peered":"2024-04-17T05:46:03.867830+0000","last_clean":"2024-04-17T05:46:03.867830+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:46:03.867830+0000","last_undegraded":"2024-04-17T05:46:03.867830+0000","last_fullsized":"2024-04-17T05:46:03.867830+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:37:59.502139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":588,"num_read_kb":11343,"num_write":1303,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.133249+0000","last_change":"2024-04-17T04:36:29.381180+0000","last_active":"2024-04-17T05:41:50.133249+0000","last_peered":"2024-04-17T05:41:50.133249+0000","last_clean":"2024-04-17T05:41:50.133249+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:41:50.133249+0000","last_undegraded":"2024-04-17T05:41:50.133249+0000","last_fullsized":"2024-04-17T05:41:50.133249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:01.625306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'340","reported_seq":945,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.289321+0000","last_change":"2024-04-17T04:36:31.406900+0000","last_active":"2024-04-17T05:46:03.289321+0000","last_peered":"2024-04-17T05:46:03.289321+0000","last_clean":"2024-04-17T05:46:03.289321+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:46:03.289321+0000","last_undegraded":"2024-04-17T05:46:03.289321+0000","last_fullsized":"2024-04-17T05:46:03.289321+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:01:47.400810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48000,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":552,"num_read_kb":1237,"num_write":440,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":490,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.135689+0000","last_change":"2024-04-17T04:36:33.412913+0000","last_active":"2024-04-17T05:41:55.135689+0000","last_peered":"2024-04-17T05:41:55.135689+0000","last_clean":"2024-04-17T05:41:55.135689+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:41:55.135689+0000","last_undegraded":"2024-04-17T05:41:55.135689+0000","last_fullsized":"2024-04-17T05:41:55.135689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:41:56.825814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'112","reported_seq":599,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.542927+0000","last_change":"2024-04-17T04:36:35.424722+0000","last_active":"2024-04-17T05:41:51.542927+0000","last_peered":"2024-04-17T05:41:51.542927+0000","last_clean":"2024-04-17T05:41:51.542927+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:41:51.542927+0000","last_undegraded":"2024-04-17T05:41:51.542927+0000","last_fullsized":"2024-04-17T05:41:51.542927+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:15:26.386012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":759,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.166609+0000","last_change":"2024-04-17T04:36:27.378417+0000","last_active":"2024-04-17T05:41:50.166609+0000","last_peered":"2024-04-17T05:41:50.166609+0000","last_clean":"2024-04-17T05:41:50.166609+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:41:50.166609+0000","last_undegraded":"2024-04-17T05:41:50.166609+0000","last_fullsized":"2024-04-17T05:41:50.166609+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:09:17.606187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'310","reported_seq":958,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.731360+0000","last_change":"2024-04-17T04:36:25.344700+0000","last_active":"2024-04-17T05:46:03.731360+0000","last_peered":"2024-04-17T05:46:03.731360+0000","last_clean":"2024-04-17T05:46:03.731360+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:46:03.731360+0000","last_undegraded":"2024-04-17T05:46:03.731360+0000","last_fullsized":"2024-04-17T05:46:03.731360+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:28:58.108460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12846794,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":445,"num_read_kb":84514,"num_write":990,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134534+0000","last_change":"2024-04-17T04:36:29.384475+0000","last_active":"2024-04-17T05:41:50.134534+0000","last_peered":"2024-04-17T05:41:50.134534+0000","last_clean":"2024-04-17T05:41:50.134534+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:41:50.134534+0000","last_undegraded":"2024-04-17T05:41:50.134534+0000","last_fullsized":"2024-04-17T05:41:50.134534+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:07:57.583947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'258","reported_seq":838,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.053094+0000","last_change":"2024-04-17T04:36:31.380383+0000","last_active":"2024-04-17T05:46:03.053094+0000","last_peered":"2024-04-17T05:46:03.053094+0000","last_clean":"2024-04-17T05:46:03.053094+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:46:03.053094+0000","last_undegraded":"2024-04-17T05:46:03.053094+0000","last_fullsized":"2024-04-17T05:46:03.053094+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:27:45.175550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47325,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":432,"num_read_kb":996,"num_write":342,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136027+0000","last_change":"2024-04-17T04:36:33.429804+0000","last_active":"2024-04-17T05:41:55.136027+0000","last_peered":"2024-04-17T05:41:55.136027+0000","last_clean":"2024-04-17T05:41:55.136027+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:41:55.136027+0000","last_undegraded":"2024-04-17T05:41:55.136027+0000","last_fullsized":"2024-04-17T05:41:55.136027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:03:10.357384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'289","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151938+0000","last_change":"2024-04-17T04:36:27.387575+0000","last_active":"2024-04-17T05:43:10.151938+0000","last_peered":"2024-04-17T05:43:10.151938+0000","last_clean":"2024-04-17T05:43:10.151938+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:43:10.151938+0000","last_undegraded":"2024-04-17T05:43:10.151938+0000","last_fullsized":"2024-04-17T05:43:10.151938+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:56:05.627702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'313","reported_seq":998,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.830005+0000","last_change":"2024-04-17T04:36:25.326760+0000","last_active":"2024-04-17T05:46:03.830005+0000","last_peered":"2024-04-17T05:46:03.830005+0000","last_clean":"2024-04-17T05:46:03.830005+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:46:03.830005+0000","last_undegraded":"2024-04-17T05:46:03.830005+0000","last_fullsized":"2024-04-17T05:46:03.830005+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:25.101546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752753,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":542,"num_read_kb":33053,"num_write":1174,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146569+0000","last_change":"2024-04-17T04:36:29.373251+0000","last_active":"2024-04-17T05:42:40.146569+0000","last_peered":"2024-04-17T05:42:40.146569+0000","last_clean":"2024-04-17T05:42:40.146569+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:42:40.146569+0000","last_undegraded":"2024-04-17T05:42:40.146569+0000","last_fullsized":"2024-04-17T05:42:40.146569+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:18:53.682774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'476","reported_seq":1127,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.980617+0000","last_change":"2024-04-17T04:36:31.397750+0000","last_active":"2024-04-17T05:46:02.980617+0000","last_peered":"2024-04-17T05:46:02.980617+0000","last_clean":"2024-04-17T05:46:02.980617+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:46:02.980617+0000","last_undegraded":"2024-04-17T05:46:02.980617+0000","last_fullsized":"2024-04-17T05:46:02.980617+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:19:14.506372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87094,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":846,"num_read_kb":2027,"num_write":720,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.135785+0000","last_change":"2024-04-17T04:36:33.418783+0000","last_active":"2024-04-17T05:41:35.135785+0000","last_peered":"2024-04-17T05:41:35.135785+0000","last_clean":"2024-04-17T05:41:35.135785+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:41:35.135785+0000","last_undegraded":"2024-04-17T05:41:35.135785+0000","last_fullsized":"2024-04-17T05:41:35.135785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:38:46.129082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'171","reported_seq":678,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.168149+0000","last_change":"2024-04-17T04:36:35.419329+0000","last_active":"2024-04-17T05:41:55.168149+0000","last_peered":"2024-04-17T05:41:55.168149+0000","last_clean":"2024-04-17T05:41:55.168149+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:41:55.168149+0000","last_undegraded":"2024-04-17T05:41:55.168149+0000","last_fullsized":"2024-04-17T05:41:55.168149+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:53:14.062860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":709,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:40.146514+0000","last_change":"2024-04-17T04:36:27.386903+0000","last_active":"2024-04-17T05:42:40.146514+0000","last_peered":"2024-04-17T05:42:40.146514+0000","last_clean":"2024-04-17T05:42:40.146514+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:42:40.146514+0000","last_undegraded":"2024-04-17T05:42:40.146514+0000","last_fullsized":"2024-04-17T05:42:40.146514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:46:53.605066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'315","reported_seq":996,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.881504+0000","last_change":"2024-04-17T04:36:25.357464+0000","last_active":"2024-04-17T05:46:03.881504+0000","last_peered":"2024-04-17T05:46:03.881504+0000","last_clean":"2024-04-17T05:46:03.881504+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:46:03.881504+0000","last_undegraded":"2024-04-17T05:46:03.881504+0000","last_fullsized":"2024-04-17T05:46:03.881504+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:12.420278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122783,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":510,"num_read_kb":112390,"num_write":1223,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151910+0000","last_change":"2024-04-17T04:36:29.376006+0000","last_active":"2024-04-17T05:43:10.151910+0000","last_peered":"2024-04-17T05:43:10.151910+0000","last_clean":"2024-04-17T05:43:10.151910+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:43:10.151910+0000","last_undegraded":"2024-04-17T05:43:10.151910+0000","last_fullsized":"2024-04-17T05:43:10.151910+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.144787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'315","reported_seq":914,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.132963+0000","last_change":"2024-04-17T04:36:31.402737+0000","last_active":"2024-04-17T05:46:03.132963+0000","last_peered":"2024-04-17T05:46:03.132963+0000","last_clean":"2024-04-17T05:46:03.132963+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:46:03.132963+0000","last_undegraded":"2024-04-17T05:46:03.132963+0000","last_fullsized":"2024-04-17T05:46:03.132963+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:20.700900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40265,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":1168,"num_write":442,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.136274+0000","last_change":"2024-04-17T04:36:33.415254+0000","last_active":"2024-04-17T05:41:35.136274+0000","last_peered":"2024-04-17T05:41:35.136274+0000","last_clean":"2024-04-17T05:41:35.136274+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:41:35.136274+0000","last_undegraded":"2024-04-17T05:41:35.136274+0000","last_fullsized":"2024-04-17T05:41:35.136274+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:20:37.629549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'175","reported_seq":711,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166706+0000","last_change":"2024-04-17T04:36:35.424191+0000","last_active":"2024-04-17T05:44:15.166706+0000","last_peered":"2024-04-17T05:44:15.166706+0000","last_clean":"2024-04-17T05:44:15.166706+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:44:15.166706+0000","last_undegraded":"2024-04-17T05:44:15.166706+0000","last_fullsized":"2024-04-17T05:44:15.166706+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:48.892746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'114","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.162530+0000","last_change":"2024-04-17T04:36:27.378054+0000","last_active":"2024-04-17T05:41:30.162530+0000","last_peered":"2024-04-17T05:41:30.162530+0000","last_clean":"2024-04-17T05:41:30.162530+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:41:30.162530+0000","last_undegraded":"2024-04-17T05:41:30.162530+0000","last_fullsized":"2024-04-17T05:41:30.162530+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:59:41.230833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'358","reported_seq":1042,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.749932+0000","last_change":"2024-04-17T04:36:25.356583+0000","last_active":"2024-04-17T05:46:03.749932+0000","last_peered":"2024-04-17T05:46:03.749932+0000","last_clean":"2024-04-17T05:46:03.749932+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:46:03.749932+0000","last_undegraded":"2024-04-17T05:46:03.749932+0000","last_fullsized":"2024-04-17T05:46:03.749932+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:38.408649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568179,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":576,"num_read_kb":10795,"num_write":1350,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":833,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.169882+0000","last_change":"2024-04-17T04:36:29.370143+0000","last_active":"2024-04-17T05:46:01.169882+0000","last_peered":"2024-04-17T05:46:01.169882+0000","last_clean":"2024-04-17T05:46:01.169882+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:46:01.169882+0000","last_undegraded":"2024-04-17T05:46:01.169882+0000","last_fullsized":"2024-04-17T05:46:01.169882+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:06:33.209495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":499,"num_read_kb":497,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'298","reported_seq":897,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.010049+0000","last_change":"2024-04-17T04:36:31.421343+0000","last_active":"2024-04-17T05:46:02.010049+0000","last_peered":"2024-04-17T05:46:02.010049+0000","last_clean":"2024-04-17T05:46:02.010049+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:46:02.010049+0000","last_undegraded":"2024-04-17T05:46:02.010049+0000","last_fullsized":"2024-04-17T05:46:02.010049+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:35:50.024878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45015,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":503,"num_read_kb":1173,"num_write":404,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:50.152311+0000","last_change":"2024-04-17T04:36:33.415113+0000","last_active":"2024-04-17T05:42:50.152311+0000","last_peered":"2024-04-17T05:42:50.152311+0000","last_clean":"2024-04-17T05:42:50.152311+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:42:50.152311+0000","last_undegraded":"2024-04-17T05:42:50.152311+0000","last_fullsized":"2024-04-17T05:42:50.152311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:41.644384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'102","reported_seq":601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.654720+0000","last_change":"2024-04-17T04:36:35.428333+0000","last_active":"2024-04-17T05:45:54.654720+0000","last_peered":"2024-04-17T05:45:54.654720+0000","last_clean":"2024-04-17T05:45:54.654720+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:45:54.654720+0000","last_undegraded":"2024-04-17T05:45:54.654720+0000","last_fullsized":"2024-04-17T05:45:54.654720+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:39.420180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.133409+0000","last_change":"2024-04-17T04:36:27.356010+0000","last_active":"2024-04-17T05:41:30.133409+0000","last_peered":"2024-04-17T05:41:30.133409+0000","last_clean":"2024-04-17T05:41:30.133409+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:41:30.133409+0000","last_undegraded":"2024-04-17T05:41:30.133409+0000","last_fullsized":"2024-04-17T05:41:30.133409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:32:42.969449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'322","reported_seq":1147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.784774+0000","last_change":"2024-04-17T04:36:25.354574+0000","last_active":"2024-04-17T05:46:03.784774+0000","last_peered":"2024-04-17T05:46:03.784774+0000","last_clean":"2024-04-17T05:46:03.784774+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:46:03.784774+0000","last_undegraded":"2024-04-17T05:46:03.784774+0000","last_fullsized":"2024-04-17T05:46:03.784774+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:55.589797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263230,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":844,"num_read_kb":11279,"num_write":1205,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.135309+0000","last_change":"2024-04-17T04:36:29.365578+0000","last_active":"2024-04-17T05:41:30.135309+0000","last_peered":"2024-04-17T05:41:30.135309+0000","last_clean":"2024-04-17T05:41:30.135309+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:41:30.135309+0000","last_undegraded":"2024-04-17T05:41:30.135309+0000","last_fullsized":"2024-04-17T05:41:30.135309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:29.189344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'119","reported_seq":666,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:45.137220+0000","last_change":"2024-04-17T04:36:31.422491+0000","last_active":"2024-04-17T05:42:45.137220+0000","last_peered":"2024-04-17T05:42:45.137220+0000","last_clean":"2024-04-17T05:42:45.137220+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:42:45.137220+0000","last_undegraded":"2024-04-17T05:42:45.137220+0000","last_fullsized":"2024-04-17T05:42:45.137220+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:16:06.145178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'539","reported_seq":2314,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.046768+0000","last_change":"2024-04-17T04:36:33.404252+0000","last_active":"2024-04-17T05:46:02.046768+0000","last_peered":"2024-04-17T05:46:02.046768+0000","last_clean":"2024-04-17T05:46:02.046768+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:46:02.046768+0000","last_undegraded":"2024-04-17T05:46:02.046768+0000","last_fullsized":"2024-04-17T05:46:02.046768+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:11.625831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'114","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.134497+0000","last_change":"2024-04-17T04:36:35.413901+0000","last_active":"2024-04-17T05:41:55.134497+0000","last_peered":"2024-04-17T05:41:55.134497+0000","last_clean":"2024-04-17T05:41:55.134497+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:41:55.134497+0000","last_undegraded":"2024-04-17T05:41:55.134497+0000","last_fullsized":"2024-04-17T05:41:55.134497+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:00:54.978078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'229","reported_seq":875,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.120332+0000","last_change":"2024-04-17T04:36:27.394864+0000","last_active":"2024-04-17T05:41:30.120332+0000","last_peered":"2024-04-17T05:41:30.120332+0000","last_clean":"2024-04-17T05:41:30.120332+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:41:30.120332+0000","last_undegraded":"2024-04-17T05:41:30.120332+0000","last_fullsized":"2024-04-17T05:41:30.120332+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:50:03.593045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'330","reported_seq":929,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.860852+0000","last_change":"2024-04-17T04:36:25.369133+0000","last_active":"2024-04-17T05:46:03.860852+0000","last_peered":"2024-04-17T05:46:03.860852+0000","last_clean":"2024-04-17T05:46:03.860852+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:46:03.860852+0000","last_undegraded":"2024-04-17T05:46:03.860852+0000","last_fullsized":"2024-04-17T05:46:03.860852+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:22.406331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244482,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":438,"num_read_kb":414,"num_write":1194,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.128372+0000","last_change":"2024-04-17T04:36:29.379152+0000","last_active":"2024-04-17T05:41:30.128372+0000","last_peered":"2024-04-17T05:41:30.128372+0000","last_clean":"2024-04-17T05:41:30.128372+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:41:30.128372+0000","last_undegraded":"2024-04-17T05:41:30.128372+0000","last_fullsized":"2024-04-17T05:41:30.128372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:52.689910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'341","reported_seq":964,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.456828+0000","last_change":"2024-04-17T04:36:31.396219+0000","last_active":"2024-04-17T05:46:02.456828+0000","last_peered":"2024-04-17T05:46:02.456828+0000","last_clean":"2024-04-17T05:46:02.456828+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:46:02.456828+0000","last_undegraded":"2024-04-17T05:46:02.456828+0000","last_fullsized":"2024-04-17T05:46:02.456828+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:39:05.869489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44869,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":547,"num_read_kb":1136,"num_write":426,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.134718+0000","last_change":"2024-04-17T04:36:33.430639+0000","last_active":"2024-04-17T05:41:50.134718+0000","last_peered":"2024-04-17T05:41:50.134718+0000","last_clean":"2024-04-17T05:41:50.134718+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:41:50.134718+0000","last_undegraded":"2024-04-17T05:41:50.134718+0000","last_fullsized":"2024-04-17T05:41:50.134718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:24:36.795912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'127","reported_seq":629,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166471+0000","last_change":"2024-04-17T04:36:35.436213+0000","last_active":"2024-04-17T05:44:15.166471+0000","last_peered":"2024-04-17T05:44:15.166471+0000","last_clean":"2024-04-17T05:44:15.166471+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:44:15.166471+0000","last_undegraded":"2024-04-17T05:44:15.166471+0000","last_fullsized":"2024-04-17T05:44:15.166471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:49:44.113556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":804,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:51.536900+0000","last_change":"2024-04-17T04:36:27.378160+0000","last_active":"2024-04-17T05:41:51.536900+0000","last_peered":"2024-04-17T05:41:51.536900+0000","last_clean":"2024-04-17T05:41:51.536900+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:41:51.536900+0000","last_undegraded":"2024-04-17T05:41:51.536900+0000","last_fullsized":"2024-04-17T05:41:51.536900+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:08:53.663348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'349","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.807321+0000","last_change":"2024-04-17T04:36:25.365976+0000","last_active":"2024-04-17T05:46:03.807321+0000","last_peered":"2024-04-17T05:46:03.807321+0000","last_clean":"2024-04-17T05:46:03.807321+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:46:03.807321+0000","last_undegraded":"2024-04-17T05:46:03.807321+0000","last_fullsized":"2024-04-17T05:46:03.807321+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:05.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":31239,"num_write":1274,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:30.134570+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:41:30.134570+0000","last_peered":"2024-04-17T05:41:30.134570+0000","last_clean":"2024-04-17T05:41:30.134570+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:41:30.134570+0000","last_undegraded":"2024-04-17T05:41:30.134570+0000","last_fullsized":"2024-04-17T05:41:30.134570+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:39.416117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'273","reported_seq":858,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.529889+0000","last_change":"2024-04-17T04:36:31.403334+0000","last_active":"2024-04-17T05:46:02.529889+0000","last_peered":"2024-04-17T05:46:02.529889+0000","last_clean":"2024-04-17T05:46:02.529889+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:46:02.529889+0000","last_undegraded":"2024-04-17T05:46:02.529889+0000","last_fullsized":"2024-04-17T05:46:02.529889+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:25.865066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46669,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":461,"num_read_kb":1085,"num_write":368,"num_write_kb":285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":489,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.136680+0000","last_change":"2024-04-17T04:36:33.420313+0000","last_active":"2024-04-17T05:42:00.136680+0000","last_peered":"2024-04-17T05:42:00.136680+0000","last_clean":"2024-04-17T05:42:00.136680+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:42:00.136680+0000","last_undegraded":"2024-04-17T05:42:00.136680+0000","last_fullsized":"2024-04-17T05:42:00.136680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:55.964612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'134","reported_seq":655,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.071549+0000","last_change":"2024-04-17T04:36:35.422747+0000","last_active":"2024-04-17T05:45:46.071549+0000","last_peered":"2024-04-17T05:45:46.071549+0000","last_clean":"2024-04-17T05:45:46.071549+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:45:46.071549+0000","last_undegraded":"2024-04-17T05:45:46.071549+0000","last_fullsized":"2024-04-17T05:45:46.071549+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:32.269106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'203","reported_seq":810,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:14.025767+0000","last_change":"2024-04-17T04:36:27.381423+0000","last_active":"2024-04-17T05:42:14.025767+0000","last_peered":"2024-04-17T05:42:14.025767+0000","last_clean":"2024-04-17T05:42:14.025767+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:42:14.025767+0000","last_undegraded":"2024-04-17T05:42:14.025767+0000","last_fullsized":"2024-04-17T05:42:14.025767+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:26:04.091357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'386","reported_seq":1049,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.859780+0000","last_change":"2024-04-17T04:36:25.374623+0000","last_active":"2024-04-17T05:46:03.859780+0000","last_peered":"2024-04-17T05:46:03.859780+0000","last_clean":"2024-04-17T05:46:03.859780+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:46:03.859780+0000","last_undegraded":"2024-04-17T05:46:03.859780+0000","last_fullsized":"2024-04-17T05:46:03.859780+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:33:52.802493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1731334,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":524,"num_read_kb":19843,"num_write":1402,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123902+0000","last_change":"2024-04-17T04:36:29.377026+0000","last_active":"2024-04-17T05:41:45.123902+0000","last_peered":"2024-04-17T05:41:45.123902+0000","last_clean":"2024-04-17T05:41:45.123902+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:41:45.123902+0000","last_undegraded":"2024-04-17T05:41:45.123902+0000","last_fullsized":"2024-04-17T05:41:45.123902+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:26:46.743881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'260","reported_seq":832,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.847097+0000","last_change":"2024-04-17T04:36:31.398159+0000","last_active":"2024-04-17T05:46:01.847097+0000","last_peered":"2024-04-17T05:46:01.847097+0000","last_clean":"2024-04-17T05:46:01.847097+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:46:01.847097+0000","last_undegraded":"2024-04-17T05:46:01.847097+0000","last_fullsized":"2024-04-17T05:46:01.847097+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:04.571296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44294,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":445,"num_read_kb":1105,"num_write":366,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.121505+0000","last_change":"2024-04-17T04:36:33.401472+0000","last_active":"2024-04-17T05:41:35.121505+0000","last_peered":"2024-04-17T05:41:35.121505+0000","last_clean":"2024-04-17T05:41:35.121505+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:41:35.121505+0000","last_undegraded":"2024-04-17T05:41:35.121505+0000","last_fullsized":"2024-04-17T05:41:35.121505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:36:49.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'141","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.700107+0000","last_change":"2024-04-17T04:36:35.423654+0000","last_active":"2024-04-17T05:45:40.700107+0000","last_peered":"2024-04-17T05:45:40.700107+0000","last_clean":"2024-04-17T05:45:40.700107+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:45:40.700107+0000","last_undegraded":"2024-04-17T05:45:40.700107+0000","last_fullsized":"2024-04-17T05:45:40.700107+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:17:36.321043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'297","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.123940+0000","last_change":"2024-04-17T04:36:27.400669+0000","last_active":"2024-04-17T05:41:45.123940+0000","last_peered":"2024-04-17T05:41:45.123940+0000","last_clean":"2024-04-17T05:41:45.123940+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:41:45.123940+0000","last_undegraded":"2024-04-17T05:41:45.123940+0000","last_fullsized":"2024-04-17T05:41:45.123940+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:59.138022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'437","reported_seq":1143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.772813+0000","last_change":"2024-04-17T04:36:25.356509+0000","last_active":"2024-04-17T05:46:03.772813+0000","last_peered":"2024-04-17T05:46:03.772813+0000","last_clean":"2024-04-17T05:46:03.772813+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:46:03.772813+0000","last_undegraded":"2024-04-17T05:46:03.772813+0000","last_fullsized":"2024-04-17T05:46:03.772813+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:11:14.238916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":515218,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":663,"num_read_kb":71297,"num_write":1686,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"35'6","reported_seq":836,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.163359+0000","last_change":"2024-04-17T04:36:29.376775+0000","last_active":"2024-04-17T05:46:01.163359+0000","last_peered":"2024-04-17T05:46:01.163359+0000","last_clean":"2024-04-17T05:46:01.163359+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:46:01.163359+0000","last_undegraded":"2024-04-17T05:46:01.163359+0000","last_fullsized":"2024-04-17T05:46:01.163359+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:09.589509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":501,"num_read_kb":814,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'319","reported_seq":929,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.099456+0000","last_change":"2024-04-17T04:36:31.391158+0000","last_active":"2024-04-17T05:46:02.099456+0000","last_peered":"2024-04-17T05:46:02.099456+0000","last_clean":"2024-04-17T05:46:02.099456+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:46:02.099456+0000","last_undegraded":"2024-04-17T05:46:02.099456+0000","last_fullsized":"2024-04-17T05:46:02.099456+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:50:43.696307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46192,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":530,"num_read_kb":1178,"num_write":420,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":491,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:35.130642+0000","last_change":"2024-04-17T04:36:33.430195+0000","last_active":"2024-04-17T05:41:35.130642+0000","last_peered":"2024-04-17T05:41:35.130642+0000","last_clean":"2024-04-17T05:41:35.130642+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:41:35.130642+0000","last_undegraded":"2024-04-17T05:41:35.130642+0000","last_fullsized":"2024-04-17T05:41:35.130642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:37.038297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'138","reported_seq":637,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.163062+0000","last_change":"2024-04-17T04:36:35.427734+0000","last_active":"2024-04-17T05:44:25.163062+0000","last_peered":"2024-04-17T05:44:25.163062+0000","last_clean":"2024-04-17T05:44:25.163062+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:44:25.163062+0000","last_undegraded":"2024-04-17T05:44:25.163062+0000","last_fullsized":"2024-04-17T05:44:25.163062+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:48:29.121374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'154","reported_seq":788,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164804+0000","last_change":"2024-04-17T04:36:27.397669+0000","last_active":"2024-04-17T05:43:40.164804+0000","last_peered":"2024-04-17T05:43:40.164804+0000","last_clean":"2024-04-17T05:43:40.164804+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:43:40.164804+0000","last_undegraded":"2024-04-17T05:43:40.164804+0000","last_fullsized":"2024-04-17T05:43:40.164804+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:46:34.375349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'420","reported_seq":1138,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.778236+0000","last_change":"2024-04-17T04:36:25.383690+0000","last_active":"2024-04-17T05:46:03.778236+0000","last_peered":"2024-04-17T05:46:03.778236+0000","last_clean":"2024-04-17T05:46:03.778236+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:46:03.778236+0000","last_undegraded":"2024-04-17T05:46:03.778236+0000","last_fullsized":"2024-04-17T05:46:03.778236+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:14.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262268,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":835,"num_read_kb":6303,"num_write":1376,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":648,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:05.142976+0000","last_change":"2024-04-17T04:37:03.630121+0000","last_active":"2024-04-17T05:42:05.142976+0000","last_peered":"2024-04-17T05:42:05.142976+0000","last_clean":"2024-04-17T05:42:05.142976+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:42:05.142976+0000","last_undegraded":"2024-04-17T05:42:05.142976+0000","last_fullsized":"2024-04-17T05:42:05.142976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:09:56.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'365","reported_seq":1031,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.823249+0000","last_change":"2024-04-17T04:36:25.355361+0000","last_active":"2024-04-17T05:46:03.823249+0000","last_peered":"2024-04-17T05:46:03.823249+0000","last_clean":"2024-04-17T05:46:03.823249+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:46:03.823249+0000","last_undegraded":"2024-04-17T05:46:03.823249+0000","last_fullsized":"2024-04-17T05:46:03.823249+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:15.756630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6192676,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":568,"num_read_kb":39832,"num_write":1198,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'148","reported_seq":762,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.163630+0000","last_change":"2024-04-17T04:36:27.388520+0000","last_active":"2024-04-17T05:43:40.163630+0000","last_peered":"2024-04-17T05:43:40.163630+0000","last_clean":"2024-04-17T05:43:40.163630+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:43:40.163630+0000","last_undegraded":"2024-04-17T05:43:40.163630+0000","last_fullsized":"2024-04-17T05:43:40.163630+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:45:31.868037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'307","reported_seq":944,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.848899+0000","last_change":"2024-04-17T04:36:25.365759+0000","last_active":"2024-04-17T05:46:03.848899+0000","last_peered":"2024-04-17T05:46:03.848899+0000","last_clean":"2024-04-17T05:46:03.848899+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:46:03.848899+0000","last_undegraded":"2024-04-17T05:46:03.848899+0000","last_fullsized":"2024-04-17T05:46:03.848899+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:25:48.226505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4341641,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":436,"num_read_kb":10784,"num_write":1165,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'94","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:40.164612+0000","last_change":"2024-04-17T04:36:27.362737+0000","last_active":"2024-04-17T05:43:40.164612+0000","last_peered":"2024-04-17T05:43:40.164612+0000","last_clean":"2024-04-17T05:43:40.164612+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:43:40.164612+0000","last_undegraded":"2024-04-17T05:43:40.164612+0000","last_fullsized":"2024-04-17T05:43:40.164612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:24:50.502747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'308","reported_seq":1034,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.886847+0000","last_change":"2024-04-17T04:36:25.355263+0000","last_active":"2024-04-17T05:46:03.886847+0000","last_peered":"2024-04-17T05:46:03.886847+0000","last_clean":"2024-04-17T05:46:03.886847+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:46:03.886847+0000","last_undegraded":"2024-04-17T05:46:03.886847+0000","last_fullsized":"2024-04-17T05:46:03.886847+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:43.114752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":709,"num_read_kb":32424,"num_write":1121,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":635,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:25.189253+0000","last_change":"2024-04-17T04:36:27.395680+0000","last_active":"2024-04-17T05:43:25.189253+0000","last_peered":"2024-04-17T05:43:25.189253+0000","last_clean":"2024-04-17T05:43:25.189253+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:43:25.189253+0000","last_undegraded":"2024-04-17T05:43:25.189253+0000","last_fullsized":"2024-04-17T05:43:25.189253+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:08:04.849323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'281","reported_seq":893,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.823812+0000","last_change":"2024-04-17T04:37:03.637190+0000","last_active":"2024-04-17T05:46:03.823812+0000","last_peered":"2024-04-17T05:46:03.823812+0000","last_clean":"2024-04-17T05:46:03.823812+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:46:03.823812+0000","last_undegraded":"2024-04-17T05:46:03.823812+0000","last_fullsized":"2024-04-17T05:46:03.823812+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:24:25.534759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5047006,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":359,"num_read_kb":5921,"num_write":992,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":632,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:09.938441+0000","last_change":"2024-04-17T04:36:27.395504+0000","last_active":"2024-04-17T05:43:09.938441+0000","last_peered":"2024-04-17T05:43:09.938441+0000","last_clean":"2024-04-17T05:43:09.938441+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:43:09.938441+0000","last_undegraded":"2024-04-17T05:43:09.938441+0000","last_fullsized":"2024-04-17T05:43:09.938441+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:27.120391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'319","reported_seq":999,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.840769+0000","last_change":"2024-04-17T04:36:25.342748+0000","last_active":"2024-04-17T05:46:03.840769+0000","last_peered":"2024-04-17T05:46:03.840769+0000","last_clean":"2024-04-17T05:46:03.840769+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:46:03.840769+0000","last_undegraded":"2024-04-17T05:46:03.840769+0000","last_fullsized":"2024-04-17T05:46:03.840769+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:25.415287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5067983,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":492,"num_read_kb":71000,"num_write":1186,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'215","reported_seq":815,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:36.452977+0000","last_change":"2024-04-17T04:36:27.384524+0000","last_active":"2024-04-17T05:43:36.452977+0000","last_peered":"2024-04-17T05:43:36.452977+0000","last_clean":"2024-04-17T05:43:36.452977+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:43:36.452977+0000","last_undegraded":"2024-04-17T05:43:36.452977+0000","last_fullsized":"2024-04-17T05:43:36.452977+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:35.098376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'371","reported_seq":1052,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.879626+0000","last_change":"2024-04-17T04:36:25.386085+0000","last_active":"2024-04-17T05:46:03.879626+0000","last_peered":"2024-04-17T05:46:03.879626+0000","last_clean":"2024-04-17T05:46:03.879626+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:46:03.879626+0000","last_undegraded":"2024-04-17T05:46:03.879626+0000","last_fullsized":"2024-04-17T05:46:03.879626+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:11:09.994649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2189648,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":654,"num_read_kb":34085,"num_write":1471,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:10.151711+0000","last_change":"2024-04-17T04:36:27.387044+0000","last_active":"2024-04-17T05:43:10.151711+0000","last_peered":"2024-04-17T05:43:10.151711+0000","last_clean":"2024-04-17T05:43:10.151711+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:43:10.151711+0000","last_undegraded":"2024-04-17T05:43:10.151711+0000","last_fullsized":"2024-04-17T05:43:10.151711+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:57:47.002935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'335","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.857756+0000","last_change":"2024-04-17T04:36:25.379835+0000","last_active":"2024-04-17T05:46:03.857756+0000","last_peered":"2024-04-17T05:46:03.857756+0000","last_clean":"2024-04-17T05:46:03.857756+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:46:03.857756+0000","last_undegraded":"2024-04-17T05:46:03.857756+0000","last_fullsized":"2024-04-17T05:46:03.857756+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:54:10.099587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534609,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":546,"num_read_kb":42711,"num_write":1290,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":738,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:49.022874+0000","last_change":"2024-04-17T04:36:27.373717+0000","last_active":"2024-04-17T05:42:49.022874+0000","last_peered":"2024-04-17T05:42:49.022874+0000","last_clean":"2024-04-17T05:42:49.022874+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:42:49.022874+0000","last_undegraded":"2024-04-17T05:42:49.022874+0000","last_fullsized":"2024-04-17T05:42:49.022874+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:38:49.102110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'300","reported_seq":934,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.787663+0000","last_change":"2024-04-17T04:36:25.361686+0000","last_active":"2024-04-17T05:46:03.787663+0000","last_peered":"2024-04-17T05:46:03.787663+0000","last_clean":"2024-04-17T05:46:03.787663+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:46:03.787663+0000","last_undegraded":"2024-04-17T05:46:03.787663+0000","last_fullsized":"2024-04-17T05:46:03.787663+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:10:40.663817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262289,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":548,"num_write":1115,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":754,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:50.125394+0000","last_change":"2024-04-17T04:36:27.378398+0000","last_active":"2024-04-17T05:41:50.125394+0000","last_peered":"2024-04-17T05:41:50.125394+0000","last_clean":"2024-04-17T05:41:50.125394+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:41:50.125394+0000","last_undegraded":"2024-04-17T05:41:50.125394+0000","last_fullsized":"2024-04-17T05:41:50.125394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:05:49.831818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'302","reported_seq":951,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.780252+0000","last_change":"2024-04-17T04:36:25.353469+0000","last_active":"2024-04-17T05:46:03.780252+0000","last_peered":"2024-04-17T05:46:03.780252+0000","last_clean":"2024-04-17T05:46:03.780252+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:46:03.780252+0000","last_undegraded":"2024-04-17T05:46:03.780252+0000","last_fullsized":"2024-04-17T05:46:03.780252+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.419090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171414,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":4352,"num_write":1098,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":837,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:42:00.139934+0000","last_change":"2024-04-17T04:36:27.398568+0000","last_active":"2024-04-17T05:42:00.139934+0000","last_peered":"2024-04-17T05:42:00.139934+0000","last_clean":"2024-04-17T05:42:00.139934+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:42:00.139934+0000","last_undegraded":"2024-04-17T05:42:00.139934+0000","last_fullsized":"2024-04-17T05:42:00.139934+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:48.128020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'317","reported_seq":928,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.850499+0000","last_change":"2024-04-17T04:37:03.632788+0000","last_active":"2024-04-17T05:46:03.850499+0000","last_peered":"2024-04-17T05:46:03.850499+0000","last_clean":"2024-04-17T05:46:03.850499+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:46:03.850499+0000","last_undegraded":"2024-04-17T05:46:03.850499+0000","last_fullsized":"2024-04-17T05:46:03.850499+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:49:33.372524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":443,"num_read_kb":16015,"num_write":1067,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'95","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.124087+0000","last_change":"2024-04-17T04:36:27.402182+0000","last_active":"2024-04-17T05:41:45.124087+0000","last_peered":"2024-04-17T05:41:45.124087+0000","last_clean":"2024-04-17T05:41:45.124087+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:41:45.124087+0000","last_undegraded":"2024-04-17T05:41:45.124087+0000","last_fullsized":"2024-04-17T05:41:45.124087+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:59.716958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'360","reported_seq":1077,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.715299+0000","last_change":"2024-04-17T04:36:25.333011+0000","last_active":"2024-04-17T05:46:03.715299+0000","last_peered":"2024-04-17T05:46:03.715299+0000","last_clean":"2024-04-17T05:46:03.715299+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:46:03.715299+0000","last_undegraded":"2024-04-17T05:46:03.715299+0000","last_fullsized":"2024-04-17T05:46:03.715299+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:18:22.239383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":746,"num_read_kb":12134,"num_write":1256,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":686,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.132122+0000","last_change":"2024-04-17T04:36:27.388791+0000","last_active":"2024-04-17T05:41:45.132122+0000","last_peered":"2024-04-17T05:41:45.132122+0000","last_clean":"2024-04-17T05:41:45.132122+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:41:45.132122+0000","last_undegraded":"2024-04-17T05:41:45.132122+0000","last_fullsized":"2024-04-17T05:41:45.132122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:19:21.837130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'357","reported_seq":993,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.751117+0000","last_change":"2024-04-17T04:36:25.373417+0000","last_active":"2024-04-17T05:46:03.751117+0000","last_peered":"2024-04-17T05:46:03.751117+0000","last_clean":"2024-04-17T05:46:03.751117+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:46:03.751117+0000","last_undegraded":"2024-04-17T05:46:03.751117+0000","last_fullsized":"2024-04-17T05:46:03.751117+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:11.729276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":501,"num_read_kb":657,"num_write":1252,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'221","reported_seq":901,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.133347+0000","last_change":"2024-04-17T04:36:27.372140+0000","last_active":"2024-04-17T05:41:45.133347+0000","last_peered":"2024-04-17T05:41:45.133347+0000","last_clean":"2024-04-17T05:41:45.133347+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:41:45.133347+0000","last_undegraded":"2024-04-17T05:41:45.133347+0000","last_fullsized":"2024-04-17T05:41:45.133347+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.338570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'420","reported_seq":1158,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.788888+0000","last_change":"2024-04-17T04:36:25.386307+0000","last_active":"2024-04-17T05:46:03.788888+0000","last_peered":"2024-04-17T05:46:03.788888+0000","last_clean":"2024-04-17T05:46:03.788888+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:46:03.788888+0000","last_undegraded":"2024-04-17T05:46:03.788888+0000","last_fullsized":"2024-04-17T05:46:03.788888+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:25:43.522371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058993,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":894,"num_read_kb":32480,"num_write":1416,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.165668+0000","last_change":"2024-04-17T04:36:27.385349+0000","last_active":"2024-04-17T05:41:45.165668+0000","last_peered":"2024-04-17T05:41:45.165668+0000","last_clean":"2024-04-17T05:41:45.165668+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:41:45.165668+0000","last_undegraded":"2024-04-17T05:41:45.165668+0000","last_fullsized":"2024-04-17T05:41:45.165668+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:27:44.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'314","reported_seq":990,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.746182+0000","last_change":"2024-04-17T04:36:25.355339+0000","last_active":"2024-04-17T05:46:03.746182+0000","last_peered":"2024-04-17T05:46:03.746182+0000","last_clean":"2024-04-17T05:46:03.746182+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:46:03.746182+0000","last_undegraded":"2024-04-17T05:46:03.746182+0000","last_fullsized":"2024-04-17T05:46:03.746182+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:47:52.068393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7281800,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":526,"num_read_kb":46303,"num_write":1296,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'321","reported_seq":1043,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:43:50.160308+0000","last_change":"2024-04-17T04:37:03.628711+0000","last_active":"2024-04-17T05:43:50.160308+0000","last_peered":"2024-04-17T05:43:50.160308+0000","last_clean":"2024-04-17T05:43:50.160308+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:43:50.160308+0000","last_undegraded":"2024-04-17T05:43:50.160308+0000","last_fullsized":"2024-04-17T05:43:50.160308+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:45.773023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'328","reported_seq":961,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.888771+0000","last_change":"2024-04-17T04:36:25.361750+0000","last_active":"2024-04-17T05:46:03.888771+0000","last_peered":"2024-04-17T05:46:03.888771+0000","last_clean":"2024-04-17T05:46:03.888771+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:46:03.888771+0000","last_undegraded":"2024-04-17T05:46:03.888771+0000","last_fullsized":"2024-04-17T05:46:03.888771+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:43:44.822213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":466,"num_read_kb":49597,"num_write":1231,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'297","reported_seq":923,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.798937+0000","last_change":"2024-04-17T04:36:25.345113+0000","last_active":"2024-04-17T05:46:03.798937+0000","last_peered":"2024-04-17T05:46:03.798937+0000","last_clean":"2024-04-17T05:46:03.798937+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:46:03.798937+0000","last_undegraded":"2024-04-17T05:46:03.798937+0000","last_fullsized":"2024-04-17T05:46:03.798937+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:36.604225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129194,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":413,"num_read_kb":7661,"num_write":1087,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'243","reported_seq":842,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:55.136202+0000","last_change":"2024-04-17T04:36:27.379068+0000","last_active":"2024-04-17T05:41:55.136202+0000","last_peered":"2024-04-17T05:41:55.136202+0000","last_clean":"2024-04-17T05:41:55.136202+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:41:55.136202+0000","last_undegraded":"2024-04-17T05:41:55.136202+0000","last_fullsized":"2024-04-17T05:41:55.136202+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:27.386167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'277","reported_seq":920,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.808541+0000","last_change":"2024-04-17T04:36:25.357445+0000","last_active":"2024-04-17T05:46:03.808541+0000","last_peered":"2024-04-17T05:46:03.808541+0000","last_clean":"2024-04-17T05:46:03.808541+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:46:03.808541+0000","last_undegraded":"2024-04-17T05:46:03.808541+0000","last_fullsized":"2024-04-17T05:46:03.808541+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:24:12.816116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":447,"num_read_kb":20933,"num_write":1123,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":819,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:41:45.138230+0000","last_change":"2024-04-17T04:36:27.378695+0000","last_active":"2024-04-17T05:41:45.138230+0000","last_peered":"2024-04-17T05:41:45.138230+0000","last_clean":"2024-04-17T05:41:45.138230+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:41:45.138230+0000","last_undegraded":"2024-04-17T05:41:45.138230+0000","last_fullsized":"2024-04-17T05:41:45.138230+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:37.636571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1280,"ondisk_log_size":1280,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1676,"ondisk_log_size":1676,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1458963,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":18010,"num_read_kb":37614,"num_write":15043,"num_write_kb":11218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1833,"ondisk_log_size":1833,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1652,"num_read_kb":1963,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3948,"ondisk_log_size":3948,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187841250,"num_objects":443,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":34833,"num_read_kb":1864137,"num_write":77494,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253607936,"data_stored":253607936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3747,"ondisk_log_size":3747,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":221363,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":145,"num_read_kb":210051,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575691,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349288,"kb_used_data":38884,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":94022552,"statfs":{"total":96636764160,"available":96279093248,"internally_reserved":0,"allocated":39817216,"data_stored":37809825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":317651402},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575692,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355712,"kb_used_data":39888,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":94016128,"statfs":{"total":96636764160,"available":96272515072,"internally_reserved":0,"allocated":40845312,"data_stored":38885655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":323353031},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575692,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206332,"kb_used_data":58384,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94165508,"statfs":{"total":96636764160,"available":96425480192,"internally_reserved":0,"allocated":59785216,"data_stored":57737844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575694,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148200,"kb_used_data":36412,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94223640,"statfs":{"total":96636764160,"available":96485007360,"internally_reserved":0,"allocated":37285888,"data_stored":35330686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575692,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298604,"kb_used_data":42448,"kb_used_omap":1,"kb_used_meta":255998,"kb_avail":94073236,"statfs":{"total":96636764160,"available":96330993664,"internally_reserved":0,"allocated":43466752,"data_stored":41436434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":262142407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575696,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159304,"kb_used_data":56000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94212536,"statfs":{"total":96636764160,"available":96473636864,"internally_reserved":0,"allocated":57344000,"data_stored":55396653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575694,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183920,"kb_used_data":36064,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94187920,"statfs":{"total":96636764160,"available":96448430080,"internally_reserved":0,"allocated":36929536,"data_stored":34946873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575692,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305384,"kb_used_data":39544,"kb_used_omap":1,"kb_used_meta":265726,"kb_avail":94066456,"statfs":{"total":96636764160,"available":96324050944,"internally_reserved":0,"allocated":40493056,"data_stored":38470704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":272103877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35614720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26390528,"data_stored":26390528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36356096,"data_stored":36356096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33927168,"data_stored":33927168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22581248,"data_stored":22581248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":38694912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31379456,"data_stored":31379456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28663808,"data_stored":28663808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:46:10.570 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.571 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.571 INFO:tasks.ceph:pgid 66.1c last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.571 INFO:tasks.ceph:pgid 65.1f last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.571 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.571 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.571 INFO:tasks.ceph:pgid 66.1d last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.572 INFO:tasks.ceph:pgid 65.1e last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.572 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.572 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.572 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.572 INFO:tasks.ceph:pgid 65.1d last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.572 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.573 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.573 INFO:tasks.ceph:pgid 66.1f last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.573 INFO:tasks.ceph:pgid 65.1c last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.573 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.573 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.573 INFO:tasks.ceph:pgid 66.18 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.574 INFO:tasks.ceph:pgid 65.1b last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.574 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.574 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.574 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.574 INFO:tasks.ceph:pgid 65.1a last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.574 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.575 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.575 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.575 INFO:tasks.ceph:pgid 65.19 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.575 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.575 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.575 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.576 INFO:tasks.ceph:pgid 65.18 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.576 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.576 INFO:tasks.ceph:pgid 67.15 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.576 INFO:tasks.ceph:pgid 66.14 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.576 INFO:tasks.ceph:pgid 65.17 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.576 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.577 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.577 INFO:tasks.ceph:pgid 66.15 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.577 INFO:tasks.ceph:pgid 65.16 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.577 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.577 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.577 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.578 INFO:tasks.ceph:pgid 65.15 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.578 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.578 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.578 INFO:tasks.ceph:pgid 66.17 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.578 INFO:tasks.ceph:pgid 65.14 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.578 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.579 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.579 INFO:tasks.ceph:pgid 66.10 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.579 INFO:tasks.ceph:pgid 65.13 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.579 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.579 INFO:tasks.ceph:pgid 67.10 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.579 INFO:tasks.ceph:pgid 66.11 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.580 INFO:tasks.ceph:pgid 65.12 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.580 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.580 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.580 INFO:tasks.ceph:pgid 66.12 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.580 INFO:tasks.ceph:pgid 65.11 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.580 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.581 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.581 INFO:tasks.ceph:pgid 66.13 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.581 INFO:tasks.ceph:pgid 65.10 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.581 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.581 INFO:tasks.ceph:pgid 67.d last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.581 INFO:tasks.ceph:pgid 66.c last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.582 INFO:tasks.ceph:pgid 65.f last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.582 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.582 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.582 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.583 INFO:tasks.ceph:pgid 65.e last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.583 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.583 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.583 INFO:tasks.ceph:pgid 66.e last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.583 INFO:tasks.ceph:pgid 65.d last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.583 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.584 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.584 INFO:tasks.ceph:pgid 66.f last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.584 INFO:tasks.ceph:pgid 65.c last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.584 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.584 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.584 INFO:tasks.ceph:pgid 66.8 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.585 INFO:tasks.ceph:pgid 65.b last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.585 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.585 INFO:tasks.ceph:pgid 67.8 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.585 INFO:tasks.ceph:pgid 66.9 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.585 INFO:tasks.ceph:pgid 65.a last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.585 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.586 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.586 INFO:tasks.ceph:pgid 66.a last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.586 INFO:tasks.ceph:pgid 65.9 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.586 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.586 INFO:tasks.ceph:pgid 67.a last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.586 INFO:tasks.ceph:pgid 66.b last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.587 INFO:tasks.ceph:pgid 65.8 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.587 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.587 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.587 INFO:tasks.ceph:pgid 66.4 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.587 INFO:tasks.ceph:pgid 65.7 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.588 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.588 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.588 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.588 INFO:tasks.ceph:pgid 65.6 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.588 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.588 INFO:tasks.ceph:pgid 67.7 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 65.5 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 66.7 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 65.4 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.589 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.590 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.590 INFO:tasks.ceph:pgid 66.0 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.590 INFO:tasks.ceph:pgid 65.3 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.590 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.590 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 66.1 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 65.2 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 66.2 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 65.1 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.591 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.592 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.592 INFO:tasks.ceph:pgid 66.3 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.592 INFO:tasks.ceph:pgid 65.0 last_scrub_stamp 2024-04-17T04:47:55.905428+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.592 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.592 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.592 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.593 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.594 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.594 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.594 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.594 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.594 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.594 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.595 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.595 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.595 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.595 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.595 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.595 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.596 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.596 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.596 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.596 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.596 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.596 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.597 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.598 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.598 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.598 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.598 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.598 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.598 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.599 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.599 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.599 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.599 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.599 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.599 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.600 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.600 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.600 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.600 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.600 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.600 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.601 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.602 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.602 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.602 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.602 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.602 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.603 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.604 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.604 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.604 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.604 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.604 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.604 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.605 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.606 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.606 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.606 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.606 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.606 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.606 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.607 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.607 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.607 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.607 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.607 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.607 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-17T04:36:11.058932+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=11, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.608 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.609 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.609 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.609 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.609 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.609 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.609 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.610 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.610 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.610 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.610 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.610 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.610 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.611 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.611 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.611 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.611 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.611 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.611 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.612 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.613 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.613 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.613 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.613 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.613 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.613 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.614 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.614 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.614 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.614 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.614 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.614 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.615 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.616 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.616 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.616 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.616 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.616 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.616 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.617 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.618 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.618 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.618 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.618 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.618 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.618 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.619 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.620 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.620 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.620 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.620 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.620 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.620 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.621 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.621 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.621 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.621 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.621 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.621 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.622 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.622 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.622 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.622 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.622 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.622 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.623 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.623 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.623 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.623 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.623 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.623 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.624 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.624 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.624 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.624 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.624 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.625 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.626 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.626 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.626 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.626 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.626 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.626 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.627 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.627 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.627 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.627 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.627 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.627 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.628 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.628 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.628 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.628 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.628 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.628 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.629 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.629 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.629 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.629 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.629 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.629 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.630 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.630 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.630 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.630 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.630 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.631 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.631 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.631 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.631 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.631 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.631 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.632 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.632 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.632 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.632 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.632 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.632 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.633 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.633 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.633 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.633 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.633 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.633 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.634 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.634 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.634 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.634 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.634 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.635 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.635 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.635 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:10.635 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T05:46:30.637 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:46:30.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:31.001 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:46:31.010 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2366,"stamp":"2024-04-17T05:46:29.297011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223466608,"num_objects":822,"num_object_clones":0,"num_object_copies":2530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":822,"num_whiteouts":0,"num_read":160833,"num_read_kb":2423241,"num_write":146364,"num_write_kb":2463359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":145162,"num_omap_keys":369,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13129,"ondisk_log_size":13129,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2028008,"kb_used_data":347624,"kb_used_omap":12,"kb_used_meta":1680307,"kb_avail":752946712,"statfs":{"total":773094113280,"available":771017433088,"internally_reserved":0,"allocated":355966976,"data_stored":340014674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1720634929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":141673,"num_omap_keys":360,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007617"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184309+0000","last_change":"2024-04-17T05:07:45.742645+0000","last_active":"2024-04-17T05:45:15.184309+0000","last_peered":"2024-04-17T05:45:15.184309+0000","last_clean":"2024-04-17T05:45:15.184309+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:45:15.184309+0000","last_undegraded":"2024-04-17T05:45:15.184309+0000","last_fullsized":"2024-04-17T05:45:15.184309+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.171659+0000","last_change":"2024-04-17T04:51:43.682048+0000","last_active":"2024-04-17T05:44:15.171659+0000","last_peered":"2024-04-17T05:44:15.171659+0000","last_clean":"2024-04-17T05:44:15.171659+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:44:15.171659+0000","last_undegraded":"2024-04-17T05:44:15.171659+0000","last_fullsized":"2024-04-17T05:44:15.171659+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.830796+0000","last_change":"2024-04-17T05:46:14.830930+0000","last_active":"2024-04-17T05:46:14.830796+0000","last_peered":"2024-04-17T05:46:14.830796+0000","last_clean":"2024-04-17T05:46:14.830796+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:46:14.830796+0000","last_undegraded":"2024-04-17T05:46:14.830796+0000","last_fullsized":"2024-04-17T05:46:14.830796+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.830571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:08.876068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":158,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.685785+0000","last_change":"2024-04-17T05:46:08.685785+0000","last_active":"2024-04-17T05:46:08.685785+0000","last_peered":"2024-04-17T05:46:08.685785+0000","last_clean":"2024-04-17T05:46:08.685785+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:46:08.685785+0000","last_undegraded":"2024-04-17T05:46:08.685785+0000","last_fullsized":"2024-04-17T05:46:08.685785+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.685550+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:05.034233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184686+0000","last_change":"2024-04-17T05:07:45.746586+0000","last_active":"2024-04-17T05:45:15.184686+0000","last_peered":"2024-04-17T05:45:15.184686+0000","last_clean":"2024-04-17T05:45:15.184686+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:45:15.184686+0000","last_undegraded":"2024-04-17T05:45:15.184686+0000","last_fullsized":"2024-04-17T05:45:15.184686+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'3","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.389714+0000","last_change":"2024-04-17T04:51:43.661871+0000","last_active":"2024-04-17T05:46:02.389714+0000","last_peered":"2024-04-17T05:46:02.389714+0000","last_clean":"2024-04-17T05:46:02.389714+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:46:02.389714+0000","last_undegraded":"2024-04-17T05:46:02.389714+0000","last_fullsized":"2024-04-17T05:46:02.389714+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.692604+0000","last_change":"2024-04-17T05:46:12.692660+0000","last_active":"2024-04-17T05:46:12.692604+0000","last_peered":"2024-04-17T05:46:12.692604+0000","last_clean":"2024-04-17T05:46:12.692604+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:46:12.692604+0000","last_undegraded":"2024-04-17T05:46:12.692604+0000","last_fullsized":"2024-04-17T05:46:12.692604+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.692409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:20.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'97","reported_seq":361,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.504742+0000","last_change":"2024-04-17T05:46:20.504828+0000","last_active":"2024-04-17T05:46:20.504742+0000","last_peered":"2024-04-17T05:46:20.504742+0000","last_clean":"2024-04-17T05:46:20.504742+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:46:20.504742+0000","last_undegraded":"2024-04-17T05:46:20.504742+0000","last_fullsized":"2024-04-17T05:46:20.504742+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.504541+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:15.125809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3493,"num_omap_keys":7,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.681194+0000","last_change":"2024-04-17T05:07:45.739412+0000","last_active":"2024-04-17T05:46:09.681194+0000","last_peered":"2024-04-17T05:46:09.681194+0000","last_clean":"2024-04-17T05:46:09.681194+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:46:09.681194+0000","last_undegraded":"2024-04-17T05:46:09.681194+0000","last_fullsized":"2024-04-17T05:46:09.681194+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.196040+0000","last_change":"2024-04-17T04:51:43.682620+0000","last_active":"2024-04-17T05:45:55.196040+0000","last_peered":"2024-04-17T05:45:55.196040+0000","last_clean":"2024-04-17T05:45:55.196040+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:45:55.196040+0000","last_undegraded":"2024-04-17T05:45:55.196040+0000","last_fullsized":"2024-04-17T05:45:55.196040+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'3","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.910939+0000","last_change":"2024-04-17T04:51:39.605291+0000","last_active":"2024-04-17T05:46:02.910939+0000","last_peered":"2024-04-17T05:46:02.910939+0000","last_clean":"2024-04-17T05:46:02.910939+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:46:02.910939+0000","last_undegraded":"2024-04-17T05:46:02.910939+0000","last_fullsized":"2024-04-17T05:46:02.910939+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":130,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.592867+0000","last_change":"2024-04-17T05:46:16.593030+0000","last_active":"2024-04-17T05:46:16.592867+0000","last_peered":"2024-04-17T05:46:16.592867+0000","last_clean":"2024-04-17T05:46:16.592867+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:46:16.592867+0000","last_undegraded":"2024-04-17T05:46:16.592867+0000","last_fullsized":"2024-04-17T05:46:16.592867+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_deep_scrub":"264'25","last_deep_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.592614+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:38.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.198100+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:44:05.198100+0000","last_peered":"2024-04-17T05:44:05.198100+0000","last_clean":"2024-04-17T05:44:05.198100+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:44:05.198100+0000","last_undegraded":"2024-04-17T05:44:05.198100+0000","last_fullsized":"2024-04-17T05:44:05.198100+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.225363+0000","last_change":"2024-04-17T04:51:43.676613+0000","last_active":"2024-04-17T05:45:55.225363+0000","last_peered":"2024-04-17T05:45:55.225363+0000","last_clean":"2024-04-17T05:45:55.225363+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:45:55.225363+0000","last_undegraded":"2024-04-17T05:45:55.225363+0000","last_fullsized":"2024-04-17T05:45:55.225363+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.770225+0000","last_change":"2024-04-17T05:46:20.770279+0000","last_active":"2024-04-17T05:46:20.770225+0000","last_peered":"2024-04-17T05:46:20.770225+0000","last_clean":"2024-04-17T05:46:20.770225+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:46:20.770225+0000","last_undegraded":"2024-04-17T05:46:20.770225+0000","last_fullsized":"2024-04-17T05:46:20.770225+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.770027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:10.097683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":169,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.814048+0000","last_change":"2024-04-17T05:46:08.814070+0000","last_active":"2024-04-17T05:46:08.814048+0000","last_peered":"2024-04-17T05:46:08.814048+0000","last_clean":"2024-04-17T05:46:08.814048+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:46:08.814048+0000","last_undegraded":"2024-04-17T05:46:08.814048+0000","last_fullsized":"2024-04-17T05:46:08.814048+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'42","last_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_deep_scrub":"264'42","last_deep_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.813969+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:41.013929+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":32,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.224140+0000","last_change":"2024-04-17T05:07:45.745164+0000","last_active":"2024-04-17T05:45:50.224140+0000","last_peered":"2024-04-17T05:45:50.224140+0000","last_clean":"2024-04-17T05:45:50.224140+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:45:50.224140+0000","last_undegraded":"2024-04-17T05:45:50.224140+0000","last_fullsized":"2024-04-17T05:45:50.224140+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'2","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.557348+0000","last_change":"2024-04-17T04:51:43.666389+0000","last_active":"2024-04-17T05:46:03.557348+0000","last_peered":"2024-04-17T05:46:03.557348+0000","last_clean":"2024-04-17T05:46:03.557348+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:46:03.557348+0000","last_undegraded":"2024-04-17T05:46:03.557348+0000","last_fullsized":"2024-04-17T05:46:03.557348+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.720633+0000","last_change":"2024-04-17T05:46:07.720653+0000","last_active":"2024-04-17T05:46:07.720633+0000","last_peered":"2024-04-17T05:46:07.720633+0000","last_clean":"2024-04-17T05:46:07.720633+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:46:07.720633+0000","last_undegraded":"2024-04-17T05:46:07.720633+0000","last_fullsized":"2024-04-17T05:46:07.720633+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.720559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:04.229433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":209,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.804847+0000","last_change":"2024-04-17T05:46:09.804897+0000","last_active":"2024-04-17T05:46:09.804847+0000","last_peered":"2024-04-17T05:46:09.804847+0000","last_clean":"2024-04-17T05:46:09.804847+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:46:09.804847+0000","last_undegraded":"2024-04-17T05:46:09.804847+0000","last_fullsized":"2024-04-17T05:46:09.804847+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.804697+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:40.852058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.189448+0000","last_change":"2024-04-17T05:07:45.754178+0000","last_active":"2024-04-17T05:45:15.189448+0000","last_peered":"2024-04-17T05:45:15.189448+0000","last_clean":"2024-04-17T05:45:15.189448+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:45:15.189448+0000","last_undegraded":"2024-04-17T05:45:15.189448+0000","last_fullsized":"2024-04-17T05:45:15.189448+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'6","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.389267+0000","last_change":"2024-04-17T04:51:43.664767+0000","last_active":"2024-04-17T05:46:02.389267+0000","last_peered":"2024-04-17T05:46:02.389267+0000","last_clean":"2024-04-17T05:46:02.389267+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:46:02.389267+0000","last_undegraded":"2024-04-17T05:46:02.389267+0000","last_fullsized":"2024-04-17T05:46:02.389267+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":22528,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"264'6","reported_seq":75,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.770136+0000","last_change":"2024-04-17T05:46:12.770216+0000","last_active":"2024-04-17T05:46:12.770136+0000","last_peered":"2024-04-17T05:46:12.770136+0000","last_clean":"2024-04-17T05:46:12.770136+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:46:12.770136+0000","last_undegraded":"2024-04-17T05:46:12.770136+0000","last_fullsized":"2024-04-17T05:46:12.770136+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.769907+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:23.154229+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'75","reported_seq":293,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.023607+0000","last_change":"2024-04-17T05:46:12.023675+0000","last_active":"2024-04-17T05:46:12.023607+0000","last_peered":"2024-04-17T05:46:12.023607+0000","last_clean":"2024-04-17T05:46:12.023607+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:46:12.023607+0000","last_undegraded":"2024-04-17T05:46:12.023607+0000","last_fullsized":"2024-04-17T05:46:12.023607+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'75","last_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_deep_scrub":"264'75","last_deep_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.023402+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.097898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3255,"num_omap_keys":7,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179323+0000","last_change":"2024-04-17T05:07:45.754885+0000","last_active":"2024-04-17T05:45:15.179323+0000","last_peered":"2024-04-17T05:45:15.179323+0000","last_clean":"2024-04-17T05:45:15.179323+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:45:15.179323+0000","last_undegraded":"2024-04-17T05:45:15.179323+0000","last_fullsized":"2024-04-17T05:45:15.179323+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166817+0000","last_change":"2024-04-17T04:51:43.664884+0000","last_active":"2024-04-17T05:44:15.166817+0000","last_peered":"2024-04-17T05:44:15.166817+0000","last_clean":"2024-04-17T05:44:15.166817+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:44:15.166817+0000","last_undegraded":"2024-04-17T05:44:15.166817+0000","last_fullsized":"2024-04-17T05:44:15.166817+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'9","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.913911+0000","last_change":"2024-04-17T04:51:39.621614+0000","last_active":"2024-04-17T05:46:02.913911+0000","last_peered":"2024-04-17T05:46:02.913911+0000","last_clean":"2024-04-17T05:46:02.913911+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:46:02.913911+0000","last_undegraded":"2024-04-17T05:46:02.913911+0000","last_fullsized":"2024-04-17T05:46:02.913911+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":16384,"num_write":24,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":84,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.595282+0000","last_change":"2024-04-17T05:46:14.595378+0000","last_active":"2024-04-17T05:46:14.595282+0000","last_peered":"2024-04-17T05:46:14.595282+0000","last_clean":"2024-04-17T05:46:14.595282+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:46:14.595282+0000","last_undegraded":"2024-04-17T05:46:14.595282+0000","last_fullsized":"2024-04-17T05:46:14.595282+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.594970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:05:26.805897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179493+0000","last_change":"2024-04-17T05:07:45.757519+0000","last_active":"2024-04-17T05:45:15.179493+0000","last_peered":"2024-04-17T05:45:15.179493+0000","last_clean":"2024-04-17T05:45:15.179493+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:45:15.179493+0000","last_undegraded":"2024-04-17T05:45:15.179493+0000","last_fullsized":"2024-04-17T05:45:15.179493+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.201004+0000","last_change":"2024-04-17T04:51:43.686621+0000","last_active":"2024-04-17T05:44:15.201004+0000","last_peered":"2024-04-17T05:44:15.201004+0000","last_clean":"2024-04-17T05:44:15.201004+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:44:15.201004+0000","last_undegraded":"2024-04-17T05:44:15.201004+0000","last_fullsized":"2024-04-17T05:44:15.201004+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.167493+0000","last_change":"2024-04-17T04:51:39.643524+0000","last_active":"2024-04-17T05:44:10.167493+0000","last_peered":"2024-04-17T05:44:10.167493+0000","last_clean":"2024-04-17T05:44:10.167493+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:44:10.167493+0000","last_undegraded":"2024-04-17T05:44:10.167493+0000","last_fullsized":"2024-04-17T05:44:10.167493+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":99,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.712592+0000","last_change":"2024-04-17T05:46:11.712658+0000","last_active":"2024-04-17T05:46:11.712592+0000","last_peered":"2024-04-17T05:46:11.712592+0000","last_clean":"2024-04-17T05:46:11.712592+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:46:11.712592+0000","last_undegraded":"2024-04-17T05:46:11.712592+0000","last_fullsized":"2024-04-17T05:46:11.712592+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'13","last_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_deep_scrub":"264'13","last_deep_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.712398+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:12.832777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.186907+0000","last_change":"2024-04-17T05:07:45.743105+0000","last_active":"2024-04-17T05:45:05.186907+0000","last_peered":"2024-04-17T05:45:05.186907+0000","last_clean":"2024-04-17T05:45:05.186907+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:45:05.186907+0000","last_undegraded":"2024-04-17T05:45:05.186907+0000","last_fullsized":"2024-04-17T05:45:05.186907+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.794712+0000","last_change":"2024-04-17T05:46:18.794764+0000","last_active":"2024-04-17T05:46:18.794712+0000","last_peered":"2024-04-17T05:46:18.794712+0000","last_clean":"2024-04-17T05:46:18.794712+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:46:18.794712+0000","last_undegraded":"2024-04-17T05:46:18.794712+0000","last_fullsized":"2024-04-17T05:46:18.794712+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.794517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:29:12.950953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.225008+0000","last_change":"2024-04-17T05:46:24.225059+0000","last_active":"2024-04-17T05:46:24.225008+0000","last_peered":"2024-04-17T05:46:24.225008+0000","last_clean":"2024-04-17T05:46:24.225008+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:46:24.225008+0000","last_undegraded":"2024-04-17T05:46:24.225008+0000","last_fullsized":"2024-04-17T05:46:24.225008+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.224809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:16.454241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":89,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.623954+0000","last_change":"2024-04-17T05:46:12.624024+0000","last_active":"2024-04-17T05:46:12.623954+0000","last_peered":"2024-04-17T05:46:12.623954+0000","last_clean":"2024-04-17T05:46:12.623954+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:46:12.623954+0000","last_undegraded":"2024-04-17T05:46:12.623954+0000","last_fullsized":"2024-04-17T05:46:12.623954+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.623741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:56:20.441738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.181969+0000","last_change":"2024-04-17T05:07:45.738001+0000","last_active":"2024-04-17T05:45:05.181969+0000","last_peered":"2024-04-17T05:45:05.181969+0000","last_clean":"2024-04-17T05:45:05.181969+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:45:05.181969+0000","last_undegraded":"2024-04-17T05:45:05.181969+0000","last_fullsized":"2024-04-17T05:45:05.181969+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:35.220513+0000","last_change":"2024-04-17T04:51:43.682839+0000","last_active":"2024-04-17T05:45:35.220513+0000","last_peered":"2024-04-17T05:45:35.220513+0000","last_clean":"2024-04-17T05:45:35.220513+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:45:35.220513+0000","last_undegraded":"2024-04-17T05:45:35.220513+0000","last_fullsized":"2024-04-17T05:45:35.220513+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'2","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.129624+0000","last_change":"2024-04-17T05:46:11.129787+0000","last_active":"2024-04-17T05:46:11.129624+0000","last_peered":"2024-04-17T05:46:11.129624+0000","last_clean":"2024-04-17T05:46:11.129624+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:46:11.129624+0000","last_undegraded":"2024-04-17T05:46:11.129624+0000","last_fullsized":"2024-04-17T05:46:11.129624+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.129393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:45.300258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":125,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.852514+0000","last_change":"2024-04-17T05:46:11.852573+0000","last_active":"2024-04-17T05:46:11.852514+0000","last_peered":"2024-04-17T05:46:11.852514+0000","last_clean":"2024-04-17T05:46:11.852514+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:46:11.852514+0000","last_undegraded":"2024-04-17T05:46:11.852514+0000","last_fullsized":"2024-04-17T05:46:11.852514+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.852327+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:08.854562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.176972+0000","last_change":"2024-04-17T05:07:45.747162+0000","last_active":"2024-04-17T05:45:05.176972+0000","last_peered":"2024-04-17T05:45:05.176972+0000","last_clean":"2024-04-17T05:45:05.176972+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:45:05.176972+0000","last_undegraded":"2024-04-17T05:45:05.176972+0000","last_fullsized":"2024-04-17T05:45:05.176972+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'5","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.734077+0000","last_change":"2024-04-17T05:46:22.734137+0000","last_active":"2024-04-17T05:46:22.734077+0000","last_peered":"2024-04-17T05:46:22.734077+0000","last_clean":"2024-04-17T05:46:22.734077+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:46:22.734077+0000","last_undegraded":"2024-04-17T05:46:22.734077+0000","last_fullsized":"2024-04-17T05:46:22.734077+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.733832+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:04.087293+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":48,"num_write":10,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'13","reported_seq":82,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.607134+0000","last_change":"2024-04-17T04:51:39.612849+0000","last_active":"2024-04-17T05:46:02.607134+0000","last_peered":"2024-04-17T05:46:02.607134+0000","last_clean":"2024-04-17T05:46:02.607134+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:46:02.607134+0000","last_undegraded":"2024-04-17T05:46:02.607134+0000","last_fullsized":"2024-04-17T05:46:02.607134+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":46082,"num_write":17,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.547405+0000","last_change":"2024-04-17T05:46:18.547463+0000","last_active":"2024-04-17T05:46:18.547405+0000","last_peered":"2024-04-17T05:46:18.547405+0000","last_clean":"2024-04-17T05:46:18.547405+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:46:18.547405+0000","last_undegraded":"2024-04-17T05:46:18.547405+0000","last_fullsized":"2024-04-17T05:46:18.547405+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'18","last_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_deep_scrub":"261'18","last_deep_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.547198+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:22.594305+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.182145+0000","last_change":"2024-04-17T05:07:45.744163+0000","last_active":"2024-04-17T05:45:05.182145+0000","last_peered":"2024-04-17T05:45:05.182145+0000","last_clean":"2024-04-17T05:45:05.182145+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:45:05.182145+0000","last_undegraded":"2024-04-17T05:45:05.182145+0000","last_fullsized":"2024-04-17T05:45:05.182145+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'11","reported_seq":73,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.556647+0000","last_change":"2024-04-17T04:51:43.667351+0000","last_active":"2024-04-17T05:46:03.556647+0000","last_peered":"2024-04-17T05:46:03.556647+0000","last_clean":"2024-04-17T05:46:03.556647+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:46:03.556647+0000","last_undegraded":"2024-04-17T05:46:03.556647+0000","last_fullsized":"2024-04-17T05:46:03.556647+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":37890,"num_write":20,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.607535+0000","last_change":"2024-04-17T05:46:23.607603+0000","last_active":"2024-04-17T05:46:23.607535+0000","last_peered":"2024-04-17T05:46:23.607535+0000","last_clean":"2024-04-17T05:46:23.607535+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:46:23.607535+0000","last_undegraded":"2024-04-17T05:46:23.607535+0000","last_fullsized":"2024-04-17T05:46:23.607535+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.607294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:20.141964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":220,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.194708+0000","last_change":"2024-04-17T05:46:18.194774+0000","last_active":"2024-04-17T05:46:18.194708+0000","last_peered":"2024-04-17T05:46:18.194708+0000","last_clean":"2024-04-17T05:46:18.194708+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:46:18.194708+0000","last_undegraded":"2024-04-17T05:46:18.194708+0000","last_fullsized":"2024-04-17T05:46:18.194708+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'65","last_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_deep_scrub":"264'65","last_deep_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.194519+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:31.405244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.277147+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:46:08.277147+0000","last_peered":"2024-04-17T05:46:08.277147+0000","last_clean":"2024-04-17T05:46:08.277147+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:46:08.277147+0000","last_undegraded":"2024-04-17T05:46:08.277147+0000","last_fullsized":"2024-04-17T05:46:08.277147+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'5","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:25.191685+0000","last_change":"2024-04-17T04:51:43.670407+0000","last_active":"2024-04-17T05:45:25.191685+0000","last_peered":"2024-04-17T05:45:25.191685+0000","last_clean":"2024-04-17T05:45:25.191685+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:45:25.191685+0000","last_undegraded":"2024-04-17T05:45:25.191685+0000","last_fullsized":"2024-04-17T05:45:25.191685+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.807390+0000","last_change":"2024-04-17T05:46:16.807574+0000","last_active":"2024-04-17T05:46:16.807390+0000","last_peered":"2024-04-17T05:46:16.807390+0000","last_clean":"2024-04-17T05:46:16.807390+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:46:16.807390+0000","last_undegraded":"2024-04-17T05:46:16.807390+0000","last_fullsized":"2024-04-17T05:46:16.807390+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.807094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:57.818718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":157,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.309760+0000","last_change":"2024-04-17T05:46:22.309837+0000","last_active":"2024-04-17T05:46:22.309760+0000","last_peered":"2024-04-17T05:46:22.309760+0000","last_clean":"2024-04-17T05:46:22.309760+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:46:22.309760+0000","last_undegraded":"2024-04-17T05:46:22.309760+0000","last_fullsized":"2024-04-17T05:46:22.309760+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.309548+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:29.733950+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.936071+0000","last_change":"2024-04-17T05:07:45.749757+0000","last_active":"2024-04-17T05:46:06.936071+0000","last_peered":"2024-04-17T05:46:06.936071+0000","last_clean":"2024-04-17T05:46:06.936071+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:46:06.936071+0000","last_undegraded":"2024-04-17T05:46:06.936071+0000","last_fullsized":"2024-04-17T05:46:06.936071+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'2","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.726875+0000","last_change":"2024-04-17T05:46:16.727020+0000","last_active":"2024-04-17T05:46:16.726875+0000","last_peered":"2024-04-17T05:46:16.726875+0000","last_clean":"2024-04-17T05:46:16.726875+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:46:16.726875+0000","last_undegraded":"2024-04-17T05:46:16.726875+0000","last_fullsized":"2024-04-17T05:46:16.726875+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.726644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:09.817384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.831759+0000","last_change":"2024-04-17T05:46:10.831811+0000","last_active":"2024-04-17T05:46:10.831759+0000","last_peered":"2024-04-17T05:46:10.831759+0000","last_clean":"2024-04-17T05:46:10.831759+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:46:10.831759+0000","last_undegraded":"2024-04-17T05:46:10.831759+0000","last_fullsized":"2024-04-17T05:46:10.831759+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.831558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:46.336010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":142,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.670845+0000","last_change":"2024-04-17T05:46:09.670845+0000","last_active":"2024-04-17T05:46:09.670845+0000","last_peered":"2024-04-17T05:46:09.670845+0000","last_clean":"2024-04-17T05:46:09.670845+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:46:09.670845+0000","last_undegraded":"2024-04-17T05:46:09.670845+0000","last_fullsized":"2024-04-17T05:46:09.670845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.670593+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:59.152267+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"263'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184793+0000","last_change":"2024-04-17T05:07:45.748765+0000","last_active":"2024-04-17T05:45:15.184793+0000","last_peered":"2024-04-17T05:45:15.184793+0000","last_clean":"2024-04-17T05:45:15.184793+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:45:15.184793+0000","last_undegraded":"2024-04-17T05:45:15.184793+0000","last_fullsized":"2024-04-17T05:45:15.184793+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'3","reported_seq":52,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.856324+0000","last_change":"2024-04-17T04:51:43.684076+0000","last_active":"2024-04-17T05:46:01.856324+0000","last_peered":"2024-04-17T05:46:01.856324+0000","last_clean":"2024-04-17T05:46:01.856324+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:46:01.856324+0000","last_undegraded":"2024-04-17T05:46:01.856324+0000","last_fullsized":"2024-04-17T05:46:01.856324+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'5","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.032735+0000","last_change":"2024-04-17T05:46:14.032796+0000","last_active":"2024-04-17T05:46:14.032735+0000","last_peered":"2024-04-17T05:46:14.032735+0000","last_clean":"2024-04-17T05:46:14.032735+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:46:14.032735+0000","last_undegraded":"2024-04-17T05:46:14.032735+0000","last_fullsized":"2024-04-17T05:46:14.032735+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.032549+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:46.453310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":111,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.793130+0000","last_change":"2024-04-17T05:46:09.793130+0000","last_active":"2024-04-17T05:46:09.793130+0000","last_peered":"2024-04-17T05:46:09.793130+0000","last_clean":"2024-04-17T05:46:09.793130+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:46:09.793130+0000","last_undegraded":"2024-04-17T05:46:09.793130+0000","last_fullsized":"2024-04-17T05:46:09.793130+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.792865+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:12:51.707714+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.277236+0000","last_change":"2024-04-17T05:07:45.741199+0000","last_active":"2024-04-17T05:46:08.277236+0000","last_peered":"2024-04-17T05:46:08.277236+0000","last_clean":"2024-04-17T05:46:08.277236+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:46:08.277236+0000","last_undegraded":"2024-04-17T05:46:08.277236+0000","last_fullsized":"2024-04-17T05:46:08.277236+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:25.182378+0000","last_change":"2024-04-17T04:51:43.673701+0000","last_active":"2024-04-17T05:45:25.182378+0000","last_peered":"2024-04-17T05:45:25.182378+0000","last_clean":"2024-04-17T05:45:25.182378+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:45:25.182378+0000","last_undegraded":"2024-04-17T05:45:25.182378+0000","last_fullsized":"2024-04-17T05:45:25.182378+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.134683+0000","last_change":"2024-04-17T05:46:20.134745+0000","last_active":"2024-04-17T05:46:20.134683+0000","last_peered":"2024-04-17T05:46:20.134683+0000","last_clean":"2024-04-17T05:46:20.134683+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:46:20.134683+0000","last_undegraded":"2024-04-17T05:46:20.134683+0000","last_fullsized":"2024-04-17T05:46:20.134683+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.134488+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:48.392260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.861464+0000","last_change":"2024-04-17T05:46:13.861464+0000","last_active":"2024-04-17T05:46:13.861464+0000","last_peered":"2024-04-17T05:46:13.861464+0000","last_clean":"2024-04-17T05:46:13.861464+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:46:13.861464+0000","last_undegraded":"2024-04-17T05:46:13.861464+0000","last_fullsized":"2024-04-17T05:46:13.861464+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'55","last_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_deep_scrub":"264'55","last_deep_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.861056+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:57.160789+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:45.177353+0000","last_change":"2024-04-17T05:07:45.745113+0000","last_active":"2024-04-17T05:44:45.177353+0000","last_peered":"2024-04-17T05:44:45.177353+0000","last_clean":"2024-04-17T05:44:45.177353+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:44:45.177353+0000","last_undegraded":"2024-04-17T05:44:45.177353+0000","last_fullsized":"2024-04-17T05:44:45.177353+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'6","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.212731+0000","last_change":"2024-04-17T04:51:43.682734+0000","last_active":"2024-04-17T05:46:03.212731+0000","last_peered":"2024-04-17T05:46:03.212731+0000","last_clean":"2024-04-17T05:46:03.212731+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:46:03.212731+0000","last_undegraded":"2024-04-17T05:46:03.212731+0000","last_fullsized":"2024-04-17T05:46:03.212731+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":11,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.170964+0000","last_change":"2024-04-17T05:46:10.171028+0000","last_active":"2024-04-17T05:46:10.170964+0000","last_peered":"2024-04-17T05:46:10.170964+0000","last_clean":"2024-04-17T05:46:10.170964+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:46:10.170964+0000","last_undegraded":"2024-04-17T05:46:10.170964+0000","last_fullsized":"2024-04-17T05:46:10.170964+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.170766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:36.494972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":142,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.881071+0000","last_change":"2024-04-17T05:46:12.881133+0000","last_active":"2024-04-17T05:46:12.881071+0000","last_peered":"2024-04-17T05:46:12.881071+0000","last_clean":"2024-04-17T05:46:12.881071+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:46:12.881071+0000","last_undegraded":"2024-04-17T05:46:12.881071+0000","last_fullsized":"2024-04-17T05:46:12.881071+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.880856+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:56.211415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.189702+0000","last_change":"2024-04-17T05:07:45.751549+0000","last_active":"2024-04-17T05:46:00.189702+0000","last_peered":"2024-04-17T05:46:00.189702+0000","last_clean":"2024-04-17T05:46:00.189702+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:46:00.189702+0000","last_undegraded":"2024-04-17T05:46:00.189702+0000","last_fullsized":"2024-04-17T05:46:00.189702+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'5","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.460050+0000","last_change":"2024-04-17T04:51:43.688891+0000","last_active":"2024-04-17T05:46:01.460050+0000","last_peered":"2024-04-17T05:46:01.460050+0000","last_clean":"2024-04-17T05:46:01.460050+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:46:01.460050+0000","last_undegraded":"2024-04-17T05:46:01.460050+0000","last_fullsized":"2024-04-17T05:46:01.460050+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.667956+0000","last_change":"2024-04-17T04:51:39.634707+0000","last_active":"2024-04-17T05:46:01.667956+0000","last_peered":"2024-04-17T05:46:01.667956+0000","last_clean":"2024-04-17T05:46:01.667956+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:46:01.667956+0000","last_undegraded":"2024-04-17T05:46:01.667956+0000","last_fullsized":"2024-04-17T05:46:01.667956+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":11264,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":162,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.131226+0000","last_change":"2024-04-17T05:46:16.131285+0000","last_active":"2024-04-17T05:46:16.131226+0000","last_peered":"2024-04-17T05:46:16.131226+0000","last_clean":"2024-04-17T05:46:16.131226+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:46:16.131226+0000","last_undegraded":"2024-04-17T05:46:16.131226+0000","last_fullsized":"2024-04-17T05:46:16.131226+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.131038+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:32:33.132984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:35.171505+0000","last_change":"2024-04-17T05:07:45.741143+0000","last_active":"2024-04-17T05:44:35.171505+0000","last_peered":"2024-04-17T05:44:35.171505+0000","last_clean":"2024-04-17T05:44:35.171505+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:44:35.171505+0000","last_undegraded":"2024-04-17T05:44:35.171505+0000","last_fullsized":"2024-04-17T05:44:35.171505+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.198648+0000","last_change":"2024-04-17T04:51:43.657675+0000","last_active":"2024-04-17T05:45:55.198648+0000","last_peered":"2024-04-17T05:45:55.198648+0000","last_clean":"2024-04-17T05:45:55.198648+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:45:55.198648+0000","last_undegraded":"2024-04-17T05:45:55.198648+0000","last_fullsized":"2024-04-17T05:45:55.198648+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.674045+0000","last_change":"2024-04-17T05:46:19.674109+0000","last_active":"2024-04-17T05:46:19.674045+0000","last_peered":"2024-04-17T05:46:19.674045+0000","last_clean":"2024-04-17T05:46:19.674045+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:46:19.674045+0000","last_undegraded":"2024-04-17T05:46:19.674045+0000","last_fullsized":"2024-04-17T05:46:19.674045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.673832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:20:55.342567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":132,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.056940+0000","last_change":"2024-04-17T05:46:11.057017+0000","last_active":"2024-04-17T05:46:11.056940+0000","last_peered":"2024-04-17T05:46:11.056940+0000","last_clean":"2024-04-17T05:46:11.056940+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:46:11.056940+0000","last_undegraded":"2024-04-17T05:46:11.056940+0000","last_fullsized":"2024-04-17T05:46:11.056940+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.056723+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:04.980611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:35.170666+0000","last_change":"2024-04-17T05:07:45.757127+0000","last_active":"2024-04-17T05:44:35.170666+0000","last_peered":"2024-04-17T05:44:35.170666+0000","last_clean":"2024-04-17T05:44:35.170666+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:44:35.170666+0000","last_undegraded":"2024-04-17T05:44:35.170666+0000","last_fullsized":"2024-04-17T05:44:35.170666+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:05.198230+0000","last_change":"2024-04-17T04:52:04.726201+0000","last_active":"2024-04-17T05:46:05.198230+0000","last_peered":"2024-04-17T05:46:05.198230+0000","last_clean":"2024-04-17T05:46:05.198230+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:46:05.198230+0000","last_undegraded":"2024-04-17T05:46:05.198230+0000","last_fullsized":"2024-04-17T05:46:05.198230+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.680401+0000","last_change":"2024-04-17T05:46:08.680438+0000","last_active":"2024-04-17T05:46:08.680401+0000","last_peered":"2024-04-17T05:46:08.680401+0000","last_clean":"2024-04-17T05:46:08.680401+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:46:08.680401+0000","last_undegraded":"2024-04-17T05:46:08.680401+0000","last_fullsized":"2024-04-17T05:46:08.680401+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.680268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:03.743450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":137,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.494274+0000","last_change":"2024-04-17T05:46:22.494346+0000","last_active":"2024-04-17T05:46:22.494274+0000","last_peered":"2024-04-17T05:46:22.494274+0000","last_clean":"2024-04-17T05:46:22.494274+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:46:22.494274+0000","last_undegraded":"2024-04-17T05:46:22.494274+0000","last_fullsized":"2024-04-17T05:46:22.494274+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_deep_scrub":"264'30","last_deep_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.494058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:08.498830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.226630+0000","last_change":"2024-04-17T05:07:45.744264+0000","last_active":"2024-04-17T05:46:00.226630+0000","last_peered":"2024-04-17T05:46:00.226630+0000","last_clean":"2024-04-17T05:46:00.226630+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:46:00.226630+0000","last_undegraded":"2024-04-17T05:46:00.226630+0000","last_fullsized":"2024-04-17T05:46:00.226630+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'7","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.833881+0000","last_change":"2024-04-17T04:51:43.661076+0000","last_active":"2024-04-17T05:46:02.833881+0000","last_peered":"2024-04-17T05:46:02.833881+0000","last_clean":"2024-04-17T05:46:02.833881+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:46:02.833881+0000","last_undegraded":"2024-04-17T05:46:02.833881+0000","last_fullsized":"2024-04-17T05:46:02.833881+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":5216,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.623849+0000","last_change":"2024-04-17T05:46:11.623908+0000","last_active":"2024-04-17T05:46:11.623849+0000","last_peered":"2024-04-17T05:46:11.623849+0000","last_clean":"2024-04-17T05:46:11.623849+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:46:11.623849+0000","last_undegraded":"2024-04-17T05:46:11.623849+0000","last_fullsized":"2024-04-17T05:46:11.623849+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.623642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:23.552160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":111,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.642003+0000","last_change":"2024-04-17T05:46:10.642068+0000","last_active":"2024-04-17T05:46:10.642003+0000","last_peered":"2024-04-17T05:46:10.642003+0000","last_clean":"2024-04-17T05:46:10.642003+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:46:10.642003+0000","last_undegraded":"2024-04-17T05:46:10.642003+0000","last_fullsized":"2024-04-17T05:46:10.642003+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.641783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:09.385082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:05.191010+0000","last_change":"2024-04-17T05:07:45.757192+0000","last_active":"2024-04-17T05:46:05.191010+0000","last_peered":"2024-04-17T05:46:05.191010+0000","last_clean":"2024-04-17T05:46:05.191010+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:46:05.191010+0000","last_undegraded":"2024-04-17T05:46:05.191010+0000","last_fullsized":"2024-04-17T05:46:05.191010+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.167379+0000","last_change":"2024-04-17T05:46:16.167438+0000","last_active":"2024-04-17T05:46:16.167379+0000","last_peered":"2024-04-17T05:46:16.167379+0000","last_clean":"2024-04-17T05:46:16.167379+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:46:16.167379+0000","last_undegraded":"2024-04-17T05:46:16.167379+0000","last_fullsized":"2024-04-17T05:46:16.167379+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.167190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:00.500071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.682742+0000","last_change":"2024-04-17T05:46:07.682778+0000","last_active":"2024-04-17T05:46:07.682742+0000","last_peered":"2024-04-17T05:46:07.682742+0000","last_clean":"2024-04-17T05:46:07.682742+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:46:07.682742+0000","last_undegraded":"2024-04-17T05:46:07.682742+0000","last_fullsized":"2024-04-17T05:46:07.682742+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.682661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:07.614824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":295,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.265842+0000","last_change":"2024-04-17T05:46:21.265908+0000","last_active":"2024-04-17T05:46:21.265842+0000","last_peered":"2024-04-17T05:46:21.265842+0000","last_clean":"2024-04-17T05:46:21.265842+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:46:21.265842+0000","last_undegraded":"2024-04-17T05:46:21.265842+0000","last_fullsized":"2024-04-17T05:46:21.265842+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'93","last_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_deep_scrub":"264'93","last_deep_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.265637+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:31.736633+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'1","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.189770+0000","last_change":"2024-04-17T05:07:45.754827+0000","last_active":"2024-04-17T05:46:00.189770+0000","last_peered":"2024-04-17T05:46:00.189770+0000","last_clean":"2024-04-17T05:46:00.189770+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:46:00.189770+0000","last_undegraded":"2024-04-17T05:46:00.189770+0000","last_fullsized":"2024-04-17T05:46:00.189770+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":47,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:45.196312+0000","last_change":"2024-04-17T04:51:43.655115+0000","last_active":"2024-04-17T05:45:45.196312+0000","last_peered":"2024-04-17T05:45:45.196312+0000","last_clean":"2024-04-17T05:45:45.196312+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:45:45.196312+0000","last_undegraded":"2024-04-17T05:45:45.196312+0000","last_fullsized":"2024-04-17T05:45:45.196312+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.584946+0000","last_change":"2024-04-17T05:46:15.585004+0000","last_active":"2024-04-17T05:46:15.584946+0000","last_peered":"2024-04-17T05:46:15.584946+0000","last_clean":"2024-04-17T05:46:15.584946+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:46:15.584946+0000","last_undegraded":"2024-04-17T05:46:15.584946+0000","last_fullsized":"2024-04-17T05:46:15.584946+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.584695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:18.890015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":115,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.833889+0000","last_change":"2024-04-17T05:46:15.833951+0000","last_active":"2024-04-17T05:46:15.833889+0000","last_peered":"2024-04-17T05:46:15.833889+0000","last_clean":"2024-04-17T05:46:15.833889+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:46:15.833889+0000","last_undegraded":"2024-04-17T05:46:15.833889+0000","last_fullsized":"2024-04-17T05:46:15.833889+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'22","last_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_deep_scrub":"264'22","last_deep_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.833697+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:34:19.233513+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.197261+0000","last_change":"2024-04-17T05:07:45.739696+0000","last_active":"2024-04-17T05:46:00.197261+0000","last_peered":"2024-04-17T05:46:00.197261+0000","last_clean":"2024-04-17T05:46:00.197261+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:46:00.197261+0000","last_undegraded":"2024-04-17T05:46:00.197261+0000","last_fullsized":"2024-04-17T05:46:00.197261+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.268742+0000","last_change":"2024-04-17T05:46:20.268796+0000","last_active":"2024-04-17T05:46:20.268742+0000","last_peered":"2024-04-17T05:46:20.268742+0000","last_clean":"2024-04-17T05:46:20.268742+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:46:20.268742+0000","last_undegraded":"2024-04-17T05:46:20.268742+0000","last_fullsized":"2024-04-17T05:46:20.268742+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.268538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:35.180807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.745329+0000","last_change":"2024-04-17T05:46:24.745393+0000","last_active":"2024-04-17T05:46:24.745329+0000","last_peered":"2024-04-17T05:46:24.745329+0000","last_clean":"2024-04-17T05:46:24.745329+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:46:24.745329+0000","last_undegraded":"2024-04-17T05:46:24.745329+0000","last_fullsized":"2024-04-17T05:46:24.745329+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.745126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:13.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":197,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.022491+0000","last_change":"2024-04-17T05:46:10.022550+0000","last_active":"2024-04-17T05:46:10.022491+0000","last_peered":"2024-04-17T05:46:10.022491+0000","last_clean":"2024-04-17T05:46:10.022491+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:46:10.022491+0000","last_undegraded":"2024-04-17T05:46:10.022491+0000","last_fullsized":"2024-04-17T05:46:10.022491+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'52","last_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_deep_scrub":"264'52","last_deep_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.022290+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.993887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179225+0000","last_change":"2024-04-17T05:07:45.747152+0000","last_active":"2024-04-17T05:45:15.179225+0000","last_peered":"2024-04-17T05:45:15.179225+0000","last_clean":"2024-04-17T05:45:15.179225+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:45:15.179225+0000","last_undegraded":"2024-04-17T05:45:15.179225+0000","last_fullsized":"2024-04-17T05:45:15.179225+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:05.198205+0000","last_change":"2024-04-17T04:51:43.682823+0000","last_active":"2024-04-17T05:46:05.198205+0000","last_peered":"2024-04-17T05:46:05.198205+0000","last_clean":"2024-04-17T05:46:05.198205+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:46:05.198205+0000","last_undegraded":"2024-04-17T05:46:05.198205+0000","last_fullsized":"2024-04-17T05:46:05.198205+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'8","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.163258+0000","last_change":"2024-04-17T05:46:15.163324+0000","last_active":"2024-04-17T05:46:15.163258+0000","last_peered":"2024-04-17T05:46:15.163258+0000","last_clean":"2024-04-17T05:46:15.163258+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:46:15.163258+0000","last_undegraded":"2024-04-17T05:46:15.163258+0000","last_fullsized":"2024-04-17T05:46:15.163258+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.163041+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:18.146680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11291,"num_write":13,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":158,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.612248+0000","last_change":"2024-04-17T05:46:13.612312+0000","last_active":"2024-04-17T05:46:13.612248+0000","last_peered":"2024-04-17T05:46:13.612248+0000","last_clean":"2024-04-17T05:46:13.612248+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:46:13.612248+0000","last_undegraded":"2024-04-17T05:46:13.612248+0000","last_fullsized":"2024-04-17T05:46:13.612248+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'39","last_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_deep_scrub":"264'39","last_deep_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.612027+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:28.378268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184387+0000","last_change":"2024-04-17T05:07:45.753495+0000","last_active":"2024-04-17T05:45:15.184387+0000","last_peered":"2024-04-17T05:45:15.184387+0000","last_clean":"2024-04-17T05:45:15.184387+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:45:15.184387+0000","last_undegraded":"2024-04-17T05:45:15.184387+0000","last_fullsized":"2024-04-17T05:45:15.184387+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.195806+0000","last_change":"2024-04-17T04:51:43.681581+0000","last_active":"2024-04-17T05:45:55.195806+0000","last_peered":"2024-04-17T05:45:55.195806+0000","last_clean":"2024-04-17T05:45:55.195806+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:45:55.195806+0000","last_undegraded":"2024-04-17T05:45:55.195806+0000","last_fullsized":"2024-04-17T05:45:55.195806+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.460406+0000","last_change":"2024-04-17T04:51:39.634757+0000","last_active":"2024-04-17T05:46:02.460406+0000","last_peered":"2024-04-17T05:46:02.460406+0000","last_clean":"2024-04-17T05:46:02.460406+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:46:02.460406+0000","last_undegraded":"2024-04-17T05:46:02.460406+0000","last_fullsized":"2024-04-17T05:46:02.460406+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.995793+0000","last_change":"2024-04-17T05:46:14.995853+0000","last_active":"2024-04-17T05:46:14.995793+0000","last_peered":"2024-04-17T05:46:14.995793+0000","last_clean":"2024-04-17T05:46:14.995793+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:46:14.995793+0000","last_undegraded":"2024-04-17T05:46:14.995793+0000","last_fullsized":"2024-04-17T05:46:14.995793+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'60","last_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_deep_scrub":"264'60","last_deep_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.995600+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:08.616032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":32,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.197250+0000","last_change":"2024-04-17T05:07:45.736217+0000","last_active":"2024-04-17T05:45:50.197250+0000","last_peered":"2024-04-17T05:45:50.197250+0000","last_clean":"2024-04-17T05:45:50.197250+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:45:50.197250+0000","last_undegraded":"2024-04-17T05:45:50.197250+0000","last_fullsized":"2024-04-17T05:45:50.197250+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'21","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.298945+0000","last_change":"2024-04-17T05:46:24.299018+0000","last_active":"2024-04-17T05:46:24.298945+0000","last_peered":"2024-04-17T05:46:24.298945+0000","last_clean":"2024-04-17T05:46:24.298945+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:46:24.298945+0000","last_undegraded":"2024-04-17T05:46:24.298945+0000","last_fullsized":"2024-04-17T05:46:24.298945+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'21","last_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_deep_scrub":"264'21","last_deep_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.298719+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:55.830264+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":54274,"num_write":19,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'10","reported_seq":80,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.192007+0000","last_change":"2024-04-17T04:51:39.635143+0000","last_active":"2024-04-17T05:45:50.192007+0000","last_peered":"2024-04-17T05:45:50.192007+0000","last_clean":"2024-04-17T05:45:50.192007+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:45:50.192007+0000","last_undegraded":"2024-04-17T05:45:50.192007+0000","last_fullsized":"2024-04-17T05:45:50.192007+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":232,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.186254+0000","last_change":"2024-04-17T05:46:17.186314+0000","last_active":"2024-04-17T05:46:17.186254+0000","last_peered":"2024-04-17T05:46:17.186254+0000","last_clean":"2024-04-17T05:46:17.186254+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:46:17.186254+0000","last_undegraded":"2024-04-17T05:46:17.186254+0000","last_fullsized":"2024-04-17T05:46:17.186254+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.186068+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:32.734141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.174058+0000","last_change":"2024-04-17T05:07:45.749227+0000","last_active":"2024-04-17T05:45:15.174058+0000","last_peered":"2024-04-17T05:45:15.174058+0000","last_clean":"2024-04-17T05:45:15.174058+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:45:15.174058+0000","last_undegraded":"2024-04-17T05:45:15.174058+0000","last_fullsized":"2024-04-17T05:45:15.174058+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.183525+0000","last_change":"2024-04-17T04:51:43.648648+0000","last_active":"2024-04-17T05:45:55.183525+0000","last_peered":"2024-04-17T05:45:55.183525+0000","last_clean":"2024-04-17T05:45:55.183525+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:45:55.183525+0000","last_undegraded":"2024-04-17T05:45:55.183525+0000","last_fullsized":"2024-04-17T05:45:55.183525+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'6","reported_seq":73,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.099801+0000","last_change":"2024-04-17T05:46:12.099865+0000","last_active":"2024-04-17T05:46:12.099801+0000","last_peered":"2024-04-17T05:46:12.099801+0000","last_clean":"2024-04-17T05:46:12.099801+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:46:12.099801+0000","last_undegraded":"2024-04-17T05:46:12.099801+0000","last_fullsized":"2024-04-17T05:46:12.099801+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.099605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:49.682658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":8192,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":120,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.665206+0000","last_change":"2024-04-17T05:46:10.665206+0000","last_active":"2024-04-17T05:46:10.665206+0000","last_peered":"2024-04-17T05:46:10.665206+0000","last_clean":"2024-04-17T05:46:10.665206+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:46:10.665206+0000","last_undegraded":"2024-04-17T05:46:10.665206+0000","last_fullsized":"2024-04-17T05:46:10.665206+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'21","last_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_deep_scrub":"261'21","last_deep_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.664937+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:18.989139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"263'1","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:35.175203+0000","last_change":"2024-04-17T05:07:45.746243+0000","last_active":"2024-04-17T05:44:35.175203+0000","last_peered":"2024-04-17T05:44:35.175203+0000","last_clean":"2024-04-17T05:44:35.175203+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:44:35.175203+0000","last_undegraded":"2024-04-17T05:44:35.175203+0000","last_fullsized":"2024-04-17T05:44:35.175203+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'8","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.210371+0000","last_change":"2024-04-17T04:51:43.675539+0000","last_active":"2024-04-17T05:46:03.210371+0000","last_peered":"2024-04-17T05:46:03.210371+0000","last_clean":"2024-04-17T05:46:03.210371+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:46:03.210371+0000","last_undegraded":"2024-04-17T05:46:03.210371+0000","last_fullsized":"2024-04-17T05:46:03.210371+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":4098,"num_write":17,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.130174+0000","last_change":"2024-04-17T05:46:13.130289+0000","last_active":"2024-04-17T05:46:13.130174+0000","last_peered":"2024-04-17T05:46:13.130174+0000","last_clean":"2024-04-17T05:46:13.130174+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:46:13.130174+0000","last_undegraded":"2024-04-17T05:46:13.130174+0000","last_fullsized":"2024-04-17T05:46:13.130174+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.129923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:48.339568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.106637+0000","last_change":"2024-04-17T05:46:12.106637+0000","last_active":"2024-04-17T05:46:12.106637+0000","last_peered":"2024-04-17T05:46:12.106637+0000","last_clean":"2024-04-17T05:46:12.106637+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:46:12.106637+0000","last_undegraded":"2024-04-17T05:46:12.106637+0000","last_fullsized":"2024-04-17T05:46:12.106637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'32","last_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_deep_scrub":"253'32","last_deep_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.106369+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:23.602749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:35.174590+0000","last_change":"2024-04-17T05:07:45.726938+0000","last_active":"2024-04-17T05:44:35.174590+0000","last_peered":"2024-04-17T05:44:35.174590+0000","last_clean":"2024-04-17T05:44:35.174590+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:44:35.174590+0000","last_undegraded":"2024-04-17T05:44:35.174590+0000","last_fullsized":"2024-04-17T05:44:35.174590+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.390031+0000","last_change":"2024-04-17T04:51:43.648636+0000","last_active":"2024-04-17T05:46:02.390031+0000","last_peered":"2024-04-17T05:46:02.390031+0000","last_clean":"2024-04-17T05:46:02.390031+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:46:02.390031+0000","last_undegraded":"2024-04-17T05:46:02.390031+0000","last_fullsized":"2024-04-17T05:46:02.390031+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.127295+0000","last_change":"2024-04-17T05:46:14.127359+0000","last_active":"2024-04-17T05:46:14.127295+0000","last_peered":"2024-04-17T05:46:14.127295+0000","last_clean":"2024-04-17T05:46:14.127295+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:46:14.127295+0000","last_undegraded":"2024-04-17T05:46:14.127295+0000","last_fullsized":"2024-04-17T05:46:14.127295+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.127097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.931818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":151,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.750372+0000","last_change":"2024-04-17T05:46:08.750372+0000","last_active":"2024-04-17T05:46:08.750372+0000","last_peered":"2024-04-17T05:46:08.750372+0000","last_clean":"2024-04-17T05:46:08.750372+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:46:08.750372+0000","last_undegraded":"2024-04-17T05:46:08.750372+0000","last_fullsized":"2024-04-17T05:46:08.750372+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'32","last_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_deep_scrub":"264'32","last_deep_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.750109+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:20.266910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184839+0000","last_change":"2024-04-17T05:07:45.757247+0000","last_active":"2024-04-17T05:45:15.184839+0000","last_peered":"2024-04-17T05:45:15.184839+0000","last_clean":"2024-04-17T05:45:15.184839+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:45:15.184839+0000","last_undegraded":"2024-04-17T05:45:15.184839+0000","last_fullsized":"2024-04-17T05:45:15.184839+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.189116+0000","last_change":"2024-04-17T04:51:43.682310+0000","last_active":"2024-04-17T05:45:55.189116+0000","last_peered":"2024-04-17T05:45:55.189116+0000","last_clean":"2024-04-17T05:45:55.189116+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:45:55.189116+0000","last_undegraded":"2024-04-17T05:45:55.189116+0000","last_fullsized":"2024-04-17T05:45:55.189116+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'2","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.643349+0000","last_change":"2024-04-17T05:46:09.643497+0000","last_active":"2024-04-17T05:46:09.643349+0000","last_peered":"2024-04-17T05:46:09.643349+0000","last_clean":"2024-04-17T05:46:09.643349+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:46:09.643349+0000","last_undegraded":"2024-04-17T05:46:09.643349+0000","last_fullsized":"2024-04-17T05:46:09.643349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.643135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:49.245538+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.049363+0000","last_change":"2024-04-17T05:46:13.049363+0000","last_active":"2024-04-17T05:46:13.049363+0000","last_peered":"2024-04-17T05:46:13.049363+0000","last_clean":"2024-04-17T05:46:13.049363+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:46:13.049363+0000","last_undegraded":"2024-04-17T05:46:13.049363+0000","last_fullsized":"2024-04-17T05:46:13.049363+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'69","last_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_deep_scrub":"264'69","last_deep_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.048938+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:49.209093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.180732+0000","last_change":"2024-04-17T05:07:45.752461+0000","last_active":"2024-04-17T05:45:20.180732+0000","last_peered":"2024-04-17T05:45:20.180732+0000","last_clean":"2024-04-17T05:45:20.180732+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:45:20.180732+0000","last_undegraded":"2024-04-17T05:45:20.180732+0000","last_fullsized":"2024-04-17T05:45:20.180732+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.183418+0000","last_change":"2024-04-17T04:51:43.651232+0000","last_active":"2024-04-17T05:45:55.183418+0000","last_peered":"2024-04-17T05:45:55.183418+0000","last_clean":"2024-04-17T05:45:55.183418+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:45:55.183418+0000","last_undegraded":"2024-04-17T05:45:55.183418+0000","last_fullsized":"2024-04-17T05:45:55.183418+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.379049+0000","last_change":"2024-04-17T04:51:39.630310+0000","last_active":"2024-04-17T05:46:03.379049+0000","last_peered":"2024-04-17T05:46:03.379049+0000","last_clean":"2024-04-17T05:46:03.379049+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:46:03.379049+0000","last_undegraded":"2024-04-17T05:46:03.379049+0000","last_fullsized":"2024-04-17T05:46:03.379049+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6192,"num_write":5,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":137,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.225148+0000","last_change":"2024-04-17T05:46:19.225296+0000","last_active":"2024-04-17T05:46:19.225148+0000","last_peered":"2024-04-17T05:46:19.225148+0000","last_clean":"2024-04-17T05:46:19.225148+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:46:19.225148+0000","last_undegraded":"2024-04-17T05:46:19.225148+0000","last_fullsized":"2024-04-17T05:46:19.225148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_deep_scrub":"264'27","last_deep_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.224892+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:29.117161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'349","reported_seq":1011,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.880896+0000","last_change":"2024-04-17T04:36:25.377106+0000","last_active":"2024-04-17T05:46:03.880896+0000","last_peered":"2024-04-17T05:46:03.880896+0000","last_clean":"2024-04-17T05:46:03.880896+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:46:03.880896+0000","last_undegraded":"2024-04-17T05:46:03.880896+0000","last_fullsized":"2024-04-17T05:46:03.880896+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":725,"num_write":1314,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":676,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.167806+0000","last_change":"2024-04-17T04:36:27.389744+0000","last_active":"2024-04-17T05:44:00.167806+0000","last_peered":"2024-04-17T05:44:00.167806+0000","last_clean":"2024-04-17T05:44:00.167806+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:44:00.167806+0000","last_undegraded":"2024-04-17T05:44:00.167806+0000","last_fullsized":"2024-04-17T05:44:00.167806+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'382","reported_seq":1055,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.852523+0000","last_change":"2024-04-17T04:36:25.357080+0000","last_active":"2024-04-17T05:46:03.852523+0000","last_peered":"2024-04-17T05:46:03.852523+0000","last_clean":"2024-04-17T05:46:03.852523+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:46:03.852523+0000","last_undegraded":"2024-04-17T05:46:03.852523+0000","last_fullsized":"2024-04-17T05:46:03.852523+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":600,"num_read_kb":10797,"num_write":1506,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":707,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.162643+0000","last_change":"2024-04-17T04:36:27.382828+0000","last_active":"2024-04-17T05:44:00.162643+0000","last_peered":"2024-04-17T05:44:00.162643+0000","last_clean":"2024-04-17T05:44:00.162643+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:44:00.162643+0000","last_undegraded":"2024-04-17T05:44:00.162643+0000","last_fullsized":"2024-04-17T05:44:00.162643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'312","reported_seq":979,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.881660+0000","last_change":"2024-04-17T04:36:25.329662+0000","last_active":"2024-04-17T05:46:03.881660+0000","last_peered":"2024-04-17T05:46:03.881660+0000","last_clean":"2024-04-17T05:46:03.881660+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:46:03.881660+0000","last_undegraded":"2024-04-17T05:46:03.881660+0000","last_fullsized":"2024-04-17T05:46:03.881660+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11583546,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":475,"num_read_kb":58403,"num_write":1188,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1206,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.156536+0000","last_change":"2024-04-17T04:36:27.385236+0000","last_active":"2024-04-17T05:44:00.156536+0000","last_peered":"2024-04-17T05:44:00.156536+0000","last_clean":"2024-04-17T05:44:00.156536+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:44:00.156536+0000","last_undegraded":"2024-04-17T05:44:00.156536+0000","last_fullsized":"2024-04-17T05:44:00.156536+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'284","reported_seq":945,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.872405+0000","last_change":"2024-04-17T04:36:25.356234+0000","last_active":"2024-04-17T05:46:03.872405+0000","last_peered":"2024-04-17T05:46:03.872405+0000","last_clean":"2024-04-17T05:46:03.872405+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:46:03.872405+0000","last_undegraded":"2024-04-17T05:46:03.872405+0000","last_fullsized":"2024-04-17T05:46:03.872405+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174737,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":460,"num_read_kb":37884,"num_write":1128,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'304","reported_seq":898,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.164939+0000","last_change":"2024-04-17T04:36:27.385726+0000","last_active":"2024-04-17T05:44:00.164939+0000","last_peered":"2024-04-17T05:44:00.164939+0000","last_clean":"2024-04-17T05:44:00.164939+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:44:00.164939+0000","last_undegraded":"2024-04-17T05:44:00.164939+0000","last_fullsized":"2024-04-17T05:44:00.164939+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'336","reported_seq":998,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155938+0000","last_change":"2024-04-17T04:36:25.377714+0000","last_active":"2024-04-17T05:46:10.155938+0000","last_peered":"2024-04-17T05:46:10.155938+0000","last_clean":"2024-04-17T05:46:10.155938+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:46:10.155938+0000","last_undegraded":"2024-04-17T05:46:10.155938+0000","last_fullsized":"2024-04-17T05:46:10.155938+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":528,"num_read_kb":13359,"num_write":1173,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5180,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.776716+0000","last_change":"2024-04-17T04:36:27.382622+0000","last_active":"2024-04-17T05:45:44.776716+0000","last_peered":"2024-04-17T05:45:44.776716+0000","last_clean":"2024-04-17T05:45:44.776716+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:45:44.776716+0000","last_undegraded":"2024-04-17T05:45:44.776716+0000","last_fullsized":"2024-04-17T05:45:44.776716+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'301","reported_seq":1016,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.212279+0000","last_change":"2024-04-17T04:36:25.352114+0000","last_active":"2024-04-17T05:46:09.212279+0000","last_peered":"2024-04-17T05:46:09.212279+0000","last_clean":"2024-04-17T05:46:09.212279+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:46:09.212279+0000","last_undegraded":"2024-04-17T05:46:09.212279+0000","last_fullsized":"2024-04-17T05:46:09.212279+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206995,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":584,"num_read_kb":64592,"num_write":977,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":775,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.174664+0000","last_change":"2024-04-17T04:36:27.385407+0000","last_active":"2024-04-17T05:45:15.174664+0000","last_peered":"2024-04-17T05:45:15.174664+0000","last_clean":"2024-04-17T05:45:15.174664+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:45:15.174664+0000","last_undegraded":"2024-04-17T05:45:15.174664+0000","last_fullsized":"2024-04-17T05:45:15.174664+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'324","reported_seq":948,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155808+0000","last_change":"2024-04-17T04:36:25.376858+0000","last_active":"2024-04-17T05:46:10.155808+0000","last_peered":"2024-04-17T05:46:10.155808+0000","last_clean":"2024-04-17T05:46:10.155808+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:46:10.155808+0000","last_undegraded":"2024-04-17T05:46:10.155808+0000","last_fullsized":"2024-04-17T05:46:10.155808+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269430,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":458,"num_read_kb":9424,"num_write":1128,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":809,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184512+0000","last_change":"2024-04-17T04:36:27.376772+0000","last_active":"2024-04-17T05:45:15.184512+0000","last_peered":"2024-04-17T05:45:15.184512+0000","last_clean":"2024-04-17T05:45:15.184512+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:45:15.184512+0000","last_undegraded":"2024-04-17T05:45:15.184512+0000","last_fullsized":"2024-04-17T05:45:15.184512+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'318","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275588+0000","last_change":"2024-04-17T04:36:25.343980+0000","last_active":"2024-04-17T05:46:08.275588+0000","last_peered":"2024-04-17T05:46:08.275588+0000","last_clean":"2024-04-17T05:46:08.275588+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:46:08.275588+0000","last_undegraded":"2024-04-17T05:46:08.275588+0000","last_fullsized":"2024-04-17T05:46:08.275588+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076711,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":427,"num_read_kb":14992,"num_write":1164,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":771,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184907+0000","last_change":"2024-04-17T04:36:27.379088+0000","last_active":"2024-04-17T05:45:15.184907+0000","last_peered":"2024-04-17T05:45:15.184907+0000","last_clean":"2024-04-17T05:45:15.184907+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:45:15.184907+0000","last_undegraded":"2024-04-17T05:45:15.184907+0000","last_fullsized":"2024-04-17T05:45:15.184907+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159364+0000","last_change":"2024-04-17T04:36:27.385695+0000","last_active":"2024-04-17T05:44:10.159364+0000","last_peered":"2024-04-17T05:44:10.159364+0000","last_clean":"2024-04-17T05:44:10.159364+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:44:10.159364+0000","last_undegraded":"2024-04-17T05:44:10.159364+0000","last_fullsized":"2024-04-17T05:44:10.159364+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'322","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.544504+0000","last_change":"2024-04-17T05:46:16.544710+0000","last_active":"2024-04-17T05:46:16.544504+0000","last_peered":"2024-04-17T05:46:16.544504+0000","last_clean":"2024-04-17T05:46:16.544504+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:46:16.544504+0000","last_undegraded":"2024-04-17T05:46:16.544504+0000","last_fullsized":"2024-04-17T05:46:16.544504+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.544168+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:59.343831+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8704691,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":453,"num_read_kb":14105,"num_write":1178,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'255","reported_seq":890,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:46.446975+0000","last_change":"2024-04-17T04:36:27.383282+0000","last_active":"2024-04-17T05:45:46.446975+0000","last_peered":"2024-04-17T05:45:46.446975+0000","last_clean":"2024-04-17T05:45:46.446975+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:45:46.446975+0000","last_undegraded":"2024-04-17T05:45:46.446975+0000","last_fullsized":"2024-04-17T05:45:46.446975+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'270","reported_seq":889,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.889856+0000","last_change":"2024-04-17T04:36:25.385368+0000","last_active":"2024-04-17T05:46:03.889856+0000","last_peered":"2024-04-17T05:46:03.889856+0000","last_clean":"2024-04-17T05:46:03.889856+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:46:03.889856+0000","last_undegraded":"2024-04-17T05:46:03.889856+0000","last_fullsized":"2024-04-17T05:46:03.889856+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1730173,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":405,"num_read_kb":10551,"num_write":968,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":724,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.173830+0000","last_change":"2024-04-17T04:36:27.376262+0000","last_active":"2024-04-17T05:44:25.173830+0000","last_peered":"2024-04-17T05:44:25.173830+0000","last_clean":"2024-04-17T05:44:25.173830+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:44:25.173830+0000","last_undegraded":"2024-04-17T05:44:25.173830+0000","last_fullsized":"2024-04-17T05:44:25.173830+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'302","reported_seq":985,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.801369+0000","last_change":"2024-04-17T04:36:25.385723+0000","last_active":"2024-04-17T05:46:03.801369+0000","last_peered":"2024-04-17T05:46:03.801369+0000","last_clean":"2024-04-17T05:46:03.801369+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:46:03.801369+0000","last_undegraded":"2024-04-17T05:46:03.801369+0000","last_fullsized":"2024-04-17T05:46:03.801369+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044776,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":456,"num_read_kb":160096,"num_write":1126,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":900,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.169063+0000","last_change":"2024-04-17T04:36:27.383859+0000","last_active":"2024-04-17T05:44:10.169063+0000","last_peered":"2024-04-17T05:44:10.169063+0000","last_clean":"2024-04-17T05:44:10.169063+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:44:10.169063+0000","last_undegraded":"2024-04-17T05:44:10.169063+0000","last_fullsized":"2024-04-17T05:44:10.169063+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'306","reported_seq":1012,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.879728+0000","last_change":"2024-04-17T04:36:25.361757+0000","last_active":"2024-04-17T05:46:03.879728+0000","last_peered":"2024-04-17T05:46:03.879728+0000","last_clean":"2024-04-17T05:46:03.879728+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:46:03.879728+0000","last_undegraded":"2024-04-17T05:46:03.879728+0000","last_fullsized":"2024-04-17T05:46:03.879728+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399248,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":586,"num_read_kb":64730,"num_write":1112,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4805,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.169152+0000","last_change":"2024-04-17T04:36:27.379593+0000","last_active":"2024-04-17T05:44:10.169152+0000","last_peered":"2024-04-17T05:44:10.169152+0000","last_clean":"2024-04-17T05:44:10.169152+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:44:10.169152+0000","last_undegraded":"2024-04-17T05:44:10.169152+0000","last_fullsized":"2024-04-17T05:44:10.169152+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'318","reported_seq":1089,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.869850+0000","last_change":"2024-04-17T04:36:25.374400+0000","last_active":"2024-04-17T05:46:03.869850+0000","last_peered":"2024-04-17T05:46:03.869850+0000","last_clean":"2024-04-17T05:46:03.869850+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:46:03.869850+0000","last_undegraded":"2024-04-17T05:46:03.869850+0000","last_fullsized":"2024-04-17T05:46:03.869850+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170354,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":810,"num_read_kb":57781,"num_write":1265,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":816,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159557+0000","last_change":"2024-04-17T04:36:27.385279+0000","last_active":"2024-04-17T05:44:10.159557+0000","last_peered":"2024-04-17T05:44:10.159557+0000","last_clean":"2024-04-17T05:44:10.159557+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:44:10.159557+0000","last_undegraded":"2024-04-17T05:44:10.159557+0000","last_fullsized":"2024-04-17T05:44:10.159557+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'346","reported_seq":981,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.885751+0000","last_change":"2024-04-17T04:36:25.355404+0000","last_active":"2024-04-17T05:46:03.885751+0000","last_peered":"2024-04-17T05:46:03.885751+0000","last_clean":"2024-04-17T05:46:03.885751+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:46:03.885751+0000","last_undegraded":"2024-04-17T05:46:03.885751+0000","last_fullsized":"2024-04-17T05:46:03.885751+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244513,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":463,"num_read_kb":836,"num_write":1174,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'155","reported_seq":680,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.797186+0000","last_change":"2024-04-17T05:46:13.797345+0000","last_active":"2024-04-17T05:46:13.797186+0000","last_peered":"2024-04-17T05:46:13.797186+0000","last_clean":"2024-04-17T05:46:13.797186+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:46:13.797186+0000","last_undegraded":"2024-04-17T05:46:13.797186+0000","last_fullsized":"2024-04-17T05:46:13.797186+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.796936+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:42.445551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":599,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.203466+0000","last_change":"2024-04-17T04:36:27.384993+0000","last_active":"2024-04-17T05:44:25.203466+0000","last_peered":"2024-04-17T05:44:25.203466+0000","last_clean":"2024-04-17T05:44:25.203466+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:44:25.203466+0000","last_undegraded":"2024-04-17T05:44:25.203466+0000","last_fullsized":"2024-04-17T05:44:25.203466+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'314","reported_seq":940,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155721+0000","last_change":"2024-04-17T04:36:25.368467+0000","last_active":"2024-04-17T05:46:10.155721+0000","last_peered":"2024-04-17T05:46:10.155721+0000","last_clean":"2024-04-17T05:46:10.155721+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:46:10.155721+0000","last_undegraded":"2024-04-17T05:46:10.155721+0000","last_fullsized":"2024-04-17T05:46:10.155721+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7313223,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":429,"num_read_kb":2527,"num_write":1169,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.170328+0000","last_change":"2024-04-17T04:36:29.388609+0000","last_active":"2024-04-17T05:44:10.170328+0000","last_peered":"2024-04-17T05:44:10.170328+0000","last_clean":"2024-04-17T05:44:10.170328+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:44:10.170328+0000","last_undegraded":"2024-04-17T05:44:10.170328+0000","last_fullsized":"2024-04-17T05:44:10.170328+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'367","reported_seq":993,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.760091+0000","last_change":"2024-04-17T04:36:31.410990+0000","last_active":"2024-04-17T05:46:03.760091+0000","last_peered":"2024-04-17T05:46:03.760091+0000","last_clean":"2024-04-17T05:46:03.760091+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:46:03.760091+0000","last_undegraded":"2024-04-17T05:46:03.760091+0000","last_fullsized":"2024-04-17T05:46:03.760091+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45299,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":596,"num_read_kb":1277,"num_write":472,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166501+0000","last_change":"2024-04-17T04:36:33.425089+0000","last_active":"2024-04-17T05:44:15.166501+0000","last_peered":"2024-04-17T05:44:15.166501+0000","last_clean":"2024-04-17T05:44:15.166501+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:44:15.166501+0000","last_undegraded":"2024-04-17T05:44:15.166501+0000","last_fullsized":"2024-04-17T05:44:15.166501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'172","reported_seq":734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.111916+0000","last_change":"2024-04-17T05:46:15.112010+0000","last_active":"2024-04-17T05:46:15.111916+0000","last_peered":"2024-04-17T05:46:15.111916+0000","last_clean":"2024-04-17T05:46:15.111916+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:46:15.111916+0000","last_undegraded":"2024-04-17T05:46:15.111916+0000","last_fullsized":"2024-04-17T05:46:15.111916+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'172","last_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_deep_scrub":"264'172","last_deep_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.111703+0000","objects_scrubbed":3,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:36:19.615360+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":674,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.170223+0000","last_change":"2024-04-17T04:36:27.389710+0000","last_active":"2024-04-17T05:44:10.170223+0000","last_peered":"2024-04-17T05:44:10.170223+0000","last_clean":"2024-04-17T05:44:10.170223+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:44:10.170223+0000","last_undegraded":"2024-04-17T05:44:10.170223+0000","last_fullsized":"2024-04-17T05:44:10.170223+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'339","reported_seq":1112,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.933964+0000","last_change":"2024-04-17T04:36:25.328153+0000","last_active":"2024-04-17T05:46:06.933964+0000","last_peered":"2024-04-17T05:46:06.933964+0000","last_clean":"2024-04-17T05:46:06.933964+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:46:06.933964+0000","last_undegraded":"2024-04-17T05:46:06.933964+0000","last_fullsized":"2024-04-17T05:46:06.933964+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1328,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":754,"num_read_kb":61435,"num_write":1363,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.203506+0000","last_change":"2024-04-17T04:36:29.373876+0000","last_active":"2024-04-17T05:44:25.203506+0000","last_peered":"2024-04-17T05:44:25.203506+0000","last_clean":"2024-04-17T05:44:25.203506+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:44:25.203506+0000","last_undegraded":"2024-04-17T05:44:25.203506+0000","last_fullsized":"2024-04-17T05:44:25.203506+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'103","reported_seq":655,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.736412+0000","last_change":"2024-04-17T05:46:22.736490+0000","last_active":"2024-04-17T05:46:22.736412+0000","last_peered":"2024-04-17T05:46:22.736412+0000","last_clean":"2024-04-17T05:46:22.736412+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:46:22.736412+0000","last_undegraded":"2024-04-17T05:46:22.736412+0000","last_fullsized":"2024-04-17T05:46:22.736412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'103","last_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_deep_scrub":"264'103","last_deep_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.736227+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:14.605978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.786351+0000","last_change":"2024-04-17T05:46:15.786406+0000","last_active":"2024-04-17T05:46:15.786351+0000","last_peered":"2024-04-17T05:46:15.786351+0000","last_clean":"2024-04-17T05:46:15.786351+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:46:15.786351+0000","last_undegraded":"2024-04-17T05:46:15.786351+0000","last_fullsized":"2024-04-17T05:46:15.786351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.786152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:05:10.717154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'129","reported_seq":641,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.649650+0000","last_change":"2024-04-17T05:46:22.649727+0000","last_active":"2024-04-17T05:46:22.649650+0000","last_peered":"2024-04-17T05:46:22.649650+0000","last_clean":"2024-04-17T05:46:22.649650+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:46:22.649650+0000","last_undegraded":"2024-04-17T05:46:22.649650+0000","last_fullsized":"2024-04-17T05:46:22.649650+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'129","last_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_deep_scrub":"264'129","last_deep_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.649467+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.914685+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":660,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.167632+0000","last_change":"2024-04-17T04:36:27.376107+0000","last_active":"2024-04-17T05:44:10.167632+0000","last_peered":"2024-04-17T05:44:10.167632+0000","last_clean":"2024-04-17T05:44:10.167632+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:44:10.167632+0000","last_undegraded":"2024-04-17T05:44:10.167632+0000","last_fullsized":"2024-04-17T05:44:10.167632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'333","reported_seq":1114,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155656+0000","last_change":"2024-04-17T04:36:25.381456+0000","last_active":"2024-04-17T05:46:10.155656+0000","last_peered":"2024-04-17T05:46:10.155656+0000","last_clean":"2024-04-17T05:46:10.155656+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:46:10.155656+0000","last_undegraded":"2024-04-17T05:46:10.155656+0000","last_fullsized":"2024-04-17T05:46:10.155656+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":672,"num_read_kb":75397,"num_write":1317,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.199888+0000","last_change":"2024-04-17T04:36:29.359822+0000","last_active":"2024-04-17T05:44:10.199888+0000","last_peered":"2024-04-17T05:44:10.199888+0000","last_clean":"2024-04-17T05:44:10.199888+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:44:10.199888+0000","last_undegraded":"2024-04-17T05:44:10.199888+0000","last_fullsized":"2024-04-17T05:44:10.199888+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'144","reported_seq":720,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.644544+0000","last_change":"2024-04-17T05:46:13.644625+0000","last_active":"2024-04-17T05:46:13.644544+0000","last_peered":"2024-04-17T05:46:13.644544+0000","last_clean":"2024-04-17T05:46:13.644544+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:46:13.644544+0000","last_undegraded":"2024-04-17T05:46:13.644544+0000","last_fullsized":"2024-04-17T05:46:13.644544+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'144","last_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_deep_scrub":"264'144","last_deep_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.644329+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:14.646427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'519","reported_seq":2417,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.902823+0000","last_change":"2024-04-17T04:36:33.420806+0000","last_active":"2024-04-17T05:46:03.902823+0000","last_peered":"2024-04-17T05:46:03.902823+0000","last_clean":"2024-04-17T05:46:03.902823+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:46:03.902823+0000","last_undegraded":"2024-04-17T05:46:03.902823+0000","last_fullsized":"2024-04-17T05:46:03.902823+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'199","reported_seq":1021,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.828784+0000","last_change":"2024-04-17T05:46:14.828863+0000","last_active":"2024-04-17T05:46:14.828784+0000","last_peered":"2024-04-17T05:46:14.828784+0000","last_clean":"2024-04-17T05:46:14.828784+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:46:14.828784+0000","last_undegraded":"2024-04-17T05:46:14.828784+0000","last_fullsized":"2024-04-17T05:46:14.828784+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'199","last_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_deep_scrub":"264'199","last_deep_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.828583+0000","objects_scrubbed":4,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:40.862007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.199859+0000","last_change":"2024-04-17T04:36:27.395613+0000","last_active":"2024-04-17T05:44:10.199859+0000","last_peered":"2024-04-17T05:44:10.199859+0000","last_clean":"2024-04-17T05:44:10.199859+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:44:10.199859+0000","last_undegraded":"2024-04-17T05:44:10.199859+0000","last_fullsized":"2024-04-17T05:44:10.199859+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'293","reported_seq":941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.679923+0000","last_change":"2024-04-17T04:36:25.384743+0000","last_active":"2024-04-17T05:46:09.679923+0000","last_peered":"2024-04-17T05:46:09.679923+0000","last_clean":"2024-04-17T05:46:09.679923+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:46:09.679923+0000","last_undegraded":"2024-04-17T05:46:09.679923+0000","last_fullsized":"2024-04-17T05:46:09.679923+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3722445,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":48921,"num_write":1143,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.170433+0000","last_change":"2024-04-17T04:36:29.391920+0000","last_active":"2024-04-17T05:44:10.170433+0000","last_peered":"2024-04-17T05:44:10.170433+0000","last_clean":"2024-04-17T05:44:10.170433+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:44:10.170433+0000","last_undegraded":"2024-04-17T05:44:10.170433+0000","last_fullsized":"2024-04-17T05:44:10.170433+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'279","reported_seq":865,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.371516+0000","last_change":"2024-04-17T04:36:31.422728+0000","last_active":"2024-04-17T05:46:03.371516+0000","last_peered":"2024-04-17T05:46:03.371516+0000","last_clean":"2024-04-17T05:46:03.371516+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:46:03.371516+0000","last_undegraded":"2024-04-17T05:46:03.371516+0000","last_fullsized":"2024-04-17T05:46:03.371516+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51171,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1108,"num_write":376,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'534","reported_seq":2303,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.902875+0000","last_change":"2024-04-17T04:36:33.422550+0000","last_active":"2024-04-17T05:46:03.902875+0000","last_peered":"2024-04-17T05:46:03.902875+0000","last_clean":"2024-04-17T05:46:03.902875+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:46:03.902875+0000","last_undegraded":"2024-04-17T05:46:03.902875+0000","last_fullsized":"2024-04-17T05:46:03.902875+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'121","reported_seq":721,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.191810+0000","last_change":"2024-04-17T04:36:27.398309+0000","last_active":"2024-04-17T05:45:50.191810+0000","last_peered":"2024-04-17T05:45:50.191810+0000","last_clean":"2024-04-17T05:45:50.191810+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:45:50.191810+0000","last_undegraded":"2024-04-17T05:45:50.191810+0000","last_fullsized":"2024-04-17T05:45:50.191810+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'281","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155605+0000","last_change":"2024-04-17T04:36:25.370799+0000","last_active":"2024-04-17T05:46:10.155605+0000","last_peered":"2024-04-17T05:46:10.155605+0000","last_clean":"2024-04-17T05:46:10.155605+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:46:10.155605+0000","last_undegraded":"2024-04-17T05:46:10.155605+0000","last_fullsized":"2024-04-17T05:46:10.155605+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6974633,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":437,"num_read_kb":24084,"num_write":1121,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.187354+0000","last_change":"2024-04-17T04:36:29.389738+0000","last_active":"2024-04-17T05:45:50.187354+0000","last_peered":"2024-04-17T05:45:50.187354+0000","last_clean":"2024-04-17T05:45:50.187354+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:45:50.187354+0000","last_undegraded":"2024-04-17T05:45:50.187354+0000","last_fullsized":"2024-04-17T05:45:50.187354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'391","reported_seq":1038,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.206831+0000","last_change":"2024-04-17T05:46:22.206940+0000","last_active":"2024-04-17T05:46:22.206831+0000","last_peered":"2024-04-17T05:46:22.206831+0000","last_clean":"2024-04-17T05:46:22.206831+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:46:22.206831+0000","last_undegraded":"2024-04-17T05:46:22.206831+0000","last_fullsized":"2024-04-17T05:46:22.206831+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'391","last_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_deep_scrub":"264'391","last_deep_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.206623+0000","objects_scrubbed":10,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:39.352908+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44506,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":619,"num_read_kb":1253,"num_write":476,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.396859+0000","last_change":"2024-04-17T04:36:33.408109+0000","last_active":"2024-04-17T05:46:07.396859+0000","last_peered":"2024-04-17T05:46:07.396859+0000","last_clean":"2024-04-17T05:46:07.396859+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:46:07.396859+0000","last_undegraded":"2024-04-17T05:46:07.396859+0000","last_fullsized":"2024-04-17T05:46:07.396859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'564","reported_seq":1397,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.160720+0000","last_change":"2024-04-17T05:46:18.160792+0000","last_active":"2024-04-17T05:46:18.160720+0000","last_peered":"2024-04-17T05:46:18.160720+0000","last_clean":"2024-04-17T05:46:18.160720+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:46:18.160720+0000","last_undegraded":"2024-04-17T05:46:18.160720+0000","last_fullsized":"2024-04-17T05:46:18.160720+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'564","last_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_deep_scrub":"264'564","last_deep_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.160537+0000","objects_scrubbed":4,"log_size":64,"log_dups_size":500,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:23.558644+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":866,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.182262+0000","last_change":"2024-04-17T04:36:27.379851+0000","last_active":"2024-04-17T05:45:50.182262+0000","last_peered":"2024-04-17T05:45:50.182262+0000","last_clean":"2024-04-17T05:45:50.182262+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:45:50.182262+0000","last_undegraded":"2024-04-17T05:45:50.182262+0000","last_fullsized":"2024-04-17T05:45:50.182262+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'395","reported_seq":1057,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155568+0000","last_change":"2024-04-17T04:36:25.371549+0000","last_active":"2024-04-17T05:46:10.155568+0000","last_peered":"2024-04-17T05:46:10.155568+0000","last_clean":"2024-04-17T05:46:10.155568+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:46:10.155568+0000","last_undegraded":"2024-04-17T05:46:10.155568+0000","last_fullsized":"2024-04-17T05:46:10.155568+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396022,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":571,"num_read_kb":14718,"num_write":1316,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.469546+0000","last_change":"2024-04-17T05:46:23.469644+0000","last_active":"2024-04-17T05:46:23.469546+0000","last_peered":"2024-04-17T05:46:23.469546+0000","last_clean":"2024-04-17T05:46:23.469546+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:46:23.469546+0000","last_undegraded":"2024-04-17T05:46:23.469546+0000","last_fullsized":"2024-04-17T05:46:23.469546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.469283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:59.151653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'269","reported_seq":860,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276587+0000","last_change":"2024-04-17T04:36:31.392065+0000","last_active":"2024-04-17T05:46:08.276587+0000","last_peered":"2024-04-17T05:46:08.276587+0000","last_clean":"2024-04-17T05:46:08.276587+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:46:08.276587+0000","last_undegraded":"2024-04-17T05:46:08.276587+0000","last_fullsized":"2024-04-17T05:46:08.276587+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42027,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":427,"num_read_kb":943,"num_write":336,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276579+0000","last_change":"2024-04-17T04:36:33.416143+0000","last_active":"2024-04-17T05:46:08.276579+0000","last_peered":"2024-04-17T05:46:08.276579+0000","last_clean":"2024-04-17T05:46:08.276579+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:46:08.276579+0000","last_undegraded":"2024-04-17T05:46:08.276579+0000","last_fullsized":"2024-04-17T05:46:08.276579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2371","reported_seq":4174,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.637166+0000","last_change":"2024-04-17T04:36:35.418228+0000","last_active":"2024-04-17T05:45:40.637166+0000","last_peered":"2024-04-17T05:45:40.637166+0000","last_clean":"2024-04-17T05:45:40.637166+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:45:40.637166+0000","last_undegraded":"2024-04-17T05:45:40.637166+0000","last_fullsized":"2024-04-17T05:45:40.637166+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":2300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":700,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.190213+0000","last_change":"2024-04-17T04:36:27.376914+0000","last_active":"2024-04-17T05:46:00.190213+0000","last_peered":"2024-04-17T05:46:00.190213+0000","last_clean":"2024-04-17T05:46:00.190213+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:46:00.190213+0000","last_undegraded":"2024-04-17T05:46:00.190213+0000","last_fullsized":"2024-04-17T05:46:00.190213+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'356","reported_seq":1032,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.156347+0000","last_change":"2024-04-17T04:36:25.386734+0000","last_active":"2024-04-17T05:46:10.156347+0000","last_peered":"2024-04-17T05:46:10.156347+0000","last_clean":"2024-04-17T05:46:10.156347+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:46:10.156347+0000","last_undegraded":"2024-04-17T05:46:10.156347+0000","last_fullsized":"2024-04-17T05:46:10.156347+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524369,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":604,"num_read_kb":65373,"num_write":1319,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.457184+0000","last_change":"2024-04-17T05:46:21.457242+0000","last_active":"2024-04-17T05:46:21.457184+0000","last_peered":"2024-04-17T05:46:21.457184+0000","last_clean":"2024-04-17T05:46:21.457184+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:46:21.457184+0000","last_undegraded":"2024-04-17T05:46:21.457184+0000","last_fullsized":"2024-04-17T05:46:21.457184+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.456977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:03.328209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'335","reported_seq":952,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.757478+0000","last_change":"2024-04-17T04:36:31.380567+0000","last_active":"2024-04-17T05:46:01.757478+0000","last_peered":"2024-04-17T05:46:01.757478+0000","last_clean":"2024-04-17T05:46:01.757478+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:46:01.757478+0000","last_undegraded":"2024-04-17T05:46:01.757478+0000","last_fullsized":"2024-04-17T05:46:01.757478+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48038,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":543,"num_read_kb":1167,"num_write":428,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'539","reported_seq":4616,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.903764+0000","last_change":"2024-04-17T04:36:33.411578+0000","last_active":"2024-04-17T05:46:03.903764+0000","last_peered":"2024-04-17T05:46:03.903764+0000","last_clean":"2024-04-17T05:46:03.903764+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:46:03.903764+0000","last_undegraded":"2024-04-17T05:46:03.903764+0000","last_fullsized":"2024-04-17T05:46:03.903764+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":630,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.594579+0000","last_change":"2024-04-17T05:46:21.594642+0000","last_active":"2024-04-17T05:46:21.594579+0000","last_peered":"2024-04-17T05:46:21.594579+0000","last_clean":"2024-04-17T05:46:21.594579+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:46:21.594579+0000","last_undegraded":"2024-04-17T05:46:21.594579+0000","last_fullsized":"2024-04-17T05:46:21.594579+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.594379+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:06.802802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'95","reported_seq":676,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934429+0000","last_change":"2024-04-17T04:36:27.370100+0000","last_active":"2024-04-17T05:46:06.934429+0000","last_peered":"2024-04-17T05:46:06.934429+0000","last_clean":"2024-04-17T05:46:06.934429+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:46:06.934429+0000","last_undegraded":"2024-04-17T05:46:06.934429+0000","last_fullsized":"2024-04-17T05:46:06.934429+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'339","reported_seq":1047,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.599447+0000","last_change":"2024-04-17T05:46:14.599664+0000","last_active":"2024-04-17T05:46:14.599447+0000","last_peered":"2024-04-17T05:46:14.599447+0000","last_clean":"2024-04-17T05:46:14.599447+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:46:14.599447+0000","last_undegraded":"2024-04-17T05:46:14.599447+0000","last_fullsized":"2024-04-17T05:46:14.599447+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'339","last_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_deep_scrub":"264'339","last_deep_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.599211+0000","objects_scrubbed":6,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:59.675254+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":662,"num_read_kb":24375,"num_write":1169,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.517281+0000","last_change":"2024-04-17T05:46:17.517338+0000","last_active":"2024-04-17T05:46:17.517281+0000","last_peered":"2024-04-17T05:46:17.517281+0000","last_clean":"2024-04-17T05:46:17.517281+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:46:17.517281+0000","last_undegraded":"2024-04-17T05:46:17.517281+0000","last_fullsized":"2024-04-17T05:46:17.517281+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.517077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:41.667573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'472","reported_seq":1130,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.825598+0000","last_change":"2024-04-17T04:36:31.409877+0000","last_active":"2024-04-17T05:46:02.825598+0000","last_peered":"2024-04-17T05:46:02.825598+0000","last_clean":"2024-04-17T05:46:02.825598+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:46:02.825598+0000","last_undegraded":"2024-04-17T05:46:02.825598+0000","last_fullsized":"2024-04-17T05:46:02.825598+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87769,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":831,"num_read_kb":1997,"num_write":666,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.734179+0000","last_change":"2024-04-17T05:46:08.734235+0000","last_active":"2024-04-17T05:46:08.734179+0000","last_peered":"2024-04-17T05:46:08.734179+0000","last_clean":"2024-04-17T05:46:08.734179+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:46:08.734179+0000","last_undegraded":"2024-04-17T05:46:08.734179+0000","last_fullsized":"2024-04-17T05:46:08.734179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.733969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:20.733334+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'150","reported_seq":700,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.732921+0000","last_change":"2024-04-17T05:46:14.733009+0000","last_active":"2024-04-17T05:46:14.732921+0000","last_peered":"2024-04-17T05:46:14.732921+0000","last_clean":"2024-04-17T05:46:14.732921+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:46:14.732921+0000","last_undegraded":"2024-04-17T05:46:14.732921+0000","last_fullsized":"2024-04-17T05:46:14.732921+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'150","last_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_deep_scrub":"264'150","last_deep_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.732679+0000","objects_scrubbed":2,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:10.487087+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":772,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.743938+0000","last_change":"2024-04-17T04:36:27.393949+0000","last_active":"2024-04-17T05:46:08.743938+0000","last_peered":"2024-04-17T05:46:08.743938+0000","last_clean":"2024-04-17T05:46:08.743938+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:46:08.743938+0000","last_undegraded":"2024-04-17T05:46:08.743938+0000","last_fullsized":"2024-04-17T05:46:08.743938+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'350","reported_seq":1020,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275408+0000","last_change":"2024-04-17T04:36:25.379472+0000","last_active":"2024-04-17T05:46:08.275408+0000","last_peered":"2024-04-17T05:46:08.275408+0000","last_clean":"2024-04-17T05:46:08.275408+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:46:08.275408+0000","last_undegraded":"2024-04-17T05:46:08.275408+0000","last_fullsized":"2024-04-17T05:46:08.275408+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1985843,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":564,"num_read_kb":9569,"num_write":1227,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.531942+0000","last_change":"2024-04-17T05:46:19.531995+0000","last_active":"2024-04-17T05:46:19.531942+0000","last_peered":"2024-04-17T05:46:19.531942+0000","last_clean":"2024-04-17T05:46:19.531942+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:46:19.531942+0000","last_undegraded":"2024-04-17T05:46:19.531942+0000","last_fullsized":"2024-04-17T05:46:19.531942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.531732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:32.490714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'319","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.744134+0000","last_change":"2024-04-17T05:46:23.744249+0000","last_active":"2024-04-17T05:46:23.744134+0000","last_peered":"2024-04-17T05:46:23.744134+0000","last_clean":"2024-04-17T05:46:23.744134+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:46:23.744134+0000","last_undegraded":"2024-04-17T05:46:23.744134+0000","last_fullsized":"2024-04-17T05:46:23.744134+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.743937+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:03.777566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":535,"num_read_kb":1188,"num_write":472,"num_write_kb":375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15128,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":495,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.169401+0000","last_change":"2024-04-17T04:36:33.430909+0000","last_active":"2024-04-17T05:44:15.169401+0000","last_peered":"2024-04-17T05:44:15.169401+0000","last_clean":"2024-04-17T05:44:15.169401+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:44:15.169401+0000","last_undegraded":"2024-04-17T05:44:15.169401+0000","last_fullsized":"2024-04-17T05:44:15.169401+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'133","reported_seq":649,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.789721+0000","last_change":"2024-04-17T05:46:19.789793+0000","last_active":"2024-04-17T05:46:19.789721+0000","last_peered":"2024-04-17T05:46:19.789721+0000","last_clean":"2024-04-17T05:46:19.789721+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:46:19.789721+0000","last_undegraded":"2024-04-17T05:46:19.789721+0000","last_fullsized":"2024-04-17T05:46:19.789721+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.789529+0000","objects_scrubbed":4,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:17.259142+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":742,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.743991+0000","last_change":"2024-04-17T04:36:27.383527+0000","last_active":"2024-04-17T05:46:08.743991+0000","last_peered":"2024-04-17T05:46:08.743991+0000","last_clean":"2024-04-17T05:46:08.743991+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:46:08.743991+0000","last_undegraded":"2024-04-17T05:46:08.743991+0000","last_fullsized":"2024-04-17T05:46:08.743991+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'365","reported_seq":1032,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.870476+0000","last_change":"2024-04-17T04:36:25.371674+0000","last_active":"2024-04-17T05:46:03.870476+0000","last_peered":"2024-04-17T05:46:03.870476+0000","last_clean":"2024-04-17T05:46:03.870476+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:46:03.870476+0000","last_undegraded":"2024-04-17T05:46:03.870476+0000","last_fullsized":"2024-04-17T05:46:03.870476+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524382,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":609,"num_read_kb":15072,"num_write":1258,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":517,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.751463+0000","last_change":"2024-04-17T05:46:19.751517+0000","last_active":"2024-04-17T05:46:19.751463+0000","last_peered":"2024-04-17T05:46:19.751463+0000","last_clean":"2024-04-17T05:46:19.751463+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:46:19.751463+0000","last_undegraded":"2024-04-17T05:46:19.751463+0000","last_fullsized":"2024-04-17T05:46:19.751463+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.751263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:08.609702+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'298","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.671190+0000","last_change":"2024-04-17T05:46:11.671356+0000","last_active":"2024-04-17T05:46:11.671190+0000","last_peered":"2024-04-17T05:46:11.671190+0000","last_clean":"2024-04-17T05:46:11.671190+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:46:11.671190+0000","last_undegraded":"2024-04-17T05:46:11.671190+0000","last_fullsized":"2024-04-17T05:46:11.671190+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.670938+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:15.760804+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47207,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":490,"num_read_kb":1044,"num_write":380,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'512","reported_seq":2453,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.695035+0000","last_change":"2024-04-17T05:46:09.695035+0000","last_active":"2024-04-17T05:46:09.695035+0000","last_peered":"2024-04-17T05:46:09.695035+0000","last_clean":"2024-04-17T05:46:09.695035+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:46:09.695035+0000","last_undegraded":"2024-04-17T05:46:09.695035+0000","last_fullsized":"2024-04-17T05:46:09.695035+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'512","last_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_deep_scrub":"264'512","last_deep_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.694925+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.559083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'102","reported_seq":617,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.732298+0000","last_change":"2024-04-17T05:46:13.732374+0000","last_active":"2024-04-17T05:46:13.732298+0000","last_peered":"2024-04-17T05:46:13.732298+0000","last_clean":"2024-04-17T05:46:13.732298+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:46:13.732298+0000","last_undegraded":"2024-04-17T05:46:13.732298+0000","last_fullsized":"2024-04-17T05:46:13.732298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.732113+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:34.325042+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":732,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.743991+0000","last_change":"2024-04-17T04:36:27.393058+0000","last_active":"2024-04-17T05:46:08.743991+0000","last_peered":"2024-04-17T05:46:08.743991+0000","last_clean":"2024-04-17T05:46:08.743991+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:46:08.743991+0000","last_undegraded":"2024-04-17T05:46:08.743991+0000","last_fullsized":"2024-04-17T05:46:08.743991+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'295","reported_seq":986,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.889660+0000","last_change":"2024-04-17T04:36:25.356100+0000","last_active":"2024-04-17T05:46:03.889660+0000","last_peered":"2024-04-17T05:46:03.889660+0000","last_clean":"2024-04-17T05:46:03.889660+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:46:03.889660+0000","last_undegraded":"2024-04-17T05:46:03.889660+0000","last_fullsized":"2024-04-17T05:46:03.889660+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243143,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":575,"num_read_kb":5671,"num_write":1192,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"28'2","reported_seq":1155,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.203833+0000","last_change":"2024-04-17T04:36:29.389132+0000","last_active":"2024-04-17T05:46:01.203833+0000","last_peered":"2024-04-17T05:46:01.203833+0000","last_clean":"2024-04-17T05:46:01.203833+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:46:01.203833+0000","last_undegraded":"2024-04-17T05:46:01.203833+0000","last_fullsized":"2024-04-17T05:46:01.203833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":652,"num_read_kb":652,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'97","reported_seq":634,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276183+0000","last_change":"2024-04-17T04:36:31.421130+0000","last_active":"2024-04-17T05:46:08.276183+0000","last_peered":"2024-04-17T05:46:08.276183+0000","last_clean":"2024-04-17T05:46:08.276183+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:46:08.276183+0000","last_undegraded":"2024-04-17T05:46:08.276183+0000","last_fullsized":"2024-04-17T05:46:08.276183+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'539","reported_seq":2249,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.680532+0000","last_change":"2024-04-17T04:36:33.421354+0000","last_active":"2024-04-17T05:46:09.680532+0000","last_peered":"2024-04-17T05:46:09.680532+0000","last_clean":"2024-04-17T05:46:09.680532+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:46:09.680532+0000","last_undegraded":"2024-04-17T05:46:09.680532+0000","last_fullsized":"2024-04-17T05:46:09.680532+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'98","reported_seq":610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.796242+0000","last_change":"2024-04-17T05:46:11.796311+0000","last_active":"2024-04-17T05:46:11.796242+0000","last_peered":"2024-04-17T05:46:11.796242+0000","last_clean":"2024-04-17T05:46:11.796242+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:46:11.796242+0000","last_undegraded":"2024-04-17T05:46:11.796242+0000","last_fullsized":"2024-04-17T05:46:11.796242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'98","last_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_deep_scrub":"264'98","last_deep_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.796022+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:25.138191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":757,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.173100+0000","last_change":"2024-04-17T04:36:27.377947+0000","last_active":"2024-04-17T05:44:15.173100+0000","last_peered":"2024-04-17T05:44:15.173100+0000","last_clean":"2024-04-17T05:44:15.173100+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:44:15.173100+0000","last_undegraded":"2024-04-17T05:44:15.173100+0000","last_fullsized":"2024-04-17T05:44:15.173100+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'296","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.879310+0000","last_change":"2024-04-17T04:36:25.353720+0000","last_active":"2024-04-17T05:46:03.879310+0000","last_peered":"2024-04-17T05:46:03.879310+0000","last_clean":"2024-04-17T05:46:03.879310+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:46:03.879310+0000","last_undegraded":"2024-04-17T05:46:03.879310+0000","last_fullsized":"2024-04-17T05:46:03.879310+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170337,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":410,"num_read_kb":36760,"num_write":993,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.744522+0000","last_change":"2024-04-17T04:36:29.389735+0000","last_active":"2024-04-17T05:46:08.744522+0000","last_peered":"2024-04-17T05:46:08.744522+0000","last_clean":"2024-04-17T05:46:08.744522+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:46:08.744522+0000","last_undegraded":"2024-04-17T05:46:08.744522+0000","last_fullsized":"2024-04-17T05:46:08.744522+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'352","reported_seq":977,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.212939+0000","last_change":"2024-04-17T04:36:31.398102+0000","last_active":"2024-04-17T05:46:09.212939+0000","last_peered":"2024-04-17T05:46:09.212939+0000","last_clean":"2024-04-17T05:46:09.212939+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:46:09.212939+0000","last_undegraded":"2024-04-17T05:46:09.212939+0000","last_fullsized":"2024-04-17T05:46:09.212939+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45806,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1167,"num_write":448,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.992307+0000","last_change":"2024-04-17T05:46:21.992363+0000","last_active":"2024-04-17T05:46:21.992307+0000","last_peered":"2024-04-17T05:46:21.992307+0000","last_clean":"2024-04-17T05:46:21.992307+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:46:21.992307+0000","last_undegraded":"2024-04-17T05:46:21.992307+0000","last_fullsized":"2024-04-17T05:46:21.992307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.992088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:40.290560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'161","reported_seq":676,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.735879+0000","last_change":"2024-04-17T05:46:15.736060+0000","last_active":"2024-04-17T05:46:15.735879+0000","last_peered":"2024-04-17T05:46:15.735879+0000","last_clean":"2024-04-17T05:46:15.735879+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:46:15.735879+0000","last_undegraded":"2024-04-17T05:46:15.735879+0000","last_fullsized":"2024-04-17T05:46:15.735879+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.735662+0000","objects_scrubbed":4,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.803225+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.006094+0000","last_change":"2024-04-17T05:46:25.006216+0000","last_active":"2024-04-17T05:46:25.006094+0000","last_peered":"2024-04-17T05:46:25.006094+0000","last_clean":"2024-04-17T05:46:25.006094+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:46:25.006094+0000","last_undegraded":"2024-04-17T05:46:25.006094+0000","last_fullsized":"2024-04-17T05:46:25.006094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.005814+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:49.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":682,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155520+0000","last_change":"2024-04-17T04:36:12.831359+0000","last_active":"2024-04-17T05:46:10.155520+0000","last_peered":"2024-04-17T05:46:10.155520+0000","last_clean":"2024-04-17T05:46:10.155520+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:46:10.155520+0000","last_undegraded":"2024-04-17T05:46:10.155520+0000","last_fullsized":"2024-04-17T05:46:10.155520+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:11.058932+0000","last_clean_scrub_stamp":"2024-04-17T04:36:11.058932+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":522,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.182271+0000","last_change":"2024-04-17T04:36:17.217323+0000","last_active":"2024-04-17T05:45:05.182271+0000","last_peered":"2024-04-17T05:45:05.182271+0000","last_clean":"2024-04-17T05:45:05.182271+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:45:05.182271+0000","last_undegraded":"2024-04-17T05:45:05.182271+0000","last_fullsized":"2024-04-17T05:45:05.182271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'327","reported_seq":979,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.766624+0000","last_change":"2024-04-17T04:36:25.364067+0000","last_active":"2024-04-17T05:46:03.766624+0000","last_peered":"2024-04-17T05:46:03.766624+0000","last_clean":"2024-04-17T05:46:03.766624+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:46:03.766624+0000","last_undegraded":"2024-04-17T05:46:03.766624+0000","last_fullsized":"2024-04-17T05:46:03.766624+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344976,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":2736,"num_write":1215,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179367+0000","last_change":"2024-04-17T04:36:29.381453+0000","last_active":"2024-04-17T05:45:15.179367+0000","last_peered":"2024-04-17T05:45:15.179367+0000","last_clean":"2024-04-17T05:45:15.179367+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:45:15.179367+0000","last_undegraded":"2024-04-17T05:45:15.179367+0000","last_fullsized":"2024-04-17T05:45:15.179367+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'728","reported_seq":1458,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.227559+0000","last_change":"2024-04-17T05:46:23.227642+0000","last_active":"2024-04-17T05:46:23.227559+0000","last_peered":"2024-04-17T05:46:23.227559+0000","last_clean":"2024-04-17T05:46:23.227559+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:46:23.227559+0000","last_undegraded":"2024-04-17T05:46:23.227559+0000","last_fullsized":"2024-04-17T05:46:23.227559+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'728","last_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_deep_scrub":"264'728","last_deep_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.227369+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":700,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:26.750099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1411,"num_read_kb":3198,"num_write":1348,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":95136,"num_omap_keys":246,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.195948+0000","last_change":"2024-04-17T04:36:33.430394+0000","last_active":"2024-04-17T05:45:55.195948+0000","last_peered":"2024-04-17T05:45:55.195948+0000","last_clean":"2024-04-17T05:45:55.195948+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:45:55.195948+0000","last_undegraded":"2024-04-17T05:45:55.195948+0000","last_fullsized":"2024-04-17T05:45:55.195948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'220","reported_seq":914,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.188970+0000","last_change":"2024-04-17T04:36:35.421183+0000","last_active":"2024-04-17T05:45:55.188970+0000","last_peered":"2024-04-17T05:45:55.188970+0000","last_clean":"2024-04-17T05:45:55.188970+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:45:55.188970+0000","last_undegraded":"2024-04-17T05:45:55.188970+0000","last_fullsized":"2024-04-17T05:45:55.188970+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":669,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.194319+0000","last_change":"2024-04-17T04:36:27.365454+0000","last_active":"2024-04-17T05:45:50.194319+0000","last_peered":"2024-04-17T05:45:50.194319+0000","last_clean":"2024-04-17T05:45:50.194319+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:45:50.194319+0000","last_undegraded":"2024-04-17T05:45:50.194319+0000","last_fullsized":"2024-04-17T05:45:50.194319+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":525,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:30.169528+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T05:44:30.169528+0000","last_peered":"2024-04-17T05:44:30.169528+0000","last_clean":"2024-04-17T05:44:30.169528+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:44:30.169528+0000","last_undegraded":"2024-04-17T05:44:30.169528+0000","last_fullsized":"2024-04-17T05:44:30.169528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'343","reported_seq":996,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.858668+0000","last_change":"2024-04-17T04:36:25.355561+0000","last_active":"2024-04-17T05:46:03.858668+0000","last_peered":"2024-04-17T05:46:03.858668+0000","last_clean":"2024-04-17T05:46:03.858668+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:46:03.858668+0000","last_undegraded":"2024-04-17T05:46:03.858668+0000","last_fullsized":"2024-04-17T05:46:03.858668+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2309902,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":570,"num_read_kb":3714,"num_write":1237,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.192092+0000","last_change":"2024-04-17T04:36:29.393231+0000","last_active":"2024-04-17T05:45:50.192092+0000","last_peered":"2024-04-17T05:45:50.192092+0000","last_clean":"2024-04-17T05:45:50.192092+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:45:50.192092+0000","last_undegraded":"2024-04-17T05:45:50.192092+0000","last_fullsized":"2024-04-17T05:45:50.192092+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1101","reported_seq":2820,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.896219+0000","last_change":"2024-04-17T04:36:31.398322+0000","last_active":"2024-04-17T05:46:03.896219+0000","last_peered":"2024-04-17T05:46:03.896219+0000","last_clean":"2024-04-17T05:46:03.896219+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:46:03.896219+0000","last_undegraded":"2024-04-17T05:46:03.896219+0000","last_fullsized":"2024-04-17T05:46:03.896219+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":1000,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1819,"num_read_kb":3719,"num_write":1122,"num_write_kb":937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:30.183102+0000","last_change":"2024-04-17T04:36:33.422428+0000","last_active":"2024-04-17T05:45:30.183102+0000","last_peered":"2024-04-17T05:45:30.183102+0000","last_clean":"2024-04-17T05:45:30.183102+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:45:30.183102+0000","last_undegraded":"2024-04-17T05:45:30.183102+0000","last_fullsized":"2024-04-17T05:45:30.183102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'114","reported_seq":633,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.750777+0000","last_change":"2024-04-17T05:46:17.750850+0000","last_active":"2024-04-17T05:46:17.750777+0000","last_peered":"2024-04-17T05:46:17.750777+0000","last_clean":"2024-04-17T05:46:17.750777+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:46:17.750777+0000","last_undegraded":"2024-04-17T05:46:17.750777+0000","last_fullsized":"2024-04-17T05:46:17.750777+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'114","last_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_deep_scrub":"264'114","last_deep_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.750563+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:00.237624+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1018,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.197523+0000","last_change":"2024-04-17T04:36:27.376154+0000","last_active":"2024-04-17T05:45:50.197523+0000","last_peered":"2024-04-17T05:45:50.197523+0000","last_clean":"2024-04-17T05:45:50.197523+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:45:50.197523+0000","last_undegraded":"2024-04-17T05:45:50.197523+0000","last_fullsized":"2024-04-17T05:45:50.197523+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":539,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.432493+0000","last_change":"2024-04-17T05:46:24.432548+0000","last_active":"2024-04-17T05:46:24.432493+0000","last_peered":"2024-04-17T05:46:24.432493+0000","last_clean":"2024-04-17T05:46:24.432493+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:46:24.432493+0000","last_undegraded":"2024-04-17T05:46:24.432493+0000","last_fullsized":"2024-04-17T05:46:24.432493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.432302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:49.304795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'298","reported_seq":955,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.869934+0000","last_change":"2024-04-17T04:36:25.343663+0000","last_active":"2024-04-17T05:46:03.869934+0000","last_peered":"2024-04-17T05:46:03.869934+0000","last_clean":"2024-04-17T05:46:03.869934+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:46:03.869934+0000","last_undegraded":"2024-04-17T05:46:03.869934+0000","last_fullsized":"2024-04-17T05:46:03.869934+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5679739,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":479,"num_read_kb":21975,"num_write":1114,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"264'2","reported_seq":522,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.548976+0000","last_change":"2024-04-17T05:46:15.549042+0000","last_active":"2024-04-17T05:46:15.548976+0000","last_peered":"2024-04-17T05:46:15.548976+0000","last_clean":"2024-04-17T05:46:15.548976+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:46:15.548976+0000","last_undegraded":"2024-04-17T05:46:15.548976+0000","last_fullsized":"2024-04-17T05:46:15.548976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.548772+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:16:50.031612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'234","reported_seq":1619,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.886275+0000","last_change":"2024-04-17T04:36:31.427048+0000","last_active":"2024-04-17T05:45:54.886275+0000","last_peered":"2024-04-17T05:45:54.886275+0000","last_clean":"2024-04-17T05:45:54.886275+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:45:54.886275+0000","last_undegraded":"2024-04-17T05:45:54.886275+0000","last_fullsized":"2024-04-17T05:45:54.886275+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.186198+0000","last_change":"2024-04-17T04:36:33.426587+0000","last_active":"2024-04-17T05:45:20.186198+0000","last_peered":"2024-04-17T05:45:20.186198+0000","last_clean":"2024-04-17T05:45:20.186198+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:45:20.186198+0000","last_undegraded":"2024-04-17T05:45:20.186198+0000","last_fullsized":"2024-04-17T05:45:20.186198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'265","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.770655+0000","last_change":"2024-04-17T05:46:24.770806+0000","last_active":"2024-04-17T05:46:24.770655+0000","last_peered":"2024-04-17T05:46:24.770655+0000","last_clean":"2024-04-17T05:46:24.770655+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:46:24.770655+0000","last_undegraded":"2024-04-17T05:46:24.770655+0000","last_fullsized":"2024-04-17T05:46:24.770655+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'265","last_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_deep_scrub":"264'265","last_deep_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.770438+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:52.656005+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":950,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.189362+0000","last_change":"2024-04-17T04:36:27.361706+0000","last_active":"2024-04-17T05:45:15.189362+0000","last_peered":"2024-04-17T05:45:15.189362+0000","last_clean":"2024-04-17T05:45:15.189362+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:45:15.189362+0000","last_undegraded":"2024-04-17T05:45:15.189362+0000","last_fullsized":"2024-04-17T05:45:15.189362+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":522,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.182226+0000","last_change":"2024-04-17T04:36:17.216772+0000","last_active":"2024-04-17T05:45:05.182226+0000","last_peered":"2024-04-17T05:45:05.182226+0000","last_clean":"2024-04-17T05:45:05.182226+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:45:05.182226+0000","last_undegraded":"2024-04-17T05:45:05.182226+0000","last_fullsized":"2024-04-17T05:45:05.182226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'341","reported_seq":1099,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.850858+0000","last_change":"2024-04-17T04:36:25.356426+0000","last_active":"2024-04-17T05:46:03.850858+0000","last_peered":"2024-04-17T05:46:03.850858+0000","last_clean":"2024-04-17T05:46:03.850858+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:46:03.850858+0000","last_undegraded":"2024-04-17T05:46:03.850858+0000","last_fullsized":"2024-04-17T05:46:03.850858+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048803,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":626,"num_read_kb":36553,"num_write":1237,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:25.187604+0000","last_change":"2024-04-17T04:36:29.371781+0000","last_active":"2024-04-17T05:45:25.187604+0000","last_peered":"2024-04-17T05:45:25.187604+0000","last_clean":"2024-04-17T05:45:25.187604+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:45:25.187604+0000","last_undegraded":"2024-04-17T05:45:25.187604+0000","last_fullsized":"2024-04-17T05:45:25.187604+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'139","reported_seq":700,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.225319+0000","last_change":"2024-04-17T04:36:31.400012+0000","last_active":"2024-04-17T05:45:55.225319+0000","last_peered":"2024-04-17T05:45:55.225319+0000","last_clean":"2024-04-17T05:45:55.225319+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:45:55.225319+0000","last_undegraded":"2024-04-17T05:45:55.225319+0000","last_fullsized":"2024-04-17T05:45:55.225319+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.968621+0000","last_change":"2024-04-17T05:46:19.968684+0000","last_active":"2024-04-17T05:46:19.968621+0000","last_peered":"2024-04-17T05:46:19.968621+0000","last_clean":"2024-04-17T05:46:19.968621+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:46:19.968621+0000","last_undegraded":"2024-04-17T05:46:19.968621+0000","last_fullsized":"2024-04-17T05:46:19.968621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.968394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:02.908417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'134","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.184018+0000","last_change":"2024-04-17T05:46:17.184135+0000","last_active":"2024-04-17T05:46:17.184018+0000","last_peered":"2024-04-17T05:46:17.184018+0000","last_clean":"2024-04-17T05:46:17.184018+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:46:17.184018+0000","last_undegraded":"2024-04-17T05:46:17.184018+0000","last_fullsized":"2024-04-17T05:46:17.184018+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.183823+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.008088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":752,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.189265+0000","last_change":"2024-04-17T04:36:27.376368+0000","last_active":"2024-04-17T05:45:15.189265+0000","last_peered":"2024-04-17T05:45:15.189265+0000","last_clean":"2024-04-17T05:45:15.189265+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:45:15.189265+0000","last_undegraded":"2024-04-17T05:45:15.189265+0000","last_fullsized":"2024-04-17T05:45:15.189265+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'300","reported_seq":963,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.874830+0000","last_change":"2024-04-17T04:36:25.368624+0000","last_active":"2024-04-17T05:46:03.874830+0000","last_peered":"2024-04-17T05:46:03.874830+0000","last_clean":"2024-04-17T05:46:03.874830+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:46:03.874830+0000","last_undegraded":"2024-04-17T05:46:03.874830+0000","last_fullsized":"2024-04-17T05:46:03.874830+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":475,"num_read_kb":1448,"num_write":1131,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":529,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.187000+0000","last_change":"2024-04-17T04:36:17.873109+0000","last_active":"2024-04-17T05:45:05.187000+0000","last_peered":"2024-04-17T05:45:05.187000+0000","last_clean":"2024-04-17T05:45:05.187000+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:45:05.187000+0000","last_undegraded":"2024-04-17T05:45:05.187000+0000","last_fullsized":"2024-04-17T05:45:05.187000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184730+0000","last_change":"2024-04-17T04:36:29.384820+0000","last_active":"2024-04-17T05:45:15.184730+0000","last_peered":"2024-04-17T05:45:15.184730+0000","last_clean":"2024-04-17T05:45:15.184730+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:45:15.184730+0000","last_undegraded":"2024-04-17T05:45:15.184730+0000","last_fullsized":"2024-04-17T05:45:15.184730+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'620","reported_seq":1317,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.447905+0000","last_change":"2024-04-17T04:36:31.411434+0000","last_active":"2024-04-17T05:46:03.447905+0000","last_peered":"2024-04-17T05:46:03.447905+0000","last_clean":"2024-04-17T05:46:03.447905+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:46:03.447905+0000","last_undegraded":"2024-04-17T05:46:03.447905+0000","last_fullsized":"2024-04-17T05:46:03.447905+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":600,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86860,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1659,"num_read_kb":2834,"num_write":2174,"num_write_kb":1973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.791699+0000","last_change":"2024-04-17T05:46:17.791757+0000","last_active":"2024-04-17T05:46:17.791699+0000","last_peered":"2024-04-17T05:46:17.791699+0000","last_clean":"2024-04-17T05:46:17.791699+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:46:17.791699+0000","last_undegraded":"2024-04-17T05:46:17.791699+0000","last_fullsized":"2024-04-17T05:46:17.791699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.791493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:50.092156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'86","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.643211+0000","last_change":"2024-04-17T05:46:18.643211+0000","last_active":"2024-04-17T05:46:18.643211+0000","last_peered":"2024-04-17T05:46:18.643211+0000","last_clean":"2024-04-17T05:46:18.643211+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:46:18.643211+0000","last_undegraded":"2024-04-17T05:46:18.643211+0000","last_fullsized":"2024-04-17T05:46:18.643211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'86","last_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_deep_scrub":"264'86","last_deep_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.642880+0000","objects_scrubbed":4,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:33.629614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":593,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184661+0000","last_change":"2024-04-17T04:36:27.376396+0000","last_active":"2024-04-17T05:45:15.184661+0000","last_peered":"2024-04-17T05:45:15.184661+0000","last_clean":"2024-04-17T05:45:15.184661+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:45:15.184661+0000","last_undegraded":"2024-04-17T05:45:15.184661+0000","last_fullsized":"2024-04-17T05:45:15.184661+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'269","reported_seq":891,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.882299+0000","last_change":"2024-04-17T04:36:25.345048+0000","last_active":"2024-04-17T05:46:03.882299+0000","last_peered":"2024-04-17T05:46:03.882299+0000","last_clean":"2024-04-17T05:46:03.882299+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:46:03.882299+0000","last_undegraded":"2024-04-17T05:46:03.882299+0000","last_fullsized":"2024-04-17T05:46:03.882299+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194313,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":396,"num_read_kb":35213,"num_write":840,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":524,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179425+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T05:45:15.179425+0000","last_peered":"2024-04-17T05:45:15.179425+0000","last_clean":"2024-04-17T05:45:15.179425+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:45:15.179425+0000","last_undegraded":"2024-04-17T05:45:15.179425+0000","last_fullsized":"2024-04-17T05:45:15.179425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.958775+0000","last_change":"2024-04-17T05:46:20.958830+0000","last_active":"2024-04-17T05:46:20.958775+0000","last_peered":"2024-04-17T05:46:20.958775+0000","last_clean":"2024-04-17T05:46:20.958775+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:46:20.958775+0000","last_undegraded":"2024-04-17T05:46:20.958775+0000","last_fullsized":"2024-04-17T05:46:20.958775+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.958569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:07.756009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'199","reported_seq":793,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.190471+0000","last_change":"2024-04-17T04:36:31.420876+0000","last_active":"2024-04-17T05:45:20.190471+0000","last_peered":"2024-04-17T05:45:20.190471+0000","last_clean":"2024-04-17T05:45:20.190471+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:45:20.190471+0000","last_undegraded":"2024-04-17T05:45:20.190471+0000","last_fullsized":"2024-04-17T05:45:20.190471+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'540","reported_seq":4367,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.162098+0000","last_change":"2024-04-17T05:46:21.162167+0000","last_active":"2024-04-17T05:46:21.162098+0000","last_peered":"2024-04-17T05:46:21.162098+0000","last_clean":"2024-04-17T05:46:21.162098+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:46:21.162098+0000","last_undegraded":"2024-04-17T05:46:21.162098+0000","last_fullsized":"2024-04-17T05:46:21.162098+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'540","last_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_deep_scrub":"264'540","last_deep_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.161902+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":500,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:14.043415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'161","reported_seq":697,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.780526+0000","last_change":"2024-04-17T05:46:23.780606+0000","last_active":"2024-04-17T05:46:23.780526+0000","last_peered":"2024-04-17T05:46:23.780526+0000","last_clean":"2024-04-17T05:46:23.780526+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:46:23.780526+0000","last_undegraded":"2024-04-17T05:46:23.780526+0000","last_fullsized":"2024-04-17T05:46:23.780526+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.780338+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:57.240957+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":800,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:45.208033+0000","last_change":"2024-04-17T04:36:27.395791+0000","last_active":"2024-04-17T05:44:45.208033+0000","last_peered":"2024-04-17T05:44:45.208033+0000","last_clean":"2024-04-17T05:44:45.208033+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:44:45.208033+0000","last_undegraded":"2024-04-17T05:44:45.208033+0000","last_fullsized":"2024-04-17T05:44:45.208033+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'379","reported_seq":1037,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.847763+0000","last_change":"2024-04-17T04:36:25.347483+0000","last_active":"2024-04-17T05:46:03.847763+0000","last_peered":"2024-04-17T05:46:03.847763+0000","last_clean":"2024-04-17T05:46:03.847763+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:46:03.847763+0000","last_undegraded":"2024-04-17T05:46:03.847763+0000","last_fullsized":"2024-04-17T05:46:03.847763+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262239,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":591,"num_read_kb":5844,"num_write":1385,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:35.193947+0000","last_change":"2024-04-17T04:36:17.212585+0000","last_active":"2024-04-17T05:45:35.193947+0000","last_peered":"2024-04-17T05:45:35.193947+0000","last_clean":"2024-04-17T05:45:35.193947+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:45:35.193947+0000","last_undegraded":"2024-04-17T05:45:35.193947+0000","last_fullsized":"2024-04-17T05:45:35.193947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.174137+0000","last_change":"2024-04-17T04:36:29.375601+0000","last_active":"2024-04-17T05:45:15.174137+0000","last_peered":"2024-04-17T05:45:15.174137+0000","last_clean":"2024-04-17T05:45:15.174137+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:45:15.174137+0000","last_undegraded":"2024-04-17T05:45:15.174137+0000","last_fullsized":"2024-04-17T05:45:15.174137+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'142","reported_seq":713,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.761857+0000","last_change":"2024-04-17T05:46:21.761940+0000","last_active":"2024-04-17T05:46:21.761857+0000","last_peered":"2024-04-17T05:46:21.761857+0000","last_clean":"2024-04-17T05:46:21.761857+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:46:21.761857+0000","last_undegraded":"2024-04-17T05:46:21.761857+0000","last_fullsized":"2024-04-17T05:46:21.761857+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'142","last_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_deep_scrub":"264'142","last_deep_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.761664+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:22.526188+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'533","reported_seq":2305,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.902024+0000","last_change":"2024-04-17T04:36:33.421012+0000","last_active":"2024-04-17T05:46:03.902024+0000","last_peered":"2024-04-17T05:46:03.902024+0000","last_clean":"2024-04-17T05:46:03.902024+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:46:03.902024+0000","last_undegraded":"2024-04-17T05:46:03.902024+0000","last_fullsized":"2024-04-17T05:46:03.902024+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":500,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'224","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.621850+0000","last_change":"2024-04-17T05:46:17.621850+0000","last_active":"2024-04-17T05:46:17.621850+0000","last_peered":"2024-04-17T05:46:17.621850+0000","last_clean":"2024-04-17T05:46:17.621850+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:46:17.621850+0000","last_undegraded":"2024-04-17T05:46:17.621850+0000","last_fullsized":"2024-04-17T05:46:17.621850+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'224","last_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_deep_scrub":"264'224","last_deep_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.621550+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:22:57.559950+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4093","reported_seq":4725,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.174109+0000","last_change":"2024-04-17T04:36:27.378432+0000","last_active":"2024-04-17T05:45:15.174109+0000","last_peered":"2024-04-17T05:45:15.174109+0000","last_clean":"2024-04-17T05:45:15.174109+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:45:15.174109+0000","last_undegraded":"2024-04-17T05:45:15.174109+0000","last_fullsized":"2024-04-17T05:45:15.174109+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":3000,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'356","reported_seq":974,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.886839+0000","last_change":"2024-04-17T04:36:25.354707+0000","last_active":"2024-04-17T05:46:03.886839+0000","last_peered":"2024-04-17T05:46:03.886839+0000","last_clean":"2024-04-17T05:46:03.886839+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:46:03.886839+0000","last_undegraded":"2024-04-17T05:46:03.886839+0000","last_fullsized":"2024-04-17T05:46:03.886839+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145969,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":466,"num_read_kb":7519,"num_write":1284,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.185742+0000","last_change":"2024-04-17T04:36:17.217883+0000","last_active":"2024-04-17T05:45:40.185742+0000","last_peered":"2024-04-17T05:45:40.185742+0000","last_clean":"2024-04-17T05:45:40.185742+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:45:40.185742+0000","last_undegraded":"2024-04-17T05:45:40.185742+0000","last_fullsized":"2024-04-17T05:45:40.185742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:45.207994+0000","last_change":"2024-04-17T04:36:29.363086+0000","last_active":"2024-04-17T05:44:45.207994+0000","last_peered":"2024-04-17T05:44:45.207994+0000","last_clean":"2024-04-17T05:44:45.207994+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:44:45.207994+0000","last_undegraded":"2024-04-17T05:44:45.207994+0000","last_fullsized":"2024-04-17T05:44:45.207994+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'730","reported_seq":1475,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.686406+0000","last_change":"2024-04-17T04:36:31.422637+0000","last_active":"2024-04-17T05:46:03.686406+0000","last_peered":"2024-04-17T05:46:03.686406+0000","last_clean":"2024-04-17T05:46:03.686406+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:46:03.686406+0000","last_undegraded":"2024-04-17T05:46:03.686406+0000","last_fullsized":"2024-04-17T05:46:03.686406+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138854,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1296,"num_read_kb":3065,"num_write":1038,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":500,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.216780+0000","last_change":"2024-04-17T04:36:33.415987+0000","last_active":"2024-04-17T05:45:20.216780+0000","last_peered":"2024-04-17T05:45:20.216780+0000","last_clean":"2024-04-17T05:45:20.216780+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:45:20.216780+0000","last_undegraded":"2024-04-17T05:45:20.216780+0000","last_fullsized":"2024-04-17T05:45:20.216780+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'105","reported_seq":604,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.069586+0000","last_change":"2024-04-17T05:46:13.069586+0000","last_active":"2024-04-17T05:46:13.069586+0000","last_peered":"2024-04-17T05:46:13.069586+0000","last_clean":"2024-04-17T05:46:13.069586+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:46:13.069586+0000","last_undegraded":"2024-04-17T05:46:13.069586+0000","last_fullsized":"2024-04-17T05:46:13.069586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.069188+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:06.656617+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'151","reported_seq":727,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.648354+0000","last_change":"2024-04-17T05:46:24.648424+0000","last_active":"2024-04-17T05:46:24.648354+0000","last_peered":"2024-04-17T05:46:24.648354+0000","last_clean":"2024-04-17T05:46:24.648354+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:46:24.648354+0000","last_undegraded":"2024-04-17T05:46:24.648354+0000","last_fullsized":"2024-04-17T05:46:24.648354+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'151","last_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_deep_scrub":"264'151","last_deep_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.648141+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:48.144202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'73","reported_seq":675,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159499+0000","last_change":"2024-04-17T04:36:27.400403+0000","last_active":"2024-04-17T05:44:10.159499+0000","last_peered":"2024-04-17T05:44:10.159499+0000","last_clean":"2024-04-17T05:44:10.159499+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:44:10.159499+0000","last_undegraded":"2024-04-17T05:44:10.159499+0000","last_fullsized":"2024-04-17T05:44:10.159499+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'281","reported_seq":920,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.878211+0000","last_change":"2024-04-17T04:36:25.357052+0000","last_active":"2024-04-17T05:46:03.878211+0000","last_peered":"2024-04-17T05:46:03.878211+0000","last_clean":"2024-04-17T05:46:03.878211+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:46:03.878211+0000","last_undegraded":"2024-04-17T05:46:03.878211+0000","last_fullsized":"2024-04-17T05:46:03.878211+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017984,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":431,"num_read_kb":11105,"num_write":979,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":515,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.773115+0000","last_change":"2024-04-17T05:46:22.773169+0000","last_active":"2024-04-17T05:46:22.773115+0000","last_peered":"2024-04-17T05:46:22.773115+0000","last_clean":"2024-04-17T05:46:22.773115+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:46:22.773115+0000","last_undegraded":"2024-04-17T05:46:22.773115+0000","last_fullsized":"2024-04-17T05:46:22.773115+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.772889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:13.164944+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'301","reported_seq":897,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.659366+0000","last_change":"2024-04-17T04:36:31.406561+0000","last_active":"2024-04-17T05:46:01.659366+0000","last_peered":"2024-04-17T05:46:01.659366+0000","last_clean":"2024-04-17T05:46:01.659366+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:46:01.659366+0000","last_undegraded":"2024-04-17T05:46:01.659366+0000","last_fullsized":"2024-04-17T05:46:01.659366+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44026,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":524,"num_read_kb":1245,"num_write":424,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":496,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.200841+0000","last_change":"2024-04-17T04:36:33.420539+0000","last_active":"2024-04-17T05:44:15.200841+0000","last_peered":"2024-04-17T05:44:15.200841+0000","last_clean":"2024-04-17T05:44:15.200841+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:44:15.200841+0000","last_undegraded":"2024-04-17T05:44:15.200841+0000","last_fullsized":"2024-04-17T05:44:15.200841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'99","reported_seq":613,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.795059+0000","last_change":"2024-04-17T05:46:17.795125+0000","last_active":"2024-04-17T05:46:17.795059+0000","last_peered":"2024-04-17T05:46:17.795059+0000","last_clean":"2024-04-17T05:46:17.795059+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:46:17.795059+0000","last_undegraded":"2024-04-17T05:46:17.795059+0000","last_fullsized":"2024-04-17T05:46:17.795059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'99","last_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_deep_scrub":"264'99","last_deep_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.794856+0000","objects_scrubbed":1,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:18.080437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'127","reported_seq":747,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.804122+0000","last_change":"2024-04-17T05:46:23.804194+0000","last_active":"2024-04-17T05:46:23.804122+0000","last_peered":"2024-04-17T05:46:23.804122+0000","last_clean":"2024-04-17T05:46:23.804122+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:46:23.804122+0000","last_undegraded":"2024-04-17T05:46:23.804122+0000","last_fullsized":"2024-04-17T05:46:23.804122+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'127","last_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_deep_scrub":"264'127","last_deep_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.803934+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:22.971464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'373","reported_seq":1034,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.683941+0000","last_change":"2024-04-17T05:46:12.684055+0000","last_active":"2024-04-17T05:46:12.683941+0000","last_peered":"2024-04-17T05:46:12.683941+0000","last_clean":"2024-04-17T05:46:12.683941+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:46:12.683941+0000","last_undegraded":"2024-04-17T05:46:12.683941+0000","last_fullsized":"2024-04-17T05:46:12.683941+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.683727+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:58.133653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":625,"num_read_kb":15946,"num_write":1543,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.199688+0000","last_change":"2024-04-17T04:36:29.371357+0000","last_active":"2024-04-17T05:44:10.199688+0000","last_peered":"2024-04-17T05:44:10.199688+0000","last_clean":"2024-04-17T05:44:10.199688+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:44:10.199688+0000","last_undegraded":"2024-04-17T05:44:10.199688+0000","last_fullsized":"2024-04-17T05:44:10.199688+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'56","reported_seq":568,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:44.783818+0000","last_change":"2024-04-17T04:36:31.404202+0000","last_active":"2024-04-17T05:45:44.783818+0000","last_peered":"2024-04-17T05:45:44.783818+0000","last_clean":"2024-04-17T05:45:44.783818+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:45:44.783818+0000","last_undegraded":"2024-04-17T05:45:44.783818+0000","last_fullsized":"2024-04-17T05:45:44.783818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":494,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166244+0000","last_change":"2024-04-17T04:36:33.416154+0000","last_active":"2024-04-17T05:44:15.166244+0000","last_peered":"2024-04-17T05:44:15.166244+0000","last_clean":"2024-04-17T05:44:15.166244+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:44:15.166244+0000","last_undegraded":"2024-04-17T05:44:15.166244+0000","last_fullsized":"2024-04-17T05:44:15.166244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'219","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.647638+0000","last_change":"2024-04-17T05:46:20.647715+0000","last_active":"2024-04-17T05:46:20.647638+0000","last_peered":"2024-04-17T05:46:20.647638+0000","last_clean":"2024-04-17T05:46:20.647638+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:46:20.647638+0000","last_undegraded":"2024-04-17T05:46:20.647638+0000","last_fullsized":"2024-04-17T05:46:20.647638+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'219","last_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_deep_scrub":"264'219","last_deep_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.647446+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:57.324481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":799,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.199545+0000","last_change":"2024-04-17T04:36:27.395464+0000","last_active":"2024-04-17T05:44:10.199545+0000","last_peered":"2024-04-17T05:44:10.199545+0000","last_clean":"2024-04-17T05:44:10.199545+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:44:10.199545+0000","last_undegraded":"2024-04-17T05:44:10.199545+0000","last_fullsized":"2024-04-17T05:44:10.199545+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'356","reported_seq":1036,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.871732+0000","last_change":"2024-04-17T04:36:25.338779+0000","last_active":"2024-04-17T05:46:03.871732+0000","last_peered":"2024-04-17T05:46:03.871732+0000","last_clean":"2024-04-17T05:46:03.871732+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:46:03.871732+0000","last_undegraded":"2024-04-17T05:46:03.871732+0000","last_fullsized":"2024-04-17T05:46:03.871732+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":588,"num_read_kb":11343,"num_write":1303,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.165163+0000","last_change":"2024-04-17T04:36:29.381180+0000","last_active":"2024-04-17T05:44:10.165163+0000","last_peered":"2024-04-17T05:44:10.165163+0000","last_clean":"2024-04-17T05:44:10.165163+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:44:10.165163+0000","last_undegraded":"2024-04-17T05:44:10.165163+0000","last_fullsized":"2024-04-17T05:44:10.165163+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'340","reported_seq":946,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.290315+0000","last_change":"2024-04-17T04:36:31.406900+0000","last_active":"2024-04-17T05:46:03.290315+0000","last_peered":"2024-04-17T05:46:03.290315+0000","last_clean":"2024-04-17T05:46:03.290315+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:46:03.290315+0000","last_undegraded":"2024-04-17T05:46:03.290315+0000","last_fullsized":"2024-04-17T05:46:03.290315+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48000,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":552,"num_read_kb":1237,"num_write":440,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.983345+0000","last_change":"2024-04-17T05:46:17.983452+0000","last_active":"2024-04-17T05:46:17.983345+0000","last_peered":"2024-04-17T05:46:17.983345+0000","last_clean":"2024-04-17T05:46:17.983345+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:46:17.983345+0000","last_undegraded":"2024-04-17T05:46:17.983345+0000","last_fullsized":"2024-04-17T05:46:17.983345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.983070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:29.782925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'113","reported_seq":609,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.111837+0000","last_change":"2024-04-17T05:46:14.111907+0000","last_active":"2024-04-17T05:46:14.111837+0000","last_peered":"2024-04-17T05:46:14.111837+0000","last_clean":"2024-04-17T05:46:14.111837+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:46:14.111837+0000","last_undegraded":"2024-04-17T05:46:14.111837+0000","last_fullsized":"2024-04-17T05:46:14.111837+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'113","last_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_deep_scrub":"264'113","last_deep_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.111624+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:35.374698+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":760,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.199464+0000","last_change":"2024-04-17T04:36:27.378417+0000","last_active":"2024-04-17T05:44:10.199464+0000","last_peered":"2024-04-17T05:44:10.199464+0000","last_clean":"2024-04-17T05:44:10.199464+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:44:10.199464+0000","last_undegraded":"2024-04-17T05:44:10.199464+0000","last_fullsized":"2024-04-17T05:44:10.199464+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'310","reported_seq":959,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.828670+0000","last_change":"2024-04-17T04:36:25.344700+0000","last_active":"2024-04-17T05:46:03.828670+0000","last_peered":"2024-04-17T05:46:03.828670+0000","last_clean":"2024-04-17T05:46:03.828670+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:46:03.828670+0000","last_undegraded":"2024-04-17T05:46:03.828670+0000","last_fullsized":"2024-04-17T05:46:03.828670+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12846794,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":445,"num_read_kb":84514,"num_write":990,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.942318+0000","last_change":"2024-04-17T05:46:16.942372+0000","last_active":"2024-04-17T05:46:16.942318+0000","last_peered":"2024-04-17T05:46:16.942318+0000","last_clean":"2024-04-17T05:46:16.942318+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:46:16.942318+0000","last_undegraded":"2024-04-17T05:46:16.942318+0000","last_fullsized":"2024-04-17T05:46:16.942318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.942121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:13.925928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'258","reported_seq":839,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.053967+0000","last_change":"2024-04-17T04:36:31.380383+0000","last_active":"2024-04-17T05:46:03.053967+0000","last_peered":"2024-04-17T05:46:03.053967+0000","last_clean":"2024-04-17T05:46:03.053967+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:46:03.053967+0000","last_undegraded":"2024-04-17T05:46:03.053967+0000","last_fullsized":"2024-04-17T05:46:03.053967+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47325,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":432,"num_read_kb":996,"num_write":342,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":494,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.169079+0000","last_change":"2024-04-17T04:36:33.429804+0000","last_active":"2024-04-17T05:44:15.169079+0000","last_peered":"2024-04-17T05:44:15.169079+0000","last_clean":"2024-04-17T05:44:15.169079+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:44:15.169079+0000","last_undegraded":"2024-04-17T05:44:15.169079+0000","last_fullsized":"2024-04-17T05:44:15.169079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'290","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.020201+0000","last_change":"2024-04-17T05:46:23.020277+0000","last_active":"2024-04-17T05:46:23.020201+0000","last_peered":"2024-04-17T05:46:23.020201+0000","last_clean":"2024-04-17T05:46:23.020201+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:46:23.020201+0000","last_undegraded":"2024-04-17T05:46:23.020201+0000","last_fullsized":"2024-04-17T05:46:23.020201+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'290","last_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_deep_scrub":"264'290","last_deep_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.019975+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:21.304396+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'313","reported_seq":999,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.882353+0000","last_change":"2024-04-17T04:36:25.326760+0000","last_active":"2024-04-17T05:46:03.882353+0000","last_peered":"2024-04-17T05:46:03.882353+0000","last_clean":"2024-04-17T05:46:03.882353+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:46:03.882353+0000","last_undegraded":"2024-04-17T05:46:03.882353+0000","last_fullsized":"2024-04-17T05:46:03.882353+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752753,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":542,"num_read_kb":33053,"num_write":1174,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:40.175911+0000","last_change":"2024-04-17T04:36:29.373251+0000","last_active":"2024-04-17T05:44:40.175911+0000","last_peered":"2024-04-17T05:44:40.175911+0000","last_clean":"2024-04-17T05:44:40.175911+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:44:40.175911+0000","last_undegraded":"2024-04-17T05:44:40.175911+0000","last_fullsized":"2024-04-17T05:44:40.175911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'480","reported_seq":1140,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.798135+0000","last_change":"2024-04-17T05:46:24.798222+0000","last_active":"2024-04-17T05:46:24.798135+0000","last_peered":"2024-04-17T05:46:24.798135+0000","last_clean":"2024-04-17T05:46:24.798135+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:46:24.798135+0000","last_undegraded":"2024-04-17T05:46:24.798135+0000","last_fullsized":"2024-04-17T05:46:24.798135+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'480","last_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_deep_scrub":"264'480","last_deep_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.797947+0000","objects_scrubbed":7,"log_size":80,"log_dups_size":400,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:12.511479+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87094,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":846,"num_read_kb":2027,"num_write":720,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22936,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.147563+0000","last_change":"2024-04-17T05:46:19.147623+0000","last_active":"2024-04-17T05:46:19.147563+0000","last_peered":"2024-04-17T05:46:19.147563+0000","last_clean":"2024-04-17T05:46:19.147563+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:46:19.147563+0000","last_undegraded":"2024-04-17T05:46:19.147563+0000","last_fullsized":"2024-04-17T05:46:19.147563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.147370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:18.842204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'174","reported_seq":690,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.787258+0000","last_change":"2024-04-17T05:46:21.787327+0000","last_active":"2024-04-17T05:46:21.787258+0000","last_peered":"2024-04-17T05:46:21.787258+0000","last_clean":"2024-04-17T05:46:21.787258+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:46:21.787258+0000","last_undegraded":"2024-04-17T05:46:21.787258+0000","last_fullsized":"2024-04-17T05:46:21.787258+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.787059+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:11.126921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":710,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:40.175836+0000","last_change":"2024-04-17T04:36:27.386903+0000","last_active":"2024-04-17T05:44:40.175836+0000","last_peered":"2024-04-17T05:44:40.175836+0000","last_clean":"2024-04-17T05:44:40.175836+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:44:40.175836+0000","last_undegraded":"2024-04-17T05:44:40.175836+0000","last_fullsized":"2024-04-17T05:44:40.175836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'315","reported_seq":997,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.882788+0000","last_change":"2024-04-17T04:36:25.357464+0000","last_active":"2024-04-17T05:46:03.882788+0000","last_peered":"2024-04-17T05:46:03.882788+0000","last_clean":"2024-04-17T05:46:03.882788+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:46:03.882788+0000","last_undegraded":"2024-04-17T05:46:03.882788+0000","last_fullsized":"2024-04-17T05:46:03.882788+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122783,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":510,"num_read_kb":112390,"num_write":1223,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.012291+0000","last_change":"2024-04-17T05:46:19.012353+0000","last_active":"2024-04-17T05:46:19.012291+0000","last_peered":"2024-04-17T05:46:19.012291+0000","last_clean":"2024-04-17T05:46:19.012291+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:46:19.012291+0000","last_undegraded":"2024-04-17T05:46:19.012291+0000","last_fullsized":"2024-04-17T05:46:19.012291+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.012071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:43.685517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'315","reported_seq":915,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.133827+0000","last_change":"2024-04-17T04:36:31.402737+0000","last_active":"2024-04-17T05:46:03.133827+0000","last_peered":"2024-04-17T05:46:03.133827+0000","last_clean":"2024-04-17T05:46:03.133827+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:46:03.133827+0000","last_undegraded":"2024-04-17T05:46:03.133827+0000","last_fullsized":"2024-04-17T05:46:03.133827+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40265,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":1168,"num_write":442,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":497,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.170837+0000","last_change":"2024-04-17T04:36:33.415254+0000","last_active":"2024-04-17T05:44:05.170837+0000","last_peered":"2024-04-17T05:44:05.170837+0000","last_clean":"2024-04-17T05:44:05.170837+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:44:05.170837+0000","last_undegraded":"2024-04-17T05:44:05.170837+0000","last_fullsized":"2024-04-17T05:44:05.170837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'175","reported_seq":712,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:39.566280+0000","last_change":"2024-04-17T04:36:35.424191+0000","last_active":"2024-04-17T05:45:39.566280+0000","last_peered":"2024-04-17T05:45:39.566280+0000","last_clean":"2024-04-17T05:45:39.566280+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:45:39.566280+0000","last_undegraded":"2024-04-17T05:45:39.566280+0000","last_fullsized":"2024-04-17T05:45:39.566280+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'114","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.196994+0000","last_change":"2024-04-17T04:36:27.378054+0000","last_active":"2024-04-17T05:44:00.196994+0000","last_peered":"2024-04-17T05:44:00.196994+0000","last_clean":"2024-04-17T05:44:00.196994+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:44:00.196994+0000","last_undegraded":"2024-04-17T05:44:00.196994+0000","last_fullsized":"2024-04-17T05:44:00.196994+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'358","reported_seq":1044,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.743644+0000","last_change":"2024-04-17T04:36:25.356583+0000","last_active":"2024-04-17T05:46:08.743644+0000","last_peered":"2024-04-17T05:46:08.743644+0000","last_clean":"2024-04-17T05:46:08.743644+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:46:08.743644+0000","last_undegraded":"2024-04-17T05:46:08.743644+0000","last_fullsized":"2024-04-17T05:46:08.743644+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568179,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":576,"num_read_kb":10795,"num_write":1350,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":834,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.204545+0000","last_change":"2024-04-17T04:36:29.370143+0000","last_active":"2024-04-17T05:46:01.204545+0000","last_peered":"2024-04-17T05:46:01.204545+0000","last_clean":"2024-04-17T05:46:01.204545+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:46:01.204545+0000","last_undegraded":"2024-04-17T05:46:01.204545+0000","last_fullsized":"2024-04-17T05:46:01.204545+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":499,"num_read_kb":497,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'298","reported_seq":898,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.011004+0000","last_change":"2024-04-17T04:36:31.421343+0000","last_active":"2024-04-17T05:46:02.011004+0000","last_peered":"2024-04-17T05:46:02.011004+0000","last_clean":"2024-04-17T05:46:02.011004+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:46:02.011004+0000","last_undegraded":"2024-04-17T05:46:02.011004+0000","last_fullsized":"2024-04-17T05:46:02.011004+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45015,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":503,"num_read_kb":1173,"num_write":404,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:50.184094+0000","last_change":"2024-04-17T04:36:33.415113+0000","last_active":"2024-04-17T05:44:50.184094+0000","last_peered":"2024-04-17T05:44:50.184094+0000","last_clean":"2024-04-17T05:44:50.184094+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:44:50.184094+0000","last_undegraded":"2024-04-17T05:44:50.184094+0000","last_fullsized":"2024-04-17T05:44:50.184094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'105","reported_seq":613,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.146503+0000","last_change":"2024-04-17T05:46:11.146585+0000","last_active":"2024-04-17T05:46:11.146503+0000","last_peered":"2024-04-17T05:46:11.146503+0000","last_clean":"2024-04-17T05:46:11.146503+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:46:11.146503+0000","last_undegraded":"2024-04-17T05:46:11.146503+0000","last_fullsized":"2024-04-17T05:46:11.146503+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.146298+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:24.914414+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":675,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.166668+0000","last_change":"2024-04-17T04:36:27.356010+0000","last_active":"2024-04-17T05:44:00.166668+0000","last_peered":"2024-04-17T05:44:00.166668+0000","last_clean":"2024-04-17T05:44:00.166668+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:44:00.166668+0000","last_undegraded":"2024-04-17T05:44:00.166668+0000","last_fullsized":"2024-04-17T05:44:00.166668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'322","reported_seq":1149,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934149+0000","last_change":"2024-04-17T04:36:25.354574+0000","last_active":"2024-04-17T05:46:06.934149+0000","last_peered":"2024-04-17T05:46:06.934149+0000","last_clean":"2024-04-17T05:46:06.934149+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:46:06.934149+0000","last_undegraded":"2024-04-17T05:46:06.934149+0000","last_fullsized":"2024-04-17T05:46:06.934149+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263230,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":844,"num_read_kb":11279,"num_write":1205,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.722399+0000","last_change":"2024-04-17T05:46:18.722456+0000","last_active":"2024-04-17T05:46:18.722399+0000","last_peered":"2024-04-17T05:46:18.722399+0000","last_clean":"2024-04-17T05:46:18.722399+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:46:18.722399+0000","last_undegraded":"2024-04-17T05:46:18.722399+0000","last_fullsized":"2024-04-17T05:46:18.722399+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.722193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:41.699090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'126","reported_seq":682,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.818411+0000","last_change":"2024-04-17T05:46:19.818501+0000","last_active":"2024-04-17T05:46:19.818411+0000","last_peered":"2024-04-17T05:46:19.818411+0000","last_clean":"2024-04-17T05:46:19.818411+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:46:19.818411+0000","last_undegraded":"2024-04-17T05:46:19.818411+0000","last_fullsized":"2024-04-17T05:46:19.818411+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'126","last_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_deep_scrub":"264'126","last_deep_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.818199+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:31.924099+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1648,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'539","reported_seq":2323,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.960848+0000","last_change":"2024-04-17T05:46:15.961007+0000","last_active":"2024-04-17T05:46:15.960848+0000","last_peered":"2024-04-17T05:46:15.960848+0000","last_clean":"2024-04-17T05:46:15.960848+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:46:15.960848+0000","last_undegraded":"2024-04-17T05:46:15.960848+0000","last_fullsized":"2024-04-17T05:46:15.960848+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.960604+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:11.021591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'114","reported_seq":615,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.166186+0000","last_change":"2024-04-17T04:36:35.413901+0000","last_active":"2024-04-17T05:44:15.166186+0000","last_peered":"2024-04-17T05:44:15.166186+0000","last_clean":"2024-04-17T05:44:15.166186+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:44:15.166186+0000","last_undegraded":"2024-04-17T05:44:15.166186+0000","last_fullsized":"2024-04-17T05:44:15.166186+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'229","reported_seq":876,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.156599+0000","last_change":"2024-04-17T04:36:27.394864+0000","last_active":"2024-04-17T05:44:00.156599+0000","last_peered":"2024-04-17T05:44:00.156599+0000","last_clean":"2024-04-17T05:44:00.156599+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:44:00.156599+0000","last_undegraded":"2024-04-17T05:44:00.156599+0000","last_fullsized":"2024-04-17T05:44:00.156599+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'330","reported_seq":931,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.211862+0000","last_change":"2024-04-17T04:36:25.369133+0000","last_active":"2024-04-17T05:46:09.211862+0000","last_peered":"2024-04-17T05:46:09.211862+0000","last_clean":"2024-04-17T05:46:09.211862+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:46:09.211862+0000","last_undegraded":"2024-04-17T05:46:09.211862+0000","last_fullsized":"2024-04-17T05:46:09.211862+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244482,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":438,"num_read_kb":414,"num_write":1194,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":500,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.162708+0000","last_change":"2024-04-17T04:36:29.379152+0000","last_active":"2024-04-17T05:44:00.162708+0000","last_peered":"2024-04-17T05:44:00.162708+0000","last_clean":"2024-04-17T05:44:00.162708+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:44:00.162708+0000","last_undegraded":"2024-04-17T05:44:00.162708+0000","last_fullsized":"2024-04-17T05:44:00.162708+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'341","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.457762+0000","last_change":"2024-04-17T04:36:31.396219+0000","last_active":"2024-04-17T05:46:02.457762+0000","last_peered":"2024-04-17T05:46:02.457762+0000","last_clean":"2024-04-17T05:46:02.457762+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:46:02.457762+0000","last_undegraded":"2024-04-17T05:46:02.457762+0000","last_fullsized":"2024-04-17T05:46:02.457762+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44869,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":547,"num_read_kb":1136,"num_write":426,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.168889+0000","last_change":"2024-04-17T04:36:33.430639+0000","last_active":"2024-04-17T05:44:15.168889+0000","last_peered":"2024-04-17T05:44:15.168889+0000","last_clean":"2024-04-17T05:44:15.168889+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:44:15.168889+0000","last_undegraded":"2024-04-17T05:44:15.168889+0000","last_fullsized":"2024-04-17T05:44:15.168889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'127","reported_seq":630,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.836478+0000","last_change":"2024-04-17T04:36:35.436213+0000","last_active":"2024-04-17T05:45:40.836478+0000","last_peered":"2024-04-17T05:45:40.836478+0000","last_clean":"2024-04-17T05:45:40.836478+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:45:40.836478+0000","last_undegraded":"2024-04-17T05:45:40.836478+0000","last_fullsized":"2024-04-17T05:45:40.836478+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":805,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.156489+0000","last_change":"2024-04-17T04:36:27.378160+0000","last_active":"2024-04-17T05:44:00.156489+0000","last_peered":"2024-04-17T05:44:00.156489+0000","last_clean":"2024-04-17T05:44:00.156489+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:44:00.156489+0000","last_undegraded":"2024-04-17T05:44:00.156489+0000","last_fullsized":"2024-04-17T05:44:00.156489+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'349","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.211884+0000","last_change":"2024-04-17T04:36:25.365976+0000","last_active":"2024-04-17T05:46:09.211884+0000","last_peered":"2024-04-17T05:46:09.211884+0000","last_clean":"2024-04-17T05:46:09.211884+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:46:09.211884+0000","last_undegraded":"2024-04-17T05:46:09.211884+0000","last_fullsized":"2024-04-17T05:46:09.211884+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":31239,"num_write":1274,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:00.167958+0000","last_change":"2024-04-17T04:36:29.393010+0000","last_active":"2024-04-17T05:44:00.167958+0000","last_peered":"2024-04-17T05:44:00.167958+0000","last_clean":"2024-04-17T05:44:00.167958+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:44:00.167958+0000","last_undegraded":"2024-04-17T05:44:00.167958+0000","last_fullsized":"2024-04-17T05:44:00.167958+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'273","reported_seq":859,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.530829+0000","last_change":"2024-04-17T04:36:31.403334+0000","last_active":"2024-04-17T05:46:02.530829+0000","last_peered":"2024-04-17T05:46:02.530829+0000","last_clean":"2024-04-17T05:46:02.530829+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:46:02.530829+0000","last_undegraded":"2024-04-17T05:46:02.530829+0000","last_fullsized":"2024-04-17T05:46:02.530829+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46669,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":461,"num_read_kb":1085,"num_write":368,"num_write_kb":285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":498,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.983212+0000","last_change":"2024-04-17T05:46:23.983269+0000","last_active":"2024-04-17T05:46:23.983212+0000","last_peered":"2024-04-17T05:46:23.983212+0000","last_clean":"2024-04-17T05:46:23.983212+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:46:23.983212+0000","last_undegraded":"2024-04-17T05:46:23.983212+0000","last_fullsized":"2024-04-17T05:46:23.983212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.983009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:24:08.626906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'136","reported_seq":666,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.796544+0000","last_change":"2024-04-17T05:46:20.796701+0000","last_active":"2024-04-17T05:46:20.796544+0000","last_peered":"2024-04-17T05:46:20.796544+0000","last_clean":"2024-04-17T05:46:20.796544+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:46:20.796544+0000","last_undegraded":"2024-04-17T05:46:20.796544+0000","last_fullsized":"2024-04-17T05:46:20.796544+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'136","last_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_deep_scrub":"264'136","last_deep_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.796340+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:15.845948+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'203","reported_seq":811,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:25.203299+0000","last_change":"2024-04-17T04:36:27.381423+0000","last_active":"2024-04-17T05:44:25.203299+0000","last_peered":"2024-04-17T05:44:25.203299+0000","last_clean":"2024-04-17T05:44:25.203299+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:44:25.203299+0000","last_undegraded":"2024-04-17T05:44:25.203299+0000","last_fullsized":"2024-04-17T05:44:25.203299+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":100,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'386","reported_seq":1050,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.881824+0000","last_change":"2024-04-17T04:36:25.374623+0000","last_active":"2024-04-17T05:46:03.881824+0000","last_peered":"2024-04-17T05:46:03.881824+0000","last_clean":"2024-04-17T05:46:03.881824+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:46:03.881824+0000","last_undegraded":"2024-04-17T05:46:03.881824+0000","last_fullsized":"2024-04-17T05:46:03.881824+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1731334,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":524,"num_read_kb":19843,"num_write":1402,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159300+0000","last_change":"2024-04-17T04:36:29.377026+0000","last_active":"2024-04-17T05:44:10.159300+0000","last_peered":"2024-04-17T05:44:10.159300+0000","last_clean":"2024-04-17T05:44:10.159300+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:44:10.159300+0000","last_undegraded":"2024-04-17T05:44:10.159300+0000","last_fullsized":"2024-04-17T05:44:10.159300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'260","reported_seq":833,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.848090+0000","last_change":"2024-04-17T04:36:31.398159+0000","last_active":"2024-04-17T05:46:01.848090+0000","last_peered":"2024-04-17T05:46:01.848090+0000","last_clean":"2024-04-17T05:46:01.848090+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:46:01.848090+0000","last_undegraded":"2024-04-17T05:46:01.848090+0000","last_fullsized":"2024-04-17T05:46:01.848090+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44294,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":445,"num_read_kb":1105,"num_write":366,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.762588+0000","last_change":"2024-04-17T05:46:18.762692+0000","last_active":"2024-04-17T05:46:18.762588+0000","last_peered":"2024-04-17T05:46:18.762588+0000","last_clean":"2024-04-17T05:46:18.762588+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:46:18.762588+0000","last_undegraded":"2024-04-17T05:46:18.762588+0000","last_fullsized":"2024-04-17T05:46:18.762588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.762296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:44.671414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'141","reported_seq":888,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:40.761553+0000","last_change":"2024-04-17T04:36:35.423654+0000","last_active":"2024-04-17T05:45:40.761553+0000","last_peered":"2024-04-17T05:45:40.761553+0000","last_clean":"2024-04-17T05:45:40.761553+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:45:40.761553+0000","last_undegraded":"2024-04-17T05:45:40.761553+0000","last_fullsized":"2024-04-17T05:45:40.761553+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:34.376213+0000","last_clean_scrub_stamp":"2024-04-17T04:36:34.376213+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'297","reported_seq":976,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159337+0000","last_change":"2024-04-17T04:36:27.400669+0000","last_active":"2024-04-17T05:44:10.159337+0000","last_peered":"2024-04-17T05:44:10.159337+0000","last_clean":"2024-04-17T05:44:10.159337+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:44:10.159337+0000","last_undegraded":"2024-04-17T05:44:10.159337+0000","last_fullsized":"2024-04-17T05:44:10.159337+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'437","reported_seq":1144,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.857015+0000","last_change":"2024-04-17T04:36:25.356509+0000","last_active":"2024-04-17T05:46:03.857015+0000","last_peered":"2024-04-17T05:46:03.857015+0000","last_clean":"2024-04-17T05:46:03.857015+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:46:03.857015+0000","last_undegraded":"2024-04-17T05:46:03.857015+0000","last_fullsized":"2024-04-17T05:46:03.857015+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":515218,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":663,"num_read_kb":71297,"num_write":1686,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"264'8","reported_seq":847,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.766255+0000","last_change":"2024-04-17T05:46:20.766334+0000","last_active":"2024-04-17T05:46:20.766255+0000","last_peered":"2024-04-17T05:46:20.766255+0000","last_clean":"2024-04-17T05:46:20.766255+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:46:20.766255+0000","last_undegraded":"2024-04-17T05:46:20.766255+0000","last_fullsized":"2024-04-17T05:46:20.766255+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.766051+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:22.853924+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":501,"num_read_kb":814,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'319","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.100466+0000","last_change":"2024-04-17T04:36:31.391158+0000","last_active":"2024-04-17T05:46:02.100466+0000","last_peered":"2024-04-17T05:46:02.100466+0000","last_clean":"2024-04-17T05:46:02.100466+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:46:02.100466+0000","last_undegraded":"2024-04-17T05:46:02.100466+0000","last_fullsized":"2024-04-17T05:46:02.100466+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46192,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":530,"num_read_kb":1178,"num_write":420,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:05.166303+0000","last_change":"2024-04-17T04:36:33.430195+0000","last_active":"2024-04-17T05:44:05.166303+0000","last_peered":"2024-04-17T05:44:05.166303+0000","last_clean":"2024-04-17T05:44:05.166303+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:44:05.166303+0000","last_undegraded":"2024-04-17T05:44:05.166303+0000","last_fullsized":"2024-04-17T05:44:05.166303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:32.365489+0000","last_clean_scrub_stamp":"2024-04-17T04:36:32.365489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'141","reported_seq":649,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.801919+0000","last_change":"2024-04-17T05:46:10.802101+0000","last_active":"2024-04-17T05:46:10.801919+0000","last_peered":"2024-04-17T05:46:10.801919+0000","last_clean":"2024-04-17T05:46:10.801919+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:46:10.801919+0000","last_undegraded":"2024-04-17T05:46:10.801919+0000","last_fullsized":"2024-04-17T05:46:10.801919+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.801687+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.852555+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'155","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.741104+0000","last_change":"2024-04-17T05:46:21.741243+0000","last_active":"2024-04-17T05:46:21.741104+0000","last_peered":"2024-04-17T05:46:21.741104+0000","last_clean":"2024-04-17T05:46:21.741104+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:46:21.741104+0000","last_undegraded":"2024-04-17T05:46:21.741104+0000","last_fullsized":"2024-04-17T05:46:21.741104+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.740847+0000","objects_scrubbed":1,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:03.471838+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'420","reported_seq":1139,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.850599+0000","last_change":"2024-04-17T04:36:25.383690+0000","last_active":"2024-04-17T05:46:03.850599+0000","last_peered":"2024-04-17T05:46:03.850599+0000","last_clean":"2024-04-17T05:46:03.850599+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:46:03.850599+0000","last_undegraded":"2024-04-17T05:46:03.850599+0000","last_fullsized":"2024-04-17T05:46:03.850599+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262268,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":835,"num_read_kb":6303,"num_write":1376,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":658,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.183156+0000","last_change":"2024-04-17T05:46:10.183156+0000","last_active":"2024-04-17T05:46:10.183156+0000","last_peered":"2024-04-17T05:46:10.183156+0000","last_clean":"2024-04-17T05:46:10.183156+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:46:10.183156+0000","last_undegraded":"2024-04-17T05:46:10.183156+0000","last_fullsized":"2024-04-17T05:46:10.183156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'90","last_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_deep_scrub":"264'90","last_deep_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.182909+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:58:10.036518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'365","reported_seq":1032,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.868818+0000","last_change":"2024-04-17T04:36:25.355361+0000","last_active":"2024-04-17T05:46:03.868818+0000","last_peered":"2024-04-17T05:46:03.868818+0000","last_clean":"2024-04-17T05:46:03.868818+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:46:03.868818+0000","last_undegraded":"2024-04-17T05:46:03.868818+0000","last_fullsized":"2024-04-17T05:46:03.868818+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6192676,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":568,"num_read_kb":39832,"num_write":1198,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'148","reported_seq":763,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.191974+0000","last_change":"2024-04-17T04:36:27.388520+0000","last_active":"2024-04-17T05:45:50.191974+0000","last_peered":"2024-04-17T05:45:50.191974+0000","last_clean":"2024-04-17T05:45:50.191974+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:45:50.191974+0000","last_undegraded":"2024-04-17T05:45:50.191974+0000","last_fullsized":"2024-04-17T05:45:50.191974+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'307","reported_seq":945,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.864318+0000","last_change":"2024-04-17T04:36:25.365759+0000","last_active":"2024-04-17T05:46:03.864318+0000","last_peered":"2024-04-17T05:46:03.864318+0000","last_clean":"2024-04-17T05:46:03.864318+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:46:03.864318+0000","last_undegraded":"2024-04-17T05:46:03.864318+0000","last_fullsized":"2024-04-17T05:46:03.864318+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4341641,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":436,"num_read_kb":10784,"num_write":1165,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'94","reported_seq":720,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.197413+0000","last_change":"2024-04-17T04:36:27.362737+0000","last_active":"2024-04-17T05:45:50.197413+0000","last_peered":"2024-04-17T05:45:50.197413+0000","last_clean":"2024-04-17T05:45:50.197413+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:45:50.197413+0000","last_undegraded":"2024-04-17T05:45:50.197413+0000","last_fullsized":"2024-04-17T05:45:50.197413+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'308","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.886992+0000","last_change":"2024-04-17T04:36:25.355263+0000","last_active":"2024-04-17T05:46:03.886992+0000","last_peered":"2024-04-17T05:46:03.886992+0000","last_clean":"2024-04-17T05:46:03.886992+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:46:03.886992+0000","last_undegraded":"2024-04-17T05:46:03.886992+0000","last_fullsized":"2024-04-17T05:46:03.886992+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":709,"num_read_kb":32424,"num_write":1121,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:30.219247+0000","last_change":"2024-04-17T04:36:27.395680+0000","last_active":"2024-04-17T05:45:30.219247+0000","last_peered":"2024-04-17T05:45:30.219247+0000","last_clean":"2024-04-17T05:45:30.219247+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:45:30.219247+0000","last_undegraded":"2024-04-17T05:45:30.219247+0000","last_fullsized":"2024-04-17T05:45:30.219247+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'281","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.814149+0000","last_change":"2024-04-17T05:46:16.814273+0000","last_active":"2024-04-17T05:46:16.814149+0000","last_peered":"2024-04-17T05:46:16.814149+0000","last_clean":"2024-04-17T05:46:16.814149+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:46:16.814149+0000","last_undegraded":"2024-04-17T05:46:16.814149+0000","last_fullsized":"2024-04-17T05:46:16.814149+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.813947+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:56.482239+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5047006,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":359,"num_read_kb":5921,"num_write":992,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":633,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.174574+0000","last_change":"2024-04-17T04:36:27.395504+0000","last_active":"2024-04-17T05:45:15.174574+0000","last_peered":"2024-04-17T05:45:15.174574+0000","last_clean":"2024-04-17T05:45:15.174574+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:45:15.174574+0000","last_undegraded":"2024-04-17T05:45:15.174574+0000","last_fullsized":"2024-04-17T05:45:15.174574+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'319","reported_seq":1000,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.882770+0000","last_change":"2024-04-17T04:36:25.342748+0000","last_active":"2024-04-17T05:46:03.882770+0000","last_peered":"2024-04-17T05:46:03.882770+0000","last_clean":"2024-04-17T05:46:03.882770+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:46:03.882770+0000","last_undegraded":"2024-04-17T05:46:03.882770+0000","last_fullsized":"2024-04-17T05:46:03.882770+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5067983,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":492,"num_read_kb":71000,"num_write":1186,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'215","reported_seq":816,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.215367+0000","last_change":"2024-04-17T04:36:27.384524+0000","last_active":"2024-04-17T05:45:15.215367+0000","last_peered":"2024-04-17T05:45:15.215367+0000","last_clean":"2024-04-17T05:45:15.215367+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:45:15.215367+0000","last_undegraded":"2024-04-17T05:45:15.215367+0000","last_fullsized":"2024-04-17T05:45:15.215367+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'371","reported_seq":1053,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.884563+0000","last_change":"2024-04-17T04:36:25.386085+0000","last_active":"2024-04-17T05:46:03.884563+0000","last_peered":"2024-04-17T05:46:03.884563+0000","last_clean":"2024-04-17T05:46:03.884563+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:46:03.884563+0000","last_undegraded":"2024-04-17T05:46:03.884563+0000","last_fullsized":"2024-04-17T05:46:03.884563+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2189648,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":654,"num_read_kb":34085,"num_write":1471,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":749,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179877+0000","last_change":"2024-04-17T04:36:27.387044+0000","last_active":"2024-04-17T05:45:15.179877+0000","last_peered":"2024-04-17T05:45:15.179877+0000","last_clean":"2024-04-17T05:45:15.179877+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:45:15.179877+0000","last_undegraded":"2024-04-17T05:45:15.179877+0000","last_fullsized":"2024-04-17T05:45:15.179877+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'335","reported_seq":1007,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.881772+0000","last_change":"2024-04-17T04:36:25.379835+0000","last_active":"2024-04-17T05:46:03.881772+0000","last_peered":"2024-04-17T05:46:03.881772+0000","last_clean":"2024-04-17T05:46:03.881772+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:46:03.881772+0000","last_undegraded":"2024-04-17T05:46:03.881772+0000","last_fullsized":"2024-04-17T05:46:03.881772+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534609,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":546,"num_read_kb":42711,"num_write":1290,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.167738+0000","last_change":"2024-04-17T04:36:27.373717+0000","last_active":"2024-04-17T05:44:10.167738+0000","last_peered":"2024-04-17T05:44:10.167738+0000","last_clean":"2024-04-17T05:44:10.167738+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:44:10.167738+0000","last_undegraded":"2024-04-17T05:44:10.167738+0000","last_fullsized":"2024-04-17T05:44:10.167738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'300","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.798568+0000","last_change":"2024-04-17T04:36:25.361686+0000","last_active":"2024-04-17T05:46:03.798568+0000","last_peered":"2024-04-17T05:46:03.798568+0000","last_clean":"2024-04-17T05:46:03.798568+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:46:03.798568+0000","last_undegraded":"2024-04-17T05:46:03.798568+0000","last_fullsized":"2024-04-17T05:46:03.798568+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262289,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":548,"num_write":1115,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":755,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.159607+0000","last_change":"2024-04-17T04:36:27.378398+0000","last_active":"2024-04-17T05:44:10.159607+0000","last_peered":"2024-04-17T05:44:10.159607+0000","last_clean":"2024-04-17T05:44:10.159607+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:44:10.159607+0000","last_undegraded":"2024-04-17T05:44:10.159607+0000","last_fullsized":"2024-04-17T05:44:10.159607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'302","reported_seq":953,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275580+0000","last_change":"2024-04-17T04:36:25.353469+0000","last_active":"2024-04-17T05:46:08.275580+0000","last_peered":"2024-04-17T05:46:08.275580+0000","last_clean":"2024-04-17T05:46:08.275580+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:46:08.275580+0000","last_undegraded":"2024-04-17T05:46:08.275580+0000","last_fullsized":"2024-04-17T05:46:08.275580+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171414,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":4352,"num_write":1098,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":838,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:15.170223+0000","last_change":"2024-04-17T04:36:27.398568+0000","last_active":"2024-04-17T05:44:15.170223+0000","last_peered":"2024-04-17T05:44:15.170223+0000","last_clean":"2024-04-17T05:44:15.170223+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:44:15.170223+0000","last_undegraded":"2024-04-17T05:44:15.170223+0000","last_fullsized":"2024-04-17T05:44:15.170223+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'317","reported_seq":937,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.663436+0000","last_change":"2024-04-17T05:46:10.663532+0000","last_active":"2024-04-17T05:46:10.663436+0000","last_peered":"2024-04-17T05:46:10.663436+0000","last_clean":"2024-04-17T05:46:10.663436+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:46:10.663436+0000","last_undegraded":"2024-04-17T05:46:10.663436+0000","last_fullsized":"2024-04-17T05:46:10.663436+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'317","last_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_deep_scrub":"264'317","last_deep_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.663219+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.001315+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":443,"num_read_kb":16015,"num_write":1067,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'95","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.744315+0000","last_change":"2024-04-17T04:36:27.402182+0000","last_active":"2024-04-17T05:46:08.744315+0000","last_peered":"2024-04-17T05:46:08.744315+0000","last_clean":"2024-04-17T05:46:08.744315+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:46:08.744315+0000","last_undegraded":"2024-04-17T05:46:08.744315+0000","last_fullsized":"2024-04-17T05:46:08.744315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'360","reported_seq":1079,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934332+0000","last_change":"2024-04-17T04:36:25.333011+0000","last_active":"2024-04-17T05:46:06.934332+0000","last_peered":"2024-04-17T05:46:06.934332+0000","last_clean":"2024-04-17T05:46:06.934332+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:46:06.934332+0000","last_undegraded":"2024-04-17T05:46:06.934332+0000","last_fullsized":"2024-04-17T05:46:06.934332+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":746,"num_read_kb":12134,"num_write":1256,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":688,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.156147+0000","last_change":"2024-04-17T04:36:27.388791+0000","last_active":"2024-04-17T05:46:10.156147+0000","last_peered":"2024-04-17T05:46:10.156147+0000","last_clean":"2024-04-17T05:46:10.156147+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:46:10.156147+0000","last_undegraded":"2024-04-17T05:46:10.156147+0000","last_fullsized":"2024-04-17T05:46:10.156147+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'357","reported_seq":995,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.679997+0000","last_change":"2024-04-17T04:36:25.373417+0000","last_active":"2024-04-17T05:46:09.679997+0000","last_peered":"2024-04-17T05:46:09.679997+0000","last_clean":"2024-04-17T05:46:09.679997+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:46:09.679997+0000","last_undegraded":"2024-04-17T05:46:09.679997+0000","last_fullsized":"2024-04-17T05:46:09.679997+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":501,"num_read_kb":657,"num_write":1252,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'221","reported_seq":903,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934551+0000","last_change":"2024-04-17T04:36:27.372140+0000","last_active":"2024-04-17T05:46:06.934551+0000","last_peered":"2024-04-17T05:46:06.934551+0000","last_clean":"2024-04-17T05:46:06.934551+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:46:06.934551+0000","last_undegraded":"2024-04-17T05:46:06.934551+0000","last_fullsized":"2024-04-17T05:46:06.934551+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'420","reported_seq":1160,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.156343+0000","last_change":"2024-04-17T04:36:25.386307+0000","last_active":"2024-04-17T05:46:10.156343+0000","last_peered":"2024-04-17T05:46:10.156343+0000","last_clean":"2024-04-17T05:46:10.156343+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:46:10.156343+0000","last_undegraded":"2024-04-17T05:46:10.156343+0000","last_fullsized":"2024-04-17T05:46:10.156343+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058993,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":894,"num_read_kb":32480,"num_write":1416,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":967,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275787+0000","last_change":"2024-04-17T04:36:27.385349+0000","last_active":"2024-04-17T05:46:08.275787+0000","last_peered":"2024-04-17T05:46:08.275787+0000","last_clean":"2024-04-17T05:46:08.275787+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:46:08.275787+0000","last_undegraded":"2024-04-17T05:46:08.275787+0000","last_fullsized":"2024-04-17T05:46:08.275787+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'314","reported_seq":992,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.843618+0000","last_change":"2024-04-17T04:36:25.355339+0000","last_active":"2024-04-17T05:46:07.843618+0000","last_peered":"2024-04-17T05:46:07.843618+0000","last_clean":"2024-04-17T05:46:07.843618+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:46:07.843618+0000","last_undegraded":"2024-04-17T05:46:07.843618+0000","last_fullsized":"2024-04-17T05:46:07.843618+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7281800,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":526,"num_read_kb":46303,"num_write":1296,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'322","reported_seq":1053,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.325081+0000","last_change":"2024-04-17T05:46:23.325152+0000","last_active":"2024-04-17T05:46:23.325081+0000","last_peered":"2024-04-17T05:46:23.325081+0000","last_clean":"2024-04-17T05:46:23.325081+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:46:23.325081+0000","last_undegraded":"2024-04-17T05:46:23.325081+0000","last_fullsized":"2024-04-17T05:46:23.325081+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.324895+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:02.515429+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'328","reported_seq":963,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.212219+0000","last_change":"2024-04-17T04:36:25.361750+0000","last_active":"2024-04-17T05:46:09.212219+0000","last_peered":"2024-04-17T05:46:09.212219+0000","last_clean":"2024-04-17T05:46:09.212219+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:46:09.212219+0000","last_undegraded":"2024-04-17T05:46:09.212219+0000","last_fullsized":"2024-04-17T05:46:09.212219+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":466,"num_read_kb":49597,"num_write":1231,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'297","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.212255+0000","last_change":"2024-04-17T04:36:25.345113+0000","last_active":"2024-04-17T05:46:09.212255+0000","last_peered":"2024-04-17T05:46:09.212255+0000","last_clean":"2024-04-17T05:46:09.212255+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:46:09.212255+0000","last_undegraded":"2024-04-17T05:46:09.212255+0000","last_fullsized":"2024-04-17T05:46:09.212255+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129194,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":413,"num_read_kb":7661,"num_write":1087,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'243","reported_seq":843,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:20.170689+0000","last_change":"2024-04-17T04:36:27.379068+0000","last_active":"2024-04-17T05:44:20.170689+0000","last_peered":"2024-04-17T05:44:20.170689+0000","last_clean":"2024-04-17T05:44:20.170689+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:44:20.170689+0000","last_undegraded":"2024-04-17T05:44:20.170689+0000","last_fullsized":"2024-04-17T05:44:20.170689+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'277","reported_seq":922,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.396551+0000","last_change":"2024-04-17T04:36:25.357445+0000","last_active":"2024-04-17T05:46:07.396551+0000","last_peered":"2024-04-17T05:46:07.396551+0000","last_clean":"2024-04-17T05:46:07.396551+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:46:07.396551+0000","last_undegraded":"2024-04-17T05:46:07.396551+0000","last_fullsized":"2024-04-17T05:46:07.396551+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":447,"num_read_kb":20933,"num_write":1123,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":820,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:44:10.170392+0000","last_change":"2024-04-17T04:36:27.378695+0000","last_active":"2024-04-17T05:44:10.170392+0000","last_peered":"2024-04-17T05:44:10.170392+0000","last_clean":"2024-04-17T05:44:10.170392+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:44:10.170392+0000","last_undegraded":"2024-04-17T05:44:10.170392+0000","last_fullsized":"2024-04-17T05:44:10.170392+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6748,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1282,"ondisk_log_size":1282,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2521,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1757,"ondisk_log_size":1757,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1458963,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":18010,"num_read_kb":37614,"num_write":15043,"num_write_kb":11218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":134848,"num_omap_keys":334,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1886,"ondisk_log_size":1886,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1652,"num_read_kb":1963,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1045,"num_omap_keys":5,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3952,"ondisk_log_size":3952,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187841250,"num_objects":443,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":34833,"num_read_kb":1864137,"num_write":77494,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253607936,"data_stored":253607936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3747,"ondisk_log_size":3747,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":221363,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":145,"num_read_kb":210051,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575695,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349100,"kb_used_data":38884,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":94022740,"statfs":{"total":96636764160,"available":96279285760,"internally_reserved":0,"allocated":39817216,"data_stored":37809825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":317651402},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575696,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355672,"kb_used_data":39888,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":94016168,"statfs":{"total":96636764160,"available":96272556032,"internally_reserved":0,"allocated":40845312,"data_stored":38885655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":323353031},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575696,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206104,"kb_used_data":58384,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94165736,"statfs":{"total":96636764160,"available":96425713664,"internally_reserved":0,"allocated":59785216,"data_stored":57737844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575698,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148100,"kb_used_data":36412,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94223740,"statfs":{"total":96636764160,"available":96485109760,"internally_reserved":0,"allocated":37285888,"data_stored":35330686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575696,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298456,"kb_used_data":42448,"kb_used_omap":1,"kb_used_meta":255998,"kb_avail":94073384,"statfs":{"total":96636764160,"available":96331145216,"internally_reserved":0,"allocated":43466752,"data_stored":41436434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":262142407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575700,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159304,"kb_used_data":56000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94212536,"statfs":{"total":96636764160,"available":96473636864,"internally_reserved":0,"allocated":57344000,"data_stored":55396653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575698,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183784,"kb_used_data":36064,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94188056,"statfs":{"total":96636764160,"available":96448569344,"internally_reserved":0,"allocated":36929536,"data_stored":34946873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575696,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327488,"kb_used_data":39544,"kb_used_omap":1,"kb_used_meta":287934,"kb_avail":94044352,"statfs":{"total":96636764160,"available":96301416448,"internally_reserved":0,"allocated":40493056,"data_stored":38470704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294844869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35614720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26390528,"data_stored":26390528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36356096,"data_stored":36356096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33927168,"data_stored":33927168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22581248,"data_stored":22581248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":38694912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31379456,"data_stored":31379456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28663808,"data_stored":28663808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:46:31.026 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.026 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.027 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.027 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.027 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.027 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.027 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.028 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.028 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.028 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.028 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.028 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.028 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.029 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.029 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.029 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.029 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.029 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.029 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.030 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.030 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.030 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.030 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.030 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.030 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.031 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.031 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.031 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.031 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.031 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.032 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.032 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.032 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.032 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.032 INFO:tasks.ceph:pgid 67.d last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.032 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.033 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.033 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.033 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.033 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.033 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.033 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.034 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.034 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.035 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.035 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.035 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.036 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.036 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.036 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.036 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.036 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.037 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.037 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.037 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.037 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.037 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.037 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.038 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.038 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.038 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.038 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.038 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.039 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.039 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.039 INFO:tasks.ceph:pgid 66.3 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.039 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.039 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.040 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.040 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.040 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.040 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.040 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.041 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.042 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.042 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.042 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.042 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.042 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.043 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.043 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.043 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.043 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.043 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.043 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.044 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.044 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.044 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.044 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.044 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.044 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.045 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.045 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.045 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.045 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.045 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.046 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.046 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.046 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.046 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.046 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.047 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.047 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.047 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.047 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.047 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.048 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.048 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.048 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.048 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.048 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.048 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.049 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.049 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.049 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.049 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.049 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.049 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.050 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.050 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.050 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.050 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.050 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.051 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.051 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.051 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.051 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.051 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.052 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.052 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.052 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.052 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.052 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.053 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-17T04:36:11.058932+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=11, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.053 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.053 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.053 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.053 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.054 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.054 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.054 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.054 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.054 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.054 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.055 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.055 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.055 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.055 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.055 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.055 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.056 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.056 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.056 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.056 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.056 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.056 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.057 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.057 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.057 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.057 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.057 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.057 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.058 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.058 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.058 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.058 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.058 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.059 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.060 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.060 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.060 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.060 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.060 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.061 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.061 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.061 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.061 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.062 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.062 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.062 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.062 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.062 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.063 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.063 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.063 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.063 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.063 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.063 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.064 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.065 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.065 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.065 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.065 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.065 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.066 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.066 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.066 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.066 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.066 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.066 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.067 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.067 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.067 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.067 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.067 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.067 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.068 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-17T04:36:34.376213+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=34, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.068 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.068 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.068 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.068 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-17T04:36:32.365489+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=32, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.068 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.069 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.069 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.069 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.069 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.069 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.069 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.070 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.070 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.070 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.070 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.070 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.070 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.071 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.071 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.071 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.071 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.071 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.071 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.072 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.072 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.072 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.072 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.072 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.072 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.073 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.073 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.073 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.073 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.074 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.074 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:31.074 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T05:46:51.076 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:46:51.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:46:51.452 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:46:51.463 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2377,"stamp":"2024-04-17T05:46:51.311550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223466608,"num_objects":822,"num_object_clones":0,"num_object_copies":2530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":822,"num_whiteouts":0,"num_read":160833,"num_read_kb":2423241,"num_write":146364,"num_write_kb":2463359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":677690,"num_omap_keys":1726,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13296,"ondisk_log_size":13296,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2028008,"kb_used_data":347624,"kb_used_omap":12,"kb_used_meta":1680307,"kb_avail":752946712,"statfs":{"total":773094113280,"available":771017433088,"internally_reserved":0,"allocated":355966976,"data_stored":340014674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1720634929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25164,"num_omap_keys":87,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007707"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184309+0000","last_change":"2024-04-17T05:07:45.742645+0000","last_active":"2024-04-17T05:45:15.184309+0000","last_peered":"2024-04-17T05:45:15.184309+0000","last_clean":"2024-04-17T05:45:15.184309+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:45:15.184309+0000","last_undegraded":"2024-04-17T05:45:15.184309+0000","last_fullsized":"2024-04-17T05:45:15.184309+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.213507+0000","last_change":"2024-04-17T04:51:43.682048+0000","last_active":"2024-04-17T05:46:09.213507+0000","last_peered":"2024-04-17T05:46:09.213507+0000","last_clean":"2024-04-17T05:46:09.213507+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:46:09.213507+0000","last_undegraded":"2024-04-17T05:46:09.213507+0000","last_fullsized":"2024-04-17T05:46:09.213507+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.830930+0000","last_change":"2024-04-17T05:46:14.830930+0000","last_active":"2024-04-17T05:46:14.830930+0000","last_peered":"2024-04-17T05:46:14.830930+0000","last_clean":"2024-04-17T05:46:14.830930+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:46:14.830930+0000","last_undegraded":"2024-04-17T05:46:14.830930+0000","last_fullsized":"2024-04-17T05:46:14.830930+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.830571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:08.876068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":158,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.685785+0000","last_change":"2024-04-17T05:46:08.685785+0000","last_active":"2024-04-17T05:46:08.685785+0000","last_peered":"2024-04-17T05:46:08.685785+0000","last_clean":"2024-04-17T05:46:08.685785+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:46:08.685785+0000","last_undegraded":"2024-04-17T05:46:08.685785+0000","last_fullsized":"2024-04-17T05:46:08.685785+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.685550+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:05.034233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.544197+0000","last_change":"2024-04-17T05:46:39.544342+0000","last_active":"2024-04-17T05:46:39.544197+0000","last_peered":"2024-04-17T05:46:39.544197+0000","last_clean":"2024-04-17T05:46:39.544197+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:46:39.544197+0000","last_undegraded":"2024-04-17T05:46:39.544197+0000","last_fullsized":"2024-04-17T05:46:39.544197+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:39.543940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:39.543940+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.543940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:06.130204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'3","reported_seq":50,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.745404+0000","last_change":"2024-04-17T04:51:43.661871+0000","last_active":"2024-04-17T05:46:08.745404+0000","last_peered":"2024-04-17T05:46:08.745404+0000","last_clean":"2024-04-17T05:46:08.745404+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:46:08.745404+0000","last_undegraded":"2024-04-17T05:46:08.745404+0000","last_fullsized":"2024-04-17T05:46:08.745404+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.692660+0000","last_change":"2024-04-17T05:46:12.692660+0000","last_active":"2024-04-17T05:46:12.692660+0000","last_peered":"2024-04-17T05:46:12.692660+0000","last_clean":"2024-04-17T05:46:12.692660+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:46:12.692660+0000","last_undegraded":"2024-04-17T05:46:12.692660+0000","last_fullsized":"2024-04-17T05:46:12.692660+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.692409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:20.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'97","reported_seq":361,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.504742+0000","last_change":"2024-04-17T05:46:20.504828+0000","last_active":"2024-04-17T05:46:20.504742+0000","last_peered":"2024-04-17T05:46:20.504742+0000","last_clean":"2024-04-17T05:46:20.504742+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:46:20.504742+0000","last_undegraded":"2024-04-17T05:46:20.504742+0000","last_fullsized":"2024-04-17T05:46:20.504742+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.504541+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:15.125809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3493,"num_omap_keys":7,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.887667+0000","last_change":"2024-04-17T05:46:45.887724+0000","last_active":"2024-04-17T05:46:45.887667+0000","last_peered":"2024-04-17T05:46:45.887667+0000","last_clean":"2024-04-17T05:46:45.887667+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:46:45.887667+0000","last_undegraded":"2024-04-17T05:46:45.887667+0000","last_fullsized":"2024-04-17T05:46:45.887667+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:45.887431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:45.887431+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.887431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:34.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.677240+0000","last_change":"2024-04-17T05:46:35.677355+0000","last_active":"2024-04-17T05:46:35.677240+0000","last_peered":"2024-04-17T05:46:35.677240+0000","last_clean":"2024-04-17T05:46:35.677240+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:46:35.677240+0000","last_undegraded":"2024-04-17T05:46:35.677240+0000","last_fullsized":"2024-04-17T05:46:35.677240+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:35.676927+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:35.676927+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.676927+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:40.289079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'3","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.493151+0000","last_change":"2024-04-17T05:46:32.493211+0000","last_active":"2024-04-17T05:46:32.493151+0000","last_peered":"2024-04-17T05:46:32.493151+0000","last_clean":"2024-04-17T05:46:32.493151+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:46:32.493151+0000","last_undegraded":"2024-04-17T05:46:32.493151+0000","last_fullsized":"2024-04-17T05:46:32.493151+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:32.492942+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:32.492942+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.492942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:29.955893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":130,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.592867+0000","last_change":"2024-04-17T05:46:16.593030+0000","last_active":"2024-04-17T05:46:16.592867+0000","last_peered":"2024-04-17T05:46:16.592867+0000","last_clean":"2024-04-17T05:46:16.592867+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:46:16.592867+0000","last_undegraded":"2024-04-17T05:46:16.592867+0000","last_fullsized":"2024-04-17T05:46:16.592867+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_deep_scrub":"264'25","last_deep_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.592614+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:38.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276684+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:46:08.276684+0000","last_peered":"2024-04-17T05:46:08.276684+0000","last_clean":"2024-04-17T05:46:08.276684+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:46:08.276684+0000","last_undegraded":"2024-04-17T05:46:08.276684+0000","last_fullsized":"2024-04-17T05:46:08.276684+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.760387+0000","last_change":"2024-04-17T05:46:26.760496+0000","last_active":"2024-04-17T05:46:26.760387+0000","last_peered":"2024-04-17T05:46:26.760387+0000","last_clean":"2024-04-17T05:46:26.760387+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:46:26.760387+0000","last_undegraded":"2024-04-17T05:46:26.760387+0000","last_fullsized":"2024-04-17T05:46:26.760387+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:26.760176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:26.760176+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.760176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:38.567211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.770225+0000","last_change":"2024-04-17T05:46:20.770279+0000","last_active":"2024-04-17T05:46:20.770225+0000","last_peered":"2024-04-17T05:46:20.770225+0000","last_clean":"2024-04-17T05:46:20.770225+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:46:20.770225+0000","last_undegraded":"2024-04-17T05:46:20.770225+0000","last_fullsized":"2024-04-17T05:46:20.770225+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.770027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:10.097683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":169,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.814048+0000","last_change":"2024-04-17T05:46:08.814070+0000","last_active":"2024-04-17T05:46:08.814048+0000","last_peered":"2024-04-17T05:46:08.814048+0000","last_clean":"2024-04-17T05:46:08.814048+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:46:08.814048+0000","last_undegraded":"2024-04-17T05:46:08.814048+0000","last_fullsized":"2024-04-17T05:46:08.814048+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'42","last_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_deep_scrub":"264'42","last_deep_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.813969+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:41.013929+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":32,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.224140+0000","last_change":"2024-04-17T05:07:45.745164+0000","last_active":"2024-04-17T05:45:50.224140+0000","last_peered":"2024-04-17T05:45:50.224140+0000","last_clean":"2024-04-17T05:45:50.224140+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:45:50.224140+0000","last_undegraded":"2024-04-17T05:45:50.224140+0000","last_fullsized":"2024-04-17T05:45:50.224140+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'2","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.557348+0000","last_change":"2024-04-17T04:51:43.666389+0000","last_active":"2024-04-17T05:46:03.557348+0000","last_peered":"2024-04-17T05:46:03.557348+0000","last_clean":"2024-04-17T05:46:03.557348+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:46:03.557348+0000","last_undegraded":"2024-04-17T05:46:03.557348+0000","last_fullsized":"2024-04-17T05:46:03.557348+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.720633+0000","last_change":"2024-04-17T05:46:07.720653+0000","last_active":"2024-04-17T05:46:07.720633+0000","last_peered":"2024-04-17T05:46:07.720633+0000","last_clean":"2024-04-17T05:46:07.720633+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:46:07.720633+0000","last_undegraded":"2024-04-17T05:46:07.720633+0000","last_fullsized":"2024-04-17T05:46:07.720633+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.720559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:04.229433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":209,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.804847+0000","last_change":"2024-04-17T05:46:09.804897+0000","last_active":"2024-04-17T05:46:09.804847+0000","last_peered":"2024-04-17T05:46:09.804847+0000","last_clean":"2024-04-17T05:46:09.804847+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:46:09.804847+0000","last_undegraded":"2024-04-17T05:46:09.804847+0000","last_fullsized":"2024-04-17T05:46:09.804847+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.804697+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:40.852058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.905733+0000","last_change":"2024-04-17T05:46:49.905791+0000","last_active":"2024-04-17T05:46:49.905733+0000","last_peered":"2024-04-17T05:46:49.905733+0000","last_clean":"2024-04-17T05:46:49.905733+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:46:49.905733+0000","last_undegraded":"2024-04-17T05:46:49.905733+0000","last_fullsized":"2024-04-17T05:46:49.905733+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:49.905522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:49.905522+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.905522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:33.396236+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'6","reported_seq":67,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.924479+0000","last_change":"2024-04-17T05:46:41.924541+0000","last_active":"2024-04-17T05:46:41.924479+0000","last_peered":"2024-04-17T05:46:41.924479+0000","last_clean":"2024-04-17T05:46:41.924479+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:46:41.924479+0000","last_undegraded":"2024-04-17T05:46:41.924479+0000","last_fullsized":"2024-04-17T05:46:41.924479+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:41.924290+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:41.924290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.924290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:42.756248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":22528,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"264'6","reported_seq":75,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.770136+0000","last_change":"2024-04-17T05:46:12.770216+0000","last_active":"2024-04-17T05:46:12.770136+0000","last_peered":"2024-04-17T05:46:12.770136+0000","last_clean":"2024-04-17T05:46:12.770136+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:46:12.770136+0000","last_undegraded":"2024-04-17T05:46:12.770136+0000","last_fullsized":"2024-04-17T05:46:12.770136+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.769907+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:23.154229+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'75","reported_seq":293,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.023607+0000","last_change":"2024-04-17T05:46:12.023675+0000","last_active":"2024-04-17T05:46:12.023607+0000","last_peered":"2024-04-17T05:46:12.023607+0000","last_clean":"2024-04-17T05:46:12.023607+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:46:12.023607+0000","last_undegraded":"2024-04-17T05:46:12.023607+0000","last_fullsized":"2024-04-17T05:46:12.023607+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'75","last_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_deep_scrub":"264'75","last_deep_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.023402+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.097898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3255,"num_omap_keys":7,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'1","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179323+0000","last_change":"2024-04-17T05:07:45.754885+0000","last_active":"2024-04-17T05:45:15.179323+0000","last_peered":"2024-04-17T05:45:15.179323+0000","last_clean":"2024-04-17T05:45:15.179323+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:45:15.179323+0000","last_undegraded":"2024-04-17T05:45:15.179323+0000","last_fullsized":"2024-04-17T05:45:15.179323+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.909074+0000","last_change":"2024-04-17T05:46:42.909074+0000","last_active":"2024-04-17T05:46:42.909074+0000","last_peered":"2024-04-17T05:46:42.909074+0000","last_clean":"2024-04-17T05:46:42.909074+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:46:42.909074+0000","last_undegraded":"2024-04-17T05:46:42.909074+0000","last_fullsized":"2024-04-17T05:46:42.909074+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.908818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.908818+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.908818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:22.365739+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'9","reported_seq":69,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.745184+0000","last_change":"2024-04-17T04:51:39.621614+0000","last_active":"2024-04-17T05:46:08.745184+0000","last_peered":"2024-04-17T05:46:08.745184+0000","last_clean":"2024-04-17T05:46:08.745184+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:46:08.745184+0000","last_undegraded":"2024-04-17T05:46:08.745184+0000","last_fullsized":"2024-04-17T05:46:08.745184+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":16384,"num_write":24,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":84,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.595282+0000","last_change":"2024-04-17T05:46:14.595378+0000","last_active":"2024-04-17T05:46:14.595282+0000","last_peered":"2024-04-17T05:46:14.595282+0000","last_clean":"2024-04-17T05:46:14.595282+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:46:14.595282+0000","last_undegraded":"2024-04-17T05:46:14.595282+0000","last_fullsized":"2024-04-17T05:46:14.595282+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.594970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:05:26.805897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":28,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179493+0000","last_change":"2024-04-17T05:07:45.757519+0000","last_active":"2024-04-17T05:45:15.179493+0000","last_peered":"2024-04-17T05:45:15.179493+0000","last_clean":"2024-04-17T05:45:15.179493+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:45:15.179493+0000","last_undegraded":"2024-04-17T05:45:15.179493+0000","last_fullsized":"2024-04-17T05:45:15.179493+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":45,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276746+0000","last_change":"2024-04-17T04:51:43.686621+0000","last_active":"2024-04-17T05:46:08.276746+0000","last_peered":"2024-04-17T05:46:08.276746+0000","last_clean":"2024-04-17T05:46:08.276746+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:46:08.276746+0000","last_undegraded":"2024-04-17T05:46:08.276746+0000","last_fullsized":"2024-04-17T05:46:08.276746+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.616272+0000","last_change":"2024-04-17T05:46:27.616272+0000","last_active":"2024-04-17T05:46:27.616272+0000","last_peered":"2024-04-17T05:46:27.616272+0000","last_clean":"2024-04-17T05:46:27.616272+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:46:27.616272+0000","last_undegraded":"2024-04-17T05:46:27.616272+0000","last_fullsized":"2024-04-17T05:46:27.616272+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-17T05:46:27.615997+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-17T05:46:27.615997+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.615997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:21:10.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":99,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.712592+0000","last_change":"2024-04-17T05:46:11.712658+0000","last_active":"2024-04-17T05:46:11.712592+0000","last_peered":"2024-04-17T05:46:11.712592+0000","last_clean":"2024-04-17T05:46:11.712592+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:46:11.712592+0000","last_undegraded":"2024-04-17T05:46:11.712592+0000","last_fullsized":"2024-04-17T05:46:11.712592+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'13","last_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_deep_scrub":"264'13","last_deep_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.712398+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:12.832777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.918864+0000","last_change":"2024-04-17T05:46:48.918919+0000","last_active":"2024-04-17T05:46:48.918864+0000","last_peered":"2024-04-17T05:46:48.918864+0000","last_clean":"2024-04-17T05:46:48.918864+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:46:48.918864+0000","last_undegraded":"2024-04-17T05:46:48.918864+0000","last_fullsized":"2024-04-17T05:46:48.918864+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:48.918658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:48.918658+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.918658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:35:22.598547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.794712+0000","last_change":"2024-04-17T05:46:18.794764+0000","last_active":"2024-04-17T05:46:18.794712+0000","last_peered":"2024-04-17T05:46:18.794712+0000","last_clean":"2024-04-17T05:46:18.794712+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:46:18.794712+0000","last_undegraded":"2024-04-17T05:46:18.794712+0000","last_fullsized":"2024-04-17T05:46:18.794712+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.794517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:29:12.950953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.225008+0000","last_change":"2024-04-17T05:46:24.225059+0000","last_active":"2024-04-17T05:46:24.225008+0000","last_peered":"2024-04-17T05:46:24.225008+0000","last_clean":"2024-04-17T05:46:24.225008+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:46:24.225008+0000","last_undegraded":"2024-04-17T05:46:24.225008+0000","last_fullsized":"2024-04-17T05:46:24.225008+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.224809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:16.454241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":89,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.623954+0000","last_change":"2024-04-17T05:46:12.624024+0000","last_active":"2024-04-17T05:46:12.623954+0000","last_peered":"2024-04-17T05:46:12.623954+0000","last_clean":"2024-04-17T05:46:12.623954+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:46:12.623954+0000","last_undegraded":"2024-04-17T05:46:12.623954+0000","last_fullsized":"2024-04-17T05:46:12.623954+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.623741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:56:20.441738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'2","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.529278+0000","last_change":"2024-04-17T05:46:36.529348+0000","last_active":"2024-04-17T05:46:36.529278+0000","last_peered":"2024-04-17T05:46:36.529278+0000","last_clean":"2024-04-17T05:46:36.529278+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:46:36.529278+0000","last_undegraded":"2024-04-17T05:46:36.529278+0000","last_fullsized":"2024-04-17T05:46:36.529278+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:36.529078+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:36.529078+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.529078+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:48.549685+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.829356+0000","last_change":"2024-04-17T05:46:34.829509+0000","last_active":"2024-04-17T05:46:34.829356+0000","last_peered":"2024-04-17T05:46:34.829356+0000","last_clean":"2024-04-17T05:46:34.829356+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:46:34.829356+0000","last_undegraded":"2024-04-17T05:46:34.829356+0000","last_fullsized":"2024-04-17T05:46:34.829356+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:34.829120+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:34.829120+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.829120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:21.304145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'2","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.129624+0000","last_change":"2024-04-17T05:46:11.129787+0000","last_active":"2024-04-17T05:46:11.129624+0000","last_peered":"2024-04-17T05:46:11.129624+0000","last_clean":"2024-04-17T05:46:11.129624+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:46:11.129624+0000","last_undegraded":"2024-04-17T05:46:11.129624+0000","last_fullsized":"2024-04-17T05:46:11.129624+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.129393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:45.300258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":125,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.852514+0000","last_change":"2024-04-17T05:46:11.852573+0000","last_active":"2024-04-17T05:46:11.852514+0000","last_peered":"2024-04-17T05:46:11.852514+0000","last_clean":"2024-04-17T05:46:11.852514+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:46:11.852514+0000","last_undegraded":"2024-04-17T05:46:11.852514+0000","last_fullsized":"2024-04-17T05:46:11.852514+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.852327+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:08.854562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.319843+0000","last_change":"2024-04-17T05:46:32.319899+0000","last_active":"2024-04-17T05:46:32.319843+0000","last_peered":"2024-04-17T05:46:32.319843+0000","last_clean":"2024-04-17T05:46:32.319843+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:46:32.319843+0000","last_undegraded":"2024-04-17T05:46:32.319843+0000","last_fullsized":"2024-04-17T05:46:32.319843+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.319638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.319638+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.319638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:10.301900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'5","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.734077+0000","last_change":"2024-04-17T05:46:22.734137+0000","last_active":"2024-04-17T05:46:22.734077+0000","last_peered":"2024-04-17T05:46:22.734077+0000","last_clean":"2024-04-17T05:46:22.734077+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:46:22.734077+0000","last_undegraded":"2024-04-17T05:46:22.734077+0000","last_fullsized":"2024-04-17T05:46:22.734077+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.733832+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:04.087293+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":48,"num_write":10,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'14","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.511830+0000","last_change":"2024-04-17T05:46:33.511896+0000","last_active":"2024-04-17T05:46:33.511830+0000","last_peered":"2024-04-17T05:46:33.511830+0000","last_clean":"2024-04-17T05:46:33.511830+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:46:33.511830+0000","last_undegraded":"2024-04-17T05:46:33.511830+0000","last_fullsized":"2024-04-17T05:46:33.511830+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'14","last_scrub_stamp":"2024-04-17T05:46:33.511695+0000","last_deep_scrub":"264'14","last_deep_scrub_stamp":"2024-04-17T05:46:33.511695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.511695+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:13.607331+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":46082,"num_write":17,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.547405+0000","last_change":"2024-04-17T05:46:18.547463+0000","last_active":"2024-04-17T05:46:18.547405+0000","last_peered":"2024-04-17T05:46:18.547405+0000","last_clean":"2024-04-17T05:46:18.547405+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:46:18.547405+0000","last_undegraded":"2024-04-17T05:46:18.547405+0000","last_fullsized":"2024-04-17T05:46:18.547405+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'18","last_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_deep_scrub":"261'18","last_deep_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.547198+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:22.594305+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:05.182145+0000","last_change":"2024-04-17T05:07:45.744163+0000","last_active":"2024-04-17T05:45:05.182145+0000","last_peered":"2024-04-17T05:45:05.182145+0000","last_clean":"2024-04-17T05:45:05.182145+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:45:05.182145+0000","last_undegraded":"2024-04-17T05:45:05.182145+0000","last_fullsized":"2024-04-17T05:45:05.182145+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'11","reported_seq":73,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.556647+0000","last_change":"2024-04-17T04:51:43.667351+0000","last_active":"2024-04-17T05:46:03.556647+0000","last_peered":"2024-04-17T05:46:03.556647+0000","last_clean":"2024-04-17T05:46:03.556647+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:46:03.556647+0000","last_undegraded":"2024-04-17T05:46:03.556647+0000","last_fullsized":"2024-04-17T05:46:03.556647+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":37890,"num_write":20,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.607535+0000","last_change":"2024-04-17T05:46:23.607603+0000","last_active":"2024-04-17T05:46:23.607535+0000","last_peered":"2024-04-17T05:46:23.607535+0000","last_clean":"2024-04-17T05:46:23.607535+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:46:23.607535+0000","last_undegraded":"2024-04-17T05:46:23.607535+0000","last_fullsized":"2024-04-17T05:46:23.607535+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.607294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:20.141964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":221,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.194774+0000","last_change":"2024-04-17T05:46:18.194774+0000","last_active":"2024-04-17T05:46:18.194774+0000","last_peered":"2024-04-17T05:46:18.194774+0000","last_clean":"2024-04-17T05:46:18.194774+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:46:18.194774+0000","last_undegraded":"2024-04-17T05:46:18.194774+0000","last_fullsized":"2024-04-17T05:46:18.194774+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'65","last_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_deep_scrub":"264'65","last_deep_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.194519+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:31.405244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.277147+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:46:08.277147+0000","last_peered":"2024-04-17T05:46:08.277147+0000","last_clean":"2024-04-17T05:46:08.277147+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:46:08.277147+0000","last_undegraded":"2024-04-17T05:46:08.277147+0000","last_fullsized":"2024-04-17T05:46:08.277147+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"264'6","reported_seq":70,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.915407+0000","last_change":"2024-04-17T05:46:45.915482+0000","last_active":"2024-04-17T05:46:45.915407+0000","last_peered":"2024-04-17T05:46:45.915407+0000","last_clean":"2024-04-17T05:46:45.915407+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:46:45.915407+0000","last_undegraded":"2024-04-17T05:46:45.915407+0000","last_fullsized":"2024-04-17T05:46:45.915407+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:45.915212+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:45.915212+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.915212+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:02.071375+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.807390+0000","last_change":"2024-04-17T05:46:16.807574+0000","last_active":"2024-04-17T05:46:16.807390+0000","last_peered":"2024-04-17T05:46:16.807390+0000","last_clean":"2024-04-17T05:46:16.807390+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:46:16.807390+0000","last_undegraded":"2024-04-17T05:46:16.807390+0000","last_fullsized":"2024-04-17T05:46:16.807390+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.807094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:57.818718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":157,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.309760+0000","last_change":"2024-04-17T05:46:22.309837+0000","last_active":"2024-04-17T05:46:22.309760+0000","last_peered":"2024-04-17T05:46:22.309760+0000","last_clean":"2024-04-17T05:46:22.309760+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:46:22.309760+0000","last_undegraded":"2024-04-17T05:46:22.309760+0000","last_fullsized":"2024-04-17T05:46:22.309760+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.309548+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:29.733950+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.936071+0000","last_change":"2024-04-17T05:07:45.749757+0000","last_active":"2024-04-17T05:46:06.936071+0000","last_peered":"2024-04-17T05:46:06.936071+0000","last_clean":"2024-04-17T05:46:06.936071+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:46:06.936071+0000","last_undegraded":"2024-04-17T05:46:06.936071+0000","last_fullsized":"2024-04-17T05:46:06.936071+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'2","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.726875+0000","last_change":"2024-04-17T05:46:16.727020+0000","last_active":"2024-04-17T05:46:16.726875+0000","last_peered":"2024-04-17T05:46:16.726875+0000","last_clean":"2024-04-17T05:46:16.726875+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:46:16.726875+0000","last_undegraded":"2024-04-17T05:46:16.726875+0000","last_fullsized":"2024-04-17T05:46:16.726875+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.726644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:09.817384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.831759+0000","last_change":"2024-04-17T05:46:10.831811+0000","last_active":"2024-04-17T05:46:10.831759+0000","last_peered":"2024-04-17T05:46:10.831759+0000","last_clean":"2024-04-17T05:46:10.831759+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:46:10.831759+0000","last_undegraded":"2024-04-17T05:46:10.831759+0000","last_fullsized":"2024-04-17T05:46:10.831759+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.831558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:46.336010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":142,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.670845+0000","last_change":"2024-04-17T05:46:09.670845+0000","last_active":"2024-04-17T05:46:09.670845+0000","last_peered":"2024-04-17T05:46:09.670845+0000","last_clean":"2024-04-17T05:46:09.670845+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:46:09.670845+0000","last_undegraded":"2024-04-17T05:46:09.670845+0000","last_fullsized":"2024-04-17T05:46:09.670845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.670593+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:59.152267+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"263'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184793+0000","last_change":"2024-04-17T05:07:45.748765+0000","last_active":"2024-04-17T05:45:15.184793+0000","last_peered":"2024-04-17T05:45:15.184793+0000","last_clean":"2024-04-17T05:45:15.184793+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:45:15.184793+0000","last_undegraded":"2024-04-17T05:45:15.184793+0000","last_fullsized":"2024-04-17T05:45:15.184793+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.712424+0000","last_change":"2024-04-17T05:46:38.712582+0000","last_active":"2024-04-17T05:46:38.712424+0000","last_peered":"2024-04-17T05:46:38.712424+0000","last_clean":"2024-04-17T05:46:38.712424+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:46:38.712424+0000","last_undegraded":"2024-04-17T05:46:38.712424+0000","last_fullsized":"2024-04-17T05:46:38.712424+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:38.712195+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:38.712195+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.712195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:58.037440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'5","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.032735+0000","last_change":"2024-04-17T05:46:14.032796+0000","last_active":"2024-04-17T05:46:14.032735+0000","last_peered":"2024-04-17T05:46:14.032735+0000","last_clean":"2024-04-17T05:46:14.032735+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:46:14.032735+0000","last_undegraded":"2024-04-17T05:46:14.032735+0000","last_fullsized":"2024-04-17T05:46:14.032735+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.032549+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:46.453310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":111,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.793130+0000","last_change":"2024-04-17T05:46:09.793130+0000","last_active":"2024-04-17T05:46:09.793130+0000","last_peered":"2024-04-17T05:46:09.793130+0000","last_clean":"2024-04-17T05:46:09.793130+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:46:09.793130+0000","last_undegraded":"2024-04-17T05:46:09.793130+0000","last_fullsized":"2024-04-17T05:46:09.793130+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.792865+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:12:51.707714+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'1","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.277236+0000","last_change":"2024-04-17T05:07:45.741199+0000","last_active":"2024-04-17T05:46:08.277236+0000","last_peered":"2024-04-17T05:46:08.277236+0000","last_clean":"2024-04-17T05:46:08.277236+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:46:08.277236+0000","last_undegraded":"2024-04-17T05:46:08.277236+0000","last_fullsized":"2024-04-17T05:46:08.277236+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.871034+0000","last_change":"2024-04-17T05:46:43.871089+0000","last_active":"2024-04-17T05:46:43.871034+0000","last_peered":"2024-04-17T05:46:43.871034+0000","last_clean":"2024-04-17T05:46:43.871034+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:46:43.871034+0000","last_undegraded":"2024-04-17T05:46:43.871034+0000","last_fullsized":"2024-04-17T05:46:43.871034+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.870838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.870838+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.870838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:23.777410+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.134683+0000","last_change":"2024-04-17T05:46:20.134745+0000","last_active":"2024-04-17T05:46:20.134683+0000","last_peered":"2024-04-17T05:46:20.134683+0000","last_clean":"2024-04-17T05:46:20.134683+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:46:20.134683+0000","last_undegraded":"2024-04-17T05:46:20.134683+0000","last_fullsized":"2024-04-17T05:46:20.134683+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.134488+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:48.392260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.861464+0000","last_change":"2024-04-17T05:46:13.861464+0000","last_active":"2024-04-17T05:46:13.861464+0000","last_peered":"2024-04-17T05:46:13.861464+0000","last_clean":"2024-04-17T05:46:13.861464+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:46:13.861464+0000","last_undegraded":"2024-04-17T05:46:13.861464+0000","last_fullsized":"2024-04-17T05:46:13.861464+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'55","last_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_deep_scrub":"264'55","last_deep_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.861056+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:57.160789+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.568509+0000","last_change":"2024-04-17T05:46:37.568563+0000","last_active":"2024-04-17T05:46:37.568509+0000","last_peered":"2024-04-17T05:46:37.568509+0000","last_clean":"2024-04-17T05:46:37.568509+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:46:37.568509+0000","last_undegraded":"2024-04-17T05:46:37.568509+0000","last_fullsized":"2024-04-17T05:46:37.568509+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:37.568302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:37.568302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.568302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:25.004821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'6","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.841237+0000","last_change":"2024-04-17T05:46:30.841297+0000","last_active":"2024-04-17T05:46:30.841237+0000","last_peered":"2024-04-17T05:46:30.841237+0000","last_clean":"2024-04-17T05:46:30.841237+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:46:30.841237+0000","last_undegraded":"2024-04-17T05:46:30.841237+0000","last_fullsized":"2024-04-17T05:46:30.841237+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:30.841048+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:30.841048+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.841048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:11.067215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":11,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.170964+0000","last_change":"2024-04-17T05:46:10.171028+0000","last_active":"2024-04-17T05:46:10.170964+0000","last_peered":"2024-04-17T05:46:10.170964+0000","last_clean":"2024-04-17T05:46:10.170964+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:46:10.170964+0000","last_undegraded":"2024-04-17T05:46:10.170964+0000","last_fullsized":"2024-04-17T05:46:10.170964+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.170766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:36.494972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.881133+0000","last_change":"2024-04-17T05:46:12.881133+0000","last_active":"2024-04-17T05:46:12.881133+0000","last_peered":"2024-04-17T05:46:12.881133+0000","last_clean":"2024-04-17T05:46:12.881133+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:46:12.881133+0000","last_undegraded":"2024-04-17T05:46:12.881133+0000","last_fullsized":"2024-04-17T05:46:12.881133+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.880856+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:56.211415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.189702+0000","last_change":"2024-04-17T05:07:45.751549+0000","last_active":"2024-04-17T05:46:00.189702+0000","last_peered":"2024-04-17T05:46:00.189702+0000","last_clean":"2024-04-17T05:46:00.189702+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:46:00.189702+0000","last_undegraded":"2024-04-17T05:46:00.189702+0000","last_fullsized":"2024-04-17T05:46:00.189702+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'5","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:01.460050+0000","last_change":"2024-04-17T04:51:43.688891+0000","last_active":"2024-04-17T05:46:01.460050+0000","last_peered":"2024-04-17T05:46:01.460050+0000","last_clean":"2024-04-17T05:46:01.460050+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:46:01.460050+0000","last_undegraded":"2024-04-17T05:46:01.460050+0000","last_fullsized":"2024-04-17T05:46:01.460050+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'5","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.505904+0000","last_change":"2024-04-17T05:46:34.506036+0000","last_active":"2024-04-17T05:46:34.505904+0000","last_peered":"2024-04-17T05:46:34.505904+0000","last_clean":"2024-04-17T05:46:34.505904+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:46:34.505904+0000","last_undegraded":"2024-04-17T05:46:34.505904+0000","last_fullsized":"2024-04-17T05:46:34.505904+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:34.505613+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:34.505613+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.505613+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:13.411372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":11264,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":163,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.131285+0000","last_change":"2024-04-17T05:46:16.131285+0000","last_active":"2024-04-17T05:46:16.131285+0000","last_peered":"2024-04-17T05:46:16.131285+0000","last_clean":"2024-04-17T05:46:16.131285+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:46:16.131285+0000","last_undegraded":"2024-04-17T05:46:16.131285+0000","last_fullsized":"2024-04-17T05:46:16.131285+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.131038+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:32:33.132984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.896222+0000","last_change":"2024-04-17T05:46:46.896279+0000","last_active":"2024-04-17T05:46:46.896222+0000","last_peered":"2024-04-17T05:46:46.896222+0000","last_clean":"2024-04-17T05:46:46.896222+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:46:46.896222+0000","last_undegraded":"2024-04-17T05:46:46.896222+0000","last_fullsized":"2024-04-17T05:46:46.896222+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:46.896013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:46.896013+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.896013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:52.872792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.908271+0000","last_change":"2024-04-17T05:46:44.908338+0000","last_active":"2024-04-17T05:46:44.908271+0000","last_peered":"2024-04-17T05:46:44.908271+0000","last_clean":"2024-04-17T05:46:44.908271+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:46:44.908271+0000","last_undegraded":"2024-04-17T05:46:44.908271+0000","last_fullsized":"2024-04-17T05:46:44.908271+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:44.908063+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:44.908063+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.908063+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:38.209073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.674045+0000","last_change":"2024-04-17T05:46:19.674109+0000","last_active":"2024-04-17T05:46:19.674045+0000","last_peered":"2024-04-17T05:46:19.674045+0000","last_clean":"2024-04-17T05:46:19.674045+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:46:19.674045+0000","last_undegraded":"2024-04-17T05:46:19.674045+0000","last_fullsized":"2024-04-17T05:46:19.674045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.673832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:20:55.342567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.057017+0000","last_change":"2024-04-17T05:46:11.057017+0000","last_active":"2024-04-17T05:46:11.057017+0000","last_peered":"2024-04-17T05:46:11.057017+0000","last_clean":"2024-04-17T05:46:11.057017+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:46:11.057017+0000","last_undegraded":"2024-04-17T05:46:11.057017+0000","last_fullsized":"2024-04-17T05:46:11.057017+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.056723+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:04.980611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.157133+0000","last_change":"2024-04-17T05:07:45.757127+0000","last_active":"2024-04-17T05:46:10.157133+0000","last_peered":"2024-04-17T05:46:10.157133+0000","last_clean":"2024-04-17T05:46:10.157133+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:46:10.157133+0000","last_undegraded":"2024-04-17T05:46:10.157133+0000","last_fullsized":"2024-04-17T05:46:10.157133+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":46,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:05.198230+0000","last_change":"2024-04-17T04:52:04.726201+0000","last_active":"2024-04-17T05:46:05.198230+0000","last_peered":"2024-04-17T05:46:05.198230+0000","last_clean":"2024-04-17T05:46:05.198230+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:46:05.198230+0000","last_undegraded":"2024-04-17T05:46:05.198230+0000","last_fullsized":"2024-04-17T05:46:05.198230+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.680401+0000","last_change":"2024-04-17T05:46:08.680438+0000","last_active":"2024-04-17T05:46:08.680401+0000","last_peered":"2024-04-17T05:46:08.680401+0000","last_clean":"2024-04-17T05:46:08.680401+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:46:08.680401+0000","last_undegraded":"2024-04-17T05:46:08.680401+0000","last_fullsized":"2024-04-17T05:46:08.680401+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.680268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:03.743450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":138,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.494346+0000","last_change":"2024-04-17T05:46:22.494346+0000","last_active":"2024-04-17T05:46:22.494346+0000","last_peered":"2024-04-17T05:46:22.494346+0000","last_clean":"2024-04-17T05:46:22.494346+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:46:22.494346+0000","last_undegraded":"2024-04-17T05:46:22.494346+0000","last_fullsized":"2024-04-17T05:46:22.494346+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_deep_scrub":"264'30","last_deep_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.494058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:08.498830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.226630+0000","last_change":"2024-04-17T05:07:45.744264+0000","last_active":"2024-04-17T05:46:00.226630+0000","last_peered":"2024-04-17T05:46:00.226630+0000","last_clean":"2024-04-17T05:46:00.226630+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:46:00.226630+0000","last_undegraded":"2024-04-17T05:46:00.226630+0000","last_fullsized":"2024-04-17T05:46:00.226630+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'7","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.833881+0000","last_change":"2024-04-17T04:51:43.661076+0000","last_active":"2024-04-17T05:46:02.833881+0000","last_peered":"2024-04-17T05:46:02.833881+0000","last_clean":"2024-04-17T05:46:02.833881+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:46:02.833881+0000","last_undegraded":"2024-04-17T05:46:02.833881+0000","last_fullsized":"2024-04-17T05:46:02.833881+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":5216,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.623849+0000","last_change":"2024-04-17T05:46:11.623908+0000","last_active":"2024-04-17T05:46:11.623849+0000","last_peered":"2024-04-17T05:46:11.623849+0000","last_clean":"2024-04-17T05:46:11.623849+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:46:11.623849+0000","last_undegraded":"2024-04-17T05:46:11.623849+0000","last_fullsized":"2024-04-17T05:46:11.623849+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.623642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:23.552160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":112,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.642068+0000","last_change":"2024-04-17T05:46:10.642068+0000","last_active":"2024-04-17T05:46:10.642068+0000","last_peered":"2024-04-17T05:46:10.642068+0000","last_clean":"2024-04-17T05:46:10.642068+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:46:10.642068+0000","last_undegraded":"2024-04-17T05:46:10.642068+0000","last_fullsized":"2024-04-17T05:46:10.642068+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.641783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:09.385082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:05.191010+0000","last_change":"2024-04-17T05:07:45.757192+0000","last_active":"2024-04-17T05:46:05.191010+0000","last_peered":"2024-04-17T05:46:05.191010+0000","last_clean":"2024-04-17T05:46:05.191010+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:46:05.191010+0000","last_undegraded":"2024-04-17T05:46:05.191010+0000","last_fullsized":"2024-04-17T05:46:05.191010+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.167379+0000","last_change":"2024-04-17T05:46:16.167438+0000","last_active":"2024-04-17T05:46:16.167379+0000","last_peered":"2024-04-17T05:46:16.167379+0000","last_clean":"2024-04-17T05:46:16.167379+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:46:16.167379+0000","last_undegraded":"2024-04-17T05:46:16.167379+0000","last_fullsized":"2024-04-17T05:46:16.167379+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.167190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:00.500071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.682742+0000","last_change":"2024-04-17T05:46:07.682778+0000","last_active":"2024-04-17T05:46:07.682742+0000","last_peered":"2024-04-17T05:46:07.682742+0000","last_clean":"2024-04-17T05:46:07.682742+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:46:07.682742+0000","last_undegraded":"2024-04-17T05:46:07.682742+0000","last_fullsized":"2024-04-17T05:46:07.682742+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.682661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:07.614824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.265908+0000","last_change":"2024-04-17T05:46:21.265908+0000","last_active":"2024-04-17T05:46:21.265908+0000","last_peered":"2024-04-17T05:46:21.265908+0000","last_clean":"2024-04-17T05:46:21.265908+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:46:21.265908+0000","last_undegraded":"2024-04-17T05:46:21.265908+0000","last_fullsized":"2024-04-17T05:46:21.265908+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'93","last_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_deep_scrub":"264'93","last_deep_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.265637+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:31.736633+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'1","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.189770+0000","last_change":"2024-04-17T05:07:45.754827+0000","last_active":"2024-04-17T05:46:00.189770+0000","last_peered":"2024-04-17T05:46:00.189770+0000","last_clean":"2024-04-17T05:46:00.189770+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:46:00.189770+0000","last_undegraded":"2024-04-17T05:46:00.189770+0000","last_fullsized":"2024-04-17T05:46:00.189770+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.884070+0000","last_change":"2024-04-17T05:46:43.884127+0000","last_active":"2024-04-17T05:46:43.884070+0000","last_peered":"2024-04-17T05:46:43.884070+0000","last_clean":"2024-04-17T05:46:43.884070+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:46:43.884070+0000","last_undegraded":"2024-04-17T05:46:43.884070+0000","last_fullsized":"2024-04-17T05:46:43.884070+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.883876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.883876+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.883876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:29.135926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.584946+0000","last_change":"2024-04-17T05:46:15.585004+0000","last_active":"2024-04-17T05:46:15.584946+0000","last_peered":"2024-04-17T05:46:15.584946+0000","last_clean":"2024-04-17T05:46:15.584946+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:46:15.584946+0000","last_undegraded":"2024-04-17T05:46:15.584946+0000","last_fullsized":"2024-04-17T05:46:15.584946+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.584695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:18.890015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.833951+0000","last_change":"2024-04-17T05:46:15.833951+0000","last_active":"2024-04-17T05:46:15.833951+0000","last_peered":"2024-04-17T05:46:15.833951+0000","last_clean":"2024-04-17T05:46:15.833951+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:46:15.833951+0000","last_undegraded":"2024-04-17T05:46:15.833951+0000","last_fullsized":"2024-04-17T05:46:15.833951+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'22","last_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_deep_scrub":"264'22","last_deep_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.833697+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:34:19.233513+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":30,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:00.197261+0000","last_change":"2024-04-17T05:07:45.739696+0000","last_active":"2024-04-17T05:46:00.197261+0000","last_peered":"2024-04-17T05:46:00.197261+0000","last_clean":"2024-04-17T05:46:00.197261+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:46:00.197261+0000","last_undegraded":"2024-04-17T05:46:00.197261+0000","last_fullsized":"2024-04-17T05:46:00.197261+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.268742+0000","last_change":"2024-04-17T05:46:20.268796+0000","last_active":"2024-04-17T05:46:20.268742+0000","last_peered":"2024-04-17T05:46:20.268742+0000","last_clean":"2024-04-17T05:46:20.268742+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:46:20.268742+0000","last_undegraded":"2024-04-17T05:46:20.268742+0000","last_fullsized":"2024-04-17T05:46:20.268742+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.268538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:35.180807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.745329+0000","last_change":"2024-04-17T05:46:24.745393+0000","last_active":"2024-04-17T05:46:24.745329+0000","last_peered":"2024-04-17T05:46:24.745329+0000","last_clean":"2024-04-17T05:46:24.745329+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:46:24.745329+0000","last_undegraded":"2024-04-17T05:46:24.745329+0000","last_fullsized":"2024-04-17T05:46:24.745329+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.745126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:13.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":198,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.022550+0000","last_change":"2024-04-17T05:46:10.022550+0000","last_active":"2024-04-17T05:46:10.022550+0000","last_peered":"2024-04-17T05:46:10.022550+0000","last_clean":"2024-04-17T05:46:10.022550+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:46:10.022550+0000","last_undegraded":"2024-04-17T05:46:10.022550+0000","last_fullsized":"2024-04-17T05:46:10.022550+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'52","last_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_deep_scrub":"264'52","last_deep_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.022290+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.993887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.273662+0000","last_change":"2024-04-17T05:46:28.273717+0000","last_active":"2024-04-17T05:46:28.273662+0000","last_peered":"2024-04-17T05:46:28.273662+0000","last_clean":"2024-04-17T05:46:28.273662+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:46:28.273662+0000","last_undegraded":"2024-04-17T05:46:28.273662+0000","last_fullsized":"2024-04-17T05:46:28.273662+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.273435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.273435+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.273435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:53.897551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:05.198205+0000","last_change":"2024-04-17T04:51:43.682823+0000","last_active":"2024-04-17T05:46:05.198205+0000","last_peered":"2024-04-17T05:46:05.198205+0000","last_clean":"2024-04-17T05:46:05.198205+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:46:05.198205+0000","last_undegraded":"2024-04-17T05:46:05.198205+0000","last_fullsized":"2024-04-17T05:46:05.198205+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'8","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.163258+0000","last_change":"2024-04-17T05:46:15.163324+0000","last_active":"2024-04-17T05:46:15.163258+0000","last_peered":"2024-04-17T05:46:15.163258+0000","last_clean":"2024-04-17T05:46:15.163258+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:46:15.163258+0000","last_undegraded":"2024-04-17T05:46:15.163258+0000","last_fullsized":"2024-04-17T05:46:15.163258+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.163041+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:18.146680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11291,"num_write":13,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":159,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.612312+0000","last_change":"2024-04-17T05:46:13.612312+0000","last_active":"2024-04-17T05:46:13.612312+0000","last_peered":"2024-04-17T05:46:13.612312+0000","last_clean":"2024-04-17T05:46:13.612312+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:46:13.612312+0000","last_undegraded":"2024-04-17T05:46:13.612312+0000","last_fullsized":"2024-04-17T05:46:13.612312+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'39","last_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_deep_scrub":"264'39","last_deep_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.612027+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:28.378268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184387+0000","last_change":"2024-04-17T05:07:45.753495+0000","last_active":"2024-04-17T05:45:15.184387+0000","last_peered":"2024-04-17T05:45:15.184387+0000","last_clean":"2024-04-17T05:45:15.184387+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:45:15.184387+0000","last_undegraded":"2024-04-17T05:45:15.184387+0000","last_fullsized":"2024-04-17T05:45:15.184387+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.614461+0000","last_change":"2024-04-17T05:46:31.614516+0000","last_active":"2024-04-17T05:46:31.614461+0000","last_peered":"2024-04-17T05:46:31.614461+0000","last_clean":"2024-04-17T05:46:31.614461+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:46:31.614461+0000","last_undegraded":"2024-04-17T05:46:31.614461+0000","last_fullsized":"2024-04-17T05:46:31.614461+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.614267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.614267+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.614267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:54:03.860590+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.554323+0000","last_change":"2024-04-17T05:46:35.554396+0000","last_active":"2024-04-17T05:46:35.554323+0000","last_peered":"2024-04-17T05:46:35.554323+0000","last_clean":"2024-04-17T05:46:35.554323+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:46:35.554323+0000","last_undegraded":"2024-04-17T05:46:35.554323+0000","last_fullsized":"2024-04-17T05:46:35.554323+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:35.554116+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:35.554116+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.554116+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:28.363602+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":206,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.995853+0000","last_change":"2024-04-17T05:46:14.995853+0000","last_active":"2024-04-17T05:46:14.995853+0000","last_peered":"2024-04-17T05:46:14.995853+0000","last_clean":"2024-04-17T05:46:14.995853+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:46:14.995853+0000","last_undegraded":"2024-04-17T05:46:14.995853+0000","last_fullsized":"2024-04-17T05:46:14.995853+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'60","last_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_deep_scrub":"264'60","last_deep_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.995600+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:08.616032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.964095+0000","last_change":"2024-04-17T05:46:47.964149+0000","last_active":"2024-04-17T05:46:47.964095+0000","last_peered":"2024-04-17T05:46:47.964095+0000","last_clean":"2024-04-17T05:46:47.964095+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:46:47.964095+0000","last_undegraded":"2024-04-17T05:46:47.964095+0000","last_fullsized":"2024-04-17T05:46:47.964095+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:47.963889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:47.963889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.963889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:33.636904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'21","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.298945+0000","last_change":"2024-04-17T05:46:24.299018+0000","last_active":"2024-04-17T05:46:24.298945+0000","last_peered":"2024-04-17T05:46:24.298945+0000","last_clean":"2024-04-17T05:46:24.298945+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:46:24.298945+0000","last_undegraded":"2024-04-17T05:46:24.298945+0000","last_fullsized":"2024-04-17T05:46:24.298945+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'21","last_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_deep_scrub":"264'21","last_deep_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.298719+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:55.830264+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":54274,"num_write":19,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'10","reported_seq":80,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:50.192007+0000","last_change":"2024-04-17T04:51:39.635143+0000","last_active":"2024-04-17T05:45:50.192007+0000","last_peered":"2024-04-17T05:45:50.192007+0000","last_clean":"2024-04-17T05:45:50.192007+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:45:50.192007+0000","last_undegraded":"2024-04-17T05:45:50.192007+0000","last_fullsized":"2024-04-17T05:45:50.192007+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:38.577167+0000","last_clean_scrub_stamp":"2024-04-17T04:51:38.577167+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":233,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.186314+0000","last_change":"2024-04-17T05:46:17.186314+0000","last_active":"2024-04-17T05:46:17.186314+0000","last_peered":"2024-04-17T05:46:17.186314+0000","last_clean":"2024-04-17T05:46:17.186314+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:46:17.186314+0000","last_undegraded":"2024-04-17T05:46:17.186314+0000","last_fullsized":"2024-04-17T05:46:17.186314+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.186068+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:32.734141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.174058+0000","last_change":"2024-04-17T05:07:45.749227+0000","last_active":"2024-04-17T05:45:15.174058+0000","last_peered":"2024-04-17T05:45:15.174058+0000","last_clean":"2024-04-17T05:45:15.174058+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:45:15.174058+0000","last_undegraded":"2024-04-17T05:45:15.174058+0000","last_fullsized":"2024-04-17T05:45:15.174058+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":48,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.183525+0000","last_change":"2024-04-17T04:51:43.648648+0000","last_active":"2024-04-17T05:45:55.183525+0000","last_peered":"2024-04-17T05:45:55.183525+0000","last_clean":"2024-04-17T05:45:55.183525+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:45:55.183525+0000","last_undegraded":"2024-04-17T05:45:55.183525+0000","last_fullsized":"2024-04-17T05:45:55.183525+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'6","reported_seq":73,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.099801+0000","last_change":"2024-04-17T05:46:12.099865+0000","last_active":"2024-04-17T05:46:12.099801+0000","last_peered":"2024-04-17T05:46:12.099801+0000","last_clean":"2024-04-17T05:46:12.099801+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:46:12.099801+0000","last_undegraded":"2024-04-17T05:46:12.099801+0000","last_fullsized":"2024-04-17T05:46:12.099801+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.099605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:49.682658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":8192,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":120,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.665206+0000","last_change":"2024-04-17T05:46:10.665206+0000","last_active":"2024-04-17T05:46:10.665206+0000","last_peered":"2024-04-17T05:46:10.665206+0000","last_clean":"2024-04-17T05:46:10.665206+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:46:10.665206+0000","last_undegraded":"2024-04-17T05:46:10.665206+0000","last_fullsized":"2024-04-17T05:46:10.665206+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'21","last_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_deep_scrub":"261'21","last_deep_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.664937+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:18.989139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'2","reported_seq":41,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.564484+0000","last_change":"2024-04-17T05:46:38.564484+0000","last_active":"2024-04-17T05:46:38.564484+0000","last_peered":"2024-04-17T05:46:38.564484+0000","last_clean":"2024-04-17T05:46:38.564484+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:46:38.564484+0000","last_undegraded":"2024-04-17T05:46:38.564484+0000","last_fullsized":"2024-04-17T05:46:38.564484+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:38.564228+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:38.564228+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.564228+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:51.812713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'9","reported_seq":72,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.926385+0000","last_change":"2024-04-17T05:46:46.926459+0000","last_active":"2024-04-17T05:46:46.926385+0000","last_peered":"2024-04-17T05:46:46.926385+0000","last_clean":"2024-04-17T05:46:46.926385+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:46:46.926385+0000","last_undegraded":"2024-04-17T05:46:46.926385+0000","last_fullsized":"2024-04-17T05:46:46.926385+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'9","last_scrub_stamp":"2024-04-17T05:46:46.926173+0000","last_deep_scrub":"264'9","last_deep_scrub_stamp":"2024-04-17T05:46:46.926173+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.926173+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:21.554513+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":4098,"num_write":17,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.130174+0000","last_change":"2024-04-17T05:46:13.130289+0000","last_active":"2024-04-17T05:46:13.130174+0000","last_peered":"2024-04-17T05:46:13.130174+0000","last_clean":"2024-04-17T05:46:13.130174+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:46:13.130174+0000","last_undegraded":"2024-04-17T05:46:13.130174+0000","last_fullsized":"2024-04-17T05:46:13.130174+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.129923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:48.339568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.106637+0000","last_change":"2024-04-17T05:46:12.106637+0000","last_active":"2024-04-17T05:46:12.106637+0000","last_peered":"2024-04-17T05:46:12.106637+0000","last_clean":"2024-04-17T05:46:12.106637+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:46:12.106637+0000","last_undegraded":"2024-04-17T05:46:12.106637+0000","last_fullsized":"2024-04-17T05:46:12.106637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'32","last_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_deep_scrub":"253'32","last_deep_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.106369+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:23.602749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.936004+0000","last_change":"2024-04-17T05:07:45.726938+0000","last_active":"2024-04-17T05:46:06.936004+0000","last_peered":"2024-04-17T05:46:06.936004+0000","last_clean":"2024-04-17T05:46:06.936004+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:46:06.936004+0000","last_undegraded":"2024-04-17T05:46:06.936004+0000","last_fullsized":"2024-04-17T05:46:06.936004+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'3","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.390031+0000","last_change":"2024-04-17T04:51:43.648636+0000","last_active":"2024-04-17T05:46:02.390031+0000","last_peered":"2024-04-17T05:46:02.390031+0000","last_clean":"2024-04-17T05:46:02.390031+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:46:02.390031+0000","last_undegraded":"2024-04-17T05:46:02.390031+0000","last_fullsized":"2024-04-17T05:46:02.390031+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.127295+0000","last_change":"2024-04-17T05:46:14.127359+0000","last_active":"2024-04-17T05:46:14.127295+0000","last_peered":"2024-04-17T05:46:14.127295+0000","last_clean":"2024-04-17T05:46:14.127295+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:46:14.127295+0000","last_undegraded":"2024-04-17T05:46:14.127295+0000","last_fullsized":"2024-04-17T05:46:14.127295+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.127097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.931818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":151,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.750372+0000","last_change":"2024-04-17T05:46:08.750372+0000","last_active":"2024-04-17T05:46:08.750372+0000","last_peered":"2024-04-17T05:46:08.750372+0000","last_clean":"2024-04-17T05:46:08.750372+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:46:08.750372+0000","last_undegraded":"2024-04-17T05:46:08.750372+0000","last_fullsized":"2024-04-17T05:46:08.750372+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'32","last_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_deep_scrub":"264'32","last_deep_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.750109+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:20.266910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.184839+0000","last_change":"2024-04-17T05:07:45.757247+0000","last_active":"2024-04-17T05:45:15.184839+0000","last_peered":"2024-04-17T05:45:15.184839+0000","last_clean":"2024-04-17T05:45:15.184839+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:45:15.184839+0000","last_undegraded":"2024-04-17T05:45:15.184839+0000","last_fullsized":"2024-04-17T05:45:15.184839+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.858323+0000","last_change":"2024-04-17T05:46:44.858377+0000","last_active":"2024-04-17T05:46:44.858323+0000","last_peered":"2024-04-17T05:46:44.858323+0000","last_clean":"2024-04-17T05:46:44.858323+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:46:44.858323+0000","last_undegraded":"2024-04-17T05:46:44.858323+0000","last_fullsized":"2024-04-17T05:46:44.858323+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:44.858126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:44.858126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.858126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:56.173235+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'2","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.643349+0000","last_change":"2024-04-17T05:46:09.643497+0000","last_active":"2024-04-17T05:46:09.643349+0000","last_peered":"2024-04-17T05:46:09.643349+0000","last_clean":"2024-04-17T05:46:09.643349+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:46:09.643349+0000","last_undegraded":"2024-04-17T05:46:09.643349+0000","last_fullsized":"2024-04-17T05:46:09.643349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.643135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:49.245538+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.049363+0000","last_change":"2024-04-17T05:46:13.049363+0000","last_active":"2024-04-17T05:46:13.049363+0000","last_peered":"2024-04-17T05:46:13.049363+0000","last_clean":"2024-04-17T05:46:13.049363+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:46:13.049363+0000","last_undegraded":"2024-04-17T05:46:13.049363+0000","last_fullsized":"2024-04-17T05:46:13.049363+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'69","last_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_deep_scrub":"264'69","last_deep_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.048938+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:49.209093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":31,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:20.180732+0000","last_change":"2024-04-17T05:07:45.752461+0000","last_active":"2024-04-17T05:45:20.180732+0000","last_peered":"2024-04-17T05:45:20.180732+0000","last_clean":"2024-04-17T05:45:20.180732+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:45:20.180732+0000","last_undegraded":"2024-04-17T05:45:20.180732+0000","last_fullsized":"2024-04-17T05:45:20.180732+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":49,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:55.183418+0000","last_change":"2024-04-17T04:51:43.651232+0000","last_active":"2024-04-17T05:45:55.183418+0000","last_peered":"2024-04-17T05:45:55.183418+0000","last_clean":"2024-04-17T05:45:55.183418+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:45:55.183418+0000","last_undegraded":"2024-04-17T05:45:55.183418+0000","last_fullsized":"2024-04-17T05:45:55.183418+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:51:42.619140+0000","last_clean_scrub_stamp":"2024-04-17T04:51:42.619140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'5","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.936901+0000","last_change":"2024-04-17T05:46:40.936964+0000","last_active":"2024-04-17T05:46:40.936901+0000","last_peered":"2024-04-17T05:46:40.936901+0000","last_clean":"2024-04-17T05:46:40.936901+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:46:40.936901+0000","last_undegraded":"2024-04-17T05:46:40.936901+0000","last_fullsized":"2024-04-17T05:46:40.936901+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:40.936715+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:40.936715+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.936715+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:06.846607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6192,"num_write":5,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":137,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.225148+0000","last_change":"2024-04-17T05:46:19.225296+0000","last_active":"2024-04-17T05:46:19.225148+0000","last_peered":"2024-04-17T05:46:19.225148+0000","last_clean":"2024-04-17T05:46:19.225148+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:46:19.225148+0000","last_undegraded":"2024-04-17T05:46:19.225148+0000","last_fullsized":"2024-04-17T05:46:19.225148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_deep_scrub":"264'27","last_deep_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.224892+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:29.117161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'349","reported_seq":1011,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.880896+0000","last_change":"2024-04-17T04:36:25.377106+0000","last_active":"2024-04-17T05:46:03.880896+0000","last_peered":"2024-04-17T05:46:03.880896+0000","last_clean":"2024-04-17T05:46:03.880896+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:46:03.880896+0000","last_undegraded":"2024-04-17T05:46:03.880896+0000","last_fullsized":"2024-04-17T05:46:03.880896+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":725,"num_write":1314,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.269919+0000","last_change":"2024-04-17T05:46:28.269919+0000","last_active":"2024-04-17T05:46:28.269919+0000","last_peered":"2024-04-17T05:46:28.269919+0000","last_clean":"2024-04-17T05:46:28.269919+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:46:28.269919+0000","last_undegraded":"2024-04-17T05:46:28.269919+0000","last_fullsized":"2024-04-17T05:46:28.269919+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'80","last_scrub_stamp":"2024-04-17T05:46:28.269667+0000","last_deep_scrub":"264'80","last_deep_scrub_stamp":"2024-04-17T05:46:28.269667+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.269667+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:21:37.474212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'382","reported_seq":1063,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.830224+0000","last_change":"2024-04-17T05:46:39.830336+0000","last_active":"2024-04-17T05:46:39.830224+0000","last_peered":"2024-04-17T05:46:39.830224+0000","last_clean":"2024-04-17T05:46:39.830224+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:46:39.830224+0000","last_undegraded":"2024-04-17T05:46:39.830224+0000","last_fullsized":"2024-04-17T05:46:39.830224+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'382","last_scrub_stamp":"2024-04-17T05:46:39.830021+0000","last_deep_scrub":"264'382","last_deep_scrub_stamp":"2024-04-17T05:46:39.830021+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.830021+0000","objects_scrubbed":6,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:40.048724+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":600,"num_read_kb":10797,"num_write":1506,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.391956+0000","last_change":"2024-04-17T05:46:39.392102+0000","last_active":"2024-04-17T05:46:39.391956+0000","last_peered":"2024-04-17T05:46:39.391956+0000","last_clean":"2024-04-17T05:46:39.391956+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:46:39.391956+0000","last_undegraded":"2024-04-17T05:46:39.391956+0000","last_fullsized":"2024-04-17T05:46:39.391956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:39.391733+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:39.391733+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.391733+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:24.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'312","reported_seq":979,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.881660+0000","last_change":"2024-04-17T04:36:25.329662+0000","last_active":"2024-04-17T05:46:03.881660+0000","last_peered":"2024-04-17T05:46:03.881660+0000","last_clean":"2024-04-17T05:46:03.881660+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:46:03.881660+0000","last_undegraded":"2024-04-17T05:46:03.881660+0000","last_fullsized":"2024-04-17T05:46:03.881660+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11583546,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":475,"num_read_kb":58403,"num_write":1188,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1215,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.608673+0000","last_change":"2024-04-17T05:46:37.608807+0000","last_active":"2024-04-17T05:46:37.608673+0000","last_peered":"2024-04-17T05:46:37.608673+0000","last_clean":"2024-04-17T05:46:37.608673+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:46:37.608673+0000","last_undegraded":"2024-04-17T05:46:37.608673+0000","last_fullsized":"2024-04-17T05:46:37.608673+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'525","last_scrub_stamp":"2024-04-17T05:46:37.608425+0000","last_deep_scrub":"264'525","last_deep_scrub_stamp":"2024-04-17T05:46:37.608425+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.608425+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:33.078323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'284","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.266538+0000","last_change":"2024-04-17T05:46:47.266902+0000","last_active":"2024-04-17T05:46:47.266538+0000","last_peered":"2024-04-17T05:46:47.266538+0000","last_clean":"2024-04-17T05:46:47.266538+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:46:47.266538+0000","last_undegraded":"2024-04-17T05:46:47.266538+0000","last_fullsized":"2024-04-17T05:46:47.266538+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'284","last_scrub_stamp":"2024-04-17T05:46:47.266303+0000","last_deep_scrub":"264'284","last_deep_scrub_stamp":"2024-04-17T05:46:47.266303+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.266303+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:04.388187+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174737,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":460,"num_read_kb":37884,"num_write":1128,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'305","reported_seq":908,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.703858+0000","last_change":"2024-04-17T05:46:39.703930+0000","last_active":"2024-04-17T05:46:39.703858+0000","last_peered":"2024-04-17T05:46:39.703858+0000","last_clean":"2024-04-17T05:46:39.703858+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:46:39.703858+0000","last_undegraded":"2024-04-17T05:46:39.703858+0000","last_fullsized":"2024-04-17T05:46:39.703858+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'305","last_scrub_stamp":"2024-04-17T05:46:39.703631+0000","last_deep_scrub":"264'305","last_deep_scrub_stamp":"2024-04-17T05:46:39.703631+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.703631+0000","objects_scrubbed":1,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:54.709159+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'336","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.112424+0000","last_change":"2024-04-17T05:46:49.112596+0000","last_active":"2024-04-17T05:46:49.112424+0000","last_peered":"2024-04-17T05:46:49.112424+0000","last_clean":"2024-04-17T05:46:49.112424+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:46:49.112424+0000","last_undegraded":"2024-04-17T05:46:49.112424+0000","last_fullsized":"2024-04-17T05:46:49.112424+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'336","last_scrub_stamp":"2024-04-17T05:46:49.112208+0000","last_deep_scrub":"264'336","last_deep_scrub_stamp":"2024-04-17T05:46:49.112208+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.112208+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:27.854631+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":528,"num_read_kb":13359,"num_write":1173,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5189,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.926781+0000","last_change":"2024-04-17T05:46:28.926781+0000","last_active":"2024-04-17T05:46:28.926781+0000","last_peered":"2024-04-17T05:46:28.926781+0000","last_clean":"2024-04-17T05:46:28.926781+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:46:28.926781+0000","last_undegraded":"2024-04-17T05:46:28.926781+0000","last_fullsized":"2024-04-17T05:46:28.926781+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4413","last_scrub_stamp":"2024-04-17T05:46:28.926530+0000","last_deep_scrub":"264'4413","last_deep_scrub_stamp":"2024-04-17T05:46:28.926530+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.926530+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:55:24.685918+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'301","reported_seq":1024,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.312758+0000","last_change":"2024-04-17T05:46:46.312861+0000","last_active":"2024-04-17T05:46:46.312758+0000","last_peered":"2024-04-17T05:46:46.312758+0000","last_clean":"2024-04-17T05:46:46.312758+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:46:46.312758+0000","last_undegraded":"2024-04-17T05:46:46.312758+0000","last_fullsized":"2024-04-17T05:46:46.312758+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'301","last_scrub_stamp":"2024-04-17T05:46:46.312563+0000","last_deep_scrub":"264'301","last_deep_scrub_stamp":"2024-04-17T05:46:46.312563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.312563+0000","objects_scrubbed":9,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:18.182664+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206995,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":584,"num_read_kb":64592,"num_write":977,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":783,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.560806+0000","last_change":"2024-04-17T05:46:33.560872+0000","last_active":"2024-04-17T05:46:33.560806+0000","last_peered":"2024-04-17T05:46:33.560806+0000","last_clean":"2024-04-17T05:46:33.560806+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:46:33.560806+0000","last_undegraded":"2024-04-17T05:46:33.560806+0000","last_fullsized":"2024-04-17T05:46:33.560806+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'157","last_scrub_stamp":"2024-04-17T05:46:33.560601+0000","last_deep_scrub":"264'157","last_deep_scrub_stamp":"2024-04-17T05:46:33.560601+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.560601+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:27.508256+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'324","reported_seq":948,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155808+0000","last_change":"2024-04-17T04:36:25.376858+0000","last_active":"2024-04-17T05:46:10.155808+0000","last_peered":"2024-04-17T05:46:10.155808+0000","last_clean":"2024-04-17T05:46:10.155808+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:46:10.155808+0000","last_undegraded":"2024-04-17T05:46:10.155808+0000","last_fullsized":"2024-04-17T05:46:10.155808+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269430,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":458,"num_read_kb":9424,"num_write":1128,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":817,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.313672+0000","last_change":"2024-04-17T05:46:31.313738+0000","last_active":"2024-04-17T05:46:31.313672+0000","last_peered":"2024-04-17T05:46:31.313672+0000","last_clean":"2024-04-17T05:46:31.313672+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:46:31.313672+0000","last_undegraded":"2024-04-17T05:46:31.313672+0000","last_fullsized":"2024-04-17T05:46:31.313672+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'165","last_scrub_stamp":"2024-04-17T05:46:31.313464+0000","last_deep_scrub":"264'165","last_deep_scrub_stamp":"2024-04-17T05:46:31.313464+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.313464+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:46.132113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'318","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275588+0000","last_change":"2024-04-17T04:36:25.343980+0000","last_active":"2024-04-17T05:46:08.275588+0000","last_peered":"2024-04-17T05:46:08.275588+0000","last_clean":"2024-04-17T05:46:08.275588+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:46:08.275588+0000","last_undegraded":"2024-04-17T05:46:08.275588+0000","last_fullsized":"2024-04-17T05:46:08.275588+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076711,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":427,"num_read_kb":14992,"num_write":1164,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":779,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.467160+0000","last_change":"2024-04-17T05:46:28.467227+0000","last_active":"2024-04-17T05:46:28.467160+0000","last_peered":"2024-04-17T05:46:28.467160+0000","last_clean":"2024-04-17T05:46:28.467160+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:46:28.467160+0000","last_undegraded":"2024-04-17T05:46:28.467160+0000","last_fullsized":"2024-04-17T05:46:28.467160+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'166","last_scrub_stamp":"2024-04-17T05:46:28.466958+0000","last_deep_scrub":"264'166","last_deep_scrub_stamp":"2024-04-17T05:46:28.466958+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.466958+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:50:48.423737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.590578+0000","last_change":"2024-04-17T05:46:31.590640+0000","last_active":"2024-04-17T05:46:31.590578+0000","last_peered":"2024-04-17T05:46:31.590578+0000","last_clean":"2024-04-17T05:46:31.590578+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:46:31.590578+0000","last_undegraded":"2024-04-17T05:46:31.590578+0000","last_fullsized":"2024-04-17T05:46:31.590578+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'128","last_scrub_stamp":"2024-04-17T05:46:31.590386+0000","last_deep_scrub":"264'128","last_deep_scrub_stamp":"2024-04-17T05:46:31.590386+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.590386+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:21.620204+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'322","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.544504+0000","last_change":"2024-04-17T05:46:16.544710+0000","last_active":"2024-04-17T05:46:16.544504+0000","last_peered":"2024-04-17T05:46:16.544504+0000","last_clean":"2024-04-17T05:46:16.544504+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:46:16.544504+0000","last_undegraded":"2024-04-17T05:46:16.544504+0000","last_fullsized":"2024-04-17T05:46:16.544504+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.544168+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:59.343831+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8704691,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":453,"num_read_kb":14105,"num_write":1178,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'256","reported_seq":900,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.357473+0000","last_change":"2024-04-17T05:46:41.357550+0000","last_active":"2024-04-17T05:46:41.357473+0000","last_peered":"2024-04-17T05:46:41.357473+0000","last_clean":"2024-04-17T05:46:41.357473+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:46:41.357473+0000","last_undegraded":"2024-04-17T05:46:41.357473+0000","last_fullsized":"2024-04-17T05:46:41.357473+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'256","last_scrub_stamp":"2024-04-17T05:46:41.357278+0000","last_deep_scrub":"264'256","last_deep_scrub_stamp":"2024-04-17T05:46:41.357278+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.357278+0000","objects_scrubbed":1,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:04:53.727788+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":506,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'270","reported_seq":898,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.886876+0000","last_change":"2024-04-17T05:46:36.886994+0000","last_active":"2024-04-17T05:46:36.886876+0000","last_peered":"2024-04-17T05:46:36.886876+0000","last_clean":"2024-04-17T05:46:36.886876+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:46:36.886876+0000","last_undegraded":"2024-04-17T05:46:36.886876+0000","last_fullsized":"2024-04-17T05:46:36.886876+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'270","last_scrub_stamp":"2024-04-17T05:46:36.886656+0000","last_deep_scrub":"264'270","last_deep_scrub_stamp":"2024-04-17T05:46:36.886656+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.886656+0000","objects_scrubbed":8,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:11.179807+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1730173,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":405,"num_read_kb":10551,"num_write":968,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":733,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.307726+0000","last_change":"2024-04-17T05:46:33.307726+0000","last_active":"2024-04-17T05:46:33.307726+0000","last_peered":"2024-04-17T05:46:33.307726+0000","last_clean":"2024-04-17T05:46:33.307726+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:46:33.307726+0000","last_undegraded":"2024-04-17T05:46:33.307726+0000","last_fullsized":"2024-04-17T05:46:33.307726+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:33.307469+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:33.307469+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.307469+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:08.248068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'302","reported_seq":994,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.891762+0000","last_change":"2024-04-17T05:46:37.892036+0000","last_active":"2024-04-17T05:46:37.891762+0000","last_peered":"2024-04-17T05:46:37.891762+0000","last_clean":"2024-04-17T05:46:37.891762+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:46:37.891762+0000","last_undegraded":"2024-04-17T05:46:37.891762+0000","last_fullsized":"2024-04-17T05:46:37.891762+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'302","last_scrub_stamp":"2024-04-17T05:46:37.891555+0000","last_deep_scrub":"264'302","last_deep_scrub_stamp":"2024-04-17T05:46:37.891555+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.891555+0000","objects_scrubbed":9,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:00.393928+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044776,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":456,"num_read_kb":160096,"num_write":1126,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":909,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.420737+0000","last_change":"2024-04-17T05:46:30.420737+0000","last_active":"2024-04-17T05:46:30.420737+0000","last_peered":"2024-04-17T05:46:30.420737+0000","last_clean":"2024-04-17T05:46:30.420737+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:46:30.420737+0000","last_undegraded":"2024-04-17T05:46:30.420737+0000","last_fullsized":"2024-04-17T05:46:30.420737+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'289","last_scrub_stamp":"2024-04-17T05:46:30.420473+0000","last_deep_scrub":"264'289","last_deep_scrub_stamp":"2024-04-17T05:46:30.420473+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.420473+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:34.221983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'306","reported_seq":1021,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.545633+0000","last_change":"2024-04-17T05:46:46.545721+0000","last_active":"2024-04-17T05:46:46.545633+0000","last_peered":"2024-04-17T05:46:46.545633+0000","last_clean":"2024-04-17T05:46:46.545633+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:46:46.545633+0000","last_undegraded":"2024-04-17T05:46:46.545633+0000","last_fullsized":"2024-04-17T05:46:46.545633+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'306","last_scrub_stamp":"2024-04-17T05:46:46.545412+0000","last_deep_scrub":"264'306","last_deep_scrub_stamp":"2024-04-17T05:46:46.545412+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.545412+0000","objects_scrubbed":7,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:21:16.960386+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399248,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":586,"num_read_kb":64730,"num_write":1112,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4814,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.442968+0000","last_change":"2024-04-17T05:46:29.442968+0000","last_active":"2024-04-17T05:46:29.442968+0000","last_peered":"2024-04-17T05:46:29.442968+0000","last_clean":"2024-04-17T05:46:29.442968+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:46:29.442968+0000","last_undegraded":"2024-04-17T05:46:29.442968+0000","last_fullsized":"2024-04-17T05:46:29.442968+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4181","last_scrub_stamp":"2024-04-17T05:46:29.442703+0000","last_deep_scrub":"264'4181","last_deep_scrub_stamp":"2024-04-17T05:46:29.442703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.442703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:51.847043+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'318","reported_seq":1090,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275759+0000","last_change":"2024-04-17T04:36:25.374400+0000","last_active":"2024-04-17T05:46:08.275759+0000","last_peered":"2024-04-17T05:46:08.275759+0000","last_clean":"2024-04-17T05:46:08.275759+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:46:08.275759+0000","last_undegraded":"2024-04-17T05:46:08.275759+0000","last_fullsized":"2024-04-17T05:46:08.275759+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170354,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":810,"num_read_kb":57781,"num_write":1265,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":825,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.625599+0000","last_change":"2024-04-17T05:46:35.625793+0000","last_active":"2024-04-17T05:46:35.625599+0000","last_peered":"2024-04-17T05:46:35.625599+0000","last_clean":"2024-04-17T05:46:35.625599+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:46:35.625599+0000","last_undegraded":"2024-04-17T05:46:35.625599+0000","last_fullsized":"2024-04-17T05:46:35.625599+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:35.625369+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:35.625369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.625369+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:22:17.124001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'346","reported_seq":982,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934044+0000","last_change":"2024-04-17T04:36:25.355404+0000","last_active":"2024-04-17T05:46:06.934044+0000","last_peered":"2024-04-17T05:46:06.934044+0000","last_clean":"2024-04-17T05:46:06.934044+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:46:06.934044+0000","last_undegraded":"2024-04-17T05:46:06.934044+0000","last_fullsized":"2024-04-17T05:46:06.934044+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244513,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":463,"num_read_kb":836,"num_write":1174,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'155","reported_seq":681,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.797345+0000","last_change":"2024-04-17T05:46:13.797345+0000","last_active":"2024-04-17T05:46:13.797345+0000","last_peered":"2024-04-17T05:46:13.797345+0000","last_clean":"2024-04-17T05:46:13.797345+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:46:13.797345+0000","last_undegraded":"2024-04-17T05:46:13.797345+0000","last_fullsized":"2024-04-17T05:46:13.797345+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.796936+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:42.445551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":608,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.733949+0000","last_change":"2024-04-17T05:46:43.733949+0000","last_active":"2024-04-17T05:46:43.733949+0000","last_peered":"2024-04-17T05:46:43.733949+0000","last_clean":"2024-04-17T05:46:43.733949+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:46:43.733949+0000","last_undegraded":"2024-04-17T05:46:43.733949+0000","last_fullsized":"2024-04-17T05:46:43.733949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:43.733678+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:43.733678+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.733678+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:13:08.252101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'314","reported_seq":940,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155721+0000","last_change":"2024-04-17T04:36:25.368467+0000","last_active":"2024-04-17T05:46:10.155721+0000","last_peered":"2024-04-17T05:46:10.155721+0000","last_clean":"2024-04-17T05:46:10.155721+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:46:10.155721+0000","last_undegraded":"2024-04-17T05:46:10.155721+0000","last_fullsized":"2024-04-17T05:46:10.155721+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7313223,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":429,"num_read_kb":2527,"num_write":1169,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.338579+0000","last_change":"2024-04-17T05:46:32.338579+0000","last_active":"2024-04-17T05:46:32.338579+0000","last_peered":"2024-04-17T05:46:32.338579+0000","last_clean":"2024-04-17T05:46:32.338579+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:46:32.338579+0000","last_undegraded":"2024-04-17T05:46:32.338579+0000","last_fullsized":"2024-04-17T05:46:32.338579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.338322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.338322+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.338322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:19.679591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'373","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.677388+0000","last_change":"2024-04-17T05:46:36.677388+0000","last_active":"2024-04-17T05:46:36.677388+0000","last_peered":"2024-04-17T05:46:36.677388+0000","last_clean":"2024-04-17T05:46:36.677388+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:46:36.677388+0000","last_undegraded":"2024-04-17T05:46:36.677388+0000","last_fullsized":"2024-04-17T05:46:36.677388+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:36.677124+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:36.677124+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.677124+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:01.400465+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45299,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":596,"num_read_kb":1277,"num_write":472,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.304719+0000","last_change":"2024-04-17T05:46:29.304719+0000","last_active":"2024-04-17T05:46:29.304719+0000","last_peered":"2024-04-17T05:46:29.304719+0000","last_clean":"2024-04-17T05:46:29.304719+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:46:29.304719+0000","last_undegraded":"2024-04-17T05:46:29.304719+0000","last_fullsized":"2024-04-17T05:46:29.304719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.304289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.304289+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.304289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:21:58.667090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'172","reported_seq":734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.111916+0000","last_change":"2024-04-17T05:46:15.112010+0000","last_active":"2024-04-17T05:46:15.111916+0000","last_peered":"2024-04-17T05:46:15.111916+0000","last_clean":"2024-04-17T05:46:15.111916+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:46:15.111916+0000","last_undegraded":"2024-04-17T05:46:15.111916+0000","last_fullsized":"2024-04-17T05:46:15.111916+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'172","last_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_deep_scrub":"264'172","last_deep_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.111703+0000","objects_scrubbed":3,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:36:19.615360+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.266652+0000","last_change":"2024-04-17T05:46:26.266652+0000","last_active":"2024-04-17T05:46:26.266652+0000","last_peered":"2024-04-17T05:46:26.266652+0000","last_clean":"2024-04-17T05:46:26.266652+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:46:26.266652+0000","last_undegraded":"2024-04-17T05:46:26.266652+0000","last_fullsized":"2024-04-17T05:46:26.266652+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:26.266405+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:26.266405+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.266405+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:34:32.444534+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'339","reported_seq":1112,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.933964+0000","last_change":"2024-04-17T04:36:25.328153+0000","last_active":"2024-04-17T05:46:06.933964+0000","last_peered":"2024-04-17T05:46:06.933964+0000","last_clean":"2024-04-17T05:46:06.933964+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:46:06.933964+0000","last_undegraded":"2024-04-17T05:46:06.933964+0000","last_fullsized":"2024-04-17T05:46:06.933964+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1328,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":754,"num_read_kb":61435,"num_write":1363,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276147+0000","last_change":"2024-04-17T04:36:29.373876+0000","last_active":"2024-04-17T05:46:08.276147+0000","last_peered":"2024-04-17T05:46:08.276147+0000","last_clean":"2024-04-17T05:46:08.276147+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:46:08.276147+0000","last_undegraded":"2024-04-17T05:46:08.276147+0000","last_fullsized":"2024-04-17T05:46:08.276147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'103","reported_seq":656,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.736490+0000","last_change":"2024-04-17T05:46:22.736490+0000","last_active":"2024-04-17T05:46:22.736490+0000","last_peered":"2024-04-17T05:46:22.736490+0000","last_clean":"2024-04-17T05:46:22.736490+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:46:22.736490+0000","last_undegraded":"2024-04-17T05:46:22.736490+0000","last_fullsized":"2024-04-17T05:46:22.736490+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'103","last_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_deep_scrub":"264'103","last_deep_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.736227+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:14.605978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.786406+0000","last_change":"2024-04-17T05:46:15.786406+0000","last_active":"2024-04-17T05:46:15.786406+0000","last_peered":"2024-04-17T05:46:15.786406+0000","last_clean":"2024-04-17T05:46:15.786406+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:46:15.786406+0000","last_undegraded":"2024-04-17T05:46:15.786406+0000","last_fullsized":"2024-04-17T05:46:15.786406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.786152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:05:10.717154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'129","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.649727+0000","last_change":"2024-04-17T05:46:22.649727+0000","last_active":"2024-04-17T05:46:22.649727+0000","last_peered":"2024-04-17T05:46:22.649727+0000","last_clean":"2024-04-17T05:46:22.649727+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:46:22.649727+0000","last_undegraded":"2024-04-17T05:46:22.649727+0000","last_fullsized":"2024-04-17T05:46:22.649727+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'129","last_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_deep_scrub":"264'129","last_deep_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.649467+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.914685+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":669,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.727502+0000","last_change":"2024-04-17T05:46:40.727563+0000","last_active":"2024-04-17T05:46:40.727502+0000","last_peered":"2024-04-17T05:46:40.727502+0000","last_clean":"2024-04-17T05:46:40.727502+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:46:40.727502+0000","last_undegraded":"2024-04-17T05:46:40.727502+0000","last_fullsized":"2024-04-17T05:46:40.727502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'85","last_scrub_stamp":"2024-04-17T05:46:40.727294+0000","last_deep_scrub":"264'85","last_deep_scrub_stamp":"2024-04-17T05:46:40.727294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.727294+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:27.135914+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'333","reported_seq":1122,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.183587+0000","last_change":"2024-04-17T05:46:47.183732+0000","last_active":"2024-04-17T05:46:47.183587+0000","last_peered":"2024-04-17T05:46:47.183587+0000","last_clean":"2024-04-17T05:46:47.183587+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:46:47.183587+0000","last_undegraded":"2024-04-17T05:46:47.183587+0000","last_fullsized":"2024-04-17T05:46:47.183587+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'333","last_scrub_stamp":"2024-04-17T05:46:47.183363+0000","last_deep_scrub":"264'333","last_deep_scrub_stamp":"2024-04-17T05:46:47.183363+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.183363+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.756516+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":672,"num_read_kb":75397,"num_write":1317,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.776716+0000","last_change":"2024-04-17T05:46:48.776790+0000","last_active":"2024-04-17T05:46:48.776716+0000","last_peered":"2024-04-17T05:46:48.776716+0000","last_clean":"2024-04-17T05:46:48.776716+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:46:48.776716+0000","last_undegraded":"2024-04-17T05:46:48.776716+0000","last_fullsized":"2024-04-17T05:46:48.776716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:48.776517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:48.776517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.776517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.117730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'144","reported_seq":721,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.644625+0000","last_change":"2024-04-17T05:46:13.644625+0000","last_active":"2024-04-17T05:46:13.644625+0000","last_peered":"2024-04-17T05:46:13.644625+0000","last_clean":"2024-04-17T05:46:13.644625+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:46:13.644625+0000","last_undegraded":"2024-04-17T05:46:13.644625+0000","last_fullsized":"2024-04-17T05:46:13.644625+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'144","last_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_deep_scrub":"264'144","last_deep_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.644329+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:14.646427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'519","reported_seq":2426,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.368300+0000","last_change":"2024-04-17T05:46:31.368300+0000","last_active":"2024-04-17T05:46:31.368300+0000","last_peered":"2024-04-17T05:46:31.368300+0000","last_clean":"2024-04-17T05:46:31.368300+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:46:31.368300+0000","last_undegraded":"2024-04-17T05:46:31.368300+0000","last_fullsized":"2024-04-17T05:46:31.368300+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'519","last_scrub_stamp":"2024-04-17T05:46:31.367883+0000","last_deep_scrub":"264'519","last_deep_scrub_stamp":"2024-04-17T05:46:31.367883+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.367883+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.078252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'199","reported_seq":1022,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.828863+0000","last_change":"2024-04-17T05:46:14.828863+0000","last_active":"2024-04-17T05:46:14.828863+0000","last_peered":"2024-04-17T05:46:14.828863+0000","last_clean":"2024-04-17T05:46:14.828863+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:46:14.828863+0000","last_undegraded":"2024-04-17T05:46:14.828863+0000","last_fullsized":"2024-04-17T05:46:14.828863+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'199","last_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_deep_scrub":"264'199","last_deep_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.828583+0000","objects_scrubbed":4,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:40.862007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.741170+0000","last_change":"2024-04-17T05:46:42.741232+0000","last_active":"2024-04-17T05:46:42.741170+0000","last_peered":"2024-04-17T05:46:42.741170+0000","last_clean":"2024-04-17T05:46:42.741170+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:46:42.741170+0000","last_undegraded":"2024-04-17T05:46:42.741170+0000","last_fullsized":"2024-04-17T05:46:42.741170+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'234","last_scrub_stamp":"2024-04-17T05:46:42.740966+0000","last_deep_scrub":"264'234","last_deep_scrub_stamp":"2024-04-17T05:46:42.740966+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.740966+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:49:41.989989+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'293","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.858878+0000","last_change":"2024-04-17T05:46:35.858991+0000","last_active":"2024-04-17T05:46:35.858878+0000","last_peered":"2024-04-17T05:46:35.858878+0000","last_clean":"2024-04-17T05:46:35.858878+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:46:35.858878+0000","last_undegraded":"2024-04-17T05:46:35.858878+0000","last_fullsized":"2024-04-17T05:46:35.858878+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'293","last_scrub_stamp":"2024-04-17T05:46:35.858673+0000","last_deep_scrub":"264'293","last_deep_scrub_stamp":"2024-04-17T05:46:35.858673+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.858673+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:47.864378+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3722445,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":48921,"num_write":1143,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.344025+0000","last_change":"2024-04-17T05:46:36.344082+0000","last_active":"2024-04-17T05:46:36.344025+0000","last_peered":"2024-04-17T05:46:36.344025+0000","last_clean":"2024-04-17T05:46:36.344025+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:46:36.344025+0000","last_undegraded":"2024-04-17T05:46:36.344025+0000","last_fullsized":"2024-04-17T05:46:36.344025+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:36.343820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:36.343820+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.343820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:48.938245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'284","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.144810+0000","last_change":"2024-04-17T05:46:48.144886+0000","last_active":"2024-04-17T05:46:48.144810+0000","last_peered":"2024-04-17T05:46:48.144810+0000","last_clean":"2024-04-17T05:46:48.144810+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:46:48.144810+0000","last_undegraded":"2024-04-17T05:46:48.144810+0000","last_fullsized":"2024-04-17T05:46:48.144810+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'284","last_scrub_stamp":"2024-04-17T05:46:48.144597+0000","last_deep_scrub":"264'284","last_deep_scrub_stamp":"2024-04-17T05:46:48.144597+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.144597+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.030105+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51171,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1108,"num_write":376,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'534","reported_seq":2312,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.727792+0000","last_change":"2024-04-17T05:46:25.727792+0000","last_active":"2024-04-17T05:46:25.727792+0000","last_peered":"2024-04-17T05:46:25.727792+0000","last_clean":"2024-04-17T05:46:25.727792+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:46:25.727792+0000","last_undegraded":"2024-04-17T05:46:25.727792+0000","last_fullsized":"2024-04-17T05:46:25.727792+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'534","last_scrub_stamp":"2024-04-17T05:46:25.727540+0000","last_deep_scrub":"264'534","last_deep_scrub_stamp":"2024-04-17T05:46:25.727540+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.727540+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:47.622101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'122","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.284199+0000","last_change":"2024-04-17T05:46:30.284267+0000","last_active":"2024-04-17T05:46:30.284199+0000","last_peered":"2024-04-17T05:46:30.284199+0000","last_clean":"2024-04-17T05:46:30.284199+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:46:30.284199+0000","last_undegraded":"2024-04-17T05:46:30.284199+0000","last_fullsized":"2024-04-17T05:46:30.284199+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'122","last_scrub_stamp":"2024-04-17T05:46:30.283976+0000","last_deep_scrub":"264'122","last_deep_scrub_stamp":"2024-04-17T05:46:30.283976+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.283976+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:13.769030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'281","reported_seq":916,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155605+0000","last_change":"2024-04-17T04:36:25.370799+0000","last_active":"2024-04-17T05:46:10.155605+0000","last_peered":"2024-04-17T05:46:10.155605+0000","last_clean":"2024-04-17T05:46:10.155605+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:46:10.155605+0000","last_undegraded":"2024-04-17T05:46:10.155605+0000","last_fullsized":"2024-04-17T05:46:10.155605+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6974633,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":437,"num_read_kb":24084,"num_write":1121,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.373791+0000","last_change":"2024-04-17T05:46:38.373891+0000","last_active":"2024-04-17T05:46:38.373791+0000","last_peered":"2024-04-17T05:46:38.373791+0000","last_clean":"2024-04-17T05:46:38.373791+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:46:38.373791+0000","last_undegraded":"2024-04-17T05:46:38.373791+0000","last_fullsized":"2024-04-17T05:46:38.373791+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.373516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.373516+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.373516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:40.616887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'391","reported_seq":1038,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.206831+0000","last_change":"2024-04-17T05:46:22.206940+0000","last_active":"2024-04-17T05:46:22.206831+0000","last_peered":"2024-04-17T05:46:22.206831+0000","last_clean":"2024-04-17T05:46:22.206831+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:46:22.206831+0000","last_undegraded":"2024-04-17T05:46:22.206831+0000","last_fullsized":"2024-04-17T05:46:22.206831+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'391","last_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_deep_scrub":"264'391","last_deep_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.206623+0000","objects_scrubbed":10,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:39.352908+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44506,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":619,"num_read_kb":1253,"num_write":476,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.860898+0000","last_change":"2024-04-17T05:46:30.860950+0000","last_active":"2024-04-17T05:46:30.860898+0000","last_peered":"2024-04-17T05:46:30.860898+0000","last_clean":"2024-04-17T05:46:30.860898+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:46:30.860898+0000","last_undegraded":"2024-04-17T05:46:30.860898+0000","last_fullsized":"2024-04-17T05:46:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.860694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.860694+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.860694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:48.164981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'564","reported_seq":1397,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.160720+0000","last_change":"2024-04-17T05:46:18.160792+0000","last_active":"2024-04-17T05:46:18.160720+0000","last_peered":"2024-04-17T05:46:18.160720+0000","last_clean":"2024-04-17T05:46:18.160720+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:46:18.160720+0000","last_undegraded":"2024-04-17T05:46:18.160720+0000","last_fullsized":"2024-04-17T05:46:18.160720+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'564","last_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_deep_scrub":"264'564","last_deep_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.160537+0000","objects_scrubbed":4,"log_size":64,"log_dups_size":500,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:23.558644+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":874,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.594125+0000","last_change":"2024-04-17T05:46:41.594194+0000","last_active":"2024-04-17T05:46:41.594125+0000","last_peered":"2024-04-17T05:46:41.594125+0000","last_clean":"2024-04-17T05:46:41.594125+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:46:41.594125+0000","last_undegraded":"2024-04-17T05:46:41.594125+0000","last_fullsized":"2024-04-17T05:46:41.594125+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'259","last_scrub_stamp":"2024-04-17T05:46:41.593897+0000","last_deep_scrub":"264'259","last_deep_scrub_stamp":"2024-04-17T05:46:41.593897+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.593897+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:27.355644+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'395","reported_seq":1057,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155568+0000","last_change":"2024-04-17T04:36:25.371549+0000","last_active":"2024-04-17T05:46:10.155568+0000","last_peered":"2024-04-17T05:46:10.155568+0000","last_clean":"2024-04-17T05:46:10.155568+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:46:10.155568+0000","last_undegraded":"2024-04-17T05:46:10.155568+0000","last_fullsized":"2024-04-17T05:46:10.155568+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396022,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":571,"num_read_kb":14718,"num_write":1316,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.469546+0000","last_change":"2024-04-17T05:46:23.469644+0000","last_active":"2024-04-17T05:46:23.469546+0000","last_peered":"2024-04-17T05:46:23.469546+0000","last_clean":"2024-04-17T05:46:23.469546+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:46:23.469546+0000","last_undegraded":"2024-04-17T05:46:23.469546+0000","last_fullsized":"2024-04-17T05:46:23.469546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.469283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:59.151653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'269","reported_seq":860,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276587+0000","last_change":"2024-04-17T04:36:31.392065+0000","last_active":"2024-04-17T05:46:08.276587+0000","last_peered":"2024-04-17T05:46:08.276587+0000","last_clean":"2024-04-17T05:46:08.276587+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:46:08.276587+0000","last_undegraded":"2024-04-17T05:46:08.276587+0000","last_fullsized":"2024-04-17T05:46:08.276587+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42027,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":427,"num_read_kb":943,"num_write":336,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.833829+0000","last_change":"2024-04-17T05:46:29.833879+0000","last_active":"2024-04-17T05:46:29.833829+0000","last_peered":"2024-04-17T05:46:29.833829+0000","last_clean":"2024-04-17T05:46:29.833829+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:46:29.833829+0000","last_undegraded":"2024-04-17T05:46:29.833829+0000","last_fullsized":"2024-04-17T05:46:29.833829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.833621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.833621+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.833621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:23.380092+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2375","reported_seq":4186,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.963343+0000","last_change":"2024-04-17T05:46:27.963419+0000","last_active":"2024-04-17T05:46:27.963343+0000","last_peered":"2024-04-17T05:46:27.963343+0000","last_clean":"2024-04-17T05:46:27.963343+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:46:27.963343+0000","last_undegraded":"2024-04-17T05:46:27.963343+0000","last_fullsized":"2024-04-17T05:46:27.963343+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2375","last_scrub_stamp":"2024-04-17T05:46:27.963136+0000","last_deep_scrub":"264'2375","last_deep_scrub_stamp":"2024-04-17T05:46:27.963136+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.963136+0000","objects_scrubbed":4,"log_size":75,"log_dups_size":2300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:51.672615+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":708,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.910107+0000","last_change":"2024-04-17T05:46:30.910177+0000","last_active":"2024-04-17T05:46:30.910107+0000","last_peered":"2024-04-17T05:46:30.910107+0000","last_clean":"2024-04-17T05:46:30.910107+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:46:30.910107+0000","last_undegraded":"2024-04-17T05:46:30.910107+0000","last_fullsized":"2024-04-17T05:46:30.910107+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'154","last_scrub_stamp":"2024-04-17T05:46:30.909913+0000","last_deep_scrub":"264'154","last_deep_scrub_stamp":"2024-04-17T05:46:30.909913+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.909913+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:00.648226+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'356","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.297884+0000","last_change":"2024-04-17T05:46:43.298040+0000","last_active":"2024-04-17T05:46:43.297884+0000","last_peered":"2024-04-17T05:46:43.297884+0000","last_clean":"2024-04-17T05:46:43.297884+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:46:43.297884+0000","last_undegraded":"2024-04-17T05:46:43.297884+0000","last_fullsized":"2024-04-17T05:46:43.297884+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:43.297672+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:43.297672+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.297672+0000","objects_scrubbed":5,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:52:50.015406+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524369,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":604,"num_read_kb":65373,"num_write":1319,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.457184+0000","last_change":"2024-04-17T05:46:21.457242+0000","last_active":"2024-04-17T05:46:21.457184+0000","last_peered":"2024-04-17T05:46:21.457184+0000","last_clean":"2024-04-17T05:46:21.457184+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:46:21.457184+0000","last_undegraded":"2024-04-17T05:46:21.457184+0000","last_fullsized":"2024-04-17T05:46:21.457184+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.456977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:03.328209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'344","reported_seq":969,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.334603+0000","last_change":"2024-04-17T05:46:42.334679+0000","last_active":"2024-04-17T05:46:42.334603+0000","last_peered":"2024-04-17T05:46:42.334603+0000","last_clean":"2024-04-17T05:46:42.334603+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:46:42.334603+0000","last_undegraded":"2024-04-17T05:46:42.334603+0000","last_fullsized":"2024-04-17T05:46:42.334603+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'344","last_scrub_stamp":"2024-04-17T05:46:42.334476+0000","last_deep_scrub":"264'344","last_deep_scrub_stamp":"2024-04-17T05:46:42.334476+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.334476+0000","objects_scrubbed":9,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:22.249150+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48038,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":543,"num_read_kb":1167,"num_write":428,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'539","reported_seq":4624,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.621904+0000","last_change":"2024-04-17T05:46:33.621973+0000","last_active":"2024-04-17T05:46:33.621904+0000","last_peered":"2024-04-17T05:46:33.621904+0000","last_clean":"2024-04-17T05:46:33.621904+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:46:33.621904+0000","last_undegraded":"2024-04-17T05:46:33.621904+0000","last_fullsized":"2024-04-17T05:46:33.621904+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:33.621706+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:33.621706+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.621706+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:44.561807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":630,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.594579+0000","last_change":"2024-04-17T05:46:21.594642+0000","last_active":"2024-04-17T05:46:21.594579+0000","last_peered":"2024-04-17T05:46:21.594579+0000","last_clean":"2024-04-17T05:46:21.594579+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:46:21.594579+0000","last_undegraded":"2024-04-17T05:46:21.594579+0000","last_fullsized":"2024-04-17T05:46:21.594579+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.594379+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:06.802802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'96","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.599050+0000","last_change":"2024-04-17T05:46:46.599114+0000","last_active":"2024-04-17T05:46:46.599050+0000","last_peered":"2024-04-17T05:46:46.599050+0000","last_clean":"2024-04-17T05:46:46.599050+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:46:46.599050+0000","last_undegraded":"2024-04-17T05:46:46.599050+0000","last_fullsized":"2024-04-17T05:46:46.599050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'96","last_scrub_stamp":"2024-04-17T05:46:46.598859+0000","last_deep_scrub":"264'96","last_deep_scrub_stamp":"2024-04-17T05:46:46.598859+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.598859+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:35.086138+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'339","reported_seq":1048,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.599664+0000","last_change":"2024-04-17T05:46:14.599664+0000","last_active":"2024-04-17T05:46:14.599664+0000","last_peered":"2024-04-17T05:46:14.599664+0000","last_clean":"2024-04-17T05:46:14.599664+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:46:14.599664+0000","last_undegraded":"2024-04-17T05:46:14.599664+0000","last_fullsized":"2024-04-17T05:46:14.599664+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'339","last_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_deep_scrub":"264'339","last_deep_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.599211+0000","objects_scrubbed":6,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:59.675254+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":662,"num_read_kb":24375,"num_write":1169,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.517281+0000","last_change":"2024-04-17T05:46:17.517338+0000","last_active":"2024-04-17T05:46:17.517281+0000","last_peered":"2024-04-17T05:46:17.517281+0000","last_clean":"2024-04-17T05:46:17.517281+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:46:17.517281+0000","last_undegraded":"2024-04-17T05:46:17.517281+0000","last_fullsized":"2024-04-17T05:46:17.517281+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.517077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:41.667573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'472","reported_seq":1130,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.825598+0000","last_change":"2024-04-17T04:36:31.409877+0000","last_active":"2024-04-17T05:46:02.825598+0000","last_peered":"2024-04-17T05:46:02.825598+0000","last_clean":"2024-04-17T05:46:02.825598+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:46:02.825598+0000","last_undegraded":"2024-04-17T05:46:02.825598+0000","last_fullsized":"2024-04-17T05:46:02.825598+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87769,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":831,"num_read_kb":1997,"num_write":666,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.734179+0000","last_change":"2024-04-17T05:46:08.734235+0000","last_active":"2024-04-17T05:46:08.734179+0000","last_peered":"2024-04-17T05:46:08.734179+0000","last_clean":"2024-04-17T05:46:08.734179+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:46:08.734179+0000","last_undegraded":"2024-04-17T05:46:08.734179+0000","last_fullsized":"2024-04-17T05:46:08.734179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.733969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:20.733334+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'150","reported_seq":700,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.732921+0000","last_change":"2024-04-17T05:46:14.733009+0000","last_active":"2024-04-17T05:46:14.732921+0000","last_peered":"2024-04-17T05:46:14.732921+0000","last_clean":"2024-04-17T05:46:14.732921+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:46:14.732921+0000","last_undegraded":"2024-04-17T05:46:14.732921+0000","last_fullsized":"2024-04-17T05:46:14.732921+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'150","last_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_deep_scrub":"264'150","last_deep_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.732679+0000","objects_scrubbed":2,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:10.487087+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":780,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.706858+0000","last_change":"2024-04-17T05:46:27.706921+0000","last_active":"2024-04-17T05:46:27.706858+0000","last_peered":"2024-04-17T05:46:27.706858+0000","last_clean":"2024-04-17T05:46:27.706858+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:46:27.706858+0000","last_undegraded":"2024-04-17T05:46:27.706858+0000","last_fullsized":"2024-04-17T05:46:27.706858+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'157","last_scrub_stamp":"2024-04-17T05:46:27.706654+0000","last_deep_scrub":"264'157","last_deep_scrub_stamp":"2024-04-17T05:46:27.706654+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.706654+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:06.380118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'350","reported_seq":1020,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275408+0000","last_change":"2024-04-17T04:36:25.379472+0000","last_active":"2024-04-17T05:46:08.275408+0000","last_peered":"2024-04-17T05:46:08.275408+0000","last_clean":"2024-04-17T05:46:08.275408+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:46:08.275408+0000","last_undegraded":"2024-04-17T05:46:08.275408+0000","last_fullsized":"2024-04-17T05:46:08.275408+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1985843,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":564,"num_read_kb":9569,"num_write":1227,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.531942+0000","last_change":"2024-04-17T05:46:19.531995+0000","last_active":"2024-04-17T05:46:19.531942+0000","last_peered":"2024-04-17T05:46:19.531942+0000","last_clean":"2024-04-17T05:46:19.531942+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:46:19.531942+0000","last_undegraded":"2024-04-17T05:46:19.531942+0000","last_fullsized":"2024-04-17T05:46:19.531942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.531732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:32.490714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'319","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.744134+0000","last_change":"2024-04-17T05:46:23.744249+0000","last_active":"2024-04-17T05:46:23.744134+0000","last_peered":"2024-04-17T05:46:23.744134+0000","last_clean":"2024-04-17T05:46:23.744134+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:46:23.744134+0000","last_undegraded":"2024-04-17T05:46:23.744134+0000","last_fullsized":"2024-04-17T05:46:23.744134+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.743937+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:03.777566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":535,"num_read_kb":1188,"num_write":472,"num_write_kb":375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15128,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.643838+0000","last_change":"2024-04-17T05:46:25.643838+0000","last_active":"2024-04-17T05:46:25.643838+0000","last_peered":"2024-04-17T05:46:25.643838+0000","last_clean":"2024-04-17T05:46:25.643838+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:46:25.643838+0000","last_undegraded":"2024-04-17T05:46:25.643838+0000","last_fullsized":"2024-04-17T05:46:25.643838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:25.643431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:25.643431+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.643431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:10.449552+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'133","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.789793+0000","last_change":"2024-04-17T05:46:19.789793+0000","last_active":"2024-04-17T05:46:19.789793+0000","last_peered":"2024-04-17T05:46:19.789793+0000","last_clean":"2024-04-17T05:46:19.789793+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:46:19.789793+0000","last_undegraded":"2024-04-17T05:46:19.789793+0000","last_fullsized":"2024-04-17T05:46:19.789793+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.789529+0000","objects_scrubbed":4,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:17.259142+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.626107+0000","last_change":"2024-04-17T05:46:39.626172+0000","last_active":"2024-04-17T05:46:39.626107+0000","last_peered":"2024-04-17T05:46:39.626107+0000","last_clean":"2024-04-17T05:46:39.626107+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:46:39.626107+0000","last_undegraded":"2024-04-17T05:46:39.626107+0000","last_fullsized":"2024-04-17T05:46:39.626107+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'149","last_scrub_stamp":"2024-04-17T05:46:39.625903+0000","last_deep_scrub":"264'149","last_deep_scrub_stamp":"2024-04-17T05:46:39.625903+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.625903+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:29.931185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'365","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.843059+0000","last_change":"2024-04-17T05:46:32.843361+0000","last_active":"2024-04-17T05:46:32.843059+0000","last_peered":"2024-04-17T05:46:32.843059+0000","last_clean":"2024-04-17T05:46:32.843059+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:46:32.843059+0000","last_undegraded":"2024-04-17T05:46:32.843059+0000","last_fullsized":"2024-04-17T05:46:32.843059+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'365","last_scrub_stamp":"2024-04-17T05:46:32.842802+0000","last_deep_scrub":"264'365","last_deep_scrub_stamp":"2024-04-17T05:46:32.842802+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.842802+0000","objects_scrubbed":7,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:42.238535+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524382,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":609,"num_read_kb":15072,"num_write":1258,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":517,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.751463+0000","last_change":"2024-04-17T05:46:19.751517+0000","last_active":"2024-04-17T05:46:19.751463+0000","last_peered":"2024-04-17T05:46:19.751463+0000","last_clean":"2024-04-17T05:46:19.751463+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:46:19.751463+0000","last_undegraded":"2024-04-17T05:46:19.751463+0000","last_fullsized":"2024-04-17T05:46:19.751463+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.751263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:08.609702+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'298","reported_seq":917,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.671356+0000","last_change":"2024-04-17T05:46:11.671356+0000","last_active":"2024-04-17T05:46:11.671356+0000","last_peered":"2024-04-17T05:46:11.671356+0000","last_clean":"2024-04-17T05:46:11.671356+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:46:11.671356+0000","last_undegraded":"2024-04-17T05:46:11.671356+0000","last_fullsized":"2024-04-17T05:46:11.671356+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.670938+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:15.760804+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47207,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":490,"num_read_kb":1044,"num_write":380,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'512","reported_seq":2453,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.695035+0000","last_change":"2024-04-17T05:46:09.695035+0000","last_active":"2024-04-17T05:46:09.695035+0000","last_peered":"2024-04-17T05:46:09.695035+0000","last_clean":"2024-04-17T05:46:09.695035+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:46:09.695035+0000","last_undegraded":"2024-04-17T05:46:09.695035+0000","last_fullsized":"2024-04-17T05:46:09.695035+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'512","last_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_deep_scrub":"264'512","last_deep_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.694925+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.559083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'102","reported_seq":617,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.732298+0000","last_change":"2024-04-17T05:46:13.732374+0000","last_active":"2024-04-17T05:46:13.732298+0000","last_peered":"2024-04-17T05:46:13.732298+0000","last_clean":"2024-04-17T05:46:13.732298+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:46:13.732298+0000","last_undegraded":"2024-04-17T05:46:13.732298+0000","last_fullsized":"2024-04-17T05:46:13.732298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.732113+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:34.325042+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.636438+0000","last_change":"2024-04-17T05:46:29.636597+0000","last_active":"2024-04-17T05:46:29.636438+0000","last_peered":"2024-04-17T05:46:29.636438+0000","last_clean":"2024-04-17T05:46:29.636438+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:46:29.636438+0000","last_undegraded":"2024-04-17T05:46:29.636438+0000","last_fullsized":"2024-04-17T05:46:29.636438+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:29.636101+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:29.636101+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.636101+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:14.328596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'295","reported_seq":994,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.600236+0000","last_change":"2024-04-17T05:46:42.600435+0000","last_active":"2024-04-17T05:46:42.600236+0000","last_peered":"2024-04-17T05:46:42.600236+0000","last_clean":"2024-04-17T05:46:42.600236+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:46:42.600236+0000","last_undegraded":"2024-04-17T05:46:42.600236+0000","last_fullsized":"2024-04-17T05:46:42.600236+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'295","last_scrub_stamp":"2024-04-17T05:46:42.600013+0000","last_deep_scrub":"264'295","last_deep_scrub_stamp":"2024-04-17T05:46:42.600013+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.600013+0000","objects_scrubbed":8,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:40.877675+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243143,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":575,"num_read_kb":5671,"num_write":1192,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"264'4","reported_seq":1166,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.330528+0000","last_change":"2024-04-17T05:46:34.330595+0000","last_active":"2024-04-17T05:46:34.330528+0000","last_peered":"2024-04-17T05:46:34.330528+0000","last_clean":"2024-04-17T05:46:34.330528+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:46:34.330528+0000","last_undegraded":"2024-04-17T05:46:34.330528+0000","last_fullsized":"2024-04-17T05:46:34.330528+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4","last_scrub_stamp":"2024-04-17T05:46:34.330337+0000","last_deep_scrub":"264'4","last_deep_scrub_stamp":"2024-04-17T05:46:34.330337+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.330337+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:21.974891+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":652,"num_read_kb":652,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'102","reported_seq":647,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.863661+0000","last_change":"2024-04-17T05:46:33.863738+0000","last_active":"2024-04-17T05:46:33.863661+0000","last_peered":"2024-04-17T05:46:33.863661+0000","last_clean":"2024-04-17T05:46:33.863661+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:46:33.863661+0000","last_undegraded":"2024-04-17T05:46:33.863661+0000","last_fullsized":"2024-04-17T05:46:33.863661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:33.863448+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:33.863448+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.863448+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:04.150489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'539","reported_seq":2257,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.997502+0000","last_change":"2024-04-17T05:46:25.997576+0000","last_active":"2024-04-17T05:46:25.997502+0000","last_peered":"2024-04-17T05:46:25.997502+0000","last_clean":"2024-04-17T05:46:25.997502+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:46:25.997502+0000","last_undegraded":"2024-04-17T05:46:25.997502+0000","last_fullsized":"2024-04-17T05:46:25.997502+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:25.997283+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:25.997283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.997283+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:16.675494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'98","reported_seq":610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.796242+0000","last_change":"2024-04-17T05:46:11.796311+0000","last_active":"2024-04-17T05:46:11.796242+0000","last_peered":"2024-04-17T05:46:11.796242+0000","last_clean":"2024-04-17T05:46:11.796242+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:46:11.796242+0000","last_undegraded":"2024-04-17T05:46:11.796242+0000","last_fullsized":"2024-04-17T05:46:11.796242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'98","last_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_deep_scrub":"264'98","last_deep_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.796022+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:25.138191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":766,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.742441+0000","last_change":"2024-04-17T05:46:25.742441+0000","last_active":"2024-04-17T05:46:25.742441+0000","last_peered":"2024-04-17T05:46:25.742441+0000","last_clean":"2024-04-17T05:46:25.742441+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:46:25.742441+0000","last_undegraded":"2024-04-17T05:46:25.742441+0000","last_fullsized":"2024-04-17T05:46:25.742441+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:25.742055+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:25.742055+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.742055+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:19.477880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'296","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.879310+0000","last_change":"2024-04-17T04:36:25.353720+0000","last_active":"2024-04-17T05:46:03.879310+0000","last_peered":"2024-04-17T05:46:03.879310+0000","last_clean":"2024-04-17T05:46:03.879310+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:46:03.879310+0000","last_undegraded":"2024-04-17T05:46:03.879310+0000","last_fullsized":"2024-04-17T05:46:03.879310+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170337,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":410,"num_read_kb":36760,"num_write":993,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.744522+0000","last_change":"2024-04-17T04:36:29.389735+0000","last_active":"2024-04-17T05:46:08.744522+0000","last_peered":"2024-04-17T05:46:08.744522+0000","last_clean":"2024-04-17T05:46:08.744522+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:46:08.744522+0000","last_undegraded":"2024-04-17T05:46:08.744522+0000","last_fullsized":"2024-04-17T05:46:08.744522+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'359","reported_seq":992,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.232070+0000","last_change":"2024-04-17T05:46:25.232126+0000","last_active":"2024-04-17T05:46:25.232070+0000","last_peered":"2024-04-17T05:46:25.232070+0000","last_clean":"2024-04-17T05:46:25.232070+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:46:25.232070+0000","last_undegraded":"2024-04-17T05:46:25.232070+0000","last_fullsized":"2024-04-17T05:46:25.232070+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'359","last_scrub_stamp":"2024-04-17T05:46:25.231966+0000","last_deep_scrub":"264'359","last_deep_scrub_stamp":"2024-04-17T05:46:25.231966+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.231966+0000","objects_scrubbed":8,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:47:43.934129+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45806,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1167,"num_write":448,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.992307+0000","last_change":"2024-04-17T05:46:21.992363+0000","last_active":"2024-04-17T05:46:21.992307+0000","last_peered":"2024-04-17T05:46:21.992307+0000","last_clean":"2024-04-17T05:46:21.992307+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:46:21.992307+0000","last_undegraded":"2024-04-17T05:46:21.992307+0000","last_fullsized":"2024-04-17T05:46:21.992307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.992088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:40.290560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'161","reported_seq":676,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.735879+0000","last_change":"2024-04-17T05:46:15.736060+0000","last_active":"2024-04-17T05:46:15.735879+0000","last_peered":"2024-04-17T05:46:15.735879+0000","last_clean":"2024-04-17T05:46:15.735879+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:46:15.735879+0000","last_undegraded":"2024-04-17T05:46:15.735879+0000","last_fullsized":"2024-04-17T05:46:15.735879+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.735662+0000","objects_scrubbed":4,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.803225+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.006094+0000","last_change":"2024-04-17T05:46:25.006216+0000","last_active":"2024-04-17T05:46:25.006094+0000","last_peered":"2024-04-17T05:46:25.006094+0000","last_clean":"2024-04-17T05:46:25.006094+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:46:25.006094+0000","last_undegraded":"2024-04-17T05:46:25.006094+0000","last_fullsized":"2024-04-17T05:46:25.006094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.005814+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:49.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"264'119","reported_seq":692,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.064201+0000","last_change":"2024-04-17T05:46:50.064279+0000","last_active":"2024-04-17T05:46:50.064201+0000","last_peered":"2024-04-17T05:46:50.064201+0000","last_clean":"2024-04-17T05:46:50.064201+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:46:50.064201+0000","last_undegraded":"2024-04-17T05:46:50.064201+0000","last_fullsized":"2024-04-17T05:46:50.064201+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:50.063983+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:50.063983+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.063983+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:28.852792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":530,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.225182+0000","last_change":"2024-04-17T05:46:43.225234+0000","last_active":"2024-04-17T05:46:43.225182+0000","last_peered":"2024-04-17T05:46:43.225182+0000","last_clean":"2024-04-17T05:46:43.225182+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:46:43.225182+0000","last_undegraded":"2024-04-17T05:46:43.225182+0000","last_fullsized":"2024-04-17T05:46:43.225182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.224981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.224981+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.224981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:16.644730+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'327","reported_seq":979,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.766624+0000","last_change":"2024-04-17T04:36:25.364067+0000","last_active":"2024-04-17T05:46:03.766624+0000","last_peered":"2024-04-17T05:46:03.766624+0000","last_clean":"2024-04-17T05:46:03.766624+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:46:03.766624+0000","last_undegraded":"2024-04-17T05:46:03.766624+0000","last_fullsized":"2024-04-17T05:46:03.766624+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344976,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":2736,"num_write":1215,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.441265+0000","last_change":"2024-04-17T05:46:36.441319+0000","last_active":"2024-04-17T05:46:36.441265+0000","last_peered":"2024-04-17T05:46:36.441265+0000","last_clean":"2024-04-17T05:46:36.441265+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:46:36.441265+0000","last_undegraded":"2024-04-17T05:46:36.441265+0000","last_fullsized":"2024-04-17T05:46:36.441265+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:36.441066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:36.441066+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.441066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:15.277478+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'728","reported_seq":1458,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.227559+0000","last_change":"2024-04-17T05:46:23.227642+0000","last_active":"2024-04-17T05:46:23.227559+0000","last_peered":"2024-04-17T05:46:23.227559+0000","last_clean":"2024-04-17T05:46:23.227559+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:46:23.227559+0000","last_undegraded":"2024-04-17T05:46:23.227559+0000","last_fullsized":"2024-04-17T05:46:23.227559+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'728","last_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_deep_scrub":"264'728","last_deep_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.227369+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":700,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:26.750099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1411,"num_read_kb":3198,"num_write":1348,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":95136,"num_omap_keys":246,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.647366+0000","last_change":"2024-04-17T05:46:28.647420+0000","last_active":"2024-04-17T05:46:28.647366+0000","last_peered":"2024-04-17T05:46:28.647366+0000","last_clean":"2024-04-17T05:46:28.647366+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:46:28.647366+0000","last_undegraded":"2024-04-17T05:46:28.647366+0000","last_fullsized":"2024-04-17T05:46:28.647366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.647167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.647167+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.647167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:59.157329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'223","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.911300+0000","last_change":"2024-04-17T05:46:29.911389+0000","last_active":"2024-04-17T05:46:29.911300+0000","last_peered":"2024-04-17T05:46:29.911300+0000","last_clean":"2024-04-17T05:46:29.911300+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:46:29.911300+0000","last_undegraded":"2024-04-17T05:46:29.911300+0000","last_fullsized":"2024-04-17T05:46:29.911300+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'223","last_scrub_stamp":"2024-04-17T05:46:29.911091+0000","last_deep_scrub":"264'223","last_deep_scrub_stamp":"2024-04-17T05:46:29.911091+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.911091+0000","objects_scrubbed":3,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:25.435946+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":677,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.627128+0000","last_change":"2024-04-17T05:46:48.627245+0000","last_active":"2024-04-17T05:46:48.627128+0000","last_peered":"2024-04-17T05:46:48.627128+0000","last_clean":"2024-04-17T05:46:48.627128+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:46:48.627128+0000","last_undegraded":"2024-04-17T05:46:48.627128+0000","last_fullsized":"2024-04-17T05:46:48.627128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'84","last_scrub_stamp":"2024-04-17T05:46:48.626891+0000","last_deep_scrub":"264'84","last_deep_scrub_stamp":"2024-04-17T05:46:48.626891+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.626891+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:55.642870+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":526,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.155540+0000","last_change":"2024-04-17T04:36:17.215826+0000","last_active":"2024-04-17T05:46:10.155540+0000","last_peered":"2024-04-17T05:46:10.155540+0000","last_clean":"2024-04-17T05:46:10.155540+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:46:10.155540+0000","last_undegraded":"2024-04-17T05:46:10.155540+0000","last_fullsized":"2024-04-17T05:46:10.155540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'343","reported_seq":996,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.858668+0000","last_change":"2024-04-17T04:36:25.355561+0000","last_active":"2024-04-17T05:46:03.858668+0000","last_peered":"2024-04-17T05:46:03.858668+0000","last_clean":"2024-04-17T05:46:03.858668+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:46:03.858668+0000","last_undegraded":"2024-04-17T05:46:03.858668+0000","last_fullsized":"2024-04-17T05:46:03.858668+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2309902,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":570,"num_read_kb":3714,"num_write":1237,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.294868+0000","last_change":"2024-04-17T05:46:40.294921+0000","last_active":"2024-04-17T05:46:40.294868+0000","last_peered":"2024-04-17T05:46:40.294868+0000","last_clean":"2024-04-17T05:46:40.294868+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:46:40.294868+0000","last_undegraded":"2024-04-17T05:46:40.294868+0000","last_fullsized":"2024-04-17T05:46:40.294868+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:40.294666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:40.294666+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.294666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:08.051052+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1109","reported_seq":2836,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.770289+0000","last_change":"2024-04-17T05:46:26.770431+0000","last_active":"2024-04-17T05:46:26.770289+0000","last_peered":"2024-04-17T05:46:26.770289+0000","last_clean":"2024-04-17T05:46:26.770289+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:46:26.770289+0000","last_undegraded":"2024-04-17T05:46:26.770289+0000","last_fullsized":"2024-04-17T05:46:26.770289+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'1109","last_scrub_stamp":"2024-04-17T05:46:26.770091+0000","last_deep_scrub":"264'1109","last_deep_scrub_stamp":"2024-04-17T05:46:26.770091+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.770091+0000","objects_scrubbed":12,"log_size":109,"log_dups_size":1000,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:23.849377+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1819,"num_read_kb":3719,"num_write":1122,"num_write_kb":937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5370,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.327552+0000","last_change":"2024-04-17T05:46:30.327657+0000","last_active":"2024-04-17T05:46:30.327552+0000","last_peered":"2024-04-17T05:46:30.327552+0000","last_clean":"2024-04-17T05:46:30.327552+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:46:30.327552+0000","last_undegraded":"2024-04-17T05:46:30.327552+0000","last_fullsized":"2024-04-17T05:46:30.327552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.327323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.327323+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.327323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:13:37.291684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'114","reported_seq":633,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.750777+0000","last_change":"2024-04-17T05:46:17.750850+0000","last_active":"2024-04-17T05:46:17.750777+0000","last_peered":"2024-04-17T05:46:17.750777+0000","last_clean":"2024-04-17T05:46:17.750777+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:46:17.750777+0000","last_undegraded":"2024-04-17T05:46:17.750777+0000","last_fullsized":"2024-04-17T05:46:17.750777+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'114","last_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_deep_scrub":"264'114","last_deep_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.750563+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:00.237624+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1026,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.816940+0000","last_change":"2024-04-17T05:46:29.817060+0000","last_active":"2024-04-17T05:46:29.816940+0000","last_peered":"2024-04-17T05:46:29.816940+0000","last_clean":"2024-04-17T05:46:29.816940+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:46:29.816940+0000","last_undegraded":"2024-04-17T05:46:29.816940+0000","last_fullsized":"2024-04-17T05:46:29.816940+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'343","last_scrub_stamp":"2024-04-17T05:46:29.816699+0000","last_deep_scrub":"264'343","last_deep_scrub_stamp":"2024-04-17T05:46:29.816699+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.816699+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:49.710831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":539,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.432493+0000","last_change":"2024-04-17T05:46:24.432548+0000","last_active":"2024-04-17T05:46:24.432493+0000","last_peered":"2024-04-17T05:46:24.432493+0000","last_clean":"2024-04-17T05:46:24.432493+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:46:24.432493+0000","last_undegraded":"2024-04-17T05:46:24.432493+0000","last_fullsized":"2024-04-17T05:46:24.432493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.432302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:49.304795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'298","reported_seq":964,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.846776+0000","last_change":"2024-04-17T05:46:37.846776+0000","last_active":"2024-04-17T05:46:37.846776+0000","last_peered":"2024-04-17T05:46:37.846776+0000","last_clean":"2024-04-17T05:46:37.846776+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:46:37.846776+0000","last_undegraded":"2024-04-17T05:46:37.846776+0000","last_fullsized":"2024-04-17T05:46:37.846776+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:37.846228+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:37.846228+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.846228+0000","objects_scrubbed":9,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:41.146058+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5679739,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":479,"num_read_kb":21975,"num_write":1114,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"264'2","reported_seq":522,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.548976+0000","last_change":"2024-04-17T05:46:15.549042+0000","last_active":"2024-04-17T05:46:15.548976+0000","last_peered":"2024-04-17T05:46:15.548976+0000","last_clean":"2024-04-17T05:46:15.548976+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:46:15.548976+0000","last_undegraded":"2024-04-17T05:46:15.548976+0000","last_fullsized":"2024-04-17T05:46:15.548976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.548772+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:16:50.031612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'234","reported_seq":1619,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:54.886275+0000","last_change":"2024-04-17T04:36:31.427048+0000","last_active":"2024-04-17T05:45:54.886275+0000","last_peered":"2024-04-17T05:45:54.886275+0000","last_clean":"2024-04-17T05:45:54.886275+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:45:54.886275+0000","last_undegraded":"2024-04-17T05:45:54.886275+0000","last_fullsized":"2024-04-17T05:45:54.886275+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.628578+0000","last_change":"2024-04-17T05:46:32.628630+0000","last_active":"2024-04-17T05:46:32.628578+0000","last_peered":"2024-04-17T05:46:32.628578+0000","last_clean":"2024-04-17T05:46:32.628578+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:46:32.628578+0000","last_undegraded":"2024-04-17T05:46:32.628578+0000","last_fullsized":"2024-04-17T05:46:32.628578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.628381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.628381+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.628381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:52.474027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'265","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.770655+0000","last_change":"2024-04-17T05:46:24.770806+0000","last_active":"2024-04-17T05:46:24.770655+0000","last_peered":"2024-04-17T05:46:24.770655+0000","last_clean":"2024-04-17T05:46:24.770655+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:46:24.770655+0000","last_undegraded":"2024-04-17T05:46:24.770655+0000","last_fullsized":"2024-04-17T05:46:24.770655+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'265","last_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_deep_scrub":"264'265","last_deep_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.770438+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:52.656005+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":958,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.751155+0000","last_change":"2024-04-17T05:46:33.751218+0000","last_active":"2024-04-17T05:46:33.751155+0000","last_peered":"2024-04-17T05:46:33.751155+0000","last_clean":"2024-04-17T05:46:33.751155+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:46:33.751155+0000","last_undegraded":"2024-04-17T05:46:33.751155+0000","last_fullsized":"2024-04-17T05:46:33.751155+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'311","last_scrub_stamp":"2024-04-17T05:46:33.750961+0000","last_deep_scrub":"264'311","last_deep_scrub_stamp":"2024-04-17T05:46:33.750961+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.750961+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:15.653327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":530,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.264955+0000","last_change":"2024-04-17T05:46:42.265009+0000","last_active":"2024-04-17T05:46:42.264955+0000","last_peered":"2024-04-17T05:46:42.264955+0000","last_clean":"2024-04-17T05:46:42.264955+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:46:42.264955+0000","last_undegraded":"2024-04-17T05:46:42.264955+0000","last_fullsized":"2024-04-17T05:46:42.264955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.264768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.264768+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.264768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:20.993345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'341","reported_seq":1107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.616779+0000","last_change":"2024-04-17T05:46:43.616921+0000","last_active":"2024-04-17T05:46:43.616779+0000","last_peered":"2024-04-17T05:46:43.616779+0000","last_clean":"2024-04-17T05:46:43.616779+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:46:43.616779+0000","last_undegraded":"2024-04-17T05:46:43.616779+0000","last_fullsized":"2024-04-17T05:46:43.616779+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'341","last_scrub_stamp":"2024-04-17T05:46:43.616551+0000","last_deep_scrub":"264'341","last_deep_scrub_stamp":"2024-04-17T05:46:43.616551+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.616551+0000","objects_scrubbed":12,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:41.283471+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048803,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":626,"num_read_kb":36553,"num_write":1237,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.635722+0000","last_change":"2024-04-17T05:46:43.635780+0000","last_active":"2024-04-17T05:46:43.635722+0000","last_peered":"2024-04-17T05:46:43.635722+0000","last_clean":"2024-04-17T05:46:43.635722+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:46:43.635722+0000","last_undegraded":"2024-04-17T05:46:43.635722+0000","last_fullsized":"2024-04-17T05:46:43.635722+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.635515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.635515+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.635515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:56.243388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'146","reported_seq":715,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.761478+0000","last_change":"2024-04-17T05:46:37.761563+0000","last_active":"2024-04-17T05:46:37.761478+0000","last_peered":"2024-04-17T05:46:37.761478+0000","last_clean":"2024-04-17T05:46:37.761478+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:46:37.761478+0000","last_undegraded":"2024-04-17T05:46:37.761478+0000","last_fullsized":"2024-04-17T05:46:37.761478+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'146","last_scrub_stamp":"2024-04-17T05:46:37.761274+0000","last_deep_scrub":"264'146","last_deep_scrub_stamp":"2024-04-17T05:46:37.761274+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.761274+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:03.176459+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1416,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.968621+0000","last_change":"2024-04-17T05:46:19.968684+0000","last_active":"2024-04-17T05:46:19.968621+0000","last_peered":"2024-04-17T05:46:19.968621+0000","last_clean":"2024-04-17T05:46:19.968621+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:46:19.968621+0000","last_undegraded":"2024-04-17T05:46:19.968621+0000","last_fullsized":"2024-04-17T05:46:19.968621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.968394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:02.908417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'134","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.184018+0000","last_change":"2024-04-17T05:46:17.184135+0000","last_active":"2024-04-17T05:46:17.184018+0000","last_peered":"2024-04-17T05:46:17.184018+0000","last_clean":"2024-04-17T05:46:17.184018+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:46:17.184018+0000","last_undegraded":"2024-04-17T05:46:17.184018+0000","last_fullsized":"2024-04-17T05:46:17.184018+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.183823+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.008088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":760,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.771890+0000","last_change":"2024-04-17T05:46:27.771954+0000","last_active":"2024-04-17T05:46:27.771890+0000","last_peered":"2024-04-17T05:46:27.771890+0000","last_clean":"2024-04-17T05:46:27.771890+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:46:27.771890+0000","last_undegraded":"2024-04-17T05:46:27.771890+0000","last_fullsized":"2024-04-17T05:46:27.771890+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'145","last_scrub_stamp":"2024-04-17T05:46:27.771685+0000","last_deep_scrub":"264'145","last_deep_scrub_stamp":"2024-04-17T05:46:27.771685+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.771685+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:13.790569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'300","reported_seq":971,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.370422+0000","last_change":"2024-04-17T05:46:37.370505+0000","last_active":"2024-04-17T05:46:37.370422+0000","last_peered":"2024-04-17T05:46:37.370422+0000","last_clean":"2024-04-17T05:46:37.370422+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:46:37.370422+0000","last_undegraded":"2024-04-17T05:46:37.370422+0000","last_fullsized":"2024-04-17T05:46:37.370422+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'300","last_scrub_stamp":"2024-04-17T05:46:37.370221+0000","last_deep_scrub":"264'300","last_deep_scrub_stamp":"2024-04-17T05:46:37.370221+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.370221+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:46:56.727712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":475,"num_read_kb":1448,"num_write":1131,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":537,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.782488+0000","last_change":"2024-04-17T05:46:35.782544+0000","last_active":"2024-04-17T05:46:35.782488+0000","last_peered":"2024-04-17T05:46:35.782488+0000","last_clean":"2024-04-17T05:46:35.782488+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:46:35.782488+0000","last_undegraded":"2024-04-17T05:46:35.782488+0000","last_fullsized":"2024-04-17T05:46:35.782488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:35.782279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:35.782279+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.782279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:01:00.464190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.605392+0000","last_change":"2024-04-17T05:46:47.605511+0000","last_active":"2024-04-17T05:46:47.605392+0000","last_peered":"2024-04-17T05:46:47.605392+0000","last_clean":"2024-04-17T05:46:47.605392+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:46:47.605392+0000","last_undegraded":"2024-04-17T05:46:47.605392+0000","last_fullsized":"2024-04-17T05:46:47.605392+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:47.605143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:47.605143+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.605143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:36.604088+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'631","reported_seq":1337,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.682650+0000","last_change":"2024-04-17T05:46:34.682650+0000","last_active":"2024-04-17T05:46:34.682650+0000","last_peered":"2024-04-17T05:46:34.682650+0000","last_clean":"2024-04-17T05:46:34.682650+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:46:34.682650+0000","last_undegraded":"2024-04-17T05:46:34.682650+0000","last_fullsized":"2024-04-17T05:46:34.682650+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'631","last_scrub_stamp":"2024-04-17T05:46:34.682357+0000","last_deep_scrub":"264'631","last_deep_scrub_stamp":"2024-04-17T05:46:34.682357+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.682357+0000","objects_scrubbed":11,"log_size":31,"log_dups_size":600,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:41.613666+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86860,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1659,"num_read_kb":2834,"num_write":2174,"num_write_kb":1973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":470034,"num_omap_keys":1124,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.791699+0000","last_change":"2024-04-17T05:46:17.791757+0000","last_active":"2024-04-17T05:46:17.791699+0000","last_peered":"2024-04-17T05:46:17.791699+0000","last_clean":"2024-04-17T05:46:17.791699+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:46:17.791699+0000","last_undegraded":"2024-04-17T05:46:17.791699+0000","last_fullsized":"2024-04-17T05:46:17.791699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.791493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:50.092156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'86","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.643211+0000","last_change":"2024-04-17T05:46:18.643211+0000","last_active":"2024-04-17T05:46:18.643211+0000","last_peered":"2024-04-17T05:46:18.643211+0000","last_clean":"2024-04-17T05:46:18.643211+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:46:18.643211+0000","last_undegraded":"2024-04-17T05:46:18.643211+0000","last_fullsized":"2024-04-17T05:46:18.643211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'86","last_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_deep_scrub":"264'86","last_deep_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.642880+0000","objects_scrubbed":4,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:33.629614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.677721+0000","last_change":"2024-04-17T05:46:38.677786+0000","last_active":"2024-04-17T05:46:38.677721+0000","last_peered":"2024-04-17T05:46:38.677721+0000","last_clean":"2024-04-17T05:46:38.677721+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:46:38.677721+0000","last_undegraded":"2024-04-17T05:46:38.677721+0000","last_fullsized":"2024-04-17T05:46:38.677721+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-04-17T05:46:38.677518+0000","last_deep_scrub":"264'34","last_deep_scrub_stamp":"2024-04-17T05:46:38.677518+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.677518+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:22.863769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'269","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.836555+0000","last_change":"2024-04-17T05:46:38.836824+0000","last_active":"2024-04-17T05:46:38.836555+0000","last_peered":"2024-04-17T05:46:38.836555+0000","last_clean":"2024-04-17T05:46:38.836555+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:46:38.836555+0000","last_undegraded":"2024-04-17T05:46:38.836555+0000","last_fullsized":"2024-04-17T05:46:38.836555+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'269","last_scrub_stamp":"2024-04-17T05:46:38.836321+0000","last_deep_scrub":"264'269","last_deep_scrub_stamp":"2024-04-17T05:46:38.836321+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.836321+0000","objects_scrubbed":4,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:25.285046+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194313,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":396,"num_read_kb":35213,"num_write":840,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":524,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:45:15.179425+0000","last_change":"2024-04-17T04:36:17.874969+0000","last_active":"2024-04-17T05:45:15.179425+0000","last_peered":"2024-04-17T05:45:15.179425+0000","last_clean":"2024-04-17T05:45:15.179425+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:45:15.179425+0000","last_undegraded":"2024-04-17T05:45:15.179425+0000","last_fullsized":"2024-04-17T05:45:15.179425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:16.191560+0000","last_clean_scrub_stamp":"2024-04-17T04:36:16.191560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.958775+0000","last_change":"2024-04-17T05:46:20.958830+0000","last_active":"2024-04-17T05:46:20.958775+0000","last_peered":"2024-04-17T05:46:20.958775+0000","last_clean":"2024-04-17T05:46:20.958775+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:46:20.958775+0000","last_undegraded":"2024-04-17T05:46:20.958775+0000","last_fullsized":"2024-04-17T05:46:20.958775+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.958569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:07.756009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'207","reported_seq":809,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.813732+0000","last_change":"2024-04-17T05:46:28.813826+0000","last_active":"2024-04-17T05:46:28.813732+0000","last_peered":"2024-04-17T05:46:28.813732+0000","last_clean":"2024-04-17T05:46:28.813732+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:46:28.813732+0000","last_undegraded":"2024-04-17T05:46:28.813732+0000","last_fullsized":"2024-04-17T05:46:28.813732+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'207","last_scrub_stamp":"2024-04-17T05:46:28.813550+0000","last_deep_scrub":"264'207","last_deep_scrub_stamp":"2024-04-17T05:46:28.813550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.813550+0000","objects_scrubbed":8,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:10.411851+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'540","reported_seq":4368,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.162167+0000","last_change":"2024-04-17T05:46:21.162167+0000","last_active":"2024-04-17T05:46:21.162167+0000","last_peered":"2024-04-17T05:46:21.162167+0000","last_clean":"2024-04-17T05:46:21.162167+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:46:21.162167+0000","last_undegraded":"2024-04-17T05:46:21.162167+0000","last_fullsized":"2024-04-17T05:46:21.162167+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'540","last_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_deep_scrub":"264'540","last_deep_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.161902+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":500,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:14.043415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'161","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.780606+0000","last_change":"2024-04-17T05:46:23.780606+0000","last_active":"2024-04-17T05:46:23.780606+0000","last_peered":"2024-04-17T05:46:23.780606+0000","last_clean":"2024-04-17T05:46:23.780606+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:46:23.780606+0000","last_undegraded":"2024-04-17T05:46:23.780606+0000","last_fullsized":"2024-04-17T05:46:23.780606+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.780338+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:57.240957+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":809,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.800965+0000","last_change":"2024-04-17T05:46:46.801029+0000","last_active":"2024-04-17T05:46:46.800965+0000","last_peered":"2024-04-17T05:46:46.800965+0000","last_clean":"2024-04-17T05:46:46.800965+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:46:46.800965+0000","last_undegraded":"2024-04-17T05:46:46.800965+0000","last_fullsized":"2024-04-17T05:46:46.800965+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'168","last_scrub_stamp":"2024-04-17T05:46:46.800742+0000","last_deep_scrub":"264'168","last_deep_scrub_stamp":"2024-04-17T05:46:46.800742+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.800742+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:35.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'379","reported_seq":1037,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.847763+0000","last_change":"2024-04-17T04:36:25.347483+0000","last_active":"2024-04-17T05:46:03.847763+0000","last_peered":"2024-04-17T05:46:03.847763+0000","last_clean":"2024-04-17T05:46:03.847763+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:46:03.847763+0000","last_undegraded":"2024-04-17T05:46:03.847763+0000","last_fullsized":"2024-04-17T05:46:03.847763+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262239,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":591,"num_read_kb":5844,"num_write":1385,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":536,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.778199+0000","last_change":"2024-04-17T05:46:34.778260+0000","last_active":"2024-04-17T05:46:34.778199+0000","last_peered":"2024-04-17T05:46:34.778199+0000","last_clean":"2024-04-17T05:46:34.778199+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:46:34.778199+0000","last_undegraded":"2024-04-17T05:46:34.778199+0000","last_fullsized":"2024-04-17T05:46:34.778199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:34.778008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:34.778008+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.778008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:33.294550+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":518,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.587013+0000","last_change":"2024-04-17T05:46:38.587078+0000","last_active":"2024-04-17T05:46:38.587013+0000","last_peered":"2024-04-17T05:46:38.587013+0000","last_clean":"2024-04-17T05:46:38.587013+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:46:38.587013+0000","last_undegraded":"2024-04-17T05:46:38.587013+0000","last_fullsized":"2024-04-17T05:46:38.587013+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.586815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.586815+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.586815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:51.535527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'142","reported_seq":713,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.761857+0000","last_change":"2024-04-17T05:46:21.761940+0000","last_active":"2024-04-17T05:46:21.761857+0000","last_peered":"2024-04-17T05:46:21.761857+0000","last_clean":"2024-04-17T05:46:21.761857+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:46:21.761857+0000","last_undegraded":"2024-04-17T05:46:21.761857+0000","last_fullsized":"2024-04-17T05:46:21.761857+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'142","last_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_deep_scrub":"264'142","last_deep_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.761664+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:22.526188+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'533","reported_seq":2313,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.802896+0000","last_change":"2024-04-17T05:46:27.803031+0000","last_active":"2024-04-17T05:46:27.802896+0000","last_peered":"2024-04-17T05:46:27.802896+0000","last_clean":"2024-04-17T05:46:27.802896+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:46:27.802896+0000","last_undegraded":"2024-04-17T05:46:27.802896+0000","last_fullsized":"2024-04-17T05:46:27.802896+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'533","last_scrub_stamp":"2024-04-17T05:46:27.802603+0000","last_deep_scrub":"264'533","last_deep_scrub_stamp":"2024-04-17T05:46:27.802603+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.802603+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":500,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:54.996107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'224","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.621850+0000","last_change":"2024-04-17T05:46:17.621850+0000","last_active":"2024-04-17T05:46:17.621850+0000","last_peered":"2024-04-17T05:46:17.621850+0000","last_clean":"2024-04-17T05:46:17.621850+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:46:17.621850+0000","last_undegraded":"2024-04-17T05:46:17.621850+0000","last_fullsized":"2024-04-17T05:46:17.621850+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'224","last_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_deep_scrub":"264'224","last_deep_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.621550+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:22:57.559950+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4094","reported_seq":4734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.497931+0000","last_change":"2024-04-17T05:46:49.498067+0000","last_active":"2024-04-17T05:46:49.497931+0000","last_peered":"2024-04-17T05:46:49.497931+0000","last_clean":"2024-04-17T05:46:49.497931+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:46:49.497931+0000","last_undegraded":"2024-04-17T05:46:49.497931+0000","last_fullsized":"2024-04-17T05:46:49.497931+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4094","last_scrub_stamp":"2024-04-17T05:46:49.497699+0000","last_deep_scrub":"264'4094","last_deep_scrub_stamp":"2024-04-17T05:46:49.497699+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.497699+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:37.910397+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'356","reported_seq":974,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.886839+0000","last_change":"2024-04-17T04:36:25.354707+0000","last_active":"2024-04-17T05:46:03.886839+0000","last_peered":"2024-04-17T05:46:03.886839+0000","last_clean":"2024-04-17T05:46:03.886839+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:46:03.886839+0000","last_undegraded":"2024-04-17T05:46:03.886839+0000","last_fullsized":"2024-04-17T05:46:03.886839+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145969,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":466,"num_read_kb":7519,"num_write":1284,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.882687+0000","last_change":"2024-04-17T05:46:31.882747+0000","last_active":"2024-04-17T05:46:31.882687+0000","last_peered":"2024-04-17T05:46:31.882687+0000","last_clean":"2024-04-17T05:46:31.882687+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:46:31.882687+0000","last_undegraded":"2024-04-17T05:46:31.882687+0000","last_fullsized":"2024-04-17T05:46:31.882687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.882477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.882477+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.882477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:43.637695+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276854+0000","last_change":"2024-04-17T04:36:29.363086+0000","last_active":"2024-04-17T05:46:08.276854+0000","last_peered":"2024-04-17T05:46:08.276854+0000","last_clean":"2024-04-17T05:46:08.276854+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:46:08.276854+0000","last_undegraded":"2024-04-17T05:46:08.276854+0000","last_fullsized":"2024-04-17T05:46:08.276854+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'739","reported_seq":1492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.240166+0000","last_change":"2024-04-17T05:46:46.240287+0000","last_active":"2024-04-17T05:46:46.240166+0000","last_peered":"2024-04-17T05:46:46.240166+0000","last_clean":"2024-04-17T05:46:46.240166+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:46:46.240166+0000","last_undegraded":"2024-04-17T05:46:46.240166+0000","last_fullsized":"2024-04-17T05:46:46.240166+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'739","last_scrub_stamp":"2024-04-17T05:46:46.239975+0000","last_deep_scrub":"264'739","last_deep_scrub_stamp":"2024-04-17T05:46:46.239975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.239975+0000","objects_scrubbed":9,"log_size":39,"log_dups_size":700,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:48.081136+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138854,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1296,"num_read_kb":3065,"num_write":1038,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.854130+0000","last_change":"2024-04-17T05:46:31.854181+0000","last_active":"2024-04-17T05:46:31.854130+0000","last_peered":"2024-04-17T05:46:31.854130+0000","last_clean":"2024-04-17T05:46:31.854130+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:46:31.854130+0000","last_undegraded":"2024-04-17T05:46:31.854130+0000","last_fullsized":"2024-04-17T05:46:31.854130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.853942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.853942+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.853942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:11.976239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'105","reported_seq":604,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.069586+0000","last_change":"2024-04-17T05:46:13.069586+0000","last_active":"2024-04-17T05:46:13.069586+0000","last_peered":"2024-04-17T05:46:13.069586+0000","last_clean":"2024-04-17T05:46:13.069586+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:46:13.069586+0000","last_undegraded":"2024-04-17T05:46:13.069586+0000","last_fullsized":"2024-04-17T05:46:13.069586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.069188+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:06.656617+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'151","reported_seq":727,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.648354+0000","last_change":"2024-04-17T05:46:24.648424+0000","last_active":"2024-04-17T05:46:24.648354+0000","last_peered":"2024-04-17T05:46:24.648354+0000","last_clean":"2024-04-17T05:46:24.648354+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:46:24.648354+0000","last_undegraded":"2024-04-17T05:46:24.648354+0000","last_fullsized":"2024-04-17T05:46:24.648354+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'151","last_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_deep_scrub":"264'151","last_deep_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.648141+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:48.144202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'74","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.642292+0000","last_change":"2024-04-17T05:46:30.642444+0000","last_active":"2024-04-17T05:46:30.642292+0000","last_peered":"2024-04-17T05:46:30.642292+0000","last_clean":"2024-04-17T05:46:30.642292+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:46:30.642292+0000","last_undegraded":"2024-04-17T05:46:30.642292+0000","last_fullsized":"2024-04-17T05:46:30.642292+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'74","last_scrub_stamp":"2024-04-17T05:46:30.642053+0000","last_deep_scrub":"264'74","last_deep_scrub_stamp":"2024-04-17T05:46:30.642053+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.642053+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:04.769101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'281","reported_seq":929,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.540711+0000","last_change":"2024-04-17T05:46:45.540862+0000","last_active":"2024-04-17T05:46:45.540711+0000","last_peered":"2024-04-17T05:46:45.540711+0000","last_clean":"2024-04-17T05:46:45.540711+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:46:45.540711+0000","last_undegraded":"2024-04-17T05:46:45.540711+0000","last_fullsized":"2024-04-17T05:46:45.540711+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:45.540487+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:45.540487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.540487+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:58.652080+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017984,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":431,"num_read_kb":11105,"num_write":979,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.773169+0000","last_change":"2024-04-17T05:46:22.773169+0000","last_active":"2024-04-17T05:46:22.773169+0000","last_peered":"2024-04-17T05:46:22.773169+0000","last_clean":"2024-04-17T05:46:22.773169+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:46:22.773169+0000","last_undegraded":"2024-04-17T05:46:22.773169+0000","last_fullsized":"2024-04-17T05:46:22.773169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.772889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:13.164944+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'307","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.772343+0000","last_change":"2024-04-17T05:46:36.772420+0000","last_active":"2024-04-17T05:46:36.772343+0000","last_peered":"2024-04-17T05:46:36.772343+0000","last_clean":"2024-04-17T05:46:36.772343+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:46:36.772343+0000","last_undegraded":"2024-04-17T05:46:36.772343+0000","last_fullsized":"2024-04-17T05:46:36.772343+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'307","last_scrub_stamp":"2024-04-17T05:46:36.772164+0000","last_deep_scrub":"264'307","last_deep_scrub_stamp":"2024-04-17T05:46:36.772164+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.772164+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.701358+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44026,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":524,"num_read_kb":1245,"num_write":424,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.845796+0000","last_change":"2024-04-17T05:46:28.845796+0000","last_active":"2024-04-17T05:46:28.845796+0000","last_peered":"2024-04-17T05:46:28.845796+0000","last_clean":"2024-04-17T05:46:28.845796+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:46:28.845796+0000","last_undegraded":"2024-04-17T05:46:28.845796+0000","last_fullsized":"2024-04-17T05:46:28.845796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.845537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.845537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.845537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:25:28.366296+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'99","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.795125+0000","last_change":"2024-04-17T05:46:17.795125+0000","last_active":"2024-04-17T05:46:17.795125+0000","last_peered":"2024-04-17T05:46:17.795125+0000","last_clean":"2024-04-17T05:46:17.795125+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:46:17.795125+0000","last_undegraded":"2024-04-17T05:46:17.795125+0000","last_fullsized":"2024-04-17T05:46:17.795125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'99","last_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_deep_scrub":"264'99","last_deep_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.794856+0000","objects_scrubbed":1,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:18.080437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'127","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.804194+0000","last_change":"2024-04-17T05:46:23.804194+0000","last_active":"2024-04-17T05:46:23.804194+0000","last_peered":"2024-04-17T05:46:23.804194+0000","last_clean":"2024-04-17T05:46:23.804194+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:46:23.804194+0000","last_undegraded":"2024-04-17T05:46:23.804194+0000","last_fullsized":"2024-04-17T05:46:23.804194+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'127","last_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_deep_scrub":"264'127","last_deep_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.803934+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:22.971464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'373","reported_seq":1034,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.683941+0000","last_change":"2024-04-17T05:46:12.684055+0000","last_active":"2024-04-17T05:46:12.683941+0000","last_peered":"2024-04-17T05:46:12.683941+0000","last_clean":"2024-04-17T05:46:12.683941+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:46:12.683941+0000","last_undegraded":"2024-04-17T05:46:12.683941+0000","last_fullsized":"2024-04-17T05:46:12.683941+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.683727+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:58.133653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":625,"num_read_kb":15946,"num_write":1543,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276060+0000","last_change":"2024-04-17T04:36:29.371357+0000","last_active":"2024-04-17T05:46:08.276060+0000","last_peered":"2024-04-17T05:46:08.276060+0000","last_clean":"2024-04-17T05:46:08.276060+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:46:08.276060+0000","last_undegraded":"2024-04-17T05:46:08.276060+0000","last_fullsized":"2024-04-17T05:46:08.276060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:28.338586+0000","last_clean_scrub_stamp":"2024-04-17T04:36:28.338586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'56","reported_seq":569,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934738+0000","last_change":"2024-04-17T04:36:31.404202+0000","last_active":"2024-04-17T05:46:06.934738+0000","last_peered":"2024-04-17T05:46:06.934738+0000","last_clean":"2024-04-17T05:46:06.934738+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:46:06.934738+0000","last_undegraded":"2024-04-17T05:46:06.934738+0000","last_fullsized":"2024-04-17T05:46:06.934738+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.325328+0000","last_change":"2024-04-17T05:46:33.325328+0000","last_active":"2024-04-17T05:46:33.325328+0000","last_peered":"2024-04-17T05:46:33.325328+0000","last_clean":"2024-04-17T05:46:33.325328+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:46:33.325328+0000","last_undegraded":"2024-04-17T05:46:33.325328+0000","last_fullsized":"2024-04-17T05:46:33.325328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:33.325058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:33.325058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.325058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:24.544642+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'219","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.647715+0000","last_change":"2024-04-17T05:46:20.647715+0000","last_active":"2024-04-17T05:46:20.647715+0000","last_peered":"2024-04-17T05:46:20.647715+0000","last_clean":"2024-04-17T05:46:20.647715+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:46:20.647715+0000","last_undegraded":"2024-04-17T05:46:20.647715+0000","last_fullsized":"2024-04-17T05:46:20.647715+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'219","last_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_deep_scrub":"264'219","last_deep_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.647446+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:57.324481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.733738+0000","last_change":"2024-04-17T05:46:40.733801+0000","last_active":"2024-04-17T05:46:40.733738+0000","last_peered":"2024-04-17T05:46:40.733738+0000","last_clean":"2024-04-17T05:46:40.733738+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:46:40.733738+0000","last_undegraded":"2024-04-17T05:46:40.733738+0000","last_fullsized":"2024-04-17T05:46:40.733738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'196","last_scrub_stamp":"2024-04-17T05:46:40.733533+0000","last_deep_scrub":"264'196","last_deep_scrub_stamp":"2024-04-17T05:46:40.733533+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.733533+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:04.953287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'356","reported_seq":1045,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.248025+0000","last_change":"2024-04-17T05:46:44.248263+0000","last_active":"2024-04-17T05:46:44.248025+0000","last_peered":"2024-04-17T05:46:44.248025+0000","last_clean":"2024-04-17T05:46:44.248025+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:46:44.248025+0000","last_undegraded":"2024-04-17T05:46:44.248025+0000","last_fullsized":"2024-04-17T05:46:44.248025+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:44.247646+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:44.247646+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.247646+0000","objects_scrubbed":6,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:11.550314+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":588,"num_read_kb":11343,"num_write":1303,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.358946+0000","last_change":"2024-04-17T05:46:34.359052+0000","last_active":"2024-04-17T05:46:34.358946+0000","last_peered":"2024-04-17T05:46:34.358946+0000","last_clean":"2024-04-17T05:46:34.358946+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:46:34.358946+0000","last_undegraded":"2024-04-17T05:46:34.358946+0000","last_fullsized":"2024-04-17T05:46:34.358946+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:34.358700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:34.358700+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.358700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:23.819994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'345","reported_seq":960,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.793714+0000","last_change":"2024-04-17T05:46:35.793792+0000","last_active":"2024-04-17T05:46:35.793714+0000","last_peered":"2024-04-17T05:46:35.793714+0000","last_clean":"2024-04-17T05:46:35.793714+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:46:35.793714+0000","last_undegraded":"2024-04-17T05:46:35.793714+0000","last_fullsized":"2024-04-17T05:46:35.793714+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'345","last_scrub_stamp":"2024-04-17T05:46:35.793519+0000","last_deep_scrub":"264'345","last_deep_scrub_stamp":"2024-04-17T05:46:35.793519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.793519+0000","objects_scrubbed":5,"log_size":45,"log_dups_size":300,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:19.755889+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48000,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":552,"num_read_kb":1237,"num_write":440,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":500,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.983452+0000","last_change":"2024-04-17T05:46:17.983452+0000","last_active":"2024-04-17T05:46:17.983452+0000","last_peered":"2024-04-17T05:46:17.983452+0000","last_clean":"2024-04-17T05:46:17.983452+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:46:17.983452+0000","last_undegraded":"2024-04-17T05:46:17.983452+0000","last_fullsized":"2024-04-17T05:46:17.983452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.983070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:29.782925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'113","reported_seq":610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.111907+0000","last_change":"2024-04-17T05:46:14.111907+0000","last_active":"2024-04-17T05:46:14.111907+0000","last_peered":"2024-04-17T05:46:14.111907+0000","last_clean":"2024-04-17T05:46:14.111907+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:46:14.111907+0000","last_undegraded":"2024-04-17T05:46:14.111907+0000","last_fullsized":"2024-04-17T05:46:14.111907+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'113","last_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_deep_scrub":"264'113","last_deep_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.111624+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:35.374698+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":769,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.750706+0000","last_change":"2024-04-17T05:46:49.750765+0000","last_active":"2024-04-17T05:46:49.750706+0000","last_peered":"2024-04-17T05:46:49.750706+0000","last_clean":"2024-04-17T05:46:49.750706+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:46:49.750706+0000","last_undegraded":"2024-04-17T05:46:49.750706+0000","last_fullsized":"2024-04-17T05:46:49.750706+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'132","last_scrub_stamp":"2024-04-17T05:46:49.750514+0000","last_deep_scrub":"264'132","last_deep_scrub_stamp":"2024-04-17T05:46:49.750514+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.750514+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:01:14.989520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'310","reported_seq":960,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.933969+0000","last_change":"2024-04-17T04:36:25.344700+0000","last_active":"2024-04-17T05:46:06.933969+0000","last_peered":"2024-04-17T05:46:06.933969+0000","last_clean":"2024-04-17T05:46:06.933969+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:46:06.933969+0000","last_undegraded":"2024-04-17T05:46:06.933969+0000","last_fullsized":"2024-04-17T05:46:06.933969+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12846794,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":445,"num_read_kb":84514,"num_write":990,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.942372+0000","last_change":"2024-04-17T05:46:16.942372+0000","last_active":"2024-04-17T05:46:16.942372+0000","last_peered":"2024-04-17T05:46:16.942372+0000","last_clean":"2024-04-17T05:46:16.942372+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:46:16.942372+0000","last_undegraded":"2024-04-17T05:46:16.942372+0000","last_fullsized":"2024-04-17T05:46:16.942372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.942121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:13.925928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'262","reported_seq":852,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.356250+0000","last_change":"2024-04-17T05:46:40.356321+0000","last_active":"2024-04-17T05:46:40.356250+0000","last_peered":"2024-04-17T05:46:40.356250+0000","last_clean":"2024-04-17T05:46:40.356250+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:46:40.356250+0000","last_undegraded":"2024-04-17T05:46:40.356250+0000","last_fullsized":"2024-04-17T05:46:40.356250+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'262","last_scrub_stamp":"2024-04-17T05:46:40.356034+0000","last_deep_scrub":"264'262","last_deep_scrub_stamp":"2024-04-17T05:46:40.356034+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.356034+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:11.830162+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47325,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":432,"num_read_kb":996,"num_write":342,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.631115+0000","last_change":"2024-04-17T05:46:30.631115+0000","last_active":"2024-04-17T05:46:30.631115+0000","last_peered":"2024-04-17T05:46:30.631115+0000","last_clean":"2024-04-17T05:46:30.631115+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:46:30.631115+0000","last_undegraded":"2024-04-17T05:46:30.631115+0000","last_fullsized":"2024-04-17T05:46:30.631115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.630868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.630868+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.630868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:07:08.526024+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'290","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.020201+0000","last_change":"2024-04-17T05:46:23.020277+0000","last_active":"2024-04-17T05:46:23.020201+0000","last_peered":"2024-04-17T05:46:23.020201+0000","last_clean":"2024-04-17T05:46:23.020201+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:46:23.020201+0000","last_undegraded":"2024-04-17T05:46:23.020201+0000","last_fullsized":"2024-04-17T05:46:23.020201+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'290","last_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_deep_scrub":"264'290","last_deep_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.019975+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:21.304396+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'313","reported_seq":1000,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276305+0000","last_change":"2024-04-17T04:36:25.326760+0000","last_active":"2024-04-17T05:46:08.276305+0000","last_peered":"2024-04-17T05:46:08.276305+0000","last_clean":"2024-04-17T05:46:08.276305+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:46:08.276305+0000","last_undegraded":"2024-04-17T05:46:08.276305+0000","last_fullsized":"2024-04-17T05:46:08.276305+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752753,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":542,"num_read_kb":33053,"num_write":1174,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.615906+0000","last_change":"2024-04-17T05:46:45.615960+0000","last_active":"2024-04-17T05:46:45.615906+0000","last_peered":"2024-04-17T05:46:45.615906+0000","last_clean":"2024-04-17T05:46:45.615906+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:46:45.615906+0000","last_undegraded":"2024-04-17T05:46:45.615906+0000","last_fullsized":"2024-04-17T05:46:45.615906+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:45.615722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:45.615722+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.615722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:21.102882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'480","reported_seq":1141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.798222+0000","last_change":"2024-04-17T05:46:24.798222+0000","last_active":"2024-04-17T05:46:24.798222+0000","last_peered":"2024-04-17T05:46:24.798222+0000","last_clean":"2024-04-17T05:46:24.798222+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:46:24.798222+0000","last_undegraded":"2024-04-17T05:46:24.798222+0000","last_fullsized":"2024-04-17T05:46:24.798222+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'480","last_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_deep_scrub":"264'480","last_deep_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.797947+0000","objects_scrubbed":7,"log_size":80,"log_dups_size":400,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:12.511479+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87094,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":846,"num_read_kb":2027,"num_write":720,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22936,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.147623+0000","last_change":"2024-04-17T05:46:19.147623+0000","last_active":"2024-04-17T05:46:19.147623+0000","last_peered":"2024-04-17T05:46:19.147623+0000","last_clean":"2024-04-17T05:46:19.147623+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:46:19.147623+0000","last_undegraded":"2024-04-17T05:46:19.147623+0000","last_fullsized":"2024-04-17T05:46:19.147623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.147370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:18.842204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'174","reported_seq":691,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.787327+0000","last_change":"2024-04-17T05:46:21.787327+0000","last_active":"2024-04-17T05:46:21.787327+0000","last_peered":"2024-04-17T05:46:21.787327+0000","last_clean":"2024-04-17T05:46:21.787327+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:46:21.787327+0000","last_undegraded":"2024-04-17T05:46:21.787327+0000","last_fullsized":"2024-04-17T05:46:21.787327+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.787059+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:11.126921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.695888+0000","last_change":"2024-04-17T05:46:41.696009+0000","last_active":"2024-04-17T05:46:41.695888+0000","last_peered":"2024-04-17T05:46:41.695888+0000","last_clean":"2024-04-17T05:46:41.695888+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:46:41.695888+0000","last_undegraded":"2024-04-17T05:46:41.695888+0000","last_fullsized":"2024-04-17T05:46:41.695888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'104","last_scrub_stamp":"2024-04-17T05:46:41.695676+0000","last_deep_scrub":"264'104","last_deep_scrub_stamp":"2024-04-17T05:46:41.695676+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.695676+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:56:52.077069+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'315","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.805432+0000","last_change":"2024-04-17T05:46:40.805432+0000","last_active":"2024-04-17T05:46:40.805432+0000","last_peered":"2024-04-17T05:46:40.805432+0000","last_clean":"2024-04-17T05:46:40.805432+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:46:40.805432+0000","last_undegraded":"2024-04-17T05:46:40.805432+0000","last_fullsized":"2024-04-17T05:46:40.805432+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'315","last_scrub_stamp":"2024-04-17T05:46:40.805067+0000","last_deep_scrub":"264'315","last_deep_scrub_stamp":"2024-04-17T05:46:40.805067+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.805067+0000","objects_scrubbed":9,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:14:53.427777+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122783,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":510,"num_read_kb":112390,"num_write":1223,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.012291+0000","last_change":"2024-04-17T05:46:19.012353+0000","last_active":"2024-04-17T05:46:19.012291+0000","last_peered":"2024-04-17T05:46:19.012291+0000","last_clean":"2024-04-17T05:46:19.012291+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:46:19.012291+0000","last_undegraded":"2024-04-17T05:46:19.012291+0000","last_fullsized":"2024-04-17T05:46:19.012291+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.012071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:43.685517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'323","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.261454+0000","last_change":"2024-04-17T05:46:44.261542+0000","last_active":"2024-04-17T05:46:44.261454+0000","last_peered":"2024-04-17T05:46:44.261454+0000","last_clean":"2024-04-17T05:46:44.261454+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:46:44.261454+0000","last_undegraded":"2024-04-17T05:46:44.261454+0000","last_fullsized":"2024-04-17T05:46:44.261454+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'323","last_scrub_stamp":"2024-04-17T05:46:44.261253+0000","last_deep_scrub":"264'323","last_deep_scrub_stamp":"2024-04-17T05:46:44.261253+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.261253+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:24.010153+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40265,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":1168,"num_write":442,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10932,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.856669+0000","last_change":"2024-04-17T05:46:42.856727+0000","last_active":"2024-04-17T05:46:42.856669+0000","last_peered":"2024-04-17T05:46:42.856669+0000","last_clean":"2024-04-17T05:46:42.856669+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:46:42.856669+0000","last_undegraded":"2024-04-17T05:46:42.856669+0000","last_fullsized":"2024-04-17T05:46:42.856669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.856474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.856474+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.856474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:20:08.791141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'180","reported_seq":726,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.904163+0000","last_change":"2024-04-17T05:46:39.904297+0000","last_active":"2024-04-17T05:46:39.904163+0000","last_peered":"2024-04-17T05:46:39.904163+0000","last_clean":"2024-04-17T05:46:39.904163+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:46:39.904163+0000","last_undegraded":"2024-04-17T05:46:39.904163+0000","last_fullsized":"2024-04-17T05:46:39.904163+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'180","last_scrub_stamp":"2024-04-17T05:46:39.903938+0000","last_deep_scrub":"264'180","last_deep_scrub_stamp":"2024-04-17T05:46:39.903938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.903938+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.200762+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'114","reported_seq":686,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275867+0000","last_change":"2024-04-17T04:36:27.378054+0000","last_active":"2024-04-17T05:46:08.275867+0000","last_peered":"2024-04-17T05:46:08.275867+0000","last_clean":"2024-04-17T05:46:08.275867+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:46:08.275867+0000","last_undegraded":"2024-04-17T05:46:08.275867+0000","last_fullsized":"2024-04-17T05:46:08.275867+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:26.331718+0000","last_clean_scrub_stamp":"2024-04-17T04:36:26.331718+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'358","reported_seq":1052,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.580780+0000","last_change":"2024-04-17T05:46:44.580927+0000","last_active":"2024-04-17T05:46:44.580780+0000","last_peered":"2024-04-17T05:46:44.580780+0000","last_clean":"2024-04-17T05:46:44.580780+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:46:44.580780+0000","last_undegraded":"2024-04-17T05:46:44.580780+0000","last_fullsized":"2024-04-17T05:46:44.580780+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'358","last_scrub_stamp":"2024-04-17T05:46:44.580574+0000","last_deep_scrub":"264'358","last_deep_scrub_stamp":"2024-04-17T05:46:44.580574+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.580574+0000","objects_scrubbed":8,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:15.221508+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568179,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":576,"num_read_kb":10795,"num_write":1350,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"264'4","reported_seq":844,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.630354+0000","last_change":"2024-04-17T05:46:36.630422+0000","last_active":"2024-04-17T05:46:36.630354+0000","last_peered":"2024-04-17T05:46:36.630354+0000","last_clean":"2024-04-17T05:46:36.630354+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:46:36.630354+0000","last_undegraded":"2024-04-17T05:46:36.630354+0000","last_fullsized":"2024-04-17T05:46:36.630354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4","last_scrub_stamp":"2024-04-17T05:46:36.630154+0000","last_deep_scrub":"264'4","last_deep_scrub_stamp":"2024-04-17T05:46:36.630154+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.630154+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:28.761949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":499,"num_read_kb":497,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'304","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.834277+0000","last_change":"2024-04-17T05:46:32.834435+0000","last_active":"2024-04-17T05:46:32.834277+0000","last_peered":"2024-04-17T05:46:32.834277+0000","last_clean":"2024-04-17T05:46:32.834277+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:46:32.834277+0000","last_undegraded":"2024-04-17T05:46:32.834277+0000","last_fullsized":"2024-04-17T05:46:32.834277+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'304","last_scrub_stamp":"2024-04-17T05:46:32.834034+0000","last_deep_scrub":"264'304","last_deep_scrub_stamp":"2024-04-17T05:46:32.834034+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.834034+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:37.797882+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45015,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":503,"num_read_kb":1173,"num_write":404,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.782872+0000","last_change":"2024-04-17T05:46:32.782872+0000","last_active":"2024-04-17T05:46:32.782872+0000","last_peered":"2024-04-17T05:46:32.782872+0000","last_clean":"2024-04-17T05:46:32.782872+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:46:32.782872+0000","last_undegraded":"2024-04-17T05:46:32.782872+0000","last_fullsized":"2024-04-17T05:46:32.782872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.782613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.782613+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.782613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:22:08.388670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'105","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.146585+0000","last_change":"2024-04-17T05:46:11.146585+0000","last_active":"2024-04-17T05:46:11.146585+0000","last_peered":"2024-04-17T05:46:11.146585+0000","last_clean":"2024-04-17T05:46:11.146585+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:46:11.146585+0000","last_undegraded":"2024-04-17T05:46:11.146585+0000","last_fullsized":"2024-04-17T05:46:11.146585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.146298+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:24.914414+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.440967+0000","last_change":"2024-04-17T05:46:27.440967+0000","last_active":"2024-04-17T05:46:27.440967+0000","last_peered":"2024-04-17T05:46:27.440967+0000","last_clean":"2024-04-17T05:46:27.440967+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:46:27.440967+0000","last_undegraded":"2024-04-17T05:46:27.440967+0000","last_fullsized":"2024-04-17T05:46:27.440967+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'84","last_scrub_stamp":"2024-04-17T05:46:27.440660+0000","last_deep_scrub":"264'84","last_deep_scrub_stamp":"2024-04-17T05:46:27.440660+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.440660+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:26.099591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'322","reported_seq":1149,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934149+0000","last_change":"2024-04-17T04:36:25.354574+0000","last_active":"2024-04-17T05:46:06.934149+0000","last_peered":"2024-04-17T05:46:06.934149+0000","last_clean":"2024-04-17T05:46:06.934149+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:46:06.934149+0000","last_undegraded":"2024-04-17T05:46:06.934149+0000","last_fullsized":"2024-04-17T05:46:06.934149+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263230,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":844,"num_read_kb":11279,"num_write":1205,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.722456+0000","last_change":"2024-04-17T05:46:18.722456+0000","last_active":"2024-04-17T05:46:18.722456+0000","last_peered":"2024-04-17T05:46:18.722456+0000","last_clean":"2024-04-17T05:46:18.722456+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:46:18.722456+0000","last_undegraded":"2024-04-17T05:46:18.722456+0000","last_fullsized":"2024-04-17T05:46:18.722456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.722193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:41.699090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'126","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.818501+0000","last_change":"2024-04-17T05:46:19.818501+0000","last_active":"2024-04-17T05:46:19.818501+0000","last_peered":"2024-04-17T05:46:19.818501+0000","last_clean":"2024-04-17T05:46:19.818501+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:46:19.818501+0000","last_undegraded":"2024-04-17T05:46:19.818501+0000","last_fullsized":"2024-04-17T05:46:19.818501+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'126","last_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_deep_scrub":"264'126","last_deep_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.818199+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:31.924099+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1648,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'539","reported_seq":2323,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.960848+0000","last_change":"2024-04-17T05:46:15.961007+0000","last_active":"2024-04-17T05:46:15.960848+0000","last_peered":"2024-04-17T05:46:15.960848+0000","last_clean":"2024-04-17T05:46:15.960848+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:46:15.960848+0000","last_undegraded":"2024-04-17T05:46:15.960848+0000","last_fullsized":"2024-04-17T05:46:15.960848+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.960604+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:11.021591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'116","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.290621+0000","last_change":"2024-04-17T05:46:27.290621+0000","last_active":"2024-04-17T05:46:27.290621+0000","last_peered":"2024-04-17T05:46:27.290621+0000","last_clean":"2024-04-17T05:46:27.290621+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:46:27.290621+0000","last_undegraded":"2024-04-17T05:46:27.290621+0000","last_fullsized":"2024-04-17T05:46:27.290621+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'116","last_scrub_stamp":"2024-04-17T05:46:27.290353+0000","last_deep_scrub":"264'116","last_deep_scrub_stamp":"2024-04-17T05:46:27.290353+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.290353+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:29.531238+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'231","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.753757+0000","last_change":"2024-04-17T05:46:26.753757+0000","last_active":"2024-04-17T05:46:26.753757+0000","last_peered":"2024-04-17T05:46:26.753757+0000","last_clean":"2024-04-17T05:46:26.753757+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:46:26.753757+0000","last_undegraded":"2024-04-17T05:46:26.753757+0000","last_fullsized":"2024-04-17T05:46:26.753757+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'231","last_scrub_stamp":"2024-04-17T05:46:26.753483+0000","last_deep_scrub":"264'231","last_deep_scrub_stamp":"2024-04-17T05:46:26.753483+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.753483+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:44.040635+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":397,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'330","reported_seq":939,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.366081+0000","last_change":"2024-04-17T05:46:35.366211+0000","last_active":"2024-04-17T05:46:35.366081+0000","last_peered":"2024-04-17T05:46:35.366081+0000","last_clean":"2024-04-17T05:46:35.366081+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:46:35.366081+0000","last_undegraded":"2024-04-17T05:46:35.366081+0000","last_fullsized":"2024-04-17T05:46:35.366081+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'330","last_scrub_stamp":"2024-04-17T05:46:35.365868+0000","last_deep_scrub":"264'330","last_deep_scrub_stamp":"2024-04-17T05:46:35.365868+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.365868+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.660066+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244482,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":438,"num_read_kb":414,"num_write":1194,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.406987+0000","last_change":"2024-04-17T05:46:35.407172+0000","last_active":"2024-04-17T05:46:35.406987+0000","last_peered":"2024-04-17T05:46:35.406987+0000","last_clean":"2024-04-17T05:46:35.406987+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:46:35.406987+0000","last_undegraded":"2024-04-17T05:46:35.406987+0000","last_fullsized":"2024-04-17T05:46:35.406987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:35.406755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:35.406755+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.406755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:18.935388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'341","reported_seq":965,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:02.457762+0000","last_change":"2024-04-17T04:36:31.396219+0000","last_active":"2024-04-17T05:46:02.457762+0000","last_peered":"2024-04-17T05:46:02.457762+0000","last_clean":"2024-04-17T05:46:02.457762+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:46:02.457762+0000","last_undegraded":"2024-04-17T05:46:02.457762+0000","last_fullsized":"2024-04-17T05:46:02.457762+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:30.359654+0000","last_clean_scrub_stamp":"2024-04-17T04:36:30.359654+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44869,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":547,"num_read_kb":1136,"num_write":426,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.659892+0000","last_change":"2024-04-17T05:46:26.659892+0000","last_active":"2024-04-17T05:46:26.659892+0000","last_peered":"2024-04-17T05:46:26.659892+0000","last_clean":"2024-04-17T05:46:26.659892+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:46:26.659892+0000","last_undegraded":"2024-04-17T05:46:26.659892+0000","last_fullsized":"2024-04-17T05:46:26.659892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:26.659492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:26.659492+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.659492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:45.664443+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'131","reported_seq":643,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.296344+0000","last_change":"2024-04-17T05:46:25.296344+0000","last_active":"2024-04-17T05:46:25.296344+0000","last_peered":"2024-04-17T05:46:25.296344+0000","last_clean":"2024-04-17T05:46:25.296344+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:46:25.296344+0000","last_undegraded":"2024-04-17T05:46:25.296344+0000","last_fullsized":"2024-04-17T05:46:25.296344+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'131","last_scrub_stamp":"2024-04-17T05:46:25.295943+0000","last_deep_scrub":"264'131","last_deep_scrub_stamp":"2024-04-17T05:46:25.295943+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.295943+0000","objects_scrubbed":4,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:33.340705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":814,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.510682+0000","last_change":"2024-04-17T05:46:47.510744+0000","last_active":"2024-04-17T05:46:47.510682+0000","last_peered":"2024-04-17T05:46:47.510682+0000","last_clean":"2024-04-17T05:46:47.510682+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:46:47.510682+0000","last_undegraded":"2024-04-17T05:46:47.510682+0000","last_fullsized":"2024-04-17T05:46:47.510682+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'162","last_scrub_stamp":"2024-04-17T05:46:47.510484+0000","last_deep_scrub":"264'162","last_deep_scrub_stamp":"2024-04-17T05:46:47.510484+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.510484+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:06.834002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'349","reported_seq":1016,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.338098+0000","last_change":"2024-04-17T05:46:39.338196+0000","last_active":"2024-04-17T05:46:39.338098+0000","last_peered":"2024-04-17T05:46:39.338098+0000","last_clean":"2024-04-17T05:46:39.338098+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:46:39.338098+0000","last_undegraded":"2024-04-17T05:46:39.338098+0000","last_fullsized":"2024-04-17T05:46:39.338098+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'349","last_scrub_stamp":"2024-04-17T05:46:39.337896+0000","last_deep_scrub":"264'349","last_deep_scrub_stamp":"2024-04-17T05:46:39.337896+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.337896+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:30.714630+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":31239,"num_write":1274,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.337367+0000","last_change":"2024-04-17T05:46:38.337476+0000","last_active":"2024-04-17T05:46:38.337367+0000","last_peered":"2024-04-17T05:46:38.337367+0000","last_clean":"2024-04-17T05:46:38.337367+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:46:38.337367+0000","last_undegraded":"2024-04-17T05:46:38.337367+0000","last_fullsized":"2024-04-17T05:46:38.337367+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.337093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.337093+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.337093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:42.557669+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'278","reported_seq":873,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.603830+0000","last_change":"2024-04-17T05:46:49.603906+0000","last_active":"2024-04-17T05:46:49.603830+0000","last_peered":"2024-04-17T05:46:49.603830+0000","last_clean":"2024-04-17T05:46:49.603830+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:46:49.603830+0000","last_undegraded":"2024-04-17T05:46:49.603830+0000","last_fullsized":"2024-04-17T05:46:49.603830+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'278","last_scrub_stamp":"2024-04-17T05:46:49.603630+0000","last_deep_scrub":"264'278","last_deep_scrub_stamp":"2024-04-17T05:46:49.603630+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.603630+0000","objects_scrubbed":5,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:15.688909+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46669,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":461,"num_read_kb":1085,"num_write":368,"num_write_kb":285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.983269+0000","last_change":"2024-04-17T05:46:23.983269+0000","last_active":"2024-04-17T05:46:23.983269+0000","last_peered":"2024-04-17T05:46:23.983269+0000","last_clean":"2024-04-17T05:46:23.983269+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:46:23.983269+0000","last_undegraded":"2024-04-17T05:46:23.983269+0000","last_fullsized":"2024-04-17T05:46:23.983269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.983009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:24:08.626906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'136","reported_seq":667,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.796701+0000","last_change":"2024-04-17T05:46:20.796701+0000","last_active":"2024-04-17T05:46:20.796701+0000","last_peered":"2024-04-17T05:46:20.796701+0000","last_clean":"2024-04-17T05:46:20.796701+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:46:20.796701+0000","last_undegraded":"2024-04-17T05:46:20.796701+0000","last_fullsized":"2024-04-17T05:46:20.796701+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'136","last_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_deep_scrub":"264'136","last_deep_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.796340+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:15.845948+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'204","reported_seq":821,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.764924+0000","last_change":"2024-04-17T05:46:47.764924+0000","last_active":"2024-04-17T05:46:47.764924+0000","last_peered":"2024-04-17T05:46:47.764924+0000","last_clean":"2024-04-17T05:46:47.764924+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:46:47.764924+0000","last_undegraded":"2024-04-17T05:46:47.764924+0000","last_fullsized":"2024-04-17T05:46:47.764924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'204","last_scrub_stamp":"2024-04-17T05:46:47.764628+0000","last_deep_scrub":"264'204","last_deep_scrub_stamp":"2024-04-17T05:46:47.764628+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.764628+0000","objects_scrubbed":1,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:52.071861+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3065,"num_omap_keys":18,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'386","reported_seq":1051,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275555+0000","last_change":"2024-04-17T04:36:25.374623+0000","last_active":"2024-04-17T05:46:08.275555+0000","last_peered":"2024-04-17T05:46:08.275555+0000","last_clean":"2024-04-17T05:46:08.275555+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:46:08.275555+0000","last_undegraded":"2024-04-17T05:46:08.275555+0000","last_fullsized":"2024-04-17T05:46:08.275555+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1731334,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":524,"num_read_kb":19843,"num_write":1402,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.586905+0000","last_change":"2024-04-17T05:46:40.586958+0000","last_active":"2024-04-17T05:46:40.586905+0000","last_peered":"2024-04-17T05:46:40.586905+0000","last_clean":"2024-04-17T05:46:40.586905+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:46:40.586905+0000","last_undegraded":"2024-04-17T05:46:40.586905+0000","last_fullsized":"2024-04-17T05:46:40.586905+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:40.586716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:40.586716+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.586716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:45:35.339099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'263","reported_seq":845,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.699498+0000","last_change":"2024-04-17T05:46:39.699578+0000","last_active":"2024-04-17T05:46:39.699498+0000","last_peered":"2024-04-17T05:46:39.699498+0000","last_clean":"2024-04-17T05:46:39.699498+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:46:39.699498+0000","last_undegraded":"2024-04-17T05:46:39.699498+0000","last_fullsized":"2024-04-17T05:46:39.699498+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'263","last_scrub_stamp":"2024-04-17T05:46:39.699314+0000","last_deep_scrub":"264'263","last_deep_scrub_stamp":"2024-04-17T05:46:39.699314+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.699314+0000","objects_scrubbed":3,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:28.949715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44294,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":445,"num_read_kb":1105,"num_write":366,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.762692+0000","last_change":"2024-04-17T05:46:18.762692+0000","last_active":"2024-04-17T05:46:18.762692+0000","last_peered":"2024-04-17T05:46:18.762692+0000","last_clean":"2024-04-17T05:46:18.762692+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:46:18.762692+0000","last_undegraded":"2024-04-17T05:46:18.762692+0000","last_fullsized":"2024-04-17T05:46:18.762692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.762296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:44.671414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'146","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.248471+0000","last_change":"2024-04-17T05:46:26.248471+0000","last_active":"2024-04-17T05:46:26.248471+0000","last_peered":"2024-04-17T05:46:26.248471+0000","last_clean":"2024-04-17T05:46:26.248471+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:46:26.248471+0000","last_undegraded":"2024-04-17T05:46:26.248471+0000","last_fullsized":"2024-04-17T05:46:26.248471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'146","last_scrub_stamp":"2024-04-17T05:46:26.248201+0000","last_deep_scrub":"264'146","last_deep_scrub_stamp":"2024-04-17T05:46:26.248201+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.248201+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:59.437725+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'299","reported_seq":987,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.566140+0000","last_change":"2024-04-17T05:46:32.566140+0000","last_active":"2024-04-17T05:46:32.566140+0000","last_peered":"2024-04-17T05:46:32.566140+0000","last_clean":"2024-04-17T05:46:32.566140+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:46:32.566140+0000","last_undegraded":"2024-04-17T05:46:32.566140+0000","last_fullsized":"2024-04-17T05:46:32.566140+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'299","last_scrub_stamp":"2024-04-17T05:46:32.565869+0000","last_deep_scrub":"264'299","last_deep_scrub_stamp":"2024-04-17T05:46:32.565869+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.565869+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:01:45.736052+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28561,"num_omap_keys":113,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'437","reported_seq":1144,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.857015+0000","last_change":"2024-04-17T04:36:25.356509+0000","last_active":"2024-04-17T05:46:03.857015+0000","last_peered":"2024-04-17T05:46:03.857015+0000","last_clean":"2024-04-17T05:46:03.857015+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:46:03.857015+0000","last_undegraded":"2024-04-17T05:46:03.857015+0000","last_fullsized":"2024-04-17T05:46:03.857015+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":515218,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":663,"num_read_kb":71297,"num_write":1686,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"264'8","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.766334+0000","last_change":"2024-04-17T05:46:20.766334+0000","last_active":"2024-04-17T05:46:20.766334+0000","last_peered":"2024-04-17T05:46:20.766334+0000","last_clean":"2024-04-17T05:46:20.766334+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:46:20.766334+0000","last_undegraded":"2024-04-17T05:46:20.766334+0000","last_fullsized":"2024-04-17T05:46:20.766334+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.766051+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:22.853924+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":501,"num_read_kb":814,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'324","reported_seq":944,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.796472+0000","last_change":"2024-04-17T05:46:25.796472+0000","last_active":"2024-04-17T05:46:25.796472+0000","last_peered":"2024-04-17T05:46:25.796472+0000","last_clean":"2024-04-17T05:46:25.796472+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:46:25.796472+0000","last_undegraded":"2024-04-17T05:46:25.796472+0000","last_fullsized":"2024-04-17T05:46:25.796472+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'324","last_scrub_stamp":"2024-04-17T05:46:25.796196+0000","last_deep_scrub":"264'324","last_deep_scrub_stamp":"2024-04-17T05:46:25.796196+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.796196+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:16.415248+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46192,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":530,"num_read_kb":1178,"num_write":420,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.616502+0000","last_change":"2024-04-17T05:46:29.616502+0000","last_active":"2024-04-17T05:46:29.616502+0000","last_peered":"2024-04-17T05:46:29.616502+0000","last_clean":"2024-04-17T05:46:29.616502+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:46:29.616502+0000","last_undegraded":"2024-04-17T05:46:29.616502+0000","last_fullsized":"2024-04-17T05:46:29.616502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.616255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.616255+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.616255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:25:32.388074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'141","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.802101+0000","last_change":"2024-04-17T05:46:10.802101+0000","last_active":"2024-04-17T05:46:10.802101+0000","last_peered":"2024-04-17T05:46:10.802101+0000","last_clean":"2024-04-17T05:46:10.802101+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:46:10.802101+0000","last_undegraded":"2024-04-17T05:46:10.802101+0000","last_fullsized":"2024-04-17T05:46:10.802101+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.801687+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.852555+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'155","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.741104+0000","last_change":"2024-04-17T05:46:21.741243+0000","last_active":"2024-04-17T05:46:21.741104+0000","last_peered":"2024-04-17T05:46:21.741104+0000","last_clean":"2024-04-17T05:46:21.741104+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:46:21.741104+0000","last_undegraded":"2024-04-17T05:46:21.741104+0000","last_fullsized":"2024-04-17T05:46:21.741104+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.740847+0000","objects_scrubbed":1,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:03.471838+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'420","reported_seq":1147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.811870+0000","last_change":"2024-04-17T05:46:34.812010+0000","last_active":"2024-04-17T05:46:34.811870+0000","last_peered":"2024-04-17T05:46:34.811870+0000","last_clean":"2024-04-17T05:46:34.811870+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:46:34.811870+0000","last_undegraded":"2024-04-17T05:46:34.811870+0000","last_fullsized":"2024-04-17T05:46:34.811870+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'420","last_scrub_stamp":"2024-04-17T05:46:34.811680+0000","last_deep_scrub":"264'420","last_deep_scrub_stamp":"2024-04-17T05:46:34.811680+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.811680+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:57.763900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262268,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":835,"num_read_kb":6303,"num_write":1376,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":658,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.183156+0000","last_change":"2024-04-17T05:46:10.183156+0000","last_active":"2024-04-17T05:46:10.183156+0000","last_peered":"2024-04-17T05:46:10.183156+0000","last_clean":"2024-04-17T05:46:10.183156+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:46:10.183156+0000","last_undegraded":"2024-04-17T05:46:10.183156+0000","last_fullsized":"2024-04-17T05:46:10.183156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'90","last_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_deep_scrub":"264'90","last_deep_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.182909+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:58:10.036518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'365","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.440689+0000","last_change":"2024-04-17T05:46:26.440875+0000","last_active":"2024-04-17T05:46:26.440689+0000","last_peered":"2024-04-17T05:46:26.440689+0000","last_clean":"2024-04-17T05:46:26.440689+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:46:26.440689+0000","last_undegraded":"2024-04-17T05:46:26.440689+0000","last_fullsized":"2024-04-17T05:46:26.440689+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'365","last_scrub_stamp":"2024-04-17T05:46:26.440480+0000","last_deep_scrub":"264'365","last_deep_scrub_stamp":"2024-04-17T05:46:26.440480+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.440480+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:58.716606+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6192676,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":568,"num_read_kb":39832,"num_write":1198,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'149","reported_seq":772,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.243871+0000","last_change":"2024-04-17T05:46:27.243940+0000","last_active":"2024-04-17T05:46:27.243871+0000","last_peered":"2024-04-17T05:46:27.243871+0000","last_clean":"2024-04-17T05:46:27.243871+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:46:27.243871+0000","last_undegraded":"2024-04-17T05:46:27.243871+0000","last_fullsized":"2024-04-17T05:46:27.243871+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'149","last_scrub_stamp":"2024-04-17T05:46:27.243689+0000","last_deep_scrub":"264'149","last_deep_scrub_stamp":"2024-04-17T05:46:27.243689+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.243689+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:03.394294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'307","reported_seq":953,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.272331+0000","last_change":"2024-04-17T05:46:41.272639+0000","last_active":"2024-04-17T05:46:41.272331+0000","last_peered":"2024-04-17T05:46:41.272331+0000","last_clean":"2024-04-17T05:46:41.272331+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:46:41.272331+0000","last_undegraded":"2024-04-17T05:46:41.272331+0000","last_fullsized":"2024-04-17T05:46:41.272331+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'307","last_scrub_stamp":"2024-04-17T05:46:41.272065+0000","last_deep_scrub":"264'307","last_deep_scrub_stamp":"2024-04-17T05:46:41.272065+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.272065+0000","objects_scrubbed":10,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:14.652269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4341641,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":436,"num_read_kb":10784,"num_write":1165,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'95","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.829126+0000","last_change":"2024-04-17T05:46:31.829195+0000","last_active":"2024-04-17T05:46:31.829126+0000","last_peered":"2024-04-17T05:46:31.829126+0000","last_clean":"2024-04-17T05:46:31.829126+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:46:31.829126+0000","last_undegraded":"2024-04-17T05:46:31.829126+0000","last_fullsized":"2024-04-17T05:46:31.829126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'95","last_scrub_stamp":"2024-04-17T05:46:31.828927+0000","last_deep_scrub":"264'95","last_deep_scrub_stamp":"2024-04-17T05:46:31.828927+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.828927+0000","objects_scrubbed":1,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:59.765753+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'308","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.886992+0000","last_change":"2024-04-17T04:36:25.355263+0000","last_active":"2024-04-17T05:46:03.886992+0000","last_peered":"2024-04-17T05:46:03.886992+0000","last_clean":"2024-04-17T05:46:03.886992+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:46:03.886992+0000","last_undegraded":"2024-04-17T05:46:03.886992+0000","last_fullsized":"2024-04-17T05:46:03.886992+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":709,"num_read_kb":32424,"num_write":1121,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":644,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.725516+0000","last_change":"2024-04-17T05:46:44.725575+0000","last_active":"2024-04-17T05:46:44.725516+0000","last_peered":"2024-04-17T05:46:44.725516+0000","last_clean":"2024-04-17T05:46:44.725516+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:46:44.725516+0000","last_undegraded":"2024-04-17T05:46:44.725516+0000","last_fullsized":"2024-04-17T05:46:44.725516+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'58","last_scrub_stamp":"2024-04-17T05:46:44.725324+0000","last_deep_scrub":"264'58","last_deep_scrub_stamp":"2024-04-17T05:46:44.725324+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.725324+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:00.884116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'281","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.814149+0000","last_change":"2024-04-17T05:46:16.814273+0000","last_active":"2024-04-17T05:46:16.814149+0000","last_peered":"2024-04-17T05:46:16.814149+0000","last_clean":"2024-04-17T05:46:16.814149+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:46:16.814149+0000","last_undegraded":"2024-04-17T05:46:16.814149+0000","last_fullsized":"2024-04-17T05:46:16.814149+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.813947+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:56.482239+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5047006,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":359,"num_read_kb":5921,"num_write":992,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":641,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.664684+0000","last_change":"2024-04-17T05:46:28.664753+0000","last_active":"2024-04-17T05:46:28.664684+0000","last_peered":"2024-04-17T05:46:28.664684+0000","last_clean":"2024-04-17T05:46:28.664684+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:46:28.664684+0000","last_undegraded":"2024-04-17T05:46:28.664684+0000","last_fullsized":"2024-04-17T05:46:28.664684+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'54","last_scrub_stamp":"2024-04-17T05:46:28.664477+0000","last_deep_scrub":"264'54","last_deep_scrub_stamp":"2024-04-17T05:46:28.664477+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.664477+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:11.472041+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'319","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.820625+0000","last_change":"2024-04-17T05:46:36.820896+0000","last_active":"2024-04-17T05:46:36.820625+0000","last_peered":"2024-04-17T05:46:36.820625+0000","last_clean":"2024-04-17T05:46:36.820625+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:46:36.820625+0000","last_undegraded":"2024-04-17T05:46:36.820625+0000","last_fullsized":"2024-04-17T05:46:36.820625+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:36.820373+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:36.820373+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.820373+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:09.390555+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5067983,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":492,"num_read_kb":71000,"num_write":1186,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'216","reported_seq":825,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.756140+0000","last_change":"2024-04-17T05:46:45.756202+0000","last_active":"2024-04-17T05:46:45.756140+0000","last_peered":"2024-04-17T05:46:45.756140+0000","last_clean":"2024-04-17T05:46:45.756140+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:46:45.756140+0000","last_undegraded":"2024-04-17T05:46:45.756140+0000","last_fullsized":"2024-04-17T05:46:45.756140+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'216","last_scrub_stamp":"2024-04-17T05:46:45.755911+0000","last_deep_scrub":"264'216","last_deep_scrub_stamp":"2024-04-17T05:46:45.755911+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.755911+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:01.103569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8079,"num_omap_keys":32,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'371","reported_seq":1061,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.914547+0000","last_change":"2024-04-17T05:46:38.914661+0000","last_active":"2024-04-17T05:46:38.914547+0000","last_peered":"2024-04-17T05:46:38.914547+0000","last_clean":"2024-04-17T05:46:38.914547+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:46:38.914547+0000","last_undegraded":"2024-04-17T05:46:38.914547+0000","last_fullsized":"2024-04-17T05:46:38.914547+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'371","last_scrub_stamp":"2024-04-17T05:46:38.914310+0000","last_deep_scrub":"264'371","last_deep_scrub_stamp":"2024-04-17T05:46:38.914310+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.914310+0000","objects_scrubbed":6,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:28.802337+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2189648,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":654,"num_read_kb":34085,"num_write":1471,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":757,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.000521+0000","last_change":"2024-04-17T05:46:27.000580+0000","last_active":"2024-04-17T05:46:27.000521+0000","last_peered":"2024-04-17T05:46:27.000521+0000","last_clean":"2024-04-17T05:46:27.000521+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:46:27.000521+0000","last_undegraded":"2024-04-17T05:46:27.000521+0000","last_fullsized":"2024-04-17T05:46:27.000521+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'135","last_scrub_stamp":"2024-04-17T05:46:27.000319+0000","last_deep_scrub":"264'135","last_deep_scrub_stamp":"2024-04-17T05:46:27.000319+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.000319+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:15.623855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'335","reported_seq":1007,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:03.881772+0000","last_change":"2024-04-17T04:36:25.379835+0000","last_active":"2024-04-17T05:46:03.881772+0000","last_peered":"2024-04-17T05:46:03.881772+0000","last_clean":"2024-04-17T05:46:03.881772+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:46:03.881772+0000","last_undegraded":"2024-04-17T05:46:03.881772+0000","last_fullsized":"2024-04-17T05:46:03.881772+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534609,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":546,"num_read_kb":42711,"num_write":1290,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.655975+0000","last_change":"2024-04-17T05:46:42.656040+0000","last_active":"2024-04-17T05:46:42.655975+0000","last_peered":"2024-04-17T05:46:42.655975+0000","last_clean":"2024-04-17T05:46:42.655975+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:46:42.655975+0000","last_undegraded":"2024-04-17T05:46:42.655975+0000","last_fullsized":"2024-04-17T05:46:42.655975+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:42.655782+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:42.655782+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.655782+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:25.917924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'300","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.230338+0000","last_change":"2024-04-17T05:46:48.230447+0000","last_active":"2024-04-17T05:46:48.230338+0000","last_peered":"2024-04-17T05:46:48.230338+0000","last_clean":"2024-04-17T05:46:48.230338+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:46:48.230338+0000","last_undegraded":"2024-04-17T05:46:48.230338+0000","last_fullsized":"2024-04-17T05:46:48.230338+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'300","last_scrub_stamp":"2024-04-17T05:46:48.230130+0000","last_deep_scrub":"264'300","last_deep_scrub_stamp":"2024-04-17T05:46:48.230130+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.230130+0000","objects_scrubbed":6,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:19.387099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262289,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":548,"num_write":1115,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":764,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.512660+0000","last_change":"2024-04-17T05:46:48.512719+0000","last_active":"2024-04-17T05:46:48.512660+0000","last_peered":"2024-04-17T05:46:48.512660+0000","last_clean":"2024-04-17T05:46:48.512660+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:46:48.512660+0000","last_undegraded":"2024-04-17T05:46:48.512660+0000","last_fullsized":"2024-04-17T05:46:48.512660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'110","last_scrub_stamp":"2024-04-17T05:46:48.512459+0000","last_deep_scrub":"264'110","last_deep_scrub_stamp":"2024-04-17T05:46:48.512459+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.512459+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:19.000329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'302","reported_seq":953,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.275580+0000","last_change":"2024-04-17T04:36:25.353469+0000","last_active":"2024-04-17T05:46:08.275580+0000","last_peered":"2024-04-17T05:46:08.275580+0000","last_clean":"2024-04-17T05:46:08.275580+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:46:08.275580+0000","last_undegraded":"2024-04-17T05:46:08.275580+0000","last_fullsized":"2024-04-17T05:46:08.275580+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171414,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":4352,"num_write":1098,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":847,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.463713+0000","last_change":"2024-04-17T05:46:31.463772+0000","last_active":"2024-04-17T05:46:31.463713+0000","last_peered":"2024-04-17T05:46:31.463713+0000","last_clean":"2024-04-17T05:46:31.463713+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:46:31.463713+0000","last_undegraded":"2024-04-17T05:46:31.463713+0000","last_fullsized":"2024-04-17T05:46:31.463713+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'202","last_scrub_stamp":"2024-04-17T05:46:31.463487+0000","last_deep_scrub":"264'202","last_deep_scrub_stamp":"2024-04-17T05:46:31.463487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.463487+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:24.787961+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'317","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.663532+0000","last_change":"2024-04-17T05:46:10.663532+0000","last_active":"2024-04-17T05:46:10.663532+0000","last_peered":"2024-04-17T05:46:10.663532+0000","last_clean":"2024-04-17T05:46:10.663532+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:46:10.663532+0000","last_undegraded":"2024-04-17T05:46:10.663532+0000","last_fullsized":"2024-04-17T05:46:10.663532+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'317","last_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_deep_scrub":"264'317","last_deep_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.663219+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.001315+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":443,"num_read_kb":16015,"num_write":1067,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'96","reported_seq":694,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.604944+0000","last_change":"2024-04-17T05:46:34.605089+0000","last_active":"2024-04-17T05:46:34.604944+0000","last_peered":"2024-04-17T05:46:34.604944+0000","last_clean":"2024-04-17T05:46:34.604944+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:46:34.604944+0000","last_undegraded":"2024-04-17T05:46:34.604944+0000","last_fullsized":"2024-04-17T05:46:34.604944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'96","last_scrub_stamp":"2024-04-17T05:46:34.604692+0000","last_deep_scrub":"264'96","last_deep_scrub_stamp":"2024-04-17T05:46:34.604692+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.604692+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:59.288725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'360","reported_seq":1079,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:06.934332+0000","last_change":"2024-04-17T04:36:25.333011+0000","last_active":"2024-04-17T05:46:06.934332+0000","last_peered":"2024-04-17T05:46:06.934332+0000","last_clean":"2024-04-17T05:46:06.934332+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:46:06.934332+0000","last_undegraded":"2024-04-17T05:46:06.934332+0000","last_fullsized":"2024-04-17T05:46:06.934332+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T04:36:24.268296+0000","last_clean_scrub_stamp":"2024-04-17T04:36:24.268296+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":746,"num_read_kb":12134,"num_write":1256,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":696,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.414667+0000","last_change":"2024-04-17T05:46:37.414725+0000","last_active":"2024-04-17T05:46:37.414667+0000","last_peered":"2024-04-17T05:46:37.414667+0000","last_clean":"2024-04-17T05:46:37.414667+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:46:37.414667+0000","last_undegraded":"2024-04-17T05:46:37.414667+0000","last_fullsized":"2024-04-17T05:46:37.414667+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:37.414492+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:37.414492+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.414492+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:18.807121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'357","reported_seq":1003,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.855024+0000","last_change":"2024-04-17T05:46:33.855258+0000","last_active":"2024-04-17T05:46:33.855024+0000","last_peered":"2024-04-17T05:46:33.855024+0000","last_clean":"2024-04-17T05:46:33.855024+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:46:33.855024+0000","last_undegraded":"2024-04-17T05:46:33.855024+0000","last_fullsized":"2024-04-17T05:46:33.855024+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'357","last_scrub_stamp":"2024-04-17T05:46:33.854698+0000","last_deep_scrub":"264'357","last_deep_scrub_stamp":"2024-04-17T05:46:33.854698+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.854698+0000","objects_scrubbed":3,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:38.333183+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":501,"num_read_kb":657,"num_write":1252,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'222","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.665956+0000","last_change":"2024-04-17T05:46:44.666020+0000","last_active":"2024-04-17T05:46:44.665956+0000","last_peered":"2024-04-17T05:46:44.665956+0000","last_clean":"2024-04-17T05:46:44.665956+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:46:44.665956+0000","last_undegraded":"2024-04-17T05:46:44.665956+0000","last_fullsized":"2024-04-17T05:46:44.665956+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'222","last_scrub_stamp":"2024-04-17T05:46:44.665753+0000","last_deep_scrub":"264'222","last_deep_scrub_stamp":"2024-04-17T05:46:44.665753+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.665753+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:05.646503+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'420","reported_seq":1168,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.230796+0000","last_change":"2024-04-17T05:46:45.230919+0000","last_active":"2024-04-17T05:46:45.230796+0000","last_peered":"2024-04-17T05:46:45.230796+0000","last_clean":"2024-04-17T05:46:45.230796+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:46:45.230796+0000","last_undegraded":"2024-04-17T05:46:45.230796+0000","last_fullsized":"2024-04-17T05:46:45.230796+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'420","last_scrub_stamp":"2024-04-17T05:46:45.230572+0000","last_deep_scrub":"264'420","last_deep_scrub_stamp":"2024-04-17T05:46:45.230572+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.230572+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:12.917895+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058993,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":894,"num_read_kb":32480,"num_write":1416,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.744514+0000","last_change":"2024-04-17T05:46:41.744578+0000","last_active":"2024-04-17T05:46:41.744514+0000","last_peered":"2024-04-17T05:46:41.744514+0000","last_clean":"2024-04-17T05:46:41.744514+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:46:41.744514+0000","last_undegraded":"2024-04-17T05:46:41.744514+0000","last_fullsized":"2024-04-17T05:46:41.744514+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'352","last_scrub_stamp":"2024-04-17T05:46:41.744313+0000","last_deep_scrub":"264'352","last_deep_scrub_stamp":"2024-04-17T05:46:41.744313+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.744313+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:33.007856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'314","reported_seq":1000,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.458853+0000","last_change":"2024-04-17T05:46:25.458952+0000","last_active":"2024-04-17T05:46:25.458853+0000","last_peered":"2024-04-17T05:46:25.458853+0000","last_clean":"2024-04-17T05:46:25.458853+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:46:25.458853+0000","last_undegraded":"2024-04-17T05:46:25.458853+0000","last_fullsized":"2024-04-17T05:46:25.458853+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'314","last_scrub_stamp":"2024-04-17T05:46:25.458647+0000","last_deep_scrub":"264'314","last_deep_scrub_stamp":"2024-04-17T05:46:25.458647+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.458647+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:02.295132+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7281800,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":526,"num_read_kb":46303,"num_write":1296,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'322","reported_seq":1053,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.325081+0000","last_change":"2024-04-17T05:46:23.325152+0000","last_active":"2024-04-17T05:46:23.325081+0000","last_peered":"2024-04-17T05:46:23.325081+0000","last_clean":"2024-04-17T05:46:23.325081+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:46:23.325081+0000","last_undegraded":"2024-04-17T05:46:23.325081+0000","last_fullsized":"2024-04-17T05:46:23.325081+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.324895+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:02.515429+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'328","reported_seq":971,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.276500+0000","last_change":"2024-04-17T05:46:49.276594+0000","last_active":"2024-04-17T05:46:49.276500+0000","last_peered":"2024-04-17T05:46:49.276500+0000","last_clean":"2024-04-17T05:46:49.276500+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:46:49.276500+0000","last_undegraded":"2024-04-17T05:46:49.276500+0000","last_fullsized":"2024-04-17T05:46:49.276500+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'328","last_scrub_stamp":"2024-04-17T05:46:49.276270+0000","last_deep_scrub":"264'328","last_deep_scrub_stamp":"2024-04-17T05:46:49.276270+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.276270+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:32.984779+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":466,"num_read_kb":49597,"num_write":1231,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'297","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.292701+0000","last_change":"2024-04-17T05:46:45.292805+0000","last_active":"2024-04-17T05:46:45.292701+0000","last_peered":"2024-04-17T05:46:45.292701+0000","last_clean":"2024-04-17T05:46:45.292701+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:46:45.292701+0000","last_undegraded":"2024-04-17T05:46:45.292701+0000","last_fullsized":"2024-04-17T05:46:45.292701+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'297","last_scrub_stamp":"2024-04-17T05:46:45.292505+0000","last_deep_scrub":"264'297","last_deep_scrub_stamp":"2024-04-17T05:46:45.292505+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.292505+0000","objects_scrubbed":8,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:54.845561+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129194,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":413,"num_read_kb":7661,"num_write":1087,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'244","reported_seq":853,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.670960+0000","last_change":"2024-04-17T05:46:37.670960+0000","last_active":"2024-04-17T05:46:37.670960+0000","last_peered":"2024-04-17T05:46:37.670960+0000","last_clean":"2024-04-17T05:46:37.670960+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:46:37.670960+0000","last_undegraded":"2024-04-17T05:46:37.670960+0000","last_fullsized":"2024-04-17T05:46:37.670960+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'244","last_scrub_stamp":"2024-04-17T05:46:37.670697+0000","last_deep_scrub":"264'244","last_deep_scrub_stamp":"2024-04-17T05:46:37.670697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.670697+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":200,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:58.583622+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'277","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.851613+0000","last_change":"2024-04-17T05:46:41.851831+0000","last_active":"2024-04-17T05:46:41.851613+0000","last_peered":"2024-04-17T05:46:41.851613+0000","last_clean":"2024-04-17T05:46:41.851613+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:46:41.851613+0000","last_undegraded":"2024-04-17T05:46:41.851613+0000","last_fullsized":"2024-04-17T05:46:41.851613+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'277","last_scrub_stamp":"2024-04-17T05:46:41.851309+0000","last_deep_scrub":"264'277","last_deep_scrub_stamp":"2024-04-17T05:46:41.851309+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.851309+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:22:42.210177+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":447,"num_read_kb":20933,"num_write":1123,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":829,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.314199+0000","last_change":"2024-04-17T05:46:29.314199+0000","last_active":"2024-04-17T05:46:29.314199+0000","last_peered":"2024-04-17T05:46:29.314199+0000","last_clean":"2024-04-17T05:46:29.314199+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:46:29.314199+0000","last_undegraded":"2024-04-17T05:46:29.314199+0000","last_fullsized":"2024-04-17T05:46:29.314199+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'198","last_scrub_stamp":"2024-04-17T05:46:29.313797+0000","last_deep_scrub":"264'198","last_deep_scrub_stamp":"2024-04-17T05:46:29.313797+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.313797+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:17.398694+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6748,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1282,"ondisk_log_size":1282,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4334,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1780,"ondisk_log_size":1780,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1458963,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":18010,"num_read_kb":37614,"num_write":15043,"num_write_kb":11218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":622600,"num_omap_keys":1502,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2003,"ondisk_log_size":2003,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1652,"num_read_kb":1963,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43632,"num_omap_keys":180,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3969,"ondisk_log_size":3969,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":376,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187841250,"num_objects":443,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":34833,"num_read_kb":1864137,"num_write":77494,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253607936,"data_stored":253607936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3747,"ondisk_log_size":3747,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":221363,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":145,"num_read_kb":210051,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575700,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349100,"kb_used_data":38884,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":94022740,"statfs":{"total":96636764160,"available":96279285760,"internally_reserved":0,"allocated":39817216,"data_stored":37809825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":317651402},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575701,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355672,"kb_used_data":39888,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":94016168,"statfs":{"total":96636764160,"available":96272556032,"internally_reserved":0,"allocated":40845312,"data_stored":38885655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":323353031},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575701,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206104,"kb_used_data":58384,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94165736,"statfs":{"total":96636764160,"available":96425713664,"internally_reserved":0,"allocated":59785216,"data_stored":57737844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575703,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148100,"kb_used_data":36412,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94223740,"statfs":{"total":96636764160,"available":96485109760,"internally_reserved":0,"allocated":37285888,"data_stored":35330686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575701,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298456,"kb_used_data":42448,"kb_used_omap":1,"kb_used_meta":255998,"kb_avail":94073384,"statfs":{"total":96636764160,"available":96331145216,"internally_reserved":0,"allocated":43466752,"data_stored":41436434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":262142407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575705,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159304,"kb_used_data":56000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94212536,"statfs":{"total":96636764160,"available":96473636864,"internally_reserved":0,"allocated":57344000,"data_stored":55396653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575703,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183784,"kb_used_data":36064,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94188056,"statfs":{"total":96636764160,"available":96448569344,"internally_reserved":0,"allocated":36929536,"data_stored":34946873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575701,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327488,"kb_used_data":39544,"kb_used_omap":1,"kb_used_meta":287934,"kb_avail":94044352,"statfs":{"total":96636764160,"available":96301416448,"internally_reserved":0,"allocated":40493056,"data_stored":38470704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294844869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35614720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26390528,"data_stored":26390528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36356096,"data_stored":36356096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33927168,"data_stored":33927168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22581248,"data_stored":22581248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":38694912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31379456,"data_stored":31379456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28663808,"data_stored":28663808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:46:51.485 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.486 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.486 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.486 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.487 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.487 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.487 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.487 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.487 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.488 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.488 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.488 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.488 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.489 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.489 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.489 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.490 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.490 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.490 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.490 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.490 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.491 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.491 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.491 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.491 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.492 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.492 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.492 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-04-17T04:51:38.577167+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=38, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.492 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.492 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.493 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.493 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.493 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.493 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.494 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-04-17T04:51:42.619140+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=51, tm_sec=42, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.494 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.494 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.494 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.495 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.495 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.495 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.495 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.495 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.496 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.496 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.496 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.497 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.497 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.497 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.497 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.498 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.498 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.498 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.498 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.499 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.499 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-17T04:36:16.191560+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=16, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.499 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.500 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.500 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.500 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-17T04:36:28.338586+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=28, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.500 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.501 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.501 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.501 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-17T04:36:26.331718+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=26, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.501 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.502 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-17T04:36:30.359654+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=30, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.502 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.502 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.503 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.503 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.503 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.503 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-17T04:36:24.268296+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=4, tm_min=36, tm_sec=24, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:46:51.504 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T05:47:11.505 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:47:11.892 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:47:11.906 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:47:11.909 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2387,"stamp":"2024-04-17T05:47:11.325660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223466608,"num_objects":822,"num_object_clones":0,"num_object_copies":2530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":822,"num_whiteouts":0,"num_read":160833,"num_read_kb":2423241,"num_write":146364,"num_write_kb":2463359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684328,"num_omap_keys":1751,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13339,"ondisk_log_size":13339,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2032168,"kb_used_data":347624,"kb_used_omap":12,"kb_used_meta":1684467,"kb_avail":752942552,"statfs":{"total":773094113280,"available":771013173248,"internally_reserved":0,"allocated":355966976,"data_stored":340014674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1724894769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5888,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007873"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.209007+0000","last_change":"2024-04-17T05:46:55.209062+0000","last_active":"2024-04-17T05:46:55.209007+0000","last_peered":"2024-04-17T05:46:55.209007+0000","last_clean":"2024-04-17T05:46:55.209007+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:46:55.209007+0000","last_undegraded":"2024-04-17T05:46:55.209007+0000","last_fullsized":"2024-04-17T05:46:55.209007+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:55.208798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:55.208798+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.208798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:05:07.820451+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.237389+0000","last_change":"2024-04-17T05:46:54.237445+0000","last_active":"2024-04-17T05:46:54.237389+0000","last_peered":"2024-04-17T05:46:54.237389+0000","last_clean":"2024-04-17T05:46:54.237389+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:46:54.237389+0000","last_undegraded":"2024-04-17T05:46:54.237389+0000","last_fullsized":"2024-04-17T05:46:54.237389+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:54.237198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:54.237198+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.237198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:22:01.742139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.830930+0000","last_change":"2024-04-17T05:46:14.830930+0000","last_active":"2024-04-17T05:46:14.830930+0000","last_peered":"2024-04-17T05:46:14.830930+0000","last_clean":"2024-04-17T05:46:14.830930+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:46:14.830930+0000","last_undegraded":"2024-04-17T05:46:14.830930+0000","last_fullsized":"2024-04-17T05:46:14.830930+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.830571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:08.876068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":158,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.685785+0000","last_change":"2024-04-17T05:46:08.685785+0000","last_active":"2024-04-17T05:46:08.685785+0000","last_peered":"2024-04-17T05:46:08.685785+0000","last_clean":"2024-04-17T05:46:08.685785+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:46:08.685785+0000","last_undegraded":"2024-04-17T05:46:08.685785+0000","last_fullsized":"2024-04-17T05:46:08.685785+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.685550+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:05.034233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.544197+0000","last_change":"2024-04-17T05:46:39.544342+0000","last_active":"2024-04-17T05:46:39.544197+0000","last_peered":"2024-04-17T05:46:39.544197+0000","last_clean":"2024-04-17T05:46:39.544197+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:46:39.544197+0000","last_undegraded":"2024-04-17T05:46:39.544197+0000","last_fullsized":"2024-04-17T05:46:39.544197+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:39.543940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:39.543940+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.543940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:06.130204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'3","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.619326+0000","last_change":"2024-04-17T05:46:55.619394+0000","last_active":"2024-04-17T05:46:55.619326+0000","last_peered":"2024-04-17T05:46:55.619326+0000","last_clean":"2024-04-17T05:46:55.619326+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:46:55.619326+0000","last_undegraded":"2024-04-17T05:46:55.619326+0000","last_fullsized":"2024-04-17T05:46:55.619326+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:55.619100+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:55.619100+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.619100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:48.858607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.692660+0000","last_change":"2024-04-17T05:46:12.692660+0000","last_active":"2024-04-17T05:46:12.692660+0000","last_peered":"2024-04-17T05:46:12.692660+0000","last_clean":"2024-04-17T05:46:12.692660+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:46:12.692660+0000","last_undegraded":"2024-04-17T05:46:12.692660+0000","last_fullsized":"2024-04-17T05:46:12.692660+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.692409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:20.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'97","reported_seq":361,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.504742+0000","last_change":"2024-04-17T05:46:20.504828+0000","last_active":"2024-04-17T05:46:20.504742+0000","last_peered":"2024-04-17T05:46:20.504742+0000","last_clean":"2024-04-17T05:46:20.504742+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:46:20.504742+0000","last_undegraded":"2024-04-17T05:46:20.504742+0000","last_fullsized":"2024-04-17T05:46:20.504742+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.504541+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:15.125809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3493,"num_omap_keys":7,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.887667+0000","last_change":"2024-04-17T05:46:45.887724+0000","last_active":"2024-04-17T05:46:45.887667+0000","last_peered":"2024-04-17T05:46:45.887667+0000","last_clean":"2024-04-17T05:46:45.887667+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:46:45.887667+0000","last_undegraded":"2024-04-17T05:46:45.887667+0000","last_fullsized":"2024-04-17T05:46:45.887667+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:45.887431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:45.887431+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.887431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:34.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.677240+0000","last_change":"2024-04-17T05:46:35.677355+0000","last_active":"2024-04-17T05:46:35.677240+0000","last_peered":"2024-04-17T05:46:35.677240+0000","last_clean":"2024-04-17T05:46:35.677240+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:46:35.677240+0000","last_undegraded":"2024-04-17T05:46:35.677240+0000","last_fullsized":"2024-04-17T05:46:35.677240+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:35.676927+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:35.676927+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.676927+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:40.289079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'3","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.493151+0000","last_change":"2024-04-17T05:46:32.493211+0000","last_active":"2024-04-17T05:46:32.493151+0000","last_peered":"2024-04-17T05:46:32.493151+0000","last_clean":"2024-04-17T05:46:32.493151+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:46:32.493151+0000","last_undegraded":"2024-04-17T05:46:32.493151+0000","last_fullsized":"2024-04-17T05:46:32.493151+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:32.492942+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:32.492942+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.492942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:29.955893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":130,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.592867+0000","last_change":"2024-04-17T05:46:16.593030+0000","last_active":"2024-04-17T05:46:16.592867+0000","last_peered":"2024-04-17T05:46:16.592867+0000","last_clean":"2024-04-17T05:46:16.592867+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:46:16.592867+0000","last_undegraded":"2024-04-17T05:46:16.592867+0000","last_fullsized":"2024-04-17T05:46:16.592867+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_deep_scrub":"264'25","last_deep_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.592614+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:38.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.276684+0000","last_change":"2024-04-17T05:07:45.746904+0000","last_active":"2024-04-17T05:46:08.276684+0000","last_peered":"2024-04-17T05:46:08.276684+0000","last_clean":"2024-04-17T05:46:08.276684+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:46:08.276684+0000","last_undegraded":"2024-04-17T05:46:08.276684+0000","last_fullsized":"2024-04-17T05:46:08.276684+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.760387+0000","last_change":"2024-04-17T05:46:26.760496+0000","last_active":"2024-04-17T05:46:26.760387+0000","last_peered":"2024-04-17T05:46:26.760387+0000","last_clean":"2024-04-17T05:46:26.760387+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:46:26.760387+0000","last_undegraded":"2024-04-17T05:46:26.760387+0000","last_fullsized":"2024-04-17T05:46:26.760387+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:26.760176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:26.760176+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.760176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:38.567211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.770225+0000","last_change":"2024-04-17T05:46:20.770279+0000","last_active":"2024-04-17T05:46:20.770225+0000","last_peered":"2024-04-17T05:46:20.770225+0000","last_clean":"2024-04-17T05:46:20.770225+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:46:20.770225+0000","last_undegraded":"2024-04-17T05:46:20.770225+0000","last_fullsized":"2024-04-17T05:46:20.770225+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.770027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:10.097683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":169,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.814048+0000","last_change":"2024-04-17T05:46:08.814070+0000","last_active":"2024-04-17T05:46:08.814048+0000","last_peered":"2024-04-17T05:46:08.814048+0000","last_clean":"2024-04-17T05:46:08.814048+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:46:08.814048+0000","last_undegraded":"2024-04-17T05:46:08.814048+0000","last_fullsized":"2024-04-17T05:46:08.814048+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'42","last_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_deep_scrub":"264'42","last_deep_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.813969+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:41.013929+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:09.609567+0000","last_change":"2024-04-17T05:47:09.609620+0000","last_active":"2024-04-17T05:47:09.609567+0000","last_peered":"2024-04-17T05:47:09.609567+0000","last_clean":"2024-04-17T05:47:09.609567+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:47:09.609567+0000","last_undegraded":"2024-04-17T05:47:09.609567+0000","last_fullsized":"2024-04-17T05:47:09.609567+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:09.609399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:09.609399+0000","last_clean_scrub_stamp":"2024-04-17T05:47:09.609399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:57:29.808680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'2","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.245317+0000","last_change":"2024-04-17T05:46:53.245387+0000","last_active":"2024-04-17T05:46:53.245317+0000","last_peered":"2024-04-17T05:46:53.245317+0000","last_clean":"2024-04-17T05:46:53.245317+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:46:53.245317+0000","last_undegraded":"2024-04-17T05:46:53.245317+0000","last_fullsized":"2024-04-17T05:46:53.245317+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:53.245100+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:53.245100+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.245100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:30.267498+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.720633+0000","last_change":"2024-04-17T05:46:07.720653+0000","last_active":"2024-04-17T05:46:07.720633+0000","last_peered":"2024-04-17T05:46:07.720633+0000","last_clean":"2024-04-17T05:46:07.720633+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:46:07.720633+0000","last_undegraded":"2024-04-17T05:46:07.720633+0000","last_fullsized":"2024-04-17T05:46:07.720633+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.720559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:04.229433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":209,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.804847+0000","last_change":"2024-04-17T05:46:09.804897+0000","last_active":"2024-04-17T05:46:09.804847+0000","last_peered":"2024-04-17T05:46:09.804847+0000","last_clean":"2024-04-17T05:46:09.804847+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:46:09.804847+0000","last_undegraded":"2024-04-17T05:46:09.804847+0000","last_fullsized":"2024-04-17T05:46:09.804847+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.804697+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:40.852058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.905733+0000","last_change":"2024-04-17T05:46:49.905791+0000","last_active":"2024-04-17T05:46:49.905733+0000","last_peered":"2024-04-17T05:46:49.905733+0000","last_clean":"2024-04-17T05:46:49.905733+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:46:49.905733+0000","last_undegraded":"2024-04-17T05:46:49.905733+0000","last_fullsized":"2024-04-17T05:46:49.905733+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:49.905522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:49.905522+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.905522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:33.396236+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'6","reported_seq":67,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.924479+0000","last_change":"2024-04-17T05:46:41.924541+0000","last_active":"2024-04-17T05:46:41.924479+0000","last_peered":"2024-04-17T05:46:41.924479+0000","last_clean":"2024-04-17T05:46:41.924479+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:46:41.924479+0000","last_undegraded":"2024-04-17T05:46:41.924479+0000","last_fullsized":"2024-04-17T05:46:41.924479+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:41.924290+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:41.924290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.924290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:42.756248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":22528,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"264'6","reported_seq":75,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.770136+0000","last_change":"2024-04-17T05:46:12.770216+0000","last_active":"2024-04-17T05:46:12.770136+0000","last_peered":"2024-04-17T05:46:12.770136+0000","last_clean":"2024-04-17T05:46:12.770136+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:46:12.770136+0000","last_undegraded":"2024-04-17T05:46:12.770136+0000","last_fullsized":"2024-04-17T05:46:12.770136+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.769907+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:23.154229+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'75","reported_seq":293,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.023607+0000","last_change":"2024-04-17T05:46:12.023675+0000","last_active":"2024-04-17T05:46:12.023607+0000","last_peered":"2024-04-17T05:46:12.023607+0000","last_clean":"2024-04-17T05:46:12.023607+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:46:12.023607+0000","last_undegraded":"2024-04-17T05:46:12.023607+0000","last_fullsized":"2024-04-17T05:46:12.023607+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'75","last_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_deep_scrub":"264'75","last_deep_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.023402+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.097898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3255,"num_omap_keys":7,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'2","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:02.337385+0000","last_change":"2024-04-17T05:47:02.337449+0000","last_active":"2024-04-17T05:47:02.337385+0000","last_peered":"2024-04-17T05:47:02.337385+0000","last_clean":"2024-04-17T05:47:02.337385+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:47:02.337385+0000","last_undegraded":"2024-04-17T05:47:02.337385+0000","last_fullsized":"2024-04-17T05:47:02.337385+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:02.337195+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:02.337195+0000","last_clean_scrub_stamp":"2024-04-17T05:47:02.337195+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:34.622453+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.909074+0000","last_change":"2024-04-17T05:46:42.909074+0000","last_active":"2024-04-17T05:46:42.909074+0000","last_peered":"2024-04-17T05:46:42.909074+0000","last_clean":"2024-04-17T05:46:42.909074+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:46:42.909074+0000","last_undegraded":"2024-04-17T05:46:42.909074+0000","last_fullsized":"2024-04-17T05:46:42.909074+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.908818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.908818+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.908818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:22.365739+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'9","reported_seq":77,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.545960+0000","last_change":"2024-04-17T05:46:51.546023+0000","last_active":"2024-04-17T05:46:51.545960+0000","last_peered":"2024-04-17T05:46:51.545960+0000","last_clean":"2024-04-17T05:46:51.545960+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:46:51.545960+0000","last_undegraded":"2024-04-17T05:46:51.545960+0000","last_fullsized":"2024-04-17T05:46:51.545960+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'9","last_scrub_stamp":"2024-04-17T05:46:51.545764+0000","last_deep_scrub":"264'9","last_deep_scrub_stamp":"2024-04-17T05:46:51.545764+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.545764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:40.874309+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":16384,"num_write":24,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":84,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.595282+0000","last_change":"2024-04-17T05:46:14.595378+0000","last_active":"2024-04-17T05:46:14.595282+0000","last_peered":"2024-04-17T05:46:14.595282+0000","last_clean":"2024-04-17T05:46:14.595282+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:46:14.595282+0000","last_undegraded":"2024-04-17T05:46:14.595282+0000","last_fullsized":"2024-04-17T05:46:14.595282+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.594970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:05:26.805897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":36,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:05.383248+0000","last_change":"2024-04-17T05:47:05.383302+0000","last_active":"2024-04-17T05:47:05.383248+0000","last_peered":"2024-04-17T05:47:05.383248+0000","last_clean":"2024-04-17T05:47:05.383248+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:47:05.383248+0000","last_undegraded":"2024-04-17T05:47:05.383248+0000","last_fullsized":"2024-04-17T05:47:05.383248+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:05.383047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:05.383047+0000","last_clean_scrub_stamp":"2024-04-17T05:47:05.383047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:15.434278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:05.709366+0000","last_change":"2024-04-17T05:47:05.709418+0000","last_active":"2024-04-17T05:47:05.709366+0000","last_peered":"2024-04-17T05:47:05.709366+0000","last_clean":"2024-04-17T05:47:05.709366+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:47:05.709366+0000","last_undegraded":"2024-04-17T05:47:05.709366+0000","last_fullsized":"2024-04-17T05:47:05.709366+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:05.709173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:05.709173+0000","last_clean_scrub_stamp":"2024-04-17T05:47:05.709173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:18.346966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.616272+0000","last_change":"2024-04-17T05:46:27.616272+0000","last_active":"2024-04-17T05:46:27.616272+0000","last_peered":"2024-04-17T05:46:27.616272+0000","last_clean":"2024-04-17T05:46:27.616272+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:46:27.616272+0000","last_undegraded":"2024-04-17T05:46:27.616272+0000","last_fullsized":"2024-04-17T05:46:27.616272+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-17T05:46:27.615997+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-17T05:46:27.615997+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.615997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:21:10.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":99,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.712592+0000","last_change":"2024-04-17T05:46:11.712658+0000","last_active":"2024-04-17T05:46:11.712592+0000","last_peered":"2024-04-17T05:46:11.712592+0000","last_clean":"2024-04-17T05:46:11.712592+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:46:11.712592+0000","last_undegraded":"2024-04-17T05:46:11.712592+0000","last_fullsized":"2024-04-17T05:46:11.712592+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'13","last_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_deep_scrub":"264'13","last_deep_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.712398+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:12.832777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.918864+0000","last_change":"2024-04-17T05:46:48.918919+0000","last_active":"2024-04-17T05:46:48.918864+0000","last_peered":"2024-04-17T05:46:48.918864+0000","last_clean":"2024-04-17T05:46:48.918864+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:46:48.918864+0000","last_undegraded":"2024-04-17T05:46:48.918864+0000","last_fullsized":"2024-04-17T05:46:48.918864+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:48.918658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:48.918658+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.918658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:35:22.598547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.794712+0000","last_change":"2024-04-17T05:46:18.794764+0000","last_active":"2024-04-17T05:46:18.794712+0000","last_peered":"2024-04-17T05:46:18.794712+0000","last_clean":"2024-04-17T05:46:18.794712+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:46:18.794712+0000","last_undegraded":"2024-04-17T05:46:18.794712+0000","last_fullsized":"2024-04-17T05:46:18.794712+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.794517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:29:12.950953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.225008+0000","last_change":"2024-04-17T05:46:24.225059+0000","last_active":"2024-04-17T05:46:24.225008+0000","last_peered":"2024-04-17T05:46:24.225008+0000","last_clean":"2024-04-17T05:46:24.225008+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:46:24.225008+0000","last_undegraded":"2024-04-17T05:46:24.225008+0000","last_fullsized":"2024-04-17T05:46:24.225008+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.224809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:16.454241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":89,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.623954+0000","last_change":"2024-04-17T05:46:12.624024+0000","last_active":"2024-04-17T05:46:12.623954+0000","last_peered":"2024-04-17T05:46:12.623954+0000","last_clean":"2024-04-17T05:46:12.623954+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:46:12.623954+0000","last_undegraded":"2024-04-17T05:46:12.623954+0000","last_fullsized":"2024-04-17T05:46:12.623954+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.623741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:56:20.441738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'2","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.529278+0000","last_change":"2024-04-17T05:46:36.529348+0000","last_active":"2024-04-17T05:46:36.529278+0000","last_peered":"2024-04-17T05:46:36.529278+0000","last_clean":"2024-04-17T05:46:36.529278+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:46:36.529278+0000","last_undegraded":"2024-04-17T05:46:36.529278+0000","last_fullsized":"2024-04-17T05:46:36.529278+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:36.529078+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:36.529078+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.529078+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:48.549685+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.829356+0000","last_change":"2024-04-17T05:46:34.829509+0000","last_active":"2024-04-17T05:46:34.829356+0000","last_peered":"2024-04-17T05:46:34.829356+0000","last_clean":"2024-04-17T05:46:34.829356+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:46:34.829356+0000","last_undegraded":"2024-04-17T05:46:34.829356+0000","last_fullsized":"2024-04-17T05:46:34.829356+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:34.829120+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:34.829120+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.829120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:21.304145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'2","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.129624+0000","last_change":"2024-04-17T05:46:11.129787+0000","last_active":"2024-04-17T05:46:11.129624+0000","last_peered":"2024-04-17T05:46:11.129624+0000","last_clean":"2024-04-17T05:46:11.129624+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:46:11.129624+0000","last_undegraded":"2024-04-17T05:46:11.129624+0000","last_fullsized":"2024-04-17T05:46:11.129624+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.129393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:45.300258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":125,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.852514+0000","last_change":"2024-04-17T05:46:11.852573+0000","last_active":"2024-04-17T05:46:11.852514+0000","last_peered":"2024-04-17T05:46:11.852514+0000","last_clean":"2024-04-17T05:46:11.852514+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:46:11.852514+0000","last_undegraded":"2024-04-17T05:46:11.852514+0000","last_fullsized":"2024-04-17T05:46:11.852514+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.852327+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:08.854562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.319843+0000","last_change":"2024-04-17T05:46:32.319899+0000","last_active":"2024-04-17T05:46:32.319843+0000","last_peered":"2024-04-17T05:46:32.319843+0000","last_clean":"2024-04-17T05:46:32.319843+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:46:32.319843+0000","last_undegraded":"2024-04-17T05:46:32.319843+0000","last_fullsized":"2024-04-17T05:46:32.319843+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.319638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.319638+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.319638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:10.301900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'5","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.734077+0000","last_change":"2024-04-17T05:46:22.734137+0000","last_active":"2024-04-17T05:46:22.734077+0000","last_peered":"2024-04-17T05:46:22.734077+0000","last_clean":"2024-04-17T05:46:22.734077+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:46:22.734077+0000","last_undegraded":"2024-04-17T05:46:22.734077+0000","last_fullsized":"2024-04-17T05:46:22.734077+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.733832+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:04.087293+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":48,"num_write":10,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'14","reported_seq":91,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.511830+0000","last_change":"2024-04-17T05:46:33.511896+0000","last_active":"2024-04-17T05:46:33.511830+0000","last_peered":"2024-04-17T05:46:33.511830+0000","last_clean":"2024-04-17T05:46:33.511830+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:46:33.511830+0000","last_undegraded":"2024-04-17T05:46:33.511830+0000","last_fullsized":"2024-04-17T05:46:33.511830+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'14","last_scrub_stamp":"2024-04-17T05:46:33.511695+0000","last_deep_scrub":"264'14","last_deep_scrub_stamp":"2024-04-17T05:46:33.511695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.511695+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:13.607331+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":46082,"num_write":17,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.547405+0000","last_change":"2024-04-17T05:46:18.547463+0000","last_active":"2024-04-17T05:46:18.547405+0000","last_peered":"2024-04-17T05:46:18.547405+0000","last_clean":"2024-04-17T05:46:18.547405+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:46:18.547405+0000","last_undegraded":"2024-04-17T05:46:18.547405+0000","last_fullsized":"2024-04-17T05:46:18.547405+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'18","last_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_deep_scrub":"261'18","last_deep_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.547198+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:22.594305+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.186308+0000","last_change":"2024-04-17T05:46:56.186371+0000","last_active":"2024-04-17T05:46:56.186308+0000","last_peered":"2024-04-17T05:46:56.186308+0000","last_clean":"2024-04-17T05:46:56.186308+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:46:56.186308+0000","last_undegraded":"2024-04-17T05:46:56.186308+0000","last_fullsized":"2024-04-17T05:46:56.186308+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:56.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:56.186078+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.186078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:51:49.151800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'12","reported_seq":82,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.644680+0000","last_change":"2024-04-17T05:46:56.644776+0000","last_active":"2024-04-17T05:46:56.644680+0000","last_peered":"2024-04-17T05:46:56.644680+0000","last_clean":"2024-04-17T05:46:56.644680+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:46:56.644680+0000","last_undegraded":"2024-04-17T05:46:56.644680+0000","last_fullsized":"2024-04-17T05:46:56.644680+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-04-17T05:46:56.644490+0000","last_deep_scrub":"264'12","last_deep_scrub_stamp":"2024-04-17T05:46:56.644490+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.644490+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:25:56.759217+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":37890,"num_write":20,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.607535+0000","last_change":"2024-04-17T05:46:23.607603+0000","last_active":"2024-04-17T05:46:23.607535+0000","last_peered":"2024-04-17T05:46:23.607535+0000","last_clean":"2024-04-17T05:46:23.607535+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:46:23.607535+0000","last_undegraded":"2024-04-17T05:46:23.607535+0000","last_fullsized":"2024-04-17T05:46:23.607535+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.607294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:20.141964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":221,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.194774+0000","last_change":"2024-04-17T05:46:18.194774+0000","last_active":"2024-04-17T05:46:18.194774+0000","last_peered":"2024-04-17T05:46:18.194774+0000","last_clean":"2024-04-17T05:46:18.194774+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:46:18.194774+0000","last_undegraded":"2024-04-17T05:46:18.194774+0000","last_fullsized":"2024-04-17T05:46:18.194774+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'65","last_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_deep_scrub":"264'65","last_deep_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.194519+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:31.405244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":29,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.277147+0000","last_change":"2024-04-17T05:07:45.749399+0000","last_active":"2024-04-17T05:46:08.277147+0000","last_peered":"2024-04-17T05:46:08.277147+0000","last_clean":"2024-04-17T05:46:08.277147+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:46:08.277147+0000","last_undegraded":"2024-04-17T05:46:08.277147+0000","last_fullsized":"2024-04-17T05:46:08.277147+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:07:44.680118+0000","last_clean_scrub_stamp":"2024-04-17T05:07:44.680118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"264'6","reported_seq":70,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.915407+0000","last_change":"2024-04-17T05:46:45.915482+0000","last_active":"2024-04-17T05:46:45.915407+0000","last_peered":"2024-04-17T05:46:45.915407+0000","last_clean":"2024-04-17T05:46:45.915407+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:46:45.915407+0000","last_undegraded":"2024-04-17T05:46:45.915407+0000","last_fullsized":"2024-04-17T05:46:45.915407+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:45.915212+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:45.915212+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.915212+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:02.071375+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.807390+0000","last_change":"2024-04-17T05:46:16.807574+0000","last_active":"2024-04-17T05:46:16.807390+0000","last_peered":"2024-04-17T05:46:16.807390+0000","last_clean":"2024-04-17T05:46:16.807390+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:46:16.807390+0000","last_undegraded":"2024-04-17T05:46:16.807390+0000","last_fullsized":"2024-04-17T05:46:16.807390+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.807094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:57.818718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":157,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.309760+0000","last_change":"2024-04-17T05:46:22.309837+0000","last_active":"2024-04-17T05:46:22.309760+0000","last_peered":"2024-04-17T05:46:22.309760+0000","last_clean":"2024-04-17T05:46:22.309760+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:46:22.309760+0000","last_undegraded":"2024-04-17T05:46:22.309760+0000","last_fullsized":"2024-04-17T05:46:22.309760+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.309548+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:29.733950+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:07.562430+0000","last_change":"2024-04-17T05:47:07.562483+0000","last_active":"2024-04-17T05:47:07.562430+0000","last_peered":"2024-04-17T05:47:07.562430+0000","last_clean":"2024-04-17T05:47:07.562430+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:47:07.562430+0000","last_undegraded":"2024-04-17T05:47:07.562430+0000","last_fullsized":"2024-04-17T05:47:07.562430+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:07.562220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:07.562220+0000","last_clean_scrub_stamp":"2024-04-17T05:47:07.562220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:16:08.871059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'2","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.726875+0000","last_change":"2024-04-17T05:46:16.727020+0000","last_active":"2024-04-17T05:46:16.726875+0000","last_peered":"2024-04-17T05:46:16.726875+0000","last_clean":"2024-04-17T05:46:16.726875+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:46:16.726875+0000","last_undegraded":"2024-04-17T05:46:16.726875+0000","last_fullsized":"2024-04-17T05:46:16.726875+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.726644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:09.817384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.831759+0000","last_change":"2024-04-17T05:46:10.831811+0000","last_active":"2024-04-17T05:46:10.831759+0000","last_peered":"2024-04-17T05:46:10.831759+0000","last_clean":"2024-04-17T05:46:10.831759+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:46:10.831759+0000","last_undegraded":"2024-04-17T05:46:10.831759+0000","last_fullsized":"2024-04-17T05:46:10.831759+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.831558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:46.336010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":142,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.670845+0000","last_change":"2024-04-17T05:46:09.670845+0000","last_active":"2024-04-17T05:46:09.670845+0000","last_peered":"2024-04-17T05:46:09.670845+0000","last_clean":"2024-04-17T05:46:09.670845+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:46:09.670845+0000","last_undegraded":"2024-04-17T05:46:09.670845+0000","last_fullsized":"2024-04-17T05:46:09.670845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.670593+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:59.152267+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'2","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:06.590495+0000","last_change":"2024-04-17T05:47:06.590561+0000","last_active":"2024-04-17T05:47:06.590495+0000","last_peered":"2024-04-17T05:47:06.590495+0000","last_clean":"2024-04-17T05:47:06.590495+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:47:06.590495+0000","last_undegraded":"2024-04-17T05:47:06.590495+0000","last_fullsized":"2024-04-17T05:47:06.590495+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:06.590290+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:06.590290+0000","last_clean_scrub_stamp":"2024-04-17T05:47:06.590290+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:12.967635+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.712424+0000","last_change":"2024-04-17T05:46:38.712582+0000","last_active":"2024-04-17T05:46:38.712424+0000","last_peered":"2024-04-17T05:46:38.712424+0000","last_clean":"2024-04-17T05:46:38.712424+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:46:38.712424+0000","last_undegraded":"2024-04-17T05:46:38.712424+0000","last_fullsized":"2024-04-17T05:46:38.712424+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:38.712195+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:38.712195+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.712195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:58.037440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'5","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.032735+0000","last_change":"2024-04-17T05:46:14.032796+0000","last_active":"2024-04-17T05:46:14.032735+0000","last_peered":"2024-04-17T05:46:14.032735+0000","last_clean":"2024-04-17T05:46:14.032735+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:46:14.032735+0000","last_undegraded":"2024-04-17T05:46:14.032735+0000","last_fullsized":"2024-04-17T05:46:14.032735+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.032549+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:46.453310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":111,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.793130+0000","last_change":"2024-04-17T05:46:09.793130+0000","last_active":"2024-04-17T05:46:09.793130+0000","last_peered":"2024-04-17T05:46:09.793130+0000","last_clean":"2024-04-17T05:46:09.793130+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:46:09.793130+0000","last_undegraded":"2024-04-17T05:46:09.793130+0000","last_fullsized":"2024-04-17T05:46:09.793130+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.792865+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:12:51.707714+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'2","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:07.664030+0000","last_change":"2024-04-17T05:47:07.664098+0000","last_active":"2024-04-17T05:47:07.664030+0000","last_peered":"2024-04-17T05:47:07.664030+0000","last_clean":"2024-04-17T05:47:07.664030+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:47:07.664030+0000","last_undegraded":"2024-04-17T05:47:07.664030+0000","last_fullsized":"2024-04-17T05:47:07.664030+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:07.663845+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:07.663845+0000","last_clean_scrub_stamp":"2024-04-17T05:47:07.663845+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:39:01.427821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.871034+0000","last_change":"2024-04-17T05:46:43.871089+0000","last_active":"2024-04-17T05:46:43.871034+0000","last_peered":"2024-04-17T05:46:43.871034+0000","last_clean":"2024-04-17T05:46:43.871034+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:46:43.871034+0000","last_undegraded":"2024-04-17T05:46:43.871034+0000","last_fullsized":"2024-04-17T05:46:43.871034+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.870838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.870838+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.870838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:23.777410+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.134683+0000","last_change":"2024-04-17T05:46:20.134745+0000","last_active":"2024-04-17T05:46:20.134683+0000","last_peered":"2024-04-17T05:46:20.134683+0000","last_clean":"2024-04-17T05:46:20.134683+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:46:20.134683+0000","last_undegraded":"2024-04-17T05:46:20.134683+0000","last_fullsized":"2024-04-17T05:46:20.134683+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.134488+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:48.392260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.861464+0000","last_change":"2024-04-17T05:46:13.861464+0000","last_active":"2024-04-17T05:46:13.861464+0000","last_peered":"2024-04-17T05:46:13.861464+0000","last_clean":"2024-04-17T05:46:13.861464+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:46:13.861464+0000","last_undegraded":"2024-04-17T05:46:13.861464+0000","last_fullsized":"2024-04-17T05:46:13.861464+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'55","last_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_deep_scrub":"264'55","last_deep_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.861056+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:57.160789+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.568563+0000","last_change":"2024-04-17T05:46:37.568563+0000","last_active":"2024-04-17T05:46:37.568563+0000","last_peered":"2024-04-17T05:46:37.568563+0000","last_clean":"2024-04-17T05:46:37.568563+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:46:37.568563+0000","last_undegraded":"2024-04-17T05:46:37.568563+0000","last_fullsized":"2024-04-17T05:46:37.568563+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:37.568302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:37.568302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.568302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:25.004821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'6","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.841237+0000","last_change":"2024-04-17T05:46:30.841297+0000","last_active":"2024-04-17T05:46:30.841237+0000","last_peered":"2024-04-17T05:46:30.841237+0000","last_clean":"2024-04-17T05:46:30.841237+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:46:30.841237+0000","last_undegraded":"2024-04-17T05:46:30.841237+0000","last_fullsized":"2024-04-17T05:46:30.841237+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:30.841048+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:30.841048+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.841048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:11.067215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":11,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.170964+0000","last_change":"2024-04-17T05:46:10.171028+0000","last_active":"2024-04-17T05:46:10.170964+0000","last_peered":"2024-04-17T05:46:10.170964+0000","last_clean":"2024-04-17T05:46:10.170964+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:46:10.170964+0000","last_undegraded":"2024-04-17T05:46:10.170964+0000","last_fullsized":"2024-04-17T05:46:10.170964+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.170766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:36.494972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.881133+0000","last_change":"2024-04-17T05:46:12.881133+0000","last_active":"2024-04-17T05:46:12.881133+0000","last_peered":"2024-04-17T05:46:12.881133+0000","last_clean":"2024-04-17T05:46:12.881133+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:46:12.881133+0000","last_undegraded":"2024-04-17T05:46:12.881133+0000","last_fullsized":"2024-04-17T05:46:12.881133+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.880856+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:56.211415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:59.267104+0000","last_change":"2024-04-17T05:46:59.267210+0000","last_active":"2024-04-17T05:46:59.267104+0000","last_peered":"2024-04-17T05:46:59.267104+0000","last_clean":"2024-04-17T05:46:59.267104+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:46:59.267104+0000","last_undegraded":"2024-04-17T05:46:59.267104+0000","last_fullsized":"2024-04-17T05:46:59.267104+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:59.266869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:59.266869+0000","last_clean_scrub_stamp":"2024-04-17T05:46:59.266869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:30.878390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:06.670567+0000","last_change":"2024-04-17T05:47:06.670625+0000","last_active":"2024-04-17T05:47:06.670567+0000","last_peered":"2024-04-17T05:47:06.670567+0000","last_clean":"2024-04-17T05:47:06.670567+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:47:06.670567+0000","last_undegraded":"2024-04-17T05:47:06.670567+0000","last_fullsized":"2024-04-17T05:47:06.670567+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:47:06.670372+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:47:06.670372+0000","last_clean_scrub_stamp":"2024-04-17T05:47:06.670372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:47:47.401249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'5","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.505904+0000","last_change":"2024-04-17T05:46:34.506036+0000","last_active":"2024-04-17T05:46:34.505904+0000","last_peered":"2024-04-17T05:46:34.505904+0000","last_clean":"2024-04-17T05:46:34.505904+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:46:34.505904+0000","last_undegraded":"2024-04-17T05:46:34.505904+0000","last_fullsized":"2024-04-17T05:46:34.505904+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:34.505613+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:34.505613+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.505613+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:13.411372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":11264,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":163,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.131285+0000","last_change":"2024-04-17T05:46:16.131285+0000","last_active":"2024-04-17T05:46:16.131285+0000","last_peered":"2024-04-17T05:46:16.131285+0000","last_clean":"2024-04-17T05:46:16.131285+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:46:16.131285+0000","last_undegraded":"2024-04-17T05:46:16.131285+0000","last_fullsized":"2024-04-17T05:46:16.131285+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.131038+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:32:33.132984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.896222+0000","last_change":"2024-04-17T05:46:46.896279+0000","last_active":"2024-04-17T05:46:46.896222+0000","last_peered":"2024-04-17T05:46:46.896222+0000","last_clean":"2024-04-17T05:46:46.896222+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:46:46.896222+0000","last_undegraded":"2024-04-17T05:46:46.896222+0000","last_fullsized":"2024-04-17T05:46:46.896222+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:46.896013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:46.896013+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.896013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:52.872792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.908271+0000","last_change":"2024-04-17T05:46:44.908338+0000","last_active":"2024-04-17T05:46:44.908271+0000","last_peered":"2024-04-17T05:46:44.908271+0000","last_clean":"2024-04-17T05:46:44.908271+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:46:44.908271+0000","last_undegraded":"2024-04-17T05:46:44.908271+0000","last_fullsized":"2024-04-17T05:46:44.908271+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:44.908063+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:44.908063+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.908063+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:38.209073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.674045+0000","last_change":"2024-04-17T05:46:19.674109+0000","last_active":"2024-04-17T05:46:19.674045+0000","last_peered":"2024-04-17T05:46:19.674045+0000","last_clean":"2024-04-17T05:46:19.674045+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:46:19.674045+0000","last_undegraded":"2024-04-17T05:46:19.674045+0000","last_fullsized":"2024-04-17T05:46:19.674045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.673832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:20:55.342567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.057017+0000","last_change":"2024-04-17T05:46:11.057017+0000","last_active":"2024-04-17T05:46:11.057017+0000","last_peered":"2024-04-17T05:46:11.057017+0000","last_clean":"2024-04-17T05:46:11.057017+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:46:11.057017+0000","last_undegraded":"2024-04-17T05:46:11.057017+0000","last_fullsized":"2024-04-17T05:46:11.057017+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.056723+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:04.980611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:03.336353+0000","last_change":"2024-04-17T05:47:03.336408+0000","last_active":"2024-04-17T05:47:03.336353+0000","last_peered":"2024-04-17T05:47:03.336353+0000","last_clean":"2024-04-17T05:47:03.336353+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:47:03.336353+0000","last_undegraded":"2024-04-17T05:47:03.336353+0000","last_fullsized":"2024-04-17T05:47:03.336353+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:03.336113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:03.336113+0000","last_clean_scrub_stamp":"2024-04-17T05:47:03.336113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:21.079049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:03.576340+0000","last_change":"2024-04-17T05:47:03.576404+0000","last_active":"2024-04-17T05:47:03.576340+0000","last_peered":"2024-04-17T05:47:03.576340+0000","last_clean":"2024-04-17T05:47:03.576340+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:47:03.576340+0000","last_undegraded":"2024-04-17T05:47:03.576340+0000","last_fullsized":"2024-04-17T05:47:03.576340+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:47:03.576128+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:47:03.576128+0000","last_clean_scrub_stamp":"2024-04-17T05:47:03.576128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:34.896558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.680401+0000","last_change":"2024-04-17T05:46:08.680438+0000","last_active":"2024-04-17T05:46:08.680401+0000","last_peered":"2024-04-17T05:46:08.680401+0000","last_clean":"2024-04-17T05:46:08.680401+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:46:08.680401+0000","last_undegraded":"2024-04-17T05:46:08.680401+0000","last_fullsized":"2024-04-17T05:46:08.680401+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.680268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:03.743450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":138,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.494346+0000","last_change":"2024-04-17T05:46:22.494346+0000","last_active":"2024-04-17T05:46:22.494346+0000","last_peered":"2024-04-17T05:46:22.494346+0000","last_clean":"2024-04-17T05:46:22.494346+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:46:22.494346+0000","last_undegraded":"2024-04-17T05:46:22.494346+0000","last_fullsized":"2024-04-17T05:46:22.494346+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_deep_scrub":"264'30","last_deep_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.494058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:08.498830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:08.637373+0000","last_change":"2024-04-17T05:47:08.637431+0000","last_active":"2024-04-17T05:47:08.637373+0000","last_peered":"2024-04-17T05:47:08.637373+0000","last_clean":"2024-04-17T05:47:08.637373+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:47:08.637373+0000","last_undegraded":"2024-04-17T05:47:08.637373+0000","last_fullsized":"2024-04-17T05:47:08.637373+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:08.637164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:08.637164+0000","last_clean_scrub_stamp":"2024-04-17T05:47:08.637164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:58.059260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'7","reported_seq":69,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.601250+0000","last_change":"2024-04-17T05:46:54.601315+0000","last_active":"2024-04-17T05:46:54.601250+0000","last_peered":"2024-04-17T05:46:54.601250+0000","last_clean":"2024-04-17T05:46:54.601250+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:46:54.601250+0000","last_undegraded":"2024-04-17T05:46:54.601250+0000","last_fullsized":"2024-04-17T05:46:54.601250+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-04-17T05:46:54.601024+0000","last_deep_scrub":"264'7","last_deep_scrub_stamp":"2024-04-17T05:46:54.601024+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.601024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:01.709982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":5216,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.623849+0000","last_change":"2024-04-17T05:46:11.623908+0000","last_active":"2024-04-17T05:46:11.623849+0000","last_peered":"2024-04-17T05:46:11.623849+0000","last_clean":"2024-04-17T05:46:11.623849+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:46:11.623849+0000","last_undegraded":"2024-04-17T05:46:11.623849+0000","last_fullsized":"2024-04-17T05:46:11.623849+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.623642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:23.552160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":112,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.642068+0000","last_change":"2024-04-17T05:46:10.642068+0000","last_active":"2024-04-17T05:46:10.642068+0000","last_peered":"2024-04-17T05:46:10.642068+0000","last_clean":"2024-04-17T05:46:10.642068+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:46:10.642068+0000","last_undegraded":"2024-04-17T05:46:10.642068+0000","last_fullsized":"2024-04-17T05:46:10.642068+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.641783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:09.385082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:04.367142+0000","last_change":"2024-04-17T05:47:04.367201+0000","last_active":"2024-04-17T05:47:04.367142+0000","last_peered":"2024-04-17T05:47:04.367142+0000","last_clean":"2024-04-17T05:47:04.367142+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:47:04.367142+0000","last_undegraded":"2024-04-17T05:47:04.367142+0000","last_fullsized":"2024-04-17T05:47:04.367142+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:04.366926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:04.366926+0000","last_clean_scrub_stamp":"2024-04-17T05:47:04.366926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:49.317729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.167379+0000","last_change":"2024-04-17T05:46:16.167438+0000","last_active":"2024-04-17T05:46:16.167379+0000","last_peered":"2024-04-17T05:46:16.167379+0000","last_clean":"2024-04-17T05:46:16.167379+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:46:16.167379+0000","last_undegraded":"2024-04-17T05:46:16.167379+0000","last_fullsized":"2024-04-17T05:46:16.167379+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.167190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:00.500071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.682742+0000","last_change":"2024-04-17T05:46:07.682778+0000","last_active":"2024-04-17T05:46:07.682742+0000","last_peered":"2024-04-17T05:46:07.682742+0000","last_clean":"2024-04-17T05:46:07.682742+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:46:07.682742+0000","last_undegraded":"2024-04-17T05:46:07.682742+0000","last_fullsized":"2024-04-17T05:46:07.682742+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.682661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:07.614824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.265908+0000","last_change":"2024-04-17T05:46:21.265908+0000","last_active":"2024-04-17T05:46:21.265908+0000","last_peered":"2024-04-17T05:46:21.265908+0000","last_clean":"2024-04-17T05:46:21.265908+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:46:21.265908+0000","last_undegraded":"2024-04-17T05:46:21.265908+0000","last_fullsized":"2024-04-17T05:46:21.265908+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'93","last_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_deep_scrub":"264'93","last_deep_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.265637+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:31.736633+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'2","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:01.316689+0000","last_change":"2024-04-17T05:47:01.316763+0000","last_active":"2024-04-17T05:47:01.316689+0000","last_peered":"2024-04-17T05:47:01.316689+0000","last_clean":"2024-04-17T05:47:01.316689+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:47:01.316689+0000","last_undegraded":"2024-04-17T05:47:01.316689+0000","last_fullsized":"2024-04-17T05:47:01.316689+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:01.316494+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:01.316494+0000","last_clean_scrub_stamp":"2024-04-17T05:47:01.316494+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:46.857428+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.884070+0000","last_change":"2024-04-17T05:46:43.884127+0000","last_active":"2024-04-17T05:46:43.884070+0000","last_peered":"2024-04-17T05:46:43.884070+0000","last_clean":"2024-04-17T05:46:43.884070+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:46:43.884070+0000","last_undegraded":"2024-04-17T05:46:43.884070+0000","last_fullsized":"2024-04-17T05:46:43.884070+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.883876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.883876+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.883876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:29.135926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.584946+0000","last_change":"2024-04-17T05:46:15.585004+0000","last_active":"2024-04-17T05:46:15.584946+0000","last_peered":"2024-04-17T05:46:15.584946+0000","last_clean":"2024-04-17T05:46:15.584946+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:46:15.584946+0000","last_undegraded":"2024-04-17T05:46:15.584946+0000","last_fullsized":"2024-04-17T05:46:15.584946+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.584695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:18.890015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.833951+0000","last_change":"2024-04-17T05:46:15.833951+0000","last_active":"2024-04-17T05:46:15.833951+0000","last_peered":"2024-04-17T05:46:15.833951+0000","last_clean":"2024-04-17T05:46:15.833951+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:46:15.833951+0000","last_undegraded":"2024-04-17T05:46:15.833951+0000","last_fullsized":"2024-04-17T05:46:15.833951+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'22","last_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_deep_scrub":"264'22","last_deep_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.833697+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:34:19.233513+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:05.601118+0000","last_change":"2024-04-17T05:47:05.601170+0000","last_active":"2024-04-17T05:47:05.601118+0000","last_peered":"2024-04-17T05:47:05.601118+0000","last_clean":"2024-04-17T05:47:05.601118+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:47:05.601118+0000","last_undegraded":"2024-04-17T05:47:05.601118+0000","last_fullsized":"2024-04-17T05:47:05.601118+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:05.600917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:05.600917+0000","last_clean_scrub_stamp":"2024-04-17T05:47:05.600917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:37.604006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.268742+0000","last_change":"2024-04-17T05:46:20.268796+0000","last_active":"2024-04-17T05:46:20.268742+0000","last_peered":"2024-04-17T05:46:20.268742+0000","last_clean":"2024-04-17T05:46:20.268742+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:46:20.268742+0000","last_undegraded":"2024-04-17T05:46:20.268742+0000","last_fullsized":"2024-04-17T05:46:20.268742+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.268538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:35.180807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.745329+0000","last_change":"2024-04-17T05:46:24.745393+0000","last_active":"2024-04-17T05:46:24.745329+0000","last_peered":"2024-04-17T05:46:24.745329+0000","last_clean":"2024-04-17T05:46:24.745329+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:46:24.745329+0000","last_undegraded":"2024-04-17T05:46:24.745329+0000","last_fullsized":"2024-04-17T05:46:24.745329+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.745126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:13.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":198,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.022550+0000","last_change":"2024-04-17T05:46:10.022550+0000","last_active":"2024-04-17T05:46:10.022550+0000","last_peered":"2024-04-17T05:46:10.022550+0000","last_clean":"2024-04-17T05:46:10.022550+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:46:10.022550+0000","last_undegraded":"2024-04-17T05:46:10.022550+0000","last_fullsized":"2024-04-17T05:46:10.022550+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'52","last_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_deep_scrub":"264'52","last_deep_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.022290+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.993887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.273662+0000","last_change":"2024-04-17T05:46:28.273717+0000","last_active":"2024-04-17T05:46:28.273662+0000","last_peered":"2024-04-17T05:46:28.273662+0000","last_clean":"2024-04-17T05:46:28.273662+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:46:28.273662+0000","last_undegraded":"2024-04-17T05:46:28.273662+0000","last_fullsized":"2024-04-17T05:46:28.273662+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.273435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.273435+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.273435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:53.897551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:02.564850+0000","last_change":"2024-04-17T05:47:02.565018+0000","last_active":"2024-04-17T05:47:02.564850+0000","last_peered":"2024-04-17T05:47:02.564850+0000","last_clean":"2024-04-17T05:47:02.564850+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:47:02.564850+0000","last_undegraded":"2024-04-17T05:47:02.564850+0000","last_fullsized":"2024-04-17T05:47:02.564850+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-17T05:47:02.564560+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-17T05:47:02.564560+0000","last_clean_scrub_stamp":"2024-04-17T05:47:02.564560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:33.650712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'8","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.163258+0000","last_change":"2024-04-17T05:46:15.163324+0000","last_active":"2024-04-17T05:46:15.163258+0000","last_peered":"2024-04-17T05:46:15.163258+0000","last_clean":"2024-04-17T05:46:15.163258+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:46:15.163258+0000","last_undegraded":"2024-04-17T05:46:15.163258+0000","last_fullsized":"2024-04-17T05:46:15.163258+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.163041+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:18.146680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11291,"num_write":13,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":159,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.612312+0000","last_change":"2024-04-17T05:46:13.612312+0000","last_active":"2024-04-17T05:46:13.612312+0000","last_peered":"2024-04-17T05:46:13.612312+0000","last_clean":"2024-04-17T05:46:13.612312+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:46:13.612312+0000","last_undegraded":"2024-04-17T05:46:13.612312+0000","last_fullsized":"2024-04-17T05:46:13.612312+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'39","last_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_deep_scrub":"264'39","last_deep_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.612027+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:28.378268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.189296+0000","last_change":"2024-04-17T05:46:57.189351+0000","last_active":"2024-04-17T05:46:57.189296+0000","last_peered":"2024-04-17T05:46:57.189296+0000","last_clean":"2024-04-17T05:46:57.189296+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:46:57.189296+0000","last_undegraded":"2024-04-17T05:46:57.189296+0000","last_fullsized":"2024-04-17T05:46:57.189296+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:57.189086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:57.189086+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.189086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:22.548179+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.614461+0000","last_change":"2024-04-17T05:46:31.614516+0000","last_active":"2024-04-17T05:46:31.614461+0000","last_peered":"2024-04-17T05:46:31.614461+0000","last_clean":"2024-04-17T05:46:31.614461+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:46:31.614461+0000","last_undegraded":"2024-04-17T05:46:31.614461+0000","last_fullsized":"2024-04-17T05:46:31.614461+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.614267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.614267+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.614267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:54:03.860590+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.554323+0000","last_change":"2024-04-17T05:46:35.554396+0000","last_active":"2024-04-17T05:46:35.554323+0000","last_peered":"2024-04-17T05:46:35.554323+0000","last_clean":"2024-04-17T05:46:35.554323+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:46:35.554323+0000","last_undegraded":"2024-04-17T05:46:35.554323+0000","last_fullsized":"2024-04-17T05:46:35.554323+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:35.554116+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:35.554116+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.554116+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:28.363602+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":206,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.995853+0000","last_change":"2024-04-17T05:46:14.995853+0000","last_active":"2024-04-17T05:46:14.995853+0000","last_peered":"2024-04-17T05:46:14.995853+0000","last_clean":"2024-04-17T05:46:14.995853+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:46:14.995853+0000","last_undegraded":"2024-04-17T05:46:14.995853+0000","last_fullsized":"2024-04-17T05:46:14.995853+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'60","last_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_deep_scrub":"264'60","last_deep_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.995600+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:08.616032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.964095+0000","last_change":"2024-04-17T05:46:47.964149+0000","last_active":"2024-04-17T05:46:47.964095+0000","last_peered":"2024-04-17T05:46:47.964095+0000","last_clean":"2024-04-17T05:46:47.964095+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:46:47.964095+0000","last_undegraded":"2024-04-17T05:46:47.964095+0000","last_fullsized":"2024-04-17T05:46:47.964095+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:47.963889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:47.963889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.963889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:33.636904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'21","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.298945+0000","last_change":"2024-04-17T05:46:24.299018+0000","last_active":"2024-04-17T05:46:24.298945+0000","last_peered":"2024-04-17T05:46:24.298945+0000","last_clean":"2024-04-17T05:46:24.298945+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:46:24.298945+0000","last_undegraded":"2024-04-17T05:46:24.298945+0000","last_fullsized":"2024-04-17T05:46:24.298945+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'21","last_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_deep_scrub":"264'21","last_deep_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.298719+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:55.830264+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":54274,"num_write":19,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"264'12","reported_seq":90,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.319932+0000","last_change":"2024-04-17T05:46:51.320013+0000","last_active":"2024-04-17T05:46:51.319932+0000","last_peered":"2024-04-17T05:46:51.319932+0000","last_clean":"2024-04-17T05:46:51.319932+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:46:51.319932+0000","last_undegraded":"2024-04-17T05:46:51.319932+0000","last_fullsized":"2024-04-17T05:46:51.319932+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-04-17T05:46:51.319740+0000","last_deep_scrub":"264'12","last_deep_scrub_stamp":"2024-04-17T05:46:51.319740+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.319740+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:47.902459+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":233,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.186314+0000","last_change":"2024-04-17T05:46:17.186314+0000","last_active":"2024-04-17T05:46:17.186314+0000","last_peered":"2024-04-17T05:46:17.186314+0000","last_clean":"2024-04-17T05:46:17.186314+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:46:17.186314+0000","last_undegraded":"2024-04-17T05:46:17.186314+0000","last_fullsized":"2024-04-17T05:46:17.186314+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.186068+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:32.734141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.652932+0000","last_change":"2024-04-17T05:46:57.652989+0000","last_active":"2024-04-17T05:46:57.652932+0000","last_peered":"2024-04-17T05:46:57.652932+0000","last_clean":"2024-04-17T05:46:57.652932+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:46:57.652932+0000","last_undegraded":"2024-04-17T05:46:57.652932+0000","last_fullsized":"2024-04-17T05:46:57.652932+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:57.652734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:57.652734+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.652734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:47:27.873329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.527408+0000","last_change":"2024-04-17T05:46:52.527462+0000","last_active":"2024-04-17T05:46:52.527408+0000","last_peered":"2024-04-17T05:46:52.527408+0000","last_clean":"2024-04-17T05:46:52.527408+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:46:52.527408+0000","last_undegraded":"2024-04-17T05:46:52.527408+0000","last_fullsized":"2024-04-17T05:46:52.527408+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:52.527195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:52.527195+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.527195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:45.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'6","reported_seq":73,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.099801+0000","last_change":"2024-04-17T05:46:12.099865+0000","last_active":"2024-04-17T05:46:12.099801+0000","last_peered":"2024-04-17T05:46:12.099801+0000","last_clean":"2024-04-17T05:46:12.099801+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:46:12.099801+0000","last_undegraded":"2024-04-17T05:46:12.099801+0000","last_fullsized":"2024-04-17T05:46:12.099801+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.099605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:49.682658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":8192,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":120,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.665206+0000","last_change":"2024-04-17T05:46:10.665206+0000","last_active":"2024-04-17T05:46:10.665206+0000","last_peered":"2024-04-17T05:46:10.665206+0000","last_clean":"2024-04-17T05:46:10.665206+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:46:10.665206+0000","last_undegraded":"2024-04-17T05:46:10.665206+0000","last_fullsized":"2024-04-17T05:46:10.665206+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'21","last_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_deep_scrub":"261'21","last_deep_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.664937+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:18.989139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'2","reported_seq":41,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.564484+0000","last_change":"2024-04-17T05:46:38.564484+0000","last_active":"2024-04-17T05:46:38.564484+0000","last_peered":"2024-04-17T05:46:38.564484+0000","last_clean":"2024-04-17T05:46:38.564484+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:46:38.564484+0000","last_undegraded":"2024-04-17T05:46:38.564484+0000","last_fullsized":"2024-04-17T05:46:38.564484+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:38.564228+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:38.564228+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.564228+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:51.812713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'9","reported_seq":72,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.926385+0000","last_change":"2024-04-17T05:46:46.926459+0000","last_active":"2024-04-17T05:46:46.926385+0000","last_peered":"2024-04-17T05:46:46.926385+0000","last_clean":"2024-04-17T05:46:46.926385+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:46:46.926385+0000","last_undegraded":"2024-04-17T05:46:46.926385+0000","last_fullsized":"2024-04-17T05:46:46.926385+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'9","last_scrub_stamp":"2024-04-17T05:46:46.926173+0000","last_deep_scrub":"264'9","last_deep_scrub_stamp":"2024-04-17T05:46:46.926173+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.926173+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:21.554513+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":4098,"num_write":17,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.130174+0000","last_change":"2024-04-17T05:46:13.130289+0000","last_active":"2024-04-17T05:46:13.130174+0000","last_peered":"2024-04-17T05:46:13.130174+0000","last_clean":"2024-04-17T05:46:13.130174+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:46:13.130174+0000","last_undegraded":"2024-04-17T05:46:13.130174+0000","last_fullsized":"2024-04-17T05:46:13.130174+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.129923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:48.339568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.106637+0000","last_change":"2024-04-17T05:46:12.106637+0000","last_active":"2024-04-17T05:46:12.106637+0000","last_peered":"2024-04-17T05:46:12.106637+0000","last_clean":"2024-04-17T05:46:12.106637+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:46:12.106637+0000","last_undegraded":"2024-04-17T05:46:12.106637+0000","last_fullsized":"2024-04-17T05:46:12.106637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'32","last_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_deep_scrub":"253'32","last_deep_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.106369+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:23.602749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:04.610139+0000","last_change":"2024-04-17T05:47:04.610193+0000","last_active":"2024-04-17T05:47:04.610139+0000","last_peered":"2024-04-17T05:47:04.610139+0000","last_clean":"2024-04-17T05:47:04.610139+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:47:04.610139+0000","last_undegraded":"2024-04-17T05:47:04.610139+0000","last_fullsized":"2024-04-17T05:47:04.610139+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:04.609938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:04.609938+0000","last_clean_scrub_stamp":"2024-04-17T05:47:04.609938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:54:57.801405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.259328+0000","last_change":"2024-04-17T05:46:52.259392+0000","last_active":"2024-04-17T05:46:52.259328+0000","last_peered":"2024-04-17T05:46:52.259328+0000","last_clean":"2024-04-17T05:46:52.259328+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:46:52.259328+0000","last_undegraded":"2024-04-17T05:46:52.259328+0000","last_fullsized":"2024-04-17T05:46:52.259328+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:52.259111+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:52.259111+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.259111+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:09.465653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.127295+0000","last_change":"2024-04-17T05:46:14.127359+0000","last_active":"2024-04-17T05:46:14.127295+0000","last_peered":"2024-04-17T05:46:14.127295+0000","last_clean":"2024-04-17T05:46:14.127295+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:46:14.127295+0000","last_undegraded":"2024-04-17T05:46:14.127295+0000","last_fullsized":"2024-04-17T05:46:14.127295+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.127097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.931818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":151,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.750372+0000","last_change":"2024-04-17T05:46:08.750372+0000","last_active":"2024-04-17T05:46:08.750372+0000","last_peered":"2024-04-17T05:46:08.750372+0000","last_clean":"2024-04-17T05:46:08.750372+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:46:08.750372+0000","last_undegraded":"2024-04-17T05:46:08.750372+0000","last_fullsized":"2024-04-17T05:46:08.750372+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'32","last_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_deep_scrub":"264'32","last_deep_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.750109+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:20.266910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:08.530021+0000","last_change":"2024-04-17T05:47:08.530076+0000","last_active":"2024-04-17T05:47:08.530021+0000","last_peered":"2024-04-17T05:47:08.530021+0000","last_clean":"2024-04-17T05:47:08.530021+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:47:08.530021+0000","last_undegraded":"2024-04-17T05:47:08.530021+0000","last_fullsized":"2024-04-17T05:47:08.530021+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:08.529819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:08.529819+0000","last_clean_scrub_stamp":"2024-04-17T05:47:08.529819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:56:32.267749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.858323+0000","last_change":"2024-04-17T05:46:44.858377+0000","last_active":"2024-04-17T05:46:44.858323+0000","last_peered":"2024-04-17T05:46:44.858323+0000","last_clean":"2024-04-17T05:46:44.858323+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:46:44.858323+0000","last_undegraded":"2024-04-17T05:46:44.858323+0000","last_fullsized":"2024-04-17T05:46:44.858323+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:44.858126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:44.858126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.858126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:56.173235+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'2","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.643349+0000","last_change":"2024-04-17T05:46:09.643497+0000","last_active":"2024-04-17T05:46:09.643349+0000","last_peered":"2024-04-17T05:46:09.643349+0000","last_clean":"2024-04-17T05:46:09.643349+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:46:09.643349+0000","last_undegraded":"2024-04-17T05:46:09.643349+0000","last_fullsized":"2024-04-17T05:46:09.643349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.643135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:49.245538+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.049363+0000","last_change":"2024-04-17T05:46:13.049363+0000","last_active":"2024-04-17T05:46:13.049363+0000","last_peered":"2024-04-17T05:46:13.049363+0000","last_clean":"2024-04-17T05:46:13.049363+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:46:13.049363+0000","last_undegraded":"2024-04-17T05:46:13.049363+0000","last_fullsized":"2024-04-17T05:46:13.049363+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'69","last_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_deep_scrub":"264'69","last_deep_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.048938+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:49.209093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:00.299351+0000","last_change":"2024-04-17T05:47:00.299410+0000","last_active":"2024-04-17T05:47:00.299351+0000","last_peered":"2024-04-17T05:47:00.299351+0000","last_clean":"2024-04-17T05:47:00.299351+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:47:00.299351+0000","last_undegraded":"2024-04-17T05:47:00.299351+0000","last_fullsized":"2024-04-17T05:47:00.299351+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:00.299142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:00.299142+0000","last_clean_scrub_stamp":"2024-04-17T05:47:00.299142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:49.331319+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.556697+0000","last_change":"2024-04-17T05:46:53.556758+0000","last_active":"2024-04-17T05:46:53.556697+0000","last_peered":"2024-04-17T05:46:53.556697+0000","last_clean":"2024-04-17T05:46:53.556697+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:46:53.556697+0000","last_undegraded":"2024-04-17T05:46:53.556697+0000","last_fullsized":"2024-04-17T05:46:53.556697+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:53.556487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:53.556487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.556487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:39.303780+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'5","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.936901+0000","last_change":"2024-04-17T05:46:40.936964+0000","last_active":"2024-04-17T05:46:40.936901+0000","last_peered":"2024-04-17T05:46:40.936901+0000","last_clean":"2024-04-17T05:46:40.936901+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:46:40.936901+0000","last_undegraded":"2024-04-17T05:46:40.936901+0000","last_fullsized":"2024-04-17T05:46:40.936901+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:40.936715+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:40.936715+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.936715+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:06.846607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6192,"num_write":5,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":137,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.225148+0000","last_change":"2024-04-17T05:46:19.225296+0000","last_active":"2024-04-17T05:46:19.225148+0000","last_peered":"2024-04-17T05:46:19.225148+0000","last_clean":"2024-04-17T05:46:19.225148+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:46:19.225148+0000","last_undegraded":"2024-04-17T05:46:19.225148+0000","last_fullsized":"2024-04-17T05:46:19.225148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_deep_scrub":"264'27","last_deep_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.224892+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:29.117161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'349","reported_seq":1019,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.239300+0000","last_change":"2024-04-17T05:46:57.239424+0000","last_active":"2024-04-17T05:46:57.239300+0000","last_peered":"2024-04-17T05:46:57.239300+0000","last_clean":"2024-04-17T05:46:57.239300+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:46:57.239300+0000","last_undegraded":"2024-04-17T05:46:57.239300+0000","last_fullsized":"2024-04-17T05:46:57.239300+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'349","last_scrub_stamp":"2024-04-17T05:46:57.239101+0000","last_deep_scrub":"264'349","last_deep_scrub_stamp":"2024-04-17T05:46:57.239101+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.239101+0000","objects_scrubbed":3,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:55.639650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":725,"num_write":1314,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.269919+0000","last_change":"2024-04-17T05:46:28.269919+0000","last_active":"2024-04-17T05:46:28.269919+0000","last_peered":"2024-04-17T05:46:28.269919+0000","last_clean":"2024-04-17T05:46:28.269919+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:46:28.269919+0000","last_undegraded":"2024-04-17T05:46:28.269919+0000","last_fullsized":"2024-04-17T05:46:28.269919+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'80","last_scrub_stamp":"2024-04-17T05:46:28.269667+0000","last_deep_scrub":"264'80","last_deep_scrub_stamp":"2024-04-17T05:46:28.269667+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.269667+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:21:37.474212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'382","reported_seq":1063,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.830224+0000","last_change":"2024-04-17T05:46:39.830336+0000","last_active":"2024-04-17T05:46:39.830224+0000","last_peered":"2024-04-17T05:46:39.830224+0000","last_clean":"2024-04-17T05:46:39.830224+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:46:39.830224+0000","last_undegraded":"2024-04-17T05:46:39.830224+0000","last_fullsized":"2024-04-17T05:46:39.830224+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'382","last_scrub_stamp":"2024-04-17T05:46:39.830021+0000","last_deep_scrub":"264'382","last_deep_scrub_stamp":"2024-04-17T05:46:39.830021+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.830021+0000","objects_scrubbed":6,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:40.048724+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":600,"num_read_kb":10797,"num_write":1506,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.391956+0000","last_change":"2024-04-17T05:46:39.392102+0000","last_active":"2024-04-17T05:46:39.391956+0000","last_peered":"2024-04-17T05:46:39.391956+0000","last_clean":"2024-04-17T05:46:39.391956+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:46:39.391956+0000","last_undegraded":"2024-04-17T05:46:39.391956+0000","last_fullsized":"2024-04-17T05:46:39.391956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:39.391733+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:39.391733+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.391733+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:24.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'312","reported_seq":987,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.651174+0000","last_change":"2024-04-17T05:46:54.651417+0000","last_active":"2024-04-17T05:46:54.651174+0000","last_peered":"2024-04-17T05:46:54.651174+0000","last_clean":"2024-04-17T05:46:54.651174+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:46:54.651174+0000","last_undegraded":"2024-04-17T05:46:54.651174+0000","last_fullsized":"2024-04-17T05:46:54.651174+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'312","last_scrub_stamp":"2024-04-17T05:46:54.650922+0000","last_deep_scrub":"264'312","last_deep_scrub_stamp":"2024-04-17T05:46:54.650922+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.650922+0000","objects_scrubbed":12,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.389913+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11583546,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":475,"num_read_kb":58403,"num_write":1188,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1215,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.608673+0000","last_change":"2024-04-17T05:46:37.608807+0000","last_active":"2024-04-17T05:46:37.608673+0000","last_peered":"2024-04-17T05:46:37.608673+0000","last_clean":"2024-04-17T05:46:37.608673+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:46:37.608673+0000","last_undegraded":"2024-04-17T05:46:37.608673+0000","last_fullsized":"2024-04-17T05:46:37.608673+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'525","last_scrub_stamp":"2024-04-17T05:46:37.608425+0000","last_deep_scrub":"264'525","last_deep_scrub_stamp":"2024-04-17T05:46:37.608425+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.608425+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:33.078323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'284","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.266538+0000","last_change":"2024-04-17T05:46:47.266902+0000","last_active":"2024-04-17T05:46:47.266538+0000","last_peered":"2024-04-17T05:46:47.266538+0000","last_clean":"2024-04-17T05:46:47.266538+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:46:47.266538+0000","last_undegraded":"2024-04-17T05:46:47.266538+0000","last_fullsized":"2024-04-17T05:46:47.266538+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'284","last_scrub_stamp":"2024-04-17T05:46:47.266303+0000","last_deep_scrub":"264'284","last_deep_scrub_stamp":"2024-04-17T05:46:47.266303+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.266303+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:04.388187+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174737,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":460,"num_read_kb":37884,"num_write":1128,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'305","reported_seq":908,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.703858+0000","last_change":"2024-04-17T05:46:39.703930+0000","last_active":"2024-04-17T05:46:39.703858+0000","last_peered":"2024-04-17T05:46:39.703858+0000","last_clean":"2024-04-17T05:46:39.703858+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:46:39.703858+0000","last_undegraded":"2024-04-17T05:46:39.703858+0000","last_fullsized":"2024-04-17T05:46:39.703858+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'305","last_scrub_stamp":"2024-04-17T05:46:39.703631+0000","last_deep_scrub":"264'305","last_deep_scrub_stamp":"2024-04-17T05:46:39.703631+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.703631+0000","objects_scrubbed":1,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:54.709159+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'336","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.112424+0000","last_change":"2024-04-17T05:46:49.112596+0000","last_active":"2024-04-17T05:46:49.112424+0000","last_peered":"2024-04-17T05:46:49.112424+0000","last_clean":"2024-04-17T05:46:49.112424+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:46:49.112424+0000","last_undegraded":"2024-04-17T05:46:49.112424+0000","last_fullsized":"2024-04-17T05:46:49.112424+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'336","last_scrub_stamp":"2024-04-17T05:46:49.112208+0000","last_deep_scrub":"264'336","last_deep_scrub_stamp":"2024-04-17T05:46:49.112208+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.112208+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:27.854631+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":528,"num_read_kb":13359,"num_write":1173,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5189,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.926781+0000","last_change":"2024-04-17T05:46:28.926781+0000","last_active":"2024-04-17T05:46:28.926781+0000","last_peered":"2024-04-17T05:46:28.926781+0000","last_clean":"2024-04-17T05:46:28.926781+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:46:28.926781+0000","last_undegraded":"2024-04-17T05:46:28.926781+0000","last_fullsized":"2024-04-17T05:46:28.926781+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4413","last_scrub_stamp":"2024-04-17T05:46:28.926530+0000","last_deep_scrub":"264'4413","last_deep_scrub_stamp":"2024-04-17T05:46:28.926530+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.926530+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:55:24.685918+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'301","reported_seq":1024,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.312758+0000","last_change":"2024-04-17T05:46:46.312861+0000","last_active":"2024-04-17T05:46:46.312758+0000","last_peered":"2024-04-17T05:46:46.312758+0000","last_clean":"2024-04-17T05:46:46.312758+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:46:46.312758+0000","last_undegraded":"2024-04-17T05:46:46.312758+0000","last_fullsized":"2024-04-17T05:46:46.312758+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'301","last_scrub_stamp":"2024-04-17T05:46:46.312563+0000","last_deep_scrub":"264'301","last_deep_scrub_stamp":"2024-04-17T05:46:46.312563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.312563+0000","objects_scrubbed":9,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:18.182664+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206995,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":584,"num_read_kb":64592,"num_write":977,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":783,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.560806+0000","last_change":"2024-04-17T05:46:33.560872+0000","last_active":"2024-04-17T05:46:33.560806+0000","last_peered":"2024-04-17T05:46:33.560806+0000","last_clean":"2024-04-17T05:46:33.560806+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:46:33.560806+0000","last_undegraded":"2024-04-17T05:46:33.560806+0000","last_fullsized":"2024-04-17T05:46:33.560806+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'157","last_scrub_stamp":"2024-04-17T05:46:33.560601+0000","last_deep_scrub":"264'157","last_deep_scrub_stamp":"2024-04-17T05:46:33.560601+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.560601+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:27.508256+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'324","reported_seq":956,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.236524+0000","last_change":"2024-04-17T05:46:56.236634+0000","last_active":"2024-04-17T05:46:56.236524+0000","last_peered":"2024-04-17T05:46:56.236524+0000","last_clean":"2024-04-17T05:46:56.236524+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:46:56.236524+0000","last_undegraded":"2024-04-17T05:46:56.236524+0000","last_fullsized":"2024-04-17T05:46:56.236524+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'324","last_scrub_stamp":"2024-04-17T05:46:56.236320+0000","last_deep_scrub":"264'324","last_deep_scrub_stamp":"2024-04-17T05:46:56.236320+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.236320+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:29:21.306325+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269430,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":458,"num_read_kb":9424,"num_write":1128,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":817,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.313672+0000","last_change":"2024-04-17T05:46:31.313738+0000","last_active":"2024-04-17T05:46:31.313672+0000","last_peered":"2024-04-17T05:46:31.313672+0000","last_clean":"2024-04-17T05:46:31.313672+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:46:31.313672+0000","last_undegraded":"2024-04-17T05:46:31.313672+0000","last_fullsized":"2024-04-17T05:46:31.313672+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'165","last_scrub_stamp":"2024-04-17T05:46:31.313464+0000","last_deep_scrub":"264'165","last_deep_scrub_stamp":"2024-04-17T05:46:31.313464+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.313464+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:46.132113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'318","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.627848+0000","last_change":"2024-04-17T05:46:57.628017+0000","last_active":"2024-04-17T05:46:57.627848+0000","last_peered":"2024-04-17T05:46:57.627848+0000","last_clean":"2024-04-17T05:46:57.627848+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:46:57.627848+0000","last_undegraded":"2024-04-17T05:46:57.627848+0000","last_fullsized":"2024-04-17T05:46:57.627848+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'318","last_scrub_stamp":"2024-04-17T05:46:57.627640+0000","last_deep_scrub":"264'318","last_deep_scrub_stamp":"2024-04-17T05:46:57.627640+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.627640+0000","objects_scrubbed":10,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:24.431177+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076711,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":427,"num_read_kb":14992,"num_write":1164,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":779,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.467160+0000","last_change":"2024-04-17T05:46:28.467227+0000","last_active":"2024-04-17T05:46:28.467160+0000","last_peered":"2024-04-17T05:46:28.467160+0000","last_clean":"2024-04-17T05:46:28.467160+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:46:28.467160+0000","last_undegraded":"2024-04-17T05:46:28.467160+0000","last_fullsized":"2024-04-17T05:46:28.467160+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'166","last_scrub_stamp":"2024-04-17T05:46:28.466958+0000","last_deep_scrub":"264'166","last_deep_scrub_stamp":"2024-04-17T05:46:28.466958+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.466958+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:50:48.423737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.590578+0000","last_change":"2024-04-17T05:46:31.590640+0000","last_active":"2024-04-17T05:46:31.590578+0000","last_peered":"2024-04-17T05:46:31.590578+0000","last_clean":"2024-04-17T05:46:31.590578+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:46:31.590578+0000","last_undegraded":"2024-04-17T05:46:31.590578+0000","last_fullsized":"2024-04-17T05:46:31.590578+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'128","last_scrub_stamp":"2024-04-17T05:46:31.590386+0000","last_deep_scrub":"264'128","last_deep_scrub_stamp":"2024-04-17T05:46:31.590386+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.590386+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:21.620204+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'322","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.544504+0000","last_change":"2024-04-17T05:46:16.544710+0000","last_active":"2024-04-17T05:46:16.544504+0000","last_peered":"2024-04-17T05:46:16.544504+0000","last_clean":"2024-04-17T05:46:16.544504+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:46:16.544504+0000","last_undegraded":"2024-04-17T05:46:16.544504+0000","last_fullsized":"2024-04-17T05:46:16.544504+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.544168+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:59.343831+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8704691,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":453,"num_read_kb":14105,"num_write":1178,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'256","reported_seq":900,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.357473+0000","last_change":"2024-04-17T05:46:41.357550+0000","last_active":"2024-04-17T05:46:41.357473+0000","last_peered":"2024-04-17T05:46:41.357473+0000","last_clean":"2024-04-17T05:46:41.357473+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:46:41.357473+0000","last_undegraded":"2024-04-17T05:46:41.357473+0000","last_fullsized":"2024-04-17T05:46:41.357473+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'256","last_scrub_stamp":"2024-04-17T05:46:41.357278+0000","last_deep_scrub":"264'256","last_deep_scrub_stamp":"2024-04-17T05:46:41.357278+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.357278+0000","objects_scrubbed":1,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:04:53.727788+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":506,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'270","reported_seq":898,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.886876+0000","last_change":"2024-04-17T05:46:36.886994+0000","last_active":"2024-04-17T05:46:36.886876+0000","last_peered":"2024-04-17T05:46:36.886876+0000","last_clean":"2024-04-17T05:46:36.886876+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:46:36.886876+0000","last_undegraded":"2024-04-17T05:46:36.886876+0000","last_fullsized":"2024-04-17T05:46:36.886876+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'270","last_scrub_stamp":"2024-04-17T05:46:36.886656+0000","last_deep_scrub":"264'270","last_deep_scrub_stamp":"2024-04-17T05:46:36.886656+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.886656+0000","objects_scrubbed":8,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:11.179807+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1730173,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":405,"num_read_kb":10551,"num_write":968,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":733,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.307726+0000","last_change":"2024-04-17T05:46:33.307726+0000","last_active":"2024-04-17T05:46:33.307726+0000","last_peered":"2024-04-17T05:46:33.307726+0000","last_clean":"2024-04-17T05:46:33.307726+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:46:33.307726+0000","last_undegraded":"2024-04-17T05:46:33.307726+0000","last_fullsized":"2024-04-17T05:46:33.307726+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:33.307469+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:33.307469+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.307469+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:08.248068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'302","reported_seq":994,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.891762+0000","last_change":"2024-04-17T05:46:37.892036+0000","last_active":"2024-04-17T05:46:37.891762+0000","last_peered":"2024-04-17T05:46:37.891762+0000","last_clean":"2024-04-17T05:46:37.891762+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:46:37.891762+0000","last_undegraded":"2024-04-17T05:46:37.891762+0000","last_fullsized":"2024-04-17T05:46:37.891762+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'302","last_scrub_stamp":"2024-04-17T05:46:37.891555+0000","last_deep_scrub":"264'302","last_deep_scrub_stamp":"2024-04-17T05:46:37.891555+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.891555+0000","objects_scrubbed":9,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:00.393928+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044776,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":456,"num_read_kb":160096,"num_write":1126,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":909,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.420737+0000","last_change":"2024-04-17T05:46:30.420737+0000","last_active":"2024-04-17T05:46:30.420737+0000","last_peered":"2024-04-17T05:46:30.420737+0000","last_clean":"2024-04-17T05:46:30.420737+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:46:30.420737+0000","last_undegraded":"2024-04-17T05:46:30.420737+0000","last_fullsized":"2024-04-17T05:46:30.420737+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'289","last_scrub_stamp":"2024-04-17T05:46:30.420473+0000","last_deep_scrub":"264'289","last_deep_scrub_stamp":"2024-04-17T05:46:30.420473+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.420473+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:34.221983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'306","reported_seq":1021,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.545633+0000","last_change":"2024-04-17T05:46:46.545721+0000","last_active":"2024-04-17T05:46:46.545633+0000","last_peered":"2024-04-17T05:46:46.545633+0000","last_clean":"2024-04-17T05:46:46.545633+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:46:46.545633+0000","last_undegraded":"2024-04-17T05:46:46.545633+0000","last_fullsized":"2024-04-17T05:46:46.545633+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'306","last_scrub_stamp":"2024-04-17T05:46:46.545412+0000","last_deep_scrub":"264'306","last_deep_scrub_stamp":"2024-04-17T05:46:46.545412+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.545412+0000","objects_scrubbed":7,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:21:16.960386+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399248,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":586,"num_read_kb":64730,"num_write":1112,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4814,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.442968+0000","last_change":"2024-04-17T05:46:29.442968+0000","last_active":"2024-04-17T05:46:29.442968+0000","last_peered":"2024-04-17T05:46:29.442968+0000","last_clean":"2024-04-17T05:46:29.442968+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:46:29.442968+0000","last_undegraded":"2024-04-17T05:46:29.442968+0000","last_fullsized":"2024-04-17T05:46:29.442968+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4181","last_scrub_stamp":"2024-04-17T05:46:29.442703+0000","last_deep_scrub":"264'4181","last_deep_scrub_stamp":"2024-04-17T05:46:29.442703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.442703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:51.847043+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'318","reported_seq":1098,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:02.696238+0000","last_change":"2024-04-17T05:47:02.696493+0000","last_active":"2024-04-17T05:47:02.696238+0000","last_peered":"2024-04-17T05:47:02.696238+0000","last_clean":"2024-04-17T05:47:02.696238+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:47:02.696238+0000","last_undegraded":"2024-04-17T05:47:02.696238+0000","last_fullsized":"2024-04-17T05:47:02.696238+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'318","last_scrub_stamp":"2024-04-17T05:47:02.695933+0000","last_deep_scrub":"264'318","last_deep_scrub_stamp":"2024-04-17T05:47:02.695933+0000","last_clean_scrub_stamp":"2024-04-17T05:47:02.695933+0000","objects_scrubbed":7,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:00.943737+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170354,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":810,"num_read_kb":57781,"num_write":1265,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":825,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.625599+0000","last_change":"2024-04-17T05:46:35.625793+0000","last_active":"2024-04-17T05:46:35.625599+0000","last_peered":"2024-04-17T05:46:35.625599+0000","last_clean":"2024-04-17T05:46:35.625599+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:46:35.625599+0000","last_undegraded":"2024-04-17T05:46:35.625599+0000","last_fullsized":"2024-04-17T05:46:35.625599+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:35.625369+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:35.625369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.625369+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:22:17.124001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'346","reported_seq":990,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:00.518075+0000","last_change":"2024-04-17T05:47:00.518281+0000","last_active":"2024-04-17T05:47:00.518075+0000","last_peered":"2024-04-17T05:47:00.518075+0000","last_clean":"2024-04-17T05:47:00.518075+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:47:00.518075+0000","last_undegraded":"2024-04-17T05:47:00.518075+0000","last_fullsized":"2024-04-17T05:47:00.518075+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'346","last_scrub_stamp":"2024-04-17T05:47:00.517862+0000","last_deep_scrub":"264'346","last_deep_scrub_stamp":"2024-04-17T05:47:00.517862+0000","last_clean_scrub_stamp":"2024-04-17T05:47:00.517862+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:17.380395+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244513,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":463,"num_read_kb":836,"num_write":1174,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'155","reported_seq":681,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.797345+0000","last_change":"2024-04-17T05:46:13.797345+0000","last_active":"2024-04-17T05:46:13.797345+0000","last_peered":"2024-04-17T05:46:13.797345+0000","last_clean":"2024-04-17T05:46:13.797345+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:46:13.797345+0000","last_undegraded":"2024-04-17T05:46:13.797345+0000","last_fullsized":"2024-04-17T05:46:13.797345+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.796936+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:42.445551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":608,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.733949+0000","last_change":"2024-04-17T05:46:43.733949+0000","last_active":"2024-04-17T05:46:43.733949+0000","last_peered":"2024-04-17T05:46:43.733949+0000","last_clean":"2024-04-17T05:46:43.733949+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:46:43.733949+0000","last_undegraded":"2024-04-17T05:46:43.733949+0000","last_fullsized":"2024-04-17T05:46:43.733949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:43.733678+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:43.733678+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.733678+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:13:08.252101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'314","reported_seq":948,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.175131+0000","last_change":"2024-04-17T05:46:53.175403+0000","last_active":"2024-04-17T05:46:53.175131+0000","last_peered":"2024-04-17T05:46:53.175131+0000","last_clean":"2024-04-17T05:46:53.175131+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:46:53.175131+0000","last_undegraded":"2024-04-17T05:46:53.175131+0000","last_fullsized":"2024-04-17T05:46:53.175131+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'314","last_scrub_stamp":"2024-04-17T05:46:53.174917+0000","last_deep_scrub":"264'314","last_deep_scrub_stamp":"2024-04-17T05:46:53.174917+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.174917+0000","objects_scrubbed":11,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:22:45.964725+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7313223,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":429,"num_read_kb":2527,"num_write":1169,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.338579+0000","last_change":"2024-04-17T05:46:32.338579+0000","last_active":"2024-04-17T05:46:32.338579+0000","last_peered":"2024-04-17T05:46:32.338579+0000","last_clean":"2024-04-17T05:46:32.338579+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:46:32.338579+0000","last_undegraded":"2024-04-17T05:46:32.338579+0000","last_fullsized":"2024-04-17T05:46:32.338579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.338322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.338322+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.338322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:19.679591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'373","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.677388+0000","last_change":"2024-04-17T05:46:36.677388+0000","last_active":"2024-04-17T05:46:36.677388+0000","last_peered":"2024-04-17T05:46:36.677388+0000","last_clean":"2024-04-17T05:46:36.677388+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:46:36.677388+0000","last_undegraded":"2024-04-17T05:46:36.677388+0000","last_fullsized":"2024-04-17T05:46:36.677388+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:36.677124+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:36.677124+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.677124+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:01.400465+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45299,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":596,"num_read_kb":1277,"num_write":472,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.304719+0000","last_change":"2024-04-17T05:46:29.304719+0000","last_active":"2024-04-17T05:46:29.304719+0000","last_peered":"2024-04-17T05:46:29.304719+0000","last_clean":"2024-04-17T05:46:29.304719+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:46:29.304719+0000","last_undegraded":"2024-04-17T05:46:29.304719+0000","last_fullsized":"2024-04-17T05:46:29.304719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.304289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.304289+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.304289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:21:58.667090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'172","reported_seq":735,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.112010+0000","last_change":"2024-04-17T05:46:15.112010+0000","last_active":"2024-04-17T05:46:15.112010+0000","last_peered":"2024-04-17T05:46:15.112010+0000","last_clean":"2024-04-17T05:46:15.112010+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:46:15.112010+0000","last_undegraded":"2024-04-17T05:46:15.112010+0000","last_fullsized":"2024-04-17T05:46:15.112010+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'172","last_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_deep_scrub":"264'172","last_deep_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.111703+0000","objects_scrubbed":3,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:36:19.615360+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.266652+0000","last_change":"2024-04-17T05:46:26.266652+0000","last_active":"2024-04-17T05:46:26.266652+0000","last_peered":"2024-04-17T05:46:26.266652+0000","last_clean":"2024-04-17T05:46:26.266652+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:46:26.266652+0000","last_undegraded":"2024-04-17T05:46:26.266652+0000","last_fullsized":"2024-04-17T05:46:26.266652+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:26.266405+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:26.266405+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.266405+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:34:32.444534+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'339","reported_seq":1120,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.618752+0000","last_change":"2024-04-17T05:46:53.618981+0000","last_active":"2024-04-17T05:46:53.618752+0000","last_peered":"2024-04-17T05:46:53.618752+0000","last_clean":"2024-04-17T05:46:53.618752+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:46:53.618752+0000","last_undegraded":"2024-04-17T05:46:53.618752+0000","last_fullsized":"2024-04-17T05:46:53.618752+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'339","last_scrub_stamp":"2024-04-17T05:46:53.618500+0000","last_deep_scrub":"264'339","last_deep_scrub_stamp":"2024-04-17T05:46:53.618500+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.618500+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:27.991022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1328,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":754,"num_read_kb":61435,"num_write":1363,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.624575+0000","last_change":"2024-04-17T05:46:54.624676+0000","last_active":"2024-04-17T05:46:54.624575+0000","last_peered":"2024-04-17T05:46:54.624575+0000","last_clean":"2024-04-17T05:46:54.624575+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:46:54.624575+0000","last_undegraded":"2024-04-17T05:46:54.624575+0000","last_fullsized":"2024-04-17T05:46:54.624575+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:54.624351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:54.624351+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.624351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:29:59.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'103","reported_seq":656,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.736490+0000","last_change":"2024-04-17T05:46:22.736490+0000","last_active":"2024-04-17T05:46:22.736490+0000","last_peered":"2024-04-17T05:46:22.736490+0000","last_clean":"2024-04-17T05:46:22.736490+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:46:22.736490+0000","last_undegraded":"2024-04-17T05:46:22.736490+0000","last_fullsized":"2024-04-17T05:46:22.736490+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'103","last_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_deep_scrub":"264'103","last_deep_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.736227+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:14.605978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.786406+0000","last_change":"2024-04-17T05:46:15.786406+0000","last_active":"2024-04-17T05:46:15.786406+0000","last_peered":"2024-04-17T05:46:15.786406+0000","last_clean":"2024-04-17T05:46:15.786406+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:46:15.786406+0000","last_undegraded":"2024-04-17T05:46:15.786406+0000","last_fullsized":"2024-04-17T05:46:15.786406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.786152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:05:10.717154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'129","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.649727+0000","last_change":"2024-04-17T05:46:22.649727+0000","last_active":"2024-04-17T05:46:22.649727+0000","last_peered":"2024-04-17T05:46:22.649727+0000","last_clean":"2024-04-17T05:46:22.649727+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:46:22.649727+0000","last_undegraded":"2024-04-17T05:46:22.649727+0000","last_fullsized":"2024-04-17T05:46:22.649727+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'129","last_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_deep_scrub":"264'129","last_deep_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.649467+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.914685+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":669,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.727502+0000","last_change":"2024-04-17T05:46:40.727563+0000","last_active":"2024-04-17T05:46:40.727502+0000","last_peered":"2024-04-17T05:46:40.727502+0000","last_clean":"2024-04-17T05:46:40.727502+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:46:40.727502+0000","last_undegraded":"2024-04-17T05:46:40.727502+0000","last_fullsized":"2024-04-17T05:46:40.727502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'85","last_scrub_stamp":"2024-04-17T05:46:40.727294+0000","last_deep_scrub":"264'85","last_deep_scrub_stamp":"2024-04-17T05:46:40.727294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.727294+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:27.135914+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'333","reported_seq":1122,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.183587+0000","last_change":"2024-04-17T05:46:47.183732+0000","last_active":"2024-04-17T05:46:47.183587+0000","last_peered":"2024-04-17T05:46:47.183587+0000","last_clean":"2024-04-17T05:46:47.183587+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:46:47.183587+0000","last_undegraded":"2024-04-17T05:46:47.183587+0000","last_fullsized":"2024-04-17T05:46:47.183587+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'333","last_scrub_stamp":"2024-04-17T05:46:47.183363+0000","last_deep_scrub":"264'333","last_deep_scrub_stamp":"2024-04-17T05:46:47.183363+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.183363+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.756516+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":672,"num_read_kb":75397,"num_write":1317,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.776716+0000","last_change":"2024-04-17T05:46:48.776790+0000","last_active":"2024-04-17T05:46:48.776716+0000","last_peered":"2024-04-17T05:46:48.776716+0000","last_clean":"2024-04-17T05:46:48.776716+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:46:48.776716+0000","last_undegraded":"2024-04-17T05:46:48.776716+0000","last_fullsized":"2024-04-17T05:46:48.776716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:48.776517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:48.776517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.776517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.117730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'144","reported_seq":721,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.644625+0000","last_change":"2024-04-17T05:46:13.644625+0000","last_active":"2024-04-17T05:46:13.644625+0000","last_peered":"2024-04-17T05:46:13.644625+0000","last_clean":"2024-04-17T05:46:13.644625+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:46:13.644625+0000","last_undegraded":"2024-04-17T05:46:13.644625+0000","last_fullsized":"2024-04-17T05:46:13.644625+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'144","last_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_deep_scrub":"264'144","last_deep_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.644329+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:14.646427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'519","reported_seq":2426,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.368300+0000","last_change":"2024-04-17T05:46:31.368300+0000","last_active":"2024-04-17T05:46:31.368300+0000","last_peered":"2024-04-17T05:46:31.368300+0000","last_clean":"2024-04-17T05:46:31.368300+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:46:31.368300+0000","last_undegraded":"2024-04-17T05:46:31.368300+0000","last_fullsized":"2024-04-17T05:46:31.368300+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'519","last_scrub_stamp":"2024-04-17T05:46:31.367883+0000","last_deep_scrub":"264'519","last_deep_scrub_stamp":"2024-04-17T05:46:31.367883+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.367883+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.078252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'199","reported_seq":1022,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.828863+0000","last_change":"2024-04-17T05:46:14.828863+0000","last_active":"2024-04-17T05:46:14.828863+0000","last_peered":"2024-04-17T05:46:14.828863+0000","last_clean":"2024-04-17T05:46:14.828863+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:46:14.828863+0000","last_undegraded":"2024-04-17T05:46:14.828863+0000","last_fullsized":"2024-04-17T05:46:14.828863+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'199","last_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_deep_scrub":"264'199","last_deep_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.828583+0000","objects_scrubbed":4,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:40.862007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.741170+0000","last_change":"2024-04-17T05:46:42.741232+0000","last_active":"2024-04-17T05:46:42.741170+0000","last_peered":"2024-04-17T05:46:42.741170+0000","last_clean":"2024-04-17T05:46:42.741170+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:46:42.741170+0000","last_undegraded":"2024-04-17T05:46:42.741170+0000","last_fullsized":"2024-04-17T05:46:42.741170+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'234","last_scrub_stamp":"2024-04-17T05:46:42.740966+0000","last_deep_scrub":"264'234","last_deep_scrub_stamp":"2024-04-17T05:46:42.740966+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.740966+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:49:41.989989+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'293","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.858878+0000","last_change":"2024-04-17T05:46:35.858991+0000","last_active":"2024-04-17T05:46:35.858878+0000","last_peered":"2024-04-17T05:46:35.858878+0000","last_clean":"2024-04-17T05:46:35.858878+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:46:35.858878+0000","last_undegraded":"2024-04-17T05:46:35.858878+0000","last_fullsized":"2024-04-17T05:46:35.858878+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'293","last_scrub_stamp":"2024-04-17T05:46:35.858673+0000","last_deep_scrub":"264'293","last_deep_scrub_stamp":"2024-04-17T05:46:35.858673+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.858673+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:47.864378+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3722445,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":48921,"num_write":1143,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.344025+0000","last_change":"2024-04-17T05:46:36.344082+0000","last_active":"2024-04-17T05:46:36.344025+0000","last_peered":"2024-04-17T05:46:36.344025+0000","last_clean":"2024-04-17T05:46:36.344025+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:46:36.344025+0000","last_undegraded":"2024-04-17T05:46:36.344025+0000","last_fullsized":"2024-04-17T05:46:36.344025+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:36.343820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:36.343820+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.343820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:48.938245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'284","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.144810+0000","last_change":"2024-04-17T05:46:48.144886+0000","last_active":"2024-04-17T05:46:48.144810+0000","last_peered":"2024-04-17T05:46:48.144810+0000","last_clean":"2024-04-17T05:46:48.144810+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:46:48.144810+0000","last_undegraded":"2024-04-17T05:46:48.144810+0000","last_fullsized":"2024-04-17T05:46:48.144810+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'284","last_scrub_stamp":"2024-04-17T05:46:48.144597+0000","last_deep_scrub":"264'284","last_deep_scrub_stamp":"2024-04-17T05:46:48.144597+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.144597+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.030105+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51171,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1108,"num_write":376,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'534","reported_seq":2312,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.727792+0000","last_change":"2024-04-17T05:46:25.727792+0000","last_active":"2024-04-17T05:46:25.727792+0000","last_peered":"2024-04-17T05:46:25.727792+0000","last_clean":"2024-04-17T05:46:25.727792+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:46:25.727792+0000","last_undegraded":"2024-04-17T05:46:25.727792+0000","last_fullsized":"2024-04-17T05:46:25.727792+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'534","last_scrub_stamp":"2024-04-17T05:46:25.727540+0000","last_deep_scrub":"264'534","last_deep_scrub_stamp":"2024-04-17T05:46:25.727540+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.727540+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:47.622101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'122","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.284199+0000","last_change":"2024-04-17T05:46:30.284267+0000","last_active":"2024-04-17T05:46:30.284199+0000","last_peered":"2024-04-17T05:46:30.284199+0000","last_clean":"2024-04-17T05:46:30.284199+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:46:30.284199+0000","last_undegraded":"2024-04-17T05:46:30.284199+0000","last_fullsized":"2024-04-17T05:46:30.284199+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'122","last_scrub_stamp":"2024-04-17T05:46:30.283976+0000","last_deep_scrub":"264'122","last_deep_scrub_stamp":"2024-04-17T05:46:30.283976+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.283976+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:13.769030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'281","reported_seq":924,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.203978+0000","last_change":"2024-04-17T05:46:54.204083+0000","last_active":"2024-04-17T05:46:54.203978+0000","last_peered":"2024-04-17T05:46:54.203978+0000","last_clean":"2024-04-17T05:46:54.203978+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:46:54.203978+0000","last_undegraded":"2024-04-17T05:46:54.203978+0000","last_fullsized":"2024-04-17T05:46:54.203978+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:54.203767+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:54.203767+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.203767+0000","objects_scrubbed":6,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:03.849010+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6974633,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":437,"num_read_kb":24084,"num_write":1121,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.373791+0000","last_change":"2024-04-17T05:46:38.373891+0000","last_active":"2024-04-17T05:46:38.373791+0000","last_peered":"2024-04-17T05:46:38.373791+0000","last_clean":"2024-04-17T05:46:38.373791+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:46:38.373791+0000","last_undegraded":"2024-04-17T05:46:38.373791+0000","last_fullsized":"2024-04-17T05:46:38.373791+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.373516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.373516+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.373516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:40.616887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'391","reported_seq":1038,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.206831+0000","last_change":"2024-04-17T05:46:22.206940+0000","last_active":"2024-04-17T05:46:22.206831+0000","last_peered":"2024-04-17T05:46:22.206831+0000","last_clean":"2024-04-17T05:46:22.206831+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:46:22.206831+0000","last_undegraded":"2024-04-17T05:46:22.206831+0000","last_fullsized":"2024-04-17T05:46:22.206831+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'391","last_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_deep_scrub":"264'391","last_deep_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.206623+0000","objects_scrubbed":10,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:39.352908+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44506,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":619,"num_read_kb":1253,"num_write":476,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.860898+0000","last_change":"2024-04-17T05:46:30.860950+0000","last_active":"2024-04-17T05:46:30.860898+0000","last_peered":"2024-04-17T05:46:30.860898+0000","last_clean":"2024-04-17T05:46:30.860898+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:46:30.860898+0000","last_undegraded":"2024-04-17T05:46:30.860898+0000","last_fullsized":"2024-04-17T05:46:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.860694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.860694+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.860694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:48.164981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'564","reported_seq":1397,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.160720+0000","last_change":"2024-04-17T05:46:18.160792+0000","last_active":"2024-04-17T05:46:18.160720+0000","last_peered":"2024-04-17T05:46:18.160720+0000","last_clean":"2024-04-17T05:46:18.160720+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:46:18.160720+0000","last_undegraded":"2024-04-17T05:46:18.160720+0000","last_fullsized":"2024-04-17T05:46:18.160720+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'564","last_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_deep_scrub":"264'564","last_deep_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.160537+0000","objects_scrubbed":4,"log_size":64,"log_dups_size":500,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:23.558644+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":874,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.594125+0000","last_change":"2024-04-17T05:46:41.594194+0000","last_active":"2024-04-17T05:46:41.594125+0000","last_peered":"2024-04-17T05:46:41.594125+0000","last_clean":"2024-04-17T05:46:41.594125+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:46:41.594125+0000","last_undegraded":"2024-04-17T05:46:41.594125+0000","last_fullsized":"2024-04-17T05:46:41.594125+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'259","last_scrub_stamp":"2024-04-17T05:46:41.593897+0000","last_deep_scrub":"264'259","last_deep_scrub_stamp":"2024-04-17T05:46:41.593897+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.593897+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:27.355644+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'395","reported_seq":1065,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.215535+0000","last_change":"2024-04-17T05:46:55.215740+0000","last_active":"2024-04-17T05:46:55.215535+0000","last_peered":"2024-04-17T05:46:55.215535+0000","last_clean":"2024-04-17T05:46:55.215535+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:46:55.215535+0000","last_undegraded":"2024-04-17T05:46:55.215535+0000","last_fullsized":"2024-04-17T05:46:55.215535+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'395","last_scrub_stamp":"2024-04-17T05:46:55.215336+0000","last_deep_scrub":"264'395","last_deep_scrub_stamp":"2024-04-17T05:46:55.215336+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.215336+0000","objects_scrubbed":8,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:01.768966+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396022,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":571,"num_read_kb":14718,"num_write":1316,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.469546+0000","last_change":"2024-04-17T05:46:23.469644+0000","last_active":"2024-04-17T05:46:23.469546+0000","last_peered":"2024-04-17T05:46:23.469546+0000","last_clean":"2024-04-17T05:46:23.469546+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:46:23.469546+0000","last_undegraded":"2024-04-17T05:46:23.469546+0000","last_fullsized":"2024-04-17T05:46:23.469546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.469283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:59.151653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'274","reported_seq":873,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:04.736724+0000","last_change":"2024-04-17T05:47:04.736825+0000","last_active":"2024-04-17T05:47:04.736724+0000","last_peered":"2024-04-17T05:47:04.736724+0000","last_clean":"2024-04-17T05:47:04.736724+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:47:04.736724+0000","last_undegraded":"2024-04-17T05:47:04.736724+0000","last_fullsized":"2024-04-17T05:47:04.736724+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'274","last_scrub_stamp":"2024-04-17T05:47:04.736535+0000","last_deep_scrub":"264'274","last_deep_scrub_stamp":"2024-04-17T05:47:04.736535+0000","last_clean_scrub_stamp":"2024-04-17T05:47:04.736535+0000","objects_scrubbed":5,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:24.653815+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42027,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":427,"num_read_kb":943,"num_write":336,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.833829+0000","last_change":"2024-04-17T05:46:29.833879+0000","last_active":"2024-04-17T05:46:29.833829+0000","last_peered":"2024-04-17T05:46:29.833829+0000","last_clean":"2024-04-17T05:46:29.833829+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:46:29.833829+0000","last_undegraded":"2024-04-17T05:46:29.833829+0000","last_fullsized":"2024-04-17T05:46:29.833829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.833621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.833621+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.833621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:23.380092+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2375","reported_seq":4186,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.963343+0000","last_change":"2024-04-17T05:46:27.963419+0000","last_active":"2024-04-17T05:46:27.963343+0000","last_peered":"2024-04-17T05:46:27.963343+0000","last_clean":"2024-04-17T05:46:27.963343+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:46:27.963343+0000","last_undegraded":"2024-04-17T05:46:27.963343+0000","last_fullsized":"2024-04-17T05:46:27.963343+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2375","last_scrub_stamp":"2024-04-17T05:46:27.963136+0000","last_deep_scrub":"264'2375","last_deep_scrub_stamp":"2024-04-17T05:46:27.963136+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.963136+0000","objects_scrubbed":4,"log_size":75,"log_dups_size":2300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:51.672615+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":708,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.910107+0000","last_change":"2024-04-17T05:46:30.910177+0000","last_active":"2024-04-17T05:46:30.910107+0000","last_peered":"2024-04-17T05:46:30.910107+0000","last_clean":"2024-04-17T05:46:30.910107+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:46:30.910107+0000","last_undegraded":"2024-04-17T05:46:30.910107+0000","last_fullsized":"2024-04-17T05:46:30.910107+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'154","last_scrub_stamp":"2024-04-17T05:46:30.909913+0000","last_deep_scrub":"264'154","last_deep_scrub_stamp":"2024-04-17T05:46:30.909913+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.909913+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:00.648226+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'356","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.297884+0000","last_change":"2024-04-17T05:46:43.298040+0000","last_active":"2024-04-17T05:46:43.297884+0000","last_peered":"2024-04-17T05:46:43.297884+0000","last_clean":"2024-04-17T05:46:43.297884+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:46:43.297884+0000","last_undegraded":"2024-04-17T05:46:43.297884+0000","last_fullsized":"2024-04-17T05:46:43.297884+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:43.297672+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:43.297672+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.297672+0000","objects_scrubbed":5,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:52:50.015406+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524369,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":604,"num_read_kb":65373,"num_write":1319,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.457184+0000","last_change":"2024-04-17T05:46:21.457242+0000","last_active":"2024-04-17T05:46:21.457184+0000","last_peered":"2024-04-17T05:46:21.457184+0000","last_clean":"2024-04-17T05:46:21.457184+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:46:21.457184+0000","last_undegraded":"2024-04-17T05:46:21.457184+0000","last_fullsized":"2024-04-17T05:46:21.457184+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.456977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:03.328209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'344","reported_seq":969,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.334603+0000","last_change":"2024-04-17T05:46:42.334679+0000","last_active":"2024-04-17T05:46:42.334603+0000","last_peered":"2024-04-17T05:46:42.334603+0000","last_clean":"2024-04-17T05:46:42.334603+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:46:42.334603+0000","last_undegraded":"2024-04-17T05:46:42.334603+0000","last_fullsized":"2024-04-17T05:46:42.334603+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'344","last_scrub_stamp":"2024-04-17T05:46:42.334476+0000","last_deep_scrub":"264'344","last_deep_scrub_stamp":"2024-04-17T05:46:42.334476+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.334476+0000","objects_scrubbed":9,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:22.249150+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48038,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":543,"num_read_kb":1167,"num_write":428,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'539","reported_seq":4624,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.621904+0000","last_change":"2024-04-17T05:46:33.621973+0000","last_active":"2024-04-17T05:46:33.621904+0000","last_peered":"2024-04-17T05:46:33.621904+0000","last_clean":"2024-04-17T05:46:33.621904+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:46:33.621904+0000","last_undegraded":"2024-04-17T05:46:33.621904+0000","last_fullsized":"2024-04-17T05:46:33.621904+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:33.621706+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:33.621706+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.621706+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:44.561807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":630,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.594579+0000","last_change":"2024-04-17T05:46:21.594642+0000","last_active":"2024-04-17T05:46:21.594579+0000","last_peered":"2024-04-17T05:46:21.594579+0000","last_clean":"2024-04-17T05:46:21.594579+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:46:21.594579+0000","last_undegraded":"2024-04-17T05:46:21.594579+0000","last_fullsized":"2024-04-17T05:46:21.594579+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.594379+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:06.802802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'96","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.599050+0000","last_change":"2024-04-17T05:46:46.599114+0000","last_active":"2024-04-17T05:46:46.599050+0000","last_peered":"2024-04-17T05:46:46.599050+0000","last_clean":"2024-04-17T05:46:46.599050+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:46:46.599050+0000","last_undegraded":"2024-04-17T05:46:46.599050+0000","last_fullsized":"2024-04-17T05:46:46.599050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'96","last_scrub_stamp":"2024-04-17T05:46:46.598859+0000","last_deep_scrub":"264'96","last_deep_scrub_stamp":"2024-04-17T05:46:46.598859+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.598859+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:35.086138+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'339","reported_seq":1048,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.599664+0000","last_change":"2024-04-17T05:46:14.599664+0000","last_active":"2024-04-17T05:46:14.599664+0000","last_peered":"2024-04-17T05:46:14.599664+0000","last_clean":"2024-04-17T05:46:14.599664+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:46:14.599664+0000","last_undegraded":"2024-04-17T05:46:14.599664+0000","last_fullsized":"2024-04-17T05:46:14.599664+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'339","last_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_deep_scrub":"264'339","last_deep_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.599211+0000","objects_scrubbed":6,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:59.675254+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":662,"num_read_kb":24375,"num_write":1169,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.517281+0000","last_change":"2024-04-17T05:46:17.517338+0000","last_active":"2024-04-17T05:46:17.517281+0000","last_peered":"2024-04-17T05:46:17.517281+0000","last_clean":"2024-04-17T05:46:17.517281+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:46:17.517281+0000","last_undegraded":"2024-04-17T05:46:17.517281+0000","last_fullsized":"2024-04-17T05:46:17.517281+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.517077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:41.667573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'479","reported_seq":1145,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:01.532888+0000","last_change":"2024-04-17T05:47:01.532969+0000","last_active":"2024-04-17T05:47:01.532888+0000","last_peered":"2024-04-17T05:47:01.532888+0000","last_clean":"2024-04-17T05:47:01.532888+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:47:01.532888+0000","last_undegraded":"2024-04-17T05:47:01.532888+0000","last_fullsized":"2024-04-17T05:47:01.532888+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'479","last_scrub_stamp":"2024-04-17T05:47:01.532657+0000","last_deep_scrub":"264'479","last_deep_scrub_stamp":"2024-04-17T05:47:01.532657+0000","last_clean_scrub_stamp":"2024-04-17T05:47:01.532657+0000","objects_scrubbed":7,"log_size":79,"log_dups_size":400,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:21.356823+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87769,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":831,"num_read_kb":1997,"num_write":666,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.734179+0000","last_change":"2024-04-17T05:46:08.734235+0000","last_active":"2024-04-17T05:46:08.734179+0000","last_peered":"2024-04-17T05:46:08.734179+0000","last_clean":"2024-04-17T05:46:08.734179+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:46:08.734179+0000","last_undegraded":"2024-04-17T05:46:08.734179+0000","last_fullsized":"2024-04-17T05:46:08.734179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.733969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:20.733334+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'150","reported_seq":700,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.732921+0000","last_change":"2024-04-17T05:46:14.733009+0000","last_active":"2024-04-17T05:46:14.732921+0000","last_peered":"2024-04-17T05:46:14.732921+0000","last_clean":"2024-04-17T05:46:14.732921+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:46:14.732921+0000","last_undegraded":"2024-04-17T05:46:14.732921+0000","last_fullsized":"2024-04-17T05:46:14.732921+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'150","last_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_deep_scrub":"264'150","last_deep_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.732679+0000","objects_scrubbed":2,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:10.487087+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":780,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.706858+0000","last_change":"2024-04-17T05:46:27.706921+0000","last_active":"2024-04-17T05:46:27.706858+0000","last_peered":"2024-04-17T05:46:27.706858+0000","last_clean":"2024-04-17T05:46:27.706858+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:46:27.706858+0000","last_undegraded":"2024-04-17T05:46:27.706858+0000","last_fullsized":"2024-04-17T05:46:27.706858+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'157","last_scrub_stamp":"2024-04-17T05:46:27.706654+0000","last_deep_scrub":"264'157","last_deep_scrub_stamp":"2024-04-17T05:46:27.706654+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.706654+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:06.380118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'350","reported_seq":1028,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.610408+0000","last_change":"2024-04-17T05:46:55.610535+0000","last_active":"2024-04-17T05:46:55.610408+0000","last_peered":"2024-04-17T05:46:55.610408+0000","last_clean":"2024-04-17T05:46:55.610408+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:46:55.610408+0000","last_undegraded":"2024-04-17T05:46:55.610408+0000","last_fullsized":"2024-04-17T05:46:55.610408+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'350","last_scrub_stamp":"2024-04-17T05:46:55.610204+0000","last_deep_scrub":"264'350","last_deep_scrub_stamp":"2024-04-17T05:46:55.610204+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.610204+0000","objects_scrubbed":9,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:29.485147+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1985843,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":564,"num_read_kb":9569,"num_write":1227,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.531942+0000","last_change":"2024-04-17T05:46:19.531995+0000","last_active":"2024-04-17T05:46:19.531942+0000","last_peered":"2024-04-17T05:46:19.531942+0000","last_clean":"2024-04-17T05:46:19.531942+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:46:19.531942+0000","last_undegraded":"2024-04-17T05:46:19.531942+0000","last_fullsized":"2024-04-17T05:46:19.531942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.531732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:32.490714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'319","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.744134+0000","last_change":"2024-04-17T05:46:23.744249+0000","last_active":"2024-04-17T05:46:23.744134+0000","last_peered":"2024-04-17T05:46:23.744134+0000","last_clean":"2024-04-17T05:46:23.744134+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:46:23.744134+0000","last_undegraded":"2024-04-17T05:46:23.744134+0000","last_fullsized":"2024-04-17T05:46:23.744134+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.743937+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:03.777566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":535,"num_read_kb":1188,"num_write":472,"num_write_kb":375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15128,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.643838+0000","last_change":"2024-04-17T05:46:25.643838+0000","last_active":"2024-04-17T05:46:25.643838+0000","last_peered":"2024-04-17T05:46:25.643838+0000","last_clean":"2024-04-17T05:46:25.643838+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:46:25.643838+0000","last_undegraded":"2024-04-17T05:46:25.643838+0000","last_fullsized":"2024-04-17T05:46:25.643838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:25.643431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:25.643431+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.643431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:10.449552+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'133","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.789793+0000","last_change":"2024-04-17T05:46:19.789793+0000","last_active":"2024-04-17T05:46:19.789793+0000","last_peered":"2024-04-17T05:46:19.789793+0000","last_clean":"2024-04-17T05:46:19.789793+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:46:19.789793+0000","last_undegraded":"2024-04-17T05:46:19.789793+0000","last_fullsized":"2024-04-17T05:46:19.789793+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.789529+0000","objects_scrubbed":4,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:17.259142+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.626107+0000","last_change":"2024-04-17T05:46:39.626172+0000","last_active":"2024-04-17T05:46:39.626107+0000","last_peered":"2024-04-17T05:46:39.626107+0000","last_clean":"2024-04-17T05:46:39.626107+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:46:39.626107+0000","last_undegraded":"2024-04-17T05:46:39.626107+0000","last_fullsized":"2024-04-17T05:46:39.626107+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'149","last_scrub_stamp":"2024-04-17T05:46:39.625903+0000","last_deep_scrub":"264'149","last_deep_scrub_stamp":"2024-04-17T05:46:39.625903+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.625903+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:29.931185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'365","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.843059+0000","last_change":"2024-04-17T05:46:32.843361+0000","last_active":"2024-04-17T05:46:32.843059+0000","last_peered":"2024-04-17T05:46:32.843059+0000","last_clean":"2024-04-17T05:46:32.843059+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:46:32.843059+0000","last_undegraded":"2024-04-17T05:46:32.843059+0000","last_fullsized":"2024-04-17T05:46:32.843059+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'365","last_scrub_stamp":"2024-04-17T05:46:32.842802+0000","last_deep_scrub":"264'365","last_deep_scrub_stamp":"2024-04-17T05:46:32.842802+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.842802+0000","objects_scrubbed":7,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:42.238535+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524382,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":609,"num_read_kb":15072,"num_write":1258,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":517,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.751463+0000","last_change":"2024-04-17T05:46:19.751517+0000","last_active":"2024-04-17T05:46:19.751463+0000","last_peered":"2024-04-17T05:46:19.751463+0000","last_clean":"2024-04-17T05:46:19.751463+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:46:19.751463+0000","last_undegraded":"2024-04-17T05:46:19.751463+0000","last_fullsized":"2024-04-17T05:46:19.751463+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.751263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:08.609702+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'298","reported_seq":917,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.671356+0000","last_change":"2024-04-17T05:46:11.671356+0000","last_active":"2024-04-17T05:46:11.671356+0000","last_peered":"2024-04-17T05:46:11.671356+0000","last_clean":"2024-04-17T05:46:11.671356+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:46:11.671356+0000","last_undegraded":"2024-04-17T05:46:11.671356+0000","last_fullsized":"2024-04-17T05:46:11.671356+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.670938+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:15.760804+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47207,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":490,"num_read_kb":1044,"num_write":380,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'512","reported_seq":2453,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.695035+0000","last_change":"2024-04-17T05:46:09.695035+0000","last_active":"2024-04-17T05:46:09.695035+0000","last_peered":"2024-04-17T05:46:09.695035+0000","last_clean":"2024-04-17T05:46:09.695035+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:46:09.695035+0000","last_undegraded":"2024-04-17T05:46:09.695035+0000","last_fullsized":"2024-04-17T05:46:09.695035+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'512","last_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_deep_scrub":"264'512","last_deep_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.694925+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.559083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'102","reported_seq":617,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.732298+0000","last_change":"2024-04-17T05:46:13.732374+0000","last_active":"2024-04-17T05:46:13.732298+0000","last_peered":"2024-04-17T05:46:13.732298+0000","last_clean":"2024-04-17T05:46:13.732298+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:46:13.732298+0000","last_undegraded":"2024-04-17T05:46:13.732298+0000","last_fullsized":"2024-04-17T05:46:13.732298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.732113+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:34.325042+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.636438+0000","last_change":"2024-04-17T05:46:29.636597+0000","last_active":"2024-04-17T05:46:29.636438+0000","last_peered":"2024-04-17T05:46:29.636438+0000","last_clean":"2024-04-17T05:46:29.636438+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:46:29.636438+0000","last_undegraded":"2024-04-17T05:46:29.636438+0000","last_fullsized":"2024-04-17T05:46:29.636438+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:29.636101+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:29.636101+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.636101+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:14.328596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'295","reported_seq":994,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.600236+0000","last_change":"2024-04-17T05:46:42.600435+0000","last_active":"2024-04-17T05:46:42.600236+0000","last_peered":"2024-04-17T05:46:42.600236+0000","last_clean":"2024-04-17T05:46:42.600236+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:46:42.600236+0000","last_undegraded":"2024-04-17T05:46:42.600236+0000","last_fullsized":"2024-04-17T05:46:42.600236+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'295","last_scrub_stamp":"2024-04-17T05:46:42.600013+0000","last_deep_scrub":"264'295","last_deep_scrub_stamp":"2024-04-17T05:46:42.600013+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.600013+0000","objects_scrubbed":8,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:40.877675+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243143,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":575,"num_read_kb":5671,"num_write":1192,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"264'4","reported_seq":1166,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.330528+0000","last_change":"2024-04-17T05:46:34.330595+0000","last_active":"2024-04-17T05:46:34.330528+0000","last_peered":"2024-04-17T05:46:34.330528+0000","last_clean":"2024-04-17T05:46:34.330528+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:46:34.330528+0000","last_undegraded":"2024-04-17T05:46:34.330528+0000","last_fullsized":"2024-04-17T05:46:34.330528+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4","last_scrub_stamp":"2024-04-17T05:46:34.330337+0000","last_deep_scrub":"264'4","last_deep_scrub_stamp":"2024-04-17T05:46:34.330337+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.330337+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:21.974891+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":652,"num_read_kb":652,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'102","reported_seq":647,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.863661+0000","last_change":"2024-04-17T05:46:33.863738+0000","last_active":"2024-04-17T05:46:33.863661+0000","last_peered":"2024-04-17T05:46:33.863661+0000","last_clean":"2024-04-17T05:46:33.863661+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:46:33.863661+0000","last_undegraded":"2024-04-17T05:46:33.863661+0000","last_fullsized":"2024-04-17T05:46:33.863661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:33.863448+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:33.863448+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.863448+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:04.150489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'539","reported_seq":2257,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.997502+0000","last_change":"2024-04-17T05:46:25.997576+0000","last_active":"2024-04-17T05:46:25.997502+0000","last_peered":"2024-04-17T05:46:25.997502+0000","last_clean":"2024-04-17T05:46:25.997502+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:46:25.997502+0000","last_undegraded":"2024-04-17T05:46:25.997502+0000","last_fullsized":"2024-04-17T05:46:25.997502+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:25.997283+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:25.997283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.997283+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:16.675494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'98","reported_seq":610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.796242+0000","last_change":"2024-04-17T05:46:11.796311+0000","last_active":"2024-04-17T05:46:11.796242+0000","last_peered":"2024-04-17T05:46:11.796242+0000","last_clean":"2024-04-17T05:46:11.796242+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:46:11.796242+0000","last_undegraded":"2024-04-17T05:46:11.796242+0000","last_fullsized":"2024-04-17T05:46:11.796242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'98","last_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_deep_scrub":"264'98","last_deep_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.796022+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:25.138191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":766,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.742441+0000","last_change":"2024-04-17T05:46:25.742441+0000","last_active":"2024-04-17T05:46:25.742441+0000","last_peered":"2024-04-17T05:46:25.742441+0000","last_clean":"2024-04-17T05:46:25.742441+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:46:25.742441+0000","last_undegraded":"2024-04-17T05:46:25.742441+0000","last_fullsized":"2024-04-17T05:46:25.742441+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:25.742055+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:25.742055+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.742055+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:19.477880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'296","reported_seq":941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:00.706078+0000","last_change":"2024-04-17T05:47:00.706286+0000","last_active":"2024-04-17T05:47:00.706078+0000","last_peered":"2024-04-17T05:47:00.706078+0000","last_clean":"2024-04-17T05:47:00.706078+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:47:00.706078+0000","last_undegraded":"2024-04-17T05:47:00.706078+0000","last_fullsized":"2024-04-17T05:47:00.706078+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'296","last_scrub_stamp":"2024-04-17T05:47:00.705786+0000","last_deep_scrub":"264'296","last_deep_scrub_stamp":"2024-04-17T05:47:00.705786+0000","last_clean_scrub_stamp":"2024-04-17T05:47:00.705786+0000","objects_scrubbed":7,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:27.453238+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170337,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":410,"num_read_kb":36760,"num_write":993,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.522731+0000","last_change":"2024-04-17T05:46:50.522786+0000","last_active":"2024-04-17T05:46:50.522731+0000","last_peered":"2024-04-17T05:46:50.522731+0000","last_clean":"2024-04-17T05:46:50.522731+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:46:50.522731+0000","last_undegraded":"2024-04-17T05:46:50.522731+0000","last_fullsized":"2024-04-17T05:46:50.522731+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:50.522512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:50.522512+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.522512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:05.669837+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'359","reported_seq":992,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.232070+0000","last_change":"2024-04-17T05:46:25.232126+0000","last_active":"2024-04-17T05:46:25.232070+0000","last_peered":"2024-04-17T05:46:25.232070+0000","last_clean":"2024-04-17T05:46:25.232070+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:46:25.232070+0000","last_undegraded":"2024-04-17T05:46:25.232070+0000","last_fullsized":"2024-04-17T05:46:25.232070+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'359","last_scrub_stamp":"2024-04-17T05:46:25.231966+0000","last_deep_scrub":"264'359","last_deep_scrub_stamp":"2024-04-17T05:46:25.231966+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.231966+0000","objects_scrubbed":8,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:47:43.934129+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45806,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1167,"num_write":448,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.992307+0000","last_change":"2024-04-17T05:46:21.992363+0000","last_active":"2024-04-17T05:46:21.992307+0000","last_peered":"2024-04-17T05:46:21.992307+0000","last_clean":"2024-04-17T05:46:21.992307+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:46:21.992307+0000","last_undegraded":"2024-04-17T05:46:21.992307+0000","last_fullsized":"2024-04-17T05:46:21.992307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.992088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:40.290560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'161","reported_seq":676,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.735879+0000","last_change":"2024-04-17T05:46:15.736060+0000","last_active":"2024-04-17T05:46:15.735879+0000","last_peered":"2024-04-17T05:46:15.735879+0000","last_clean":"2024-04-17T05:46:15.735879+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:46:15.735879+0000","last_undegraded":"2024-04-17T05:46:15.735879+0000","last_fullsized":"2024-04-17T05:46:15.735879+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.735662+0000","objects_scrubbed":4,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.803225+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":704,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.006094+0000","last_change":"2024-04-17T05:46:25.006216+0000","last_active":"2024-04-17T05:46:25.006094+0000","last_peered":"2024-04-17T05:46:25.006094+0000","last_clean":"2024-04-17T05:46:25.006094+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:46:25.006094+0000","last_undegraded":"2024-04-17T05:46:25.006094+0000","last_fullsized":"2024-04-17T05:46:25.006094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.005814+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:49.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"264'119","reported_seq":692,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.064201+0000","last_change":"2024-04-17T05:46:50.064279+0000","last_active":"2024-04-17T05:46:50.064201+0000","last_peered":"2024-04-17T05:46:50.064201+0000","last_clean":"2024-04-17T05:46:50.064201+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:46:50.064201+0000","last_undegraded":"2024-04-17T05:46:50.064201+0000","last_fullsized":"2024-04-17T05:46:50.064201+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:50.063983+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:50.063983+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.063983+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:28.852792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":531,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.225234+0000","last_change":"2024-04-17T05:46:43.225234+0000","last_active":"2024-04-17T05:46:43.225234+0000","last_peered":"2024-04-17T05:46:43.225234+0000","last_clean":"2024-04-17T05:46:43.225234+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:46:43.225234+0000","last_undegraded":"2024-04-17T05:46:43.225234+0000","last_fullsized":"2024-04-17T05:46:43.225234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.224981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.224981+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.224981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:16.644730+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'327","reported_seq":987,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:01.735424+0000","last_change":"2024-04-17T05:47:01.735526+0000","last_active":"2024-04-17T05:47:01.735424+0000","last_peered":"2024-04-17T05:47:01.735424+0000","last_clean":"2024-04-17T05:47:01.735424+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:47:01.735424+0000","last_undegraded":"2024-04-17T05:47:01.735424+0000","last_fullsized":"2024-04-17T05:47:01.735424+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'327","last_scrub_stamp":"2024-04-17T05:47:01.735208+0000","last_deep_scrub":"264'327","last_deep_scrub_stamp":"2024-04-17T05:47:01.735208+0000","last_clean_scrub_stamp":"2024-04-17T05:47:01.735208+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:19.437082+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344976,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":2736,"num_write":1215,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.441265+0000","last_change":"2024-04-17T05:46:36.441319+0000","last_active":"2024-04-17T05:46:36.441265+0000","last_peered":"2024-04-17T05:46:36.441265+0000","last_clean":"2024-04-17T05:46:36.441265+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:46:36.441265+0000","last_undegraded":"2024-04-17T05:46:36.441265+0000","last_fullsized":"2024-04-17T05:46:36.441265+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:36.441066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:36.441066+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.441066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:15.277478+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'728","reported_seq":1458,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.227559+0000","last_change":"2024-04-17T05:46:23.227642+0000","last_active":"2024-04-17T05:46:23.227559+0000","last_peered":"2024-04-17T05:46:23.227559+0000","last_clean":"2024-04-17T05:46:23.227559+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:46:23.227559+0000","last_undegraded":"2024-04-17T05:46:23.227559+0000","last_fullsized":"2024-04-17T05:46:23.227559+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'728","last_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_deep_scrub":"264'728","last_deep_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.227369+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":700,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:26.750099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1411,"num_read_kb":3198,"num_write":1348,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":95136,"num_omap_keys":246,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.647366+0000","last_change":"2024-04-17T05:46:28.647420+0000","last_active":"2024-04-17T05:46:28.647366+0000","last_peered":"2024-04-17T05:46:28.647366+0000","last_clean":"2024-04-17T05:46:28.647366+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:46:28.647366+0000","last_undegraded":"2024-04-17T05:46:28.647366+0000","last_fullsized":"2024-04-17T05:46:28.647366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.647167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.647167+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.647167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:59.157329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'223","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.911300+0000","last_change":"2024-04-17T05:46:29.911389+0000","last_active":"2024-04-17T05:46:29.911300+0000","last_peered":"2024-04-17T05:46:29.911300+0000","last_clean":"2024-04-17T05:46:29.911300+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:46:29.911300+0000","last_undegraded":"2024-04-17T05:46:29.911300+0000","last_fullsized":"2024-04-17T05:46:29.911300+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'223","last_scrub_stamp":"2024-04-17T05:46:29.911091+0000","last_deep_scrub":"264'223","last_deep_scrub_stamp":"2024-04-17T05:46:29.911091+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.911091+0000","objects_scrubbed":3,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:25.435946+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":677,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.627128+0000","last_change":"2024-04-17T05:46:48.627245+0000","last_active":"2024-04-17T05:46:48.627128+0000","last_peered":"2024-04-17T05:46:48.627128+0000","last_clean":"2024-04-17T05:46:48.627128+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:46:48.627128+0000","last_undegraded":"2024-04-17T05:46:48.627128+0000","last_fullsized":"2024-04-17T05:46:48.627128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'84","last_scrub_stamp":"2024-04-17T05:46:48.626891+0000","last_deep_scrub":"264'84","last_deep_scrub_stamp":"2024-04-17T05:46:48.626891+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.626891+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:55.642870+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":534,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.100977+0000","last_change":"2024-04-17T05:46:51.101031+0000","last_active":"2024-04-17T05:46:51.100977+0000","last_peered":"2024-04-17T05:46:51.100977+0000","last_clean":"2024-04-17T05:46:51.100977+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:46:51.100977+0000","last_undegraded":"2024-04-17T05:46:51.100977+0000","last_fullsized":"2024-04-17T05:46:51.100977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:51.100774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:51.100774+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.100774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:20:01.756488+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'343","reported_seq":1004,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.587007+0000","last_change":"2024-04-17T05:46:52.587177+0000","last_active":"2024-04-17T05:46:52.587007+0000","last_peered":"2024-04-17T05:46:52.587007+0000","last_clean":"2024-04-17T05:46:52.587007+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:46:52.587007+0000","last_undegraded":"2024-04-17T05:46:52.587007+0000","last_fullsized":"2024-04-17T05:46:52.587007+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'343","last_scrub_stamp":"2024-04-17T05:46:52.586788+0000","last_deep_scrub":"264'343","last_deep_scrub_stamp":"2024-04-17T05:46:52.586788+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.586788+0000","objects_scrubbed":11,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:32.792717+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2309902,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":570,"num_read_kb":3714,"num_write":1237,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.294868+0000","last_change":"2024-04-17T05:46:40.294921+0000","last_active":"2024-04-17T05:46:40.294868+0000","last_peered":"2024-04-17T05:46:40.294868+0000","last_clean":"2024-04-17T05:46:40.294868+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:46:40.294868+0000","last_undegraded":"2024-04-17T05:46:40.294868+0000","last_fullsized":"2024-04-17T05:46:40.294868+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:40.294666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:40.294666+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.294666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:08.051052+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1109","reported_seq":2836,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.770289+0000","last_change":"2024-04-17T05:46:26.770431+0000","last_active":"2024-04-17T05:46:26.770289+0000","last_peered":"2024-04-17T05:46:26.770289+0000","last_clean":"2024-04-17T05:46:26.770289+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:46:26.770289+0000","last_undegraded":"2024-04-17T05:46:26.770289+0000","last_fullsized":"2024-04-17T05:46:26.770289+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'1109","last_scrub_stamp":"2024-04-17T05:46:26.770091+0000","last_deep_scrub":"264'1109","last_deep_scrub_stamp":"2024-04-17T05:46:26.770091+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.770091+0000","objects_scrubbed":12,"log_size":109,"log_dups_size":1000,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:23.849377+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1819,"num_read_kb":3719,"num_write":1122,"num_write_kb":937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5370,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.327552+0000","last_change":"2024-04-17T05:46:30.327657+0000","last_active":"2024-04-17T05:46:30.327552+0000","last_peered":"2024-04-17T05:46:30.327552+0000","last_clean":"2024-04-17T05:46:30.327552+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:46:30.327552+0000","last_undegraded":"2024-04-17T05:46:30.327552+0000","last_fullsized":"2024-04-17T05:46:30.327552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.327323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.327323+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.327323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:13:37.291684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'114","reported_seq":633,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.750777+0000","last_change":"2024-04-17T05:46:17.750850+0000","last_active":"2024-04-17T05:46:17.750777+0000","last_peered":"2024-04-17T05:46:17.750777+0000","last_clean":"2024-04-17T05:46:17.750777+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:46:17.750777+0000","last_undegraded":"2024-04-17T05:46:17.750777+0000","last_fullsized":"2024-04-17T05:46:17.750777+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'114","last_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_deep_scrub":"264'114","last_deep_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.750563+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:00.237624+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1026,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.816940+0000","last_change":"2024-04-17T05:46:29.817060+0000","last_active":"2024-04-17T05:46:29.816940+0000","last_peered":"2024-04-17T05:46:29.816940+0000","last_clean":"2024-04-17T05:46:29.816940+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:46:29.816940+0000","last_undegraded":"2024-04-17T05:46:29.816940+0000","last_fullsized":"2024-04-17T05:46:29.816940+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'343","last_scrub_stamp":"2024-04-17T05:46:29.816699+0000","last_deep_scrub":"264'343","last_deep_scrub_stamp":"2024-04-17T05:46:29.816699+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.816699+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:49.710831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":540,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.432548+0000","last_change":"2024-04-17T05:46:24.432548+0000","last_active":"2024-04-17T05:46:24.432548+0000","last_peered":"2024-04-17T05:46:24.432548+0000","last_clean":"2024-04-17T05:46:24.432548+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:46:24.432548+0000","last_undegraded":"2024-04-17T05:46:24.432548+0000","last_fullsized":"2024-04-17T05:46:24.432548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.432302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:49.304795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'298","reported_seq":964,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.846776+0000","last_change":"2024-04-17T05:46:37.846776+0000","last_active":"2024-04-17T05:46:37.846776+0000","last_peered":"2024-04-17T05:46:37.846776+0000","last_clean":"2024-04-17T05:46:37.846776+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:46:37.846776+0000","last_undegraded":"2024-04-17T05:46:37.846776+0000","last_fullsized":"2024-04-17T05:46:37.846776+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:37.846228+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:37.846228+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.846228+0000","objects_scrubbed":9,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:41.146058+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5679739,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":479,"num_read_kb":21975,"num_write":1114,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"264'2","reported_seq":522,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.548976+0000","last_change":"2024-04-17T05:46:15.549042+0000","last_active":"2024-04-17T05:46:15.548976+0000","last_peered":"2024-04-17T05:46:15.548976+0000","last_clean":"2024-04-17T05:46:15.548976+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:46:15.548976+0000","last_undegraded":"2024-04-17T05:46:15.548976+0000","last_fullsized":"2024-04-17T05:46:15.548976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.548772+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:16:50.031612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'243","reported_seq":1636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:58.265285+0000","last_change":"2024-04-17T05:46:58.265370+0000","last_active":"2024-04-17T05:46:58.265285+0000","last_peered":"2024-04-17T05:46:58.265285+0000","last_clean":"2024-04-17T05:46:58.265285+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:46:58.265285+0000","last_undegraded":"2024-04-17T05:46:58.265285+0000","last_fullsized":"2024-04-17T05:46:58.265285+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'243","last_scrub_stamp":"2024-04-17T05:46:58.265097+0000","last_deep_scrub":"264'243","last_deep_scrub_stamp":"2024-04-17T05:46:58.265097+0000","last_clean_scrub_stamp":"2024-04-17T05:46:58.265097+0000","objects_scrubbed":9,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:38.453917+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5136,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.628578+0000","last_change":"2024-04-17T05:46:32.628630+0000","last_active":"2024-04-17T05:46:32.628578+0000","last_peered":"2024-04-17T05:46:32.628578+0000","last_clean":"2024-04-17T05:46:32.628578+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:46:32.628578+0000","last_undegraded":"2024-04-17T05:46:32.628578+0000","last_fullsized":"2024-04-17T05:46:32.628578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.628381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.628381+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.628381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:52.474027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'265","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.770655+0000","last_change":"2024-04-17T05:46:24.770806+0000","last_active":"2024-04-17T05:46:24.770655+0000","last_peered":"2024-04-17T05:46:24.770655+0000","last_clean":"2024-04-17T05:46:24.770655+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:46:24.770655+0000","last_undegraded":"2024-04-17T05:46:24.770655+0000","last_fullsized":"2024-04-17T05:46:24.770655+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'265","last_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_deep_scrub":"264'265","last_deep_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.770438+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:52.656005+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":958,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.751155+0000","last_change":"2024-04-17T05:46:33.751218+0000","last_active":"2024-04-17T05:46:33.751155+0000","last_peered":"2024-04-17T05:46:33.751155+0000","last_clean":"2024-04-17T05:46:33.751155+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:46:33.751155+0000","last_undegraded":"2024-04-17T05:46:33.751155+0000","last_fullsized":"2024-04-17T05:46:33.751155+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'311","last_scrub_stamp":"2024-04-17T05:46:33.750961+0000","last_deep_scrub":"264'311","last_deep_scrub_stamp":"2024-04-17T05:46:33.750961+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.750961+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:15.653327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":531,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.265009+0000","last_change":"2024-04-17T05:46:42.265009+0000","last_active":"2024-04-17T05:46:42.265009+0000","last_peered":"2024-04-17T05:46:42.265009+0000","last_clean":"2024-04-17T05:46:42.265009+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:46:42.265009+0000","last_undegraded":"2024-04-17T05:46:42.265009+0000","last_fullsized":"2024-04-17T05:46:42.265009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.264768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.264768+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.264768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:20.993345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'341","reported_seq":1107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.616779+0000","last_change":"2024-04-17T05:46:43.616921+0000","last_active":"2024-04-17T05:46:43.616779+0000","last_peered":"2024-04-17T05:46:43.616779+0000","last_clean":"2024-04-17T05:46:43.616779+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:46:43.616779+0000","last_undegraded":"2024-04-17T05:46:43.616779+0000","last_fullsized":"2024-04-17T05:46:43.616779+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'341","last_scrub_stamp":"2024-04-17T05:46:43.616551+0000","last_deep_scrub":"264'341","last_deep_scrub_stamp":"2024-04-17T05:46:43.616551+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.616551+0000","objects_scrubbed":12,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:41.283471+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048803,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":626,"num_read_kb":36553,"num_write":1237,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.635722+0000","last_change":"2024-04-17T05:46:43.635780+0000","last_active":"2024-04-17T05:46:43.635722+0000","last_peered":"2024-04-17T05:46:43.635722+0000","last_clean":"2024-04-17T05:46:43.635722+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:46:43.635722+0000","last_undegraded":"2024-04-17T05:46:43.635722+0000","last_fullsized":"2024-04-17T05:46:43.635722+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.635515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.635515+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.635515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:56.243388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'146","reported_seq":715,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.761478+0000","last_change":"2024-04-17T05:46:37.761563+0000","last_active":"2024-04-17T05:46:37.761478+0000","last_peered":"2024-04-17T05:46:37.761478+0000","last_clean":"2024-04-17T05:46:37.761478+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:46:37.761478+0000","last_undegraded":"2024-04-17T05:46:37.761478+0000","last_fullsized":"2024-04-17T05:46:37.761478+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'146","last_scrub_stamp":"2024-04-17T05:46:37.761274+0000","last_deep_scrub":"264'146","last_deep_scrub_stamp":"2024-04-17T05:46:37.761274+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.761274+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:03.176459+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1416,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.968621+0000","last_change":"2024-04-17T05:46:19.968684+0000","last_active":"2024-04-17T05:46:19.968621+0000","last_peered":"2024-04-17T05:46:19.968621+0000","last_clean":"2024-04-17T05:46:19.968621+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:46:19.968621+0000","last_undegraded":"2024-04-17T05:46:19.968621+0000","last_fullsized":"2024-04-17T05:46:19.968621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.968394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:02.908417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'134","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.184018+0000","last_change":"2024-04-17T05:46:17.184135+0000","last_active":"2024-04-17T05:46:17.184018+0000","last_peered":"2024-04-17T05:46:17.184018+0000","last_clean":"2024-04-17T05:46:17.184018+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:46:17.184018+0000","last_undegraded":"2024-04-17T05:46:17.184018+0000","last_fullsized":"2024-04-17T05:46:17.184018+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.183823+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.008088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":760,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.771890+0000","last_change":"2024-04-17T05:46:27.771954+0000","last_active":"2024-04-17T05:46:27.771890+0000","last_peered":"2024-04-17T05:46:27.771890+0000","last_clean":"2024-04-17T05:46:27.771890+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:46:27.771890+0000","last_undegraded":"2024-04-17T05:46:27.771890+0000","last_fullsized":"2024-04-17T05:46:27.771890+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'145","last_scrub_stamp":"2024-04-17T05:46:27.771685+0000","last_deep_scrub":"264'145","last_deep_scrub_stamp":"2024-04-17T05:46:27.771685+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.771685+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:13.790569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'300","reported_seq":971,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.370422+0000","last_change":"2024-04-17T05:46:37.370505+0000","last_active":"2024-04-17T05:46:37.370422+0000","last_peered":"2024-04-17T05:46:37.370422+0000","last_clean":"2024-04-17T05:46:37.370422+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:46:37.370422+0000","last_undegraded":"2024-04-17T05:46:37.370422+0000","last_fullsized":"2024-04-17T05:46:37.370422+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'300","last_scrub_stamp":"2024-04-17T05:46:37.370221+0000","last_deep_scrub":"264'300","last_deep_scrub_stamp":"2024-04-17T05:46:37.370221+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.370221+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:46:56.727712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":475,"num_read_kb":1448,"num_write":1131,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":538,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.782544+0000","last_change":"2024-04-17T05:46:35.782544+0000","last_active":"2024-04-17T05:46:35.782544+0000","last_peered":"2024-04-17T05:46:35.782544+0000","last_clean":"2024-04-17T05:46:35.782544+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:46:35.782544+0000","last_undegraded":"2024-04-17T05:46:35.782544+0000","last_fullsized":"2024-04-17T05:46:35.782544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:35.782279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:35.782279+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.782279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:01:00.464190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.605392+0000","last_change":"2024-04-17T05:46:47.605511+0000","last_active":"2024-04-17T05:46:47.605392+0000","last_peered":"2024-04-17T05:46:47.605392+0000","last_clean":"2024-04-17T05:46:47.605392+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:46:47.605392+0000","last_undegraded":"2024-04-17T05:46:47.605392+0000","last_fullsized":"2024-04-17T05:46:47.605392+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:47.605143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:47.605143+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.605143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:36.604088+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'631","reported_seq":1337,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.682650+0000","last_change":"2024-04-17T05:46:34.682650+0000","last_active":"2024-04-17T05:46:34.682650+0000","last_peered":"2024-04-17T05:46:34.682650+0000","last_clean":"2024-04-17T05:46:34.682650+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:46:34.682650+0000","last_undegraded":"2024-04-17T05:46:34.682650+0000","last_fullsized":"2024-04-17T05:46:34.682650+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'631","last_scrub_stamp":"2024-04-17T05:46:34.682357+0000","last_deep_scrub":"264'631","last_deep_scrub_stamp":"2024-04-17T05:46:34.682357+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.682357+0000","objects_scrubbed":11,"log_size":31,"log_dups_size":600,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:41.613666+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86860,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1659,"num_read_kb":2834,"num_write":2174,"num_write_kb":1973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":470034,"num_omap_keys":1124,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":510,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.791699+0000","last_change":"2024-04-17T05:46:17.791757+0000","last_active":"2024-04-17T05:46:17.791699+0000","last_peered":"2024-04-17T05:46:17.791699+0000","last_clean":"2024-04-17T05:46:17.791699+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:46:17.791699+0000","last_undegraded":"2024-04-17T05:46:17.791699+0000","last_fullsized":"2024-04-17T05:46:17.791699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.791493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:50.092156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'86","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.643211+0000","last_change":"2024-04-17T05:46:18.643211+0000","last_active":"2024-04-17T05:46:18.643211+0000","last_peered":"2024-04-17T05:46:18.643211+0000","last_clean":"2024-04-17T05:46:18.643211+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:46:18.643211+0000","last_undegraded":"2024-04-17T05:46:18.643211+0000","last_fullsized":"2024-04-17T05:46:18.643211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'86","last_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_deep_scrub":"264'86","last_deep_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.642880+0000","objects_scrubbed":4,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:33.629614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":601,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.677721+0000","last_change":"2024-04-17T05:46:38.677786+0000","last_active":"2024-04-17T05:46:38.677721+0000","last_peered":"2024-04-17T05:46:38.677721+0000","last_clean":"2024-04-17T05:46:38.677721+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:46:38.677721+0000","last_undegraded":"2024-04-17T05:46:38.677721+0000","last_fullsized":"2024-04-17T05:46:38.677721+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-04-17T05:46:38.677518+0000","last_deep_scrub":"264'34","last_deep_scrub_stamp":"2024-04-17T05:46:38.677518+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.677518+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:22.863769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'269","reported_seq":899,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.836555+0000","last_change":"2024-04-17T05:46:38.836824+0000","last_active":"2024-04-17T05:46:38.836555+0000","last_peered":"2024-04-17T05:46:38.836555+0000","last_clean":"2024-04-17T05:46:38.836555+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:46:38.836555+0000","last_undegraded":"2024-04-17T05:46:38.836555+0000","last_fullsized":"2024-04-17T05:46:38.836555+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'269","last_scrub_stamp":"2024-04-17T05:46:38.836321+0000","last_deep_scrub":"264'269","last_deep_scrub_stamp":"2024-04-17T05:46:38.836321+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.836321+0000","objects_scrubbed":4,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:25.285046+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194313,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":396,"num_read_kb":35213,"num_write":840,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":532,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.148689+0000","last_change":"2024-04-17T05:46:52.148749+0000","last_active":"2024-04-17T05:46:52.148689+0000","last_peered":"2024-04-17T05:46:52.148689+0000","last_clean":"2024-04-17T05:46:52.148689+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:46:52.148689+0000","last_undegraded":"2024-04-17T05:46:52.148689+0000","last_fullsized":"2024-04-17T05:46:52.148689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:52.148489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:52.148489+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.148489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:00:21.186802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.958775+0000","last_change":"2024-04-17T05:46:20.958830+0000","last_active":"2024-04-17T05:46:20.958775+0000","last_peered":"2024-04-17T05:46:20.958775+0000","last_clean":"2024-04-17T05:46:20.958775+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:46:20.958775+0000","last_undegraded":"2024-04-17T05:46:20.958775+0000","last_fullsized":"2024-04-17T05:46:20.958775+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.958569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:07.756009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'207","reported_seq":809,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.813732+0000","last_change":"2024-04-17T05:46:28.813826+0000","last_active":"2024-04-17T05:46:28.813732+0000","last_peered":"2024-04-17T05:46:28.813732+0000","last_clean":"2024-04-17T05:46:28.813732+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:46:28.813732+0000","last_undegraded":"2024-04-17T05:46:28.813732+0000","last_fullsized":"2024-04-17T05:46:28.813732+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'207","last_scrub_stamp":"2024-04-17T05:46:28.813550+0000","last_deep_scrub":"264'207","last_deep_scrub_stamp":"2024-04-17T05:46:28.813550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.813550+0000","objects_scrubbed":8,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:10.411851+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'540","reported_seq":4368,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.162167+0000","last_change":"2024-04-17T05:46:21.162167+0000","last_active":"2024-04-17T05:46:21.162167+0000","last_peered":"2024-04-17T05:46:21.162167+0000","last_clean":"2024-04-17T05:46:21.162167+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:46:21.162167+0000","last_undegraded":"2024-04-17T05:46:21.162167+0000","last_fullsized":"2024-04-17T05:46:21.162167+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'540","last_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_deep_scrub":"264'540","last_deep_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.161902+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":500,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:14.043415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'161","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.780606+0000","last_change":"2024-04-17T05:46:23.780606+0000","last_active":"2024-04-17T05:46:23.780606+0000","last_peered":"2024-04-17T05:46:23.780606+0000","last_clean":"2024-04-17T05:46:23.780606+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:46:23.780606+0000","last_undegraded":"2024-04-17T05:46:23.780606+0000","last_fullsized":"2024-04-17T05:46:23.780606+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.780338+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:57.240957+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":809,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.800965+0000","last_change":"2024-04-17T05:46:46.801029+0000","last_active":"2024-04-17T05:46:46.800965+0000","last_peered":"2024-04-17T05:46:46.800965+0000","last_clean":"2024-04-17T05:46:46.800965+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:46:46.800965+0000","last_undegraded":"2024-04-17T05:46:46.800965+0000","last_fullsized":"2024-04-17T05:46:46.800965+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'168","last_scrub_stamp":"2024-04-17T05:46:46.800742+0000","last_deep_scrub":"264'168","last_deep_scrub_stamp":"2024-04-17T05:46:46.800742+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.800742+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:35.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'379","reported_seq":1045,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:58.653020+0000","last_change":"2024-04-17T05:46:58.653101+0000","last_active":"2024-04-17T05:46:58.653020+0000","last_peered":"2024-04-17T05:46:58.653020+0000","last_clean":"2024-04-17T05:46:58.653020+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:46:58.653020+0000","last_undegraded":"2024-04-17T05:46:58.653020+0000","last_fullsized":"2024-04-17T05:46:58.653020+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'379","last_scrub_stamp":"2024-04-17T05:46:58.652814+0000","last_deep_scrub":"264'379","last_deep_scrub_stamp":"2024-04-17T05:46:58.652814+0000","last_clean_scrub_stamp":"2024-04-17T05:46:58.652814+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:02.434927+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262239,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":591,"num_read_kb":5844,"num_write":1385,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":536,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.778199+0000","last_change":"2024-04-17T05:46:34.778260+0000","last_active":"2024-04-17T05:46:34.778199+0000","last_peered":"2024-04-17T05:46:34.778199+0000","last_clean":"2024-04-17T05:46:34.778199+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:46:34.778199+0000","last_undegraded":"2024-04-17T05:46:34.778199+0000","last_fullsized":"2024-04-17T05:46:34.778199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:34.778008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:34.778008+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.778008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:33.294550+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":518,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.587013+0000","last_change":"2024-04-17T05:46:38.587078+0000","last_active":"2024-04-17T05:46:38.587013+0000","last_peered":"2024-04-17T05:46:38.587013+0000","last_clean":"2024-04-17T05:46:38.587013+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:46:38.587013+0000","last_undegraded":"2024-04-17T05:46:38.587013+0000","last_fullsized":"2024-04-17T05:46:38.587013+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.586815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.586815+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.586815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:51.535527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'142","reported_seq":713,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.761857+0000","last_change":"2024-04-17T05:46:21.761940+0000","last_active":"2024-04-17T05:46:21.761857+0000","last_peered":"2024-04-17T05:46:21.761857+0000","last_clean":"2024-04-17T05:46:21.761857+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:46:21.761857+0000","last_undegraded":"2024-04-17T05:46:21.761857+0000","last_fullsized":"2024-04-17T05:46:21.761857+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'142","last_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_deep_scrub":"264'142","last_deep_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.761664+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:22.526188+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'533","reported_seq":2313,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.802896+0000","last_change":"2024-04-17T05:46:27.803031+0000","last_active":"2024-04-17T05:46:27.802896+0000","last_peered":"2024-04-17T05:46:27.802896+0000","last_clean":"2024-04-17T05:46:27.802896+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:46:27.802896+0000","last_undegraded":"2024-04-17T05:46:27.802896+0000","last_fullsized":"2024-04-17T05:46:27.802896+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'533","last_scrub_stamp":"2024-04-17T05:46:27.802603+0000","last_deep_scrub":"264'533","last_deep_scrub_stamp":"2024-04-17T05:46:27.802603+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.802603+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":500,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:54.996107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'224","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.621850+0000","last_change":"2024-04-17T05:46:17.621850+0000","last_active":"2024-04-17T05:46:17.621850+0000","last_peered":"2024-04-17T05:46:17.621850+0000","last_clean":"2024-04-17T05:46:17.621850+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:46:17.621850+0000","last_undegraded":"2024-04-17T05:46:17.621850+0000","last_fullsized":"2024-04-17T05:46:17.621850+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'224","last_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_deep_scrub":"264'224","last_deep_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.621550+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:22:57.559950+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4094","reported_seq":4734,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.497931+0000","last_change":"2024-04-17T05:46:49.498067+0000","last_active":"2024-04-17T05:46:49.497931+0000","last_peered":"2024-04-17T05:46:49.497931+0000","last_clean":"2024-04-17T05:46:49.497931+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:46:49.497931+0000","last_undegraded":"2024-04-17T05:46:49.497931+0000","last_fullsized":"2024-04-17T05:46:49.497931+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4094","last_scrub_stamp":"2024-04-17T05:46:49.497699+0000","last_deep_scrub":"264'4094","last_deep_scrub_stamp":"2024-04-17T05:46:49.497699+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.497699+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:37.910397+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'356","reported_seq":982,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:58.582947+0000","last_change":"2024-04-17T05:46:58.583154+0000","last_active":"2024-04-17T05:46:58.582947+0000","last_peered":"2024-04-17T05:46:58.582947+0000","last_clean":"2024-04-17T05:46:58.582947+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:46:58.582947+0000","last_undegraded":"2024-04-17T05:46:58.582947+0000","last_fullsized":"2024-04-17T05:46:58.582947+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:58.582703+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:58.582703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:58.582703+0000","objects_scrubbed":7,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:52:04.025146+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145969,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":466,"num_read_kb":7519,"num_write":1284,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.882687+0000","last_change":"2024-04-17T05:46:31.882747+0000","last_active":"2024-04-17T05:46:31.882687+0000","last_peered":"2024-04-17T05:46:31.882687+0000","last_clean":"2024-04-17T05:46:31.882687+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:46:31.882687+0000","last_undegraded":"2024-04-17T05:46:31.882687+0000","last_fullsized":"2024-04-17T05:46:31.882687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.882477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.882477+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.882477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:43.637695+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":518,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.758092+0000","last_change":"2024-04-17T05:46:50.758190+0000","last_active":"2024-04-17T05:46:50.758092+0000","last_peered":"2024-04-17T05:46:50.758092+0000","last_clean":"2024-04-17T05:46:50.758092+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:46:50.758092+0000","last_undegraded":"2024-04-17T05:46:50.758092+0000","last_fullsized":"2024-04-17T05:46:50.758092+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:50.757743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:50.757743+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.757743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:15.565878+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'739","reported_seq":1492,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.240166+0000","last_change":"2024-04-17T05:46:46.240287+0000","last_active":"2024-04-17T05:46:46.240166+0000","last_peered":"2024-04-17T05:46:46.240166+0000","last_clean":"2024-04-17T05:46:46.240166+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:46:46.240166+0000","last_undegraded":"2024-04-17T05:46:46.240166+0000","last_fullsized":"2024-04-17T05:46:46.240166+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'739","last_scrub_stamp":"2024-04-17T05:46:46.239975+0000","last_deep_scrub":"264'739","last_deep_scrub_stamp":"2024-04-17T05:46:46.239975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.239975+0000","objects_scrubbed":9,"log_size":39,"log_dups_size":700,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:48.081136+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138854,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1296,"num_read_kb":3065,"num_write":1038,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.854130+0000","last_change":"2024-04-17T05:46:31.854181+0000","last_active":"2024-04-17T05:46:31.854130+0000","last_peered":"2024-04-17T05:46:31.854130+0000","last_clean":"2024-04-17T05:46:31.854130+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:46:31.854130+0000","last_undegraded":"2024-04-17T05:46:31.854130+0000","last_fullsized":"2024-04-17T05:46:31.854130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.853942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.853942+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.853942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:11.976239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'105","reported_seq":604,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.069586+0000","last_change":"2024-04-17T05:46:13.069586+0000","last_active":"2024-04-17T05:46:13.069586+0000","last_peered":"2024-04-17T05:46:13.069586+0000","last_clean":"2024-04-17T05:46:13.069586+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:46:13.069586+0000","last_undegraded":"2024-04-17T05:46:13.069586+0000","last_fullsized":"2024-04-17T05:46:13.069586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.069188+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:06.656617+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'151","reported_seq":727,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.648354+0000","last_change":"2024-04-17T05:46:24.648424+0000","last_active":"2024-04-17T05:46:24.648354+0000","last_peered":"2024-04-17T05:46:24.648354+0000","last_clean":"2024-04-17T05:46:24.648354+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:46:24.648354+0000","last_undegraded":"2024-04-17T05:46:24.648354+0000","last_fullsized":"2024-04-17T05:46:24.648354+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'151","last_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_deep_scrub":"264'151","last_deep_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.648141+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:48.144202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'74","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.642292+0000","last_change":"2024-04-17T05:46:30.642444+0000","last_active":"2024-04-17T05:46:30.642292+0000","last_peered":"2024-04-17T05:46:30.642292+0000","last_clean":"2024-04-17T05:46:30.642292+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:46:30.642292+0000","last_undegraded":"2024-04-17T05:46:30.642292+0000","last_fullsized":"2024-04-17T05:46:30.642292+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'74","last_scrub_stamp":"2024-04-17T05:46:30.642053+0000","last_deep_scrub":"264'74","last_deep_scrub_stamp":"2024-04-17T05:46:30.642053+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.642053+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:04.769101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'281","reported_seq":929,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.540711+0000","last_change":"2024-04-17T05:46:45.540862+0000","last_active":"2024-04-17T05:46:45.540711+0000","last_peered":"2024-04-17T05:46:45.540711+0000","last_clean":"2024-04-17T05:46:45.540711+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:46:45.540711+0000","last_undegraded":"2024-04-17T05:46:45.540711+0000","last_fullsized":"2024-04-17T05:46:45.540711+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:45.540487+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:45.540487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.540487+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:58.652080+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017984,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":431,"num_read_kb":11105,"num_write":979,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.773169+0000","last_change":"2024-04-17T05:46:22.773169+0000","last_active":"2024-04-17T05:46:22.773169+0000","last_peered":"2024-04-17T05:46:22.773169+0000","last_clean":"2024-04-17T05:46:22.773169+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:46:22.773169+0000","last_undegraded":"2024-04-17T05:46:22.773169+0000","last_fullsized":"2024-04-17T05:46:22.773169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.772889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:13.164944+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'307","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.772343+0000","last_change":"2024-04-17T05:46:36.772420+0000","last_active":"2024-04-17T05:46:36.772343+0000","last_peered":"2024-04-17T05:46:36.772343+0000","last_clean":"2024-04-17T05:46:36.772343+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:46:36.772343+0000","last_undegraded":"2024-04-17T05:46:36.772343+0000","last_fullsized":"2024-04-17T05:46:36.772343+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'307","last_scrub_stamp":"2024-04-17T05:46:36.772164+0000","last_deep_scrub":"264'307","last_deep_scrub_stamp":"2024-04-17T05:46:36.772164+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.772164+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.701358+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44026,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":524,"num_read_kb":1245,"num_write":424,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.845796+0000","last_change":"2024-04-17T05:46:28.845796+0000","last_active":"2024-04-17T05:46:28.845796+0000","last_peered":"2024-04-17T05:46:28.845796+0000","last_clean":"2024-04-17T05:46:28.845796+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:46:28.845796+0000","last_undegraded":"2024-04-17T05:46:28.845796+0000","last_fullsized":"2024-04-17T05:46:28.845796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.845537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.845537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.845537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:25:28.366296+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'99","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.795125+0000","last_change":"2024-04-17T05:46:17.795125+0000","last_active":"2024-04-17T05:46:17.795125+0000","last_peered":"2024-04-17T05:46:17.795125+0000","last_clean":"2024-04-17T05:46:17.795125+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:46:17.795125+0000","last_undegraded":"2024-04-17T05:46:17.795125+0000","last_fullsized":"2024-04-17T05:46:17.795125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'99","last_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_deep_scrub":"264'99","last_deep_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.794856+0000","objects_scrubbed":1,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:18.080437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'127","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.804194+0000","last_change":"2024-04-17T05:46:23.804194+0000","last_active":"2024-04-17T05:46:23.804194+0000","last_peered":"2024-04-17T05:46:23.804194+0000","last_clean":"2024-04-17T05:46:23.804194+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:46:23.804194+0000","last_undegraded":"2024-04-17T05:46:23.804194+0000","last_fullsized":"2024-04-17T05:46:23.804194+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'127","last_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_deep_scrub":"264'127","last_deep_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.803934+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:22.971464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'373","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.684055+0000","last_change":"2024-04-17T05:46:12.684055+0000","last_active":"2024-04-17T05:46:12.684055+0000","last_peered":"2024-04-17T05:46:12.684055+0000","last_clean":"2024-04-17T05:46:12.684055+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:46:12.684055+0000","last_undegraded":"2024-04-17T05:46:12.684055+0000","last_fullsized":"2024-04-17T05:46:12.684055+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.683727+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:58.133653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":625,"num_read_kb":15946,"num_write":1543,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.697466+0000","last_change":"2024-04-17T05:46:52.697520+0000","last_active":"2024-04-17T05:46:52.697466+0000","last_peered":"2024-04-17T05:46:52.697466+0000","last_clean":"2024-04-17T05:46:52.697466+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:46:52.697466+0000","last_undegraded":"2024-04-17T05:46:52.697466+0000","last_fullsized":"2024-04-17T05:46:52.697466+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:52.697265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:52.697265+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.697265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:56.004700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'59","reported_seq":580,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.558443+0000","last_change":"2024-04-17T05:46:51.558538+0000","last_active":"2024-04-17T05:46:51.558443+0000","last_peered":"2024-04-17T05:46:51.558443+0000","last_clean":"2024-04-17T05:46:51.558443+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:46:51.558443+0000","last_undegraded":"2024-04-17T05:46:51.558443+0000","last_fullsized":"2024-04-17T05:46:51.558443+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'59","last_scrub_stamp":"2024-04-17T05:46:51.558239+0000","last_deep_scrub":"264'59","last_deep_scrub_stamp":"2024-04-17T05:46:51.558239+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.558239+0000","objects_scrubbed":3,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:08.889728+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":500,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.325328+0000","last_change":"2024-04-17T05:46:33.325328+0000","last_active":"2024-04-17T05:46:33.325328+0000","last_peered":"2024-04-17T05:46:33.325328+0000","last_clean":"2024-04-17T05:46:33.325328+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:46:33.325328+0000","last_undegraded":"2024-04-17T05:46:33.325328+0000","last_fullsized":"2024-04-17T05:46:33.325328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:33.325058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:33.325058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.325058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:24.544642+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'219","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.647715+0000","last_change":"2024-04-17T05:46:20.647715+0000","last_active":"2024-04-17T05:46:20.647715+0000","last_peered":"2024-04-17T05:46:20.647715+0000","last_clean":"2024-04-17T05:46:20.647715+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:46:20.647715+0000","last_undegraded":"2024-04-17T05:46:20.647715+0000","last_fullsized":"2024-04-17T05:46:20.647715+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'219","last_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_deep_scrub":"264'219","last_deep_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.647446+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:57.324481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.733738+0000","last_change":"2024-04-17T05:46:40.733801+0000","last_active":"2024-04-17T05:46:40.733738+0000","last_peered":"2024-04-17T05:46:40.733738+0000","last_clean":"2024-04-17T05:46:40.733738+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:46:40.733738+0000","last_undegraded":"2024-04-17T05:46:40.733738+0000","last_fullsized":"2024-04-17T05:46:40.733738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'196","last_scrub_stamp":"2024-04-17T05:46:40.733533+0000","last_deep_scrub":"264'196","last_deep_scrub_stamp":"2024-04-17T05:46:40.733533+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.733533+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:04.953287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'356","reported_seq":1045,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.248025+0000","last_change":"2024-04-17T05:46:44.248263+0000","last_active":"2024-04-17T05:46:44.248025+0000","last_peered":"2024-04-17T05:46:44.248025+0000","last_clean":"2024-04-17T05:46:44.248025+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:46:44.248025+0000","last_undegraded":"2024-04-17T05:46:44.248025+0000","last_fullsized":"2024-04-17T05:46:44.248025+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:44.247646+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:44.247646+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.247646+0000","objects_scrubbed":6,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:11.550314+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":588,"num_read_kb":11343,"num_write":1303,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.358946+0000","last_change":"2024-04-17T05:46:34.359052+0000","last_active":"2024-04-17T05:46:34.358946+0000","last_peered":"2024-04-17T05:46:34.358946+0000","last_clean":"2024-04-17T05:46:34.358946+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:46:34.358946+0000","last_undegraded":"2024-04-17T05:46:34.358946+0000","last_fullsized":"2024-04-17T05:46:34.358946+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:34.358700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:34.358700+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.358700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:23.819994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'345","reported_seq":960,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.793714+0000","last_change":"2024-04-17T05:46:35.793792+0000","last_active":"2024-04-17T05:46:35.793714+0000","last_peered":"2024-04-17T05:46:35.793714+0000","last_clean":"2024-04-17T05:46:35.793714+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:46:35.793714+0000","last_undegraded":"2024-04-17T05:46:35.793714+0000","last_fullsized":"2024-04-17T05:46:35.793714+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'345","last_scrub_stamp":"2024-04-17T05:46:35.793519+0000","last_deep_scrub":"264'345","last_deep_scrub_stamp":"2024-04-17T05:46:35.793519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.793519+0000","objects_scrubbed":5,"log_size":45,"log_dups_size":300,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:19.755889+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48000,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":552,"num_read_kb":1237,"num_write":440,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":500,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.983452+0000","last_change":"2024-04-17T05:46:17.983452+0000","last_active":"2024-04-17T05:46:17.983452+0000","last_peered":"2024-04-17T05:46:17.983452+0000","last_clean":"2024-04-17T05:46:17.983452+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:46:17.983452+0000","last_undegraded":"2024-04-17T05:46:17.983452+0000","last_fullsized":"2024-04-17T05:46:17.983452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.983070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:29.782925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'113","reported_seq":610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.111907+0000","last_change":"2024-04-17T05:46:14.111907+0000","last_active":"2024-04-17T05:46:14.111907+0000","last_peered":"2024-04-17T05:46:14.111907+0000","last_clean":"2024-04-17T05:46:14.111907+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:46:14.111907+0000","last_undegraded":"2024-04-17T05:46:14.111907+0000","last_fullsized":"2024-04-17T05:46:14.111907+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'113","last_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_deep_scrub":"264'113","last_deep_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.111624+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:35.374698+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":769,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.750706+0000","last_change":"2024-04-17T05:46:49.750765+0000","last_active":"2024-04-17T05:46:49.750706+0000","last_peered":"2024-04-17T05:46:49.750706+0000","last_clean":"2024-04-17T05:46:49.750706+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:46:49.750706+0000","last_undegraded":"2024-04-17T05:46:49.750706+0000","last_fullsized":"2024-04-17T05:46:49.750706+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'132","last_scrub_stamp":"2024-04-17T05:46:49.750514+0000","last_deep_scrub":"264'132","last_deep_scrub_stamp":"2024-04-17T05:46:49.750514+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.750514+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:01:14.989520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'310","reported_seq":968,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.608799+0000","last_change":"2024-04-17T05:46:56.608932+0000","last_active":"2024-04-17T05:46:56.608799+0000","last_peered":"2024-04-17T05:46:56.608799+0000","last_clean":"2024-04-17T05:46:56.608799+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:46:56.608799+0000","last_undegraded":"2024-04-17T05:46:56.608799+0000","last_fullsized":"2024-04-17T05:46:56.608799+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'310","last_scrub_stamp":"2024-04-17T05:46:56.608557+0000","last_deep_scrub":"264'310","last_deep_scrub_stamp":"2024-04-17T05:46:56.608557+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.608557+0000","objects_scrubbed":9,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:29.339994+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12846794,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":445,"num_read_kb":84514,"num_write":990,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.942372+0000","last_change":"2024-04-17T05:46:16.942372+0000","last_active":"2024-04-17T05:46:16.942372+0000","last_peered":"2024-04-17T05:46:16.942372+0000","last_clean":"2024-04-17T05:46:16.942372+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:46:16.942372+0000","last_undegraded":"2024-04-17T05:46:16.942372+0000","last_fullsized":"2024-04-17T05:46:16.942372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.942121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:13.925928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'262","reported_seq":852,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.356250+0000","last_change":"2024-04-17T05:46:40.356321+0000","last_active":"2024-04-17T05:46:40.356250+0000","last_peered":"2024-04-17T05:46:40.356250+0000","last_clean":"2024-04-17T05:46:40.356250+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:46:40.356250+0000","last_undegraded":"2024-04-17T05:46:40.356250+0000","last_fullsized":"2024-04-17T05:46:40.356250+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'262","last_scrub_stamp":"2024-04-17T05:46:40.356034+0000","last_deep_scrub":"264'262","last_deep_scrub_stamp":"2024-04-17T05:46:40.356034+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.356034+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:11.830162+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47325,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":432,"num_read_kb":996,"num_write":342,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.631115+0000","last_change":"2024-04-17T05:46:30.631115+0000","last_active":"2024-04-17T05:46:30.631115+0000","last_peered":"2024-04-17T05:46:30.631115+0000","last_clean":"2024-04-17T05:46:30.631115+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:46:30.631115+0000","last_undegraded":"2024-04-17T05:46:30.631115+0000","last_fullsized":"2024-04-17T05:46:30.631115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.630868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.630868+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.630868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:07:08.526024+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'290","reported_seq":926,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.020201+0000","last_change":"2024-04-17T05:46:23.020277+0000","last_active":"2024-04-17T05:46:23.020201+0000","last_peered":"2024-04-17T05:46:23.020201+0000","last_clean":"2024-04-17T05:46:23.020201+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:46:23.020201+0000","last_undegraded":"2024-04-17T05:46:23.020201+0000","last_fullsized":"2024-04-17T05:46:23.020201+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'290","last_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_deep_scrub":"264'290","last_deep_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.019975+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:21.304396+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'313","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.620802+0000","last_change":"2024-04-17T05:46:56.620978+0000","last_active":"2024-04-17T05:46:56.620802+0000","last_peered":"2024-04-17T05:46:56.620802+0000","last_clean":"2024-04-17T05:46:56.620802+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:46:56.620802+0000","last_undegraded":"2024-04-17T05:46:56.620802+0000","last_fullsized":"2024-04-17T05:46:56.620802+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'313","last_scrub_stamp":"2024-04-17T05:46:56.620571+0000","last_deep_scrub":"264'313","last_deep_scrub_stamp":"2024-04-17T05:46:56.620571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.620571+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:43.525135+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752753,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":542,"num_read_kb":33053,"num_write":1174,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.615906+0000","last_change":"2024-04-17T05:46:45.615960+0000","last_active":"2024-04-17T05:46:45.615906+0000","last_peered":"2024-04-17T05:46:45.615906+0000","last_clean":"2024-04-17T05:46:45.615906+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:46:45.615906+0000","last_undegraded":"2024-04-17T05:46:45.615906+0000","last_fullsized":"2024-04-17T05:46:45.615906+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:45.615722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:45.615722+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.615722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:21.102882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'480","reported_seq":1141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.798222+0000","last_change":"2024-04-17T05:46:24.798222+0000","last_active":"2024-04-17T05:46:24.798222+0000","last_peered":"2024-04-17T05:46:24.798222+0000","last_clean":"2024-04-17T05:46:24.798222+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:46:24.798222+0000","last_undegraded":"2024-04-17T05:46:24.798222+0000","last_fullsized":"2024-04-17T05:46:24.798222+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'480","last_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_deep_scrub":"264'480","last_deep_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.797947+0000","objects_scrubbed":7,"log_size":80,"log_dups_size":400,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:12.511479+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87094,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":846,"num_read_kb":2027,"num_write":720,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22936,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.147623+0000","last_change":"2024-04-17T05:46:19.147623+0000","last_active":"2024-04-17T05:46:19.147623+0000","last_peered":"2024-04-17T05:46:19.147623+0000","last_clean":"2024-04-17T05:46:19.147623+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:46:19.147623+0000","last_undegraded":"2024-04-17T05:46:19.147623+0000","last_fullsized":"2024-04-17T05:46:19.147623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.147370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:18.842204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'174","reported_seq":691,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.787327+0000","last_change":"2024-04-17T05:46:21.787327+0000","last_active":"2024-04-17T05:46:21.787327+0000","last_peered":"2024-04-17T05:46:21.787327+0000","last_clean":"2024-04-17T05:46:21.787327+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:46:21.787327+0000","last_undegraded":"2024-04-17T05:46:21.787327+0000","last_fullsized":"2024-04-17T05:46:21.787327+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.787059+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:11.126921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.695888+0000","last_change":"2024-04-17T05:46:41.696009+0000","last_active":"2024-04-17T05:46:41.695888+0000","last_peered":"2024-04-17T05:46:41.695888+0000","last_clean":"2024-04-17T05:46:41.695888+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:46:41.695888+0000","last_undegraded":"2024-04-17T05:46:41.695888+0000","last_fullsized":"2024-04-17T05:46:41.695888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'104","last_scrub_stamp":"2024-04-17T05:46:41.695676+0000","last_deep_scrub":"264'104","last_deep_scrub_stamp":"2024-04-17T05:46:41.695676+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.695676+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:56:52.077069+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'315","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.805432+0000","last_change":"2024-04-17T05:46:40.805432+0000","last_active":"2024-04-17T05:46:40.805432+0000","last_peered":"2024-04-17T05:46:40.805432+0000","last_clean":"2024-04-17T05:46:40.805432+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:46:40.805432+0000","last_undegraded":"2024-04-17T05:46:40.805432+0000","last_fullsized":"2024-04-17T05:46:40.805432+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'315","last_scrub_stamp":"2024-04-17T05:46:40.805067+0000","last_deep_scrub":"264'315","last_deep_scrub_stamp":"2024-04-17T05:46:40.805067+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.805067+0000","objects_scrubbed":9,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:14:53.427777+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122783,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":510,"num_read_kb":112390,"num_write":1223,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.012291+0000","last_change":"2024-04-17T05:46:19.012353+0000","last_active":"2024-04-17T05:46:19.012291+0000","last_peered":"2024-04-17T05:46:19.012291+0000","last_clean":"2024-04-17T05:46:19.012291+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:46:19.012291+0000","last_undegraded":"2024-04-17T05:46:19.012291+0000","last_fullsized":"2024-04-17T05:46:19.012291+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.012071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:43.685517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'323","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.261454+0000","last_change":"2024-04-17T05:46:44.261542+0000","last_active":"2024-04-17T05:46:44.261454+0000","last_peered":"2024-04-17T05:46:44.261454+0000","last_clean":"2024-04-17T05:46:44.261454+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:46:44.261454+0000","last_undegraded":"2024-04-17T05:46:44.261454+0000","last_fullsized":"2024-04-17T05:46:44.261454+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'323","last_scrub_stamp":"2024-04-17T05:46:44.261253+0000","last_deep_scrub":"264'323","last_deep_scrub_stamp":"2024-04-17T05:46:44.261253+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.261253+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:24.010153+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40265,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":1168,"num_write":442,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10932,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.856669+0000","last_change":"2024-04-17T05:46:42.856727+0000","last_active":"2024-04-17T05:46:42.856669+0000","last_peered":"2024-04-17T05:46:42.856669+0000","last_clean":"2024-04-17T05:46:42.856669+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:46:42.856669+0000","last_undegraded":"2024-04-17T05:46:42.856669+0000","last_fullsized":"2024-04-17T05:46:42.856669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.856474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.856474+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.856474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:20:08.791141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'180","reported_seq":726,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.904163+0000","last_change":"2024-04-17T05:46:39.904297+0000","last_active":"2024-04-17T05:46:39.904163+0000","last_peered":"2024-04-17T05:46:39.904163+0000","last_clean":"2024-04-17T05:46:39.904163+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:46:39.904163+0000","last_undegraded":"2024-04-17T05:46:39.904163+0000","last_fullsized":"2024-04-17T05:46:39.904163+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'180","last_scrub_stamp":"2024-04-17T05:46:39.903938+0000","last_deep_scrub":"264'180","last_deep_scrub_stamp":"2024-04-17T05:46:39.903938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.903938+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.200762+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'115","reported_seq":695,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.716735+0000","last_change":"2024-04-17T05:46:51.716821+0000","last_active":"2024-04-17T05:46:51.716735+0000","last_peered":"2024-04-17T05:46:51.716735+0000","last_clean":"2024-04-17T05:46:51.716735+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:46:51.716735+0000","last_undegraded":"2024-04-17T05:46:51.716735+0000","last_fullsized":"2024-04-17T05:46:51.716735+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'115","last_scrub_stamp":"2024-04-17T05:46:51.716517+0000","last_deep_scrub":"264'115","last_deep_scrub_stamp":"2024-04-17T05:46:51.716517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.716517+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":100,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:30.417919+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'358","reported_seq":1052,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.580780+0000","last_change":"2024-04-17T05:46:44.580927+0000","last_active":"2024-04-17T05:46:44.580780+0000","last_peered":"2024-04-17T05:46:44.580780+0000","last_clean":"2024-04-17T05:46:44.580780+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:46:44.580780+0000","last_undegraded":"2024-04-17T05:46:44.580780+0000","last_fullsized":"2024-04-17T05:46:44.580780+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'358","last_scrub_stamp":"2024-04-17T05:46:44.580574+0000","last_deep_scrub":"264'358","last_deep_scrub_stamp":"2024-04-17T05:46:44.580574+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.580574+0000","objects_scrubbed":8,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:15.221508+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568179,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":576,"num_read_kb":10795,"num_write":1350,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"264'4","reported_seq":844,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.630354+0000","last_change":"2024-04-17T05:46:36.630422+0000","last_active":"2024-04-17T05:46:36.630354+0000","last_peered":"2024-04-17T05:46:36.630354+0000","last_clean":"2024-04-17T05:46:36.630354+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:46:36.630354+0000","last_undegraded":"2024-04-17T05:46:36.630354+0000","last_fullsized":"2024-04-17T05:46:36.630354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4","last_scrub_stamp":"2024-04-17T05:46:36.630154+0000","last_deep_scrub":"264'4","last_deep_scrub_stamp":"2024-04-17T05:46:36.630154+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.630154+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:28.761949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":499,"num_read_kb":497,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'304","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.834277+0000","last_change":"2024-04-17T05:46:32.834435+0000","last_active":"2024-04-17T05:46:32.834277+0000","last_peered":"2024-04-17T05:46:32.834277+0000","last_clean":"2024-04-17T05:46:32.834277+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:46:32.834277+0000","last_undegraded":"2024-04-17T05:46:32.834277+0000","last_fullsized":"2024-04-17T05:46:32.834277+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'304","last_scrub_stamp":"2024-04-17T05:46:32.834034+0000","last_deep_scrub":"264'304","last_deep_scrub_stamp":"2024-04-17T05:46:32.834034+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.834034+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:37.797882+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45015,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":503,"num_read_kb":1173,"num_write":404,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.782872+0000","last_change":"2024-04-17T05:46:32.782872+0000","last_active":"2024-04-17T05:46:32.782872+0000","last_peered":"2024-04-17T05:46:32.782872+0000","last_clean":"2024-04-17T05:46:32.782872+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:46:32.782872+0000","last_undegraded":"2024-04-17T05:46:32.782872+0000","last_fullsized":"2024-04-17T05:46:32.782872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.782613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.782613+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.782613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:22:08.388670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'105","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.146585+0000","last_change":"2024-04-17T05:46:11.146585+0000","last_active":"2024-04-17T05:46:11.146585+0000","last_peered":"2024-04-17T05:46:11.146585+0000","last_clean":"2024-04-17T05:46:11.146585+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:46:11.146585+0000","last_undegraded":"2024-04-17T05:46:11.146585+0000","last_fullsized":"2024-04-17T05:46:11.146585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.146298+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:24.914414+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.440967+0000","last_change":"2024-04-17T05:46:27.440967+0000","last_active":"2024-04-17T05:46:27.440967+0000","last_peered":"2024-04-17T05:46:27.440967+0000","last_clean":"2024-04-17T05:46:27.440967+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:46:27.440967+0000","last_undegraded":"2024-04-17T05:46:27.440967+0000","last_fullsized":"2024-04-17T05:46:27.440967+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'84","last_scrub_stamp":"2024-04-17T05:46:27.440660+0000","last_deep_scrub":"264'84","last_deep_scrub_stamp":"2024-04-17T05:46:27.440660+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.440660+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:26.099591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'322","reported_seq":1157,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.598893+0000","last_change":"2024-04-17T05:46:57.599081+0000","last_active":"2024-04-17T05:46:57.598893+0000","last_peered":"2024-04-17T05:46:57.598893+0000","last_clean":"2024-04-17T05:46:57.598893+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:46:57.598893+0000","last_undegraded":"2024-04-17T05:46:57.598893+0000","last_fullsized":"2024-04-17T05:46:57.598893+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:57.598571+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:57.598571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.598571+0000","objects_scrubbed":3,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:51.547796+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263230,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":844,"num_read_kb":11279,"num_write":1205,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.722456+0000","last_change":"2024-04-17T05:46:18.722456+0000","last_active":"2024-04-17T05:46:18.722456+0000","last_peered":"2024-04-17T05:46:18.722456+0000","last_clean":"2024-04-17T05:46:18.722456+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:46:18.722456+0000","last_undegraded":"2024-04-17T05:46:18.722456+0000","last_fullsized":"2024-04-17T05:46:18.722456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.722193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:41.699090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'126","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.818501+0000","last_change":"2024-04-17T05:46:19.818501+0000","last_active":"2024-04-17T05:46:19.818501+0000","last_peered":"2024-04-17T05:46:19.818501+0000","last_clean":"2024-04-17T05:46:19.818501+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:46:19.818501+0000","last_undegraded":"2024-04-17T05:46:19.818501+0000","last_fullsized":"2024-04-17T05:46:19.818501+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'126","last_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_deep_scrub":"264'126","last_deep_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.818199+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:31.924099+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1648,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'539","reported_seq":2323,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.960848+0000","last_change":"2024-04-17T05:46:15.961007+0000","last_active":"2024-04-17T05:46:15.960848+0000","last_peered":"2024-04-17T05:46:15.960848+0000","last_clean":"2024-04-17T05:46:15.960848+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:46:15.960848+0000","last_undegraded":"2024-04-17T05:46:15.960848+0000","last_fullsized":"2024-04-17T05:46:15.960848+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.960604+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:11.021591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'116","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.290621+0000","last_change":"2024-04-17T05:46:27.290621+0000","last_active":"2024-04-17T05:46:27.290621+0000","last_peered":"2024-04-17T05:46:27.290621+0000","last_clean":"2024-04-17T05:46:27.290621+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:46:27.290621+0000","last_undegraded":"2024-04-17T05:46:27.290621+0000","last_fullsized":"2024-04-17T05:46:27.290621+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'116","last_scrub_stamp":"2024-04-17T05:46:27.290353+0000","last_deep_scrub":"264'116","last_deep_scrub_stamp":"2024-04-17T05:46:27.290353+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.290353+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:29.531238+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'231","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.753757+0000","last_change":"2024-04-17T05:46:26.753757+0000","last_active":"2024-04-17T05:46:26.753757+0000","last_peered":"2024-04-17T05:46:26.753757+0000","last_clean":"2024-04-17T05:46:26.753757+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:46:26.753757+0000","last_undegraded":"2024-04-17T05:46:26.753757+0000","last_fullsized":"2024-04-17T05:46:26.753757+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'231","last_scrub_stamp":"2024-04-17T05:46:26.753483+0000","last_deep_scrub":"264'231","last_deep_scrub_stamp":"2024-04-17T05:46:26.753483+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.753483+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:44.040635+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":397,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'330","reported_seq":939,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.366081+0000","last_change":"2024-04-17T05:46:35.366211+0000","last_active":"2024-04-17T05:46:35.366081+0000","last_peered":"2024-04-17T05:46:35.366081+0000","last_clean":"2024-04-17T05:46:35.366081+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:46:35.366081+0000","last_undegraded":"2024-04-17T05:46:35.366081+0000","last_fullsized":"2024-04-17T05:46:35.366081+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'330","last_scrub_stamp":"2024-04-17T05:46:35.365868+0000","last_deep_scrub":"264'330","last_deep_scrub_stamp":"2024-04-17T05:46:35.365868+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.365868+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.660066+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244482,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":438,"num_read_kb":414,"num_write":1194,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.406987+0000","last_change":"2024-04-17T05:46:35.407172+0000","last_active":"2024-04-17T05:46:35.406987+0000","last_peered":"2024-04-17T05:46:35.406987+0000","last_clean":"2024-04-17T05:46:35.406987+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:46:35.406987+0000","last_undegraded":"2024-04-17T05:46:35.406987+0000","last_fullsized":"2024-04-17T05:46:35.406987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:35.406755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:35.406755+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.406755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:18.935388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'352","reported_seq":985,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.303937+0000","last_change":"2024-04-17T05:46:50.303937+0000","last_active":"2024-04-17T05:46:50.303937+0000","last_peered":"2024-04-17T05:46:50.303937+0000","last_clean":"2024-04-17T05:46:50.303937+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:46:50.303937+0000","last_undegraded":"2024-04-17T05:46:50.303937+0000","last_fullsized":"2024-04-17T05:46:50.303937+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'352","last_scrub_stamp":"2024-04-17T05:46:50.303664+0000","last_deep_scrub":"264'352","last_deep_scrub_stamp":"2024-04-17T05:46:50.303664+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.303664+0000","objects_scrubbed":11,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:50.032317+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44869,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":547,"num_read_kb":1136,"num_write":426,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.659892+0000","last_change":"2024-04-17T05:46:26.659892+0000","last_active":"2024-04-17T05:46:26.659892+0000","last_peered":"2024-04-17T05:46:26.659892+0000","last_clean":"2024-04-17T05:46:26.659892+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:46:26.659892+0000","last_undegraded":"2024-04-17T05:46:26.659892+0000","last_fullsized":"2024-04-17T05:46:26.659892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:26.659492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:26.659492+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.659492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:45.664443+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'131","reported_seq":643,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.296344+0000","last_change":"2024-04-17T05:46:25.296344+0000","last_active":"2024-04-17T05:46:25.296344+0000","last_peered":"2024-04-17T05:46:25.296344+0000","last_clean":"2024-04-17T05:46:25.296344+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:46:25.296344+0000","last_undegraded":"2024-04-17T05:46:25.296344+0000","last_fullsized":"2024-04-17T05:46:25.296344+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'131","last_scrub_stamp":"2024-04-17T05:46:25.295943+0000","last_deep_scrub":"264'131","last_deep_scrub_stamp":"2024-04-17T05:46:25.295943+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.295943+0000","objects_scrubbed":4,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:33.340705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":814,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.510682+0000","last_change":"2024-04-17T05:46:47.510744+0000","last_active":"2024-04-17T05:46:47.510682+0000","last_peered":"2024-04-17T05:46:47.510682+0000","last_clean":"2024-04-17T05:46:47.510682+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:46:47.510682+0000","last_undegraded":"2024-04-17T05:46:47.510682+0000","last_fullsized":"2024-04-17T05:46:47.510682+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'162","last_scrub_stamp":"2024-04-17T05:46:47.510484+0000","last_deep_scrub":"264'162","last_deep_scrub_stamp":"2024-04-17T05:46:47.510484+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.510484+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:06.834002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'349","reported_seq":1016,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.338098+0000","last_change":"2024-04-17T05:46:39.338196+0000","last_active":"2024-04-17T05:46:39.338098+0000","last_peered":"2024-04-17T05:46:39.338098+0000","last_clean":"2024-04-17T05:46:39.338098+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:46:39.338098+0000","last_undegraded":"2024-04-17T05:46:39.338098+0000","last_fullsized":"2024-04-17T05:46:39.338098+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'349","last_scrub_stamp":"2024-04-17T05:46:39.337896+0000","last_deep_scrub":"264'349","last_deep_scrub_stamp":"2024-04-17T05:46:39.337896+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.337896+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:30.714630+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":31239,"num_write":1274,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.337367+0000","last_change":"2024-04-17T05:46:38.337476+0000","last_active":"2024-04-17T05:46:38.337367+0000","last_peered":"2024-04-17T05:46:38.337367+0000","last_clean":"2024-04-17T05:46:38.337367+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:46:38.337367+0000","last_undegraded":"2024-04-17T05:46:38.337367+0000","last_fullsized":"2024-04-17T05:46:38.337367+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.337093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.337093+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.337093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:42.557669+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'278","reported_seq":873,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.603830+0000","last_change":"2024-04-17T05:46:49.603906+0000","last_active":"2024-04-17T05:46:49.603830+0000","last_peered":"2024-04-17T05:46:49.603830+0000","last_clean":"2024-04-17T05:46:49.603830+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:46:49.603830+0000","last_undegraded":"2024-04-17T05:46:49.603830+0000","last_fullsized":"2024-04-17T05:46:49.603830+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'278","last_scrub_stamp":"2024-04-17T05:46:49.603630+0000","last_deep_scrub":"264'278","last_deep_scrub_stamp":"2024-04-17T05:46:49.603630+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.603630+0000","objects_scrubbed":5,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:15.688909+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46669,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":461,"num_read_kb":1085,"num_write":368,"num_write_kb":285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.983269+0000","last_change":"2024-04-17T05:46:23.983269+0000","last_active":"2024-04-17T05:46:23.983269+0000","last_peered":"2024-04-17T05:46:23.983269+0000","last_clean":"2024-04-17T05:46:23.983269+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:46:23.983269+0000","last_undegraded":"2024-04-17T05:46:23.983269+0000","last_fullsized":"2024-04-17T05:46:23.983269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.983009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:24:08.626906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'136","reported_seq":667,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.796701+0000","last_change":"2024-04-17T05:46:20.796701+0000","last_active":"2024-04-17T05:46:20.796701+0000","last_peered":"2024-04-17T05:46:20.796701+0000","last_clean":"2024-04-17T05:46:20.796701+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:46:20.796701+0000","last_undegraded":"2024-04-17T05:46:20.796701+0000","last_fullsized":"2024-04-17T05:46:20.796701+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'136","last_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_deep_scrub":"264'136","last_deep_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.796340+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:15.845948+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'204","reported_seq":821,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.764924+0000","last_change":"2024-04-17T05:46:47.764924+0000","last_active":"2024-04-17T05:46:47.764924+0000","last_peered":"2024-04-17T05:46:47.764924+0000","last_clean":"2024-04-17T05:46:47.764924+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:46:47.764924+0000","last_undegraded":"2024-04-17T05:46:47.764924+0000","last_fullsized":"2024-04-17T05:46:47.764924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'204","last_scrub_stamp":"2024-04-17T05:46:47.764628+0000","last_deep_scrub":"264'204","last_deep_scrub_stamp":"2024-04-17T05:46:47.764628+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.764628+0000","objects_scrubbed":1,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:52.071861+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3065,"num_omap_keys":18,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'386","reported_seq":1059,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:03.718544+0000","last_change":"2024-04-17T05:47:03.718673+0000","last_active":"2024-04-17T05:47:03.718544+0000","last_peered":"2024-04-17T05:47:03.718544+0000","last_clean":"2024-04-17T05:47:03.718544+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:47:03.718544+0000","last_undegraded":"2024-04-17T05:47:03.718544+0000","last_fullsized":"2024-04-17T05:47:03.718544+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'386","last_scrub_stamp":"2024-04-17T05:47:03.718335+0000","last_deep_scrub":"264'386","last_deep_scrub_stamp":"2024-04-17T05:47:03.718335+0000","last_clean_scrub_stamp":"2024-04-17T05:47:03.718335+0000","objects_scrubbed":11,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:07.484945+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1731334,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":524,"num_read_kb":19843,"num_write":1402,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.586905+0000","last_change":"2024-04-17T05:46:40.586958+0000","last_active":"2024-04-17T05:46:40.586905+0000","last_peered":"2024-04-17T05:46:40.586905+0000","last_clean":"2024-04-17T05:46:40.586905+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:46:40.586905+0000","last_undegraded":"2024-04-17T05:46:40.586905+0000","last_fullsized":"2024-04-17T05:46:40.586905+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:40.586716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:40.586716+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.586716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:45:35.339099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'263","reported_seq":845,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.699498+0000","last_change":"2024-04-17T05:46:39.699578+0000","last_active":"2024-04-17T05:46:39.699498+0000","last_peered":"2024-04-17T05:46:39.699498+0000","last_clean":"2024-04-17T05:46:39.699498+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:46:39.699498+0000","last_undegraded":"2024-04-17T05:46:39.699498+0000","last_fullsized":"2024-04-17T05:46:39.699498+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'263","last_scrub_stamp":"2024-04-17T05:46:39.699314+0000","last_deep_scrub":"264'263","last_deep_scrub_stamp":"2024-04-17T05:46:39.699314+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.699314+0000","objects_scrubbed":3,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:28.949715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44294,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":445,"num_read_kb":1105,"num_write":366,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.762692+0000","last_change":"2024-04-17T05:46:18.762692+0000","last_active":"2024-04-17T05:46:18.762692+0000","last_peered":"2024-04-17T05:46:18.762692+0000","last_clean":"2024-04-17T05:46:18.762692+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:46:18.762692+0000","last_undegraded":"2024-04-17T05:46:18.762692+0000","last_fullsized":"2024-04-17T05:46:18.762692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.762296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:44.671414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'146","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.248471+0000","last_change":"2024-04-17T05:46:26.248471+0000","last_active":"2024-04-17T05:46:26.248471+0000","last_peered":"2024-04-17T05:46:26.248471+0000","last_clean":"2024-04-17T05:46:26.248471+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:46:26.248471+0000","last_undegraded":"2024-04-17T05:46:26.248471+0000","last_fullsized":"2024-04-17T05:46:26.248471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'146","last_scrub_stamp":"2024-04-17T05:46:26.248201+0000","last_deep_scrub":"264'146","last_deep_scrub_stamp":"2024-04-17T05:46:26.248201+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.248201+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:59.437725+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'299","reported_seq":987,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.566140+0000","last_change":"2024-04-17T05:46:32.566140+0000","last_active":"2024-04-17T05:46:32.566140+0000","last_peered":"2024-04-17T05:46:32.566140+0000","last_clean":"2024-04-17T05:46:32.566140+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:46:32.566140+0000","last_undegraded":"2024-04-17T05:46:32.566140+0000","last_fullsized":"2024-04-17T05:46:32.566140+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'299","last_scrub_stamp":"2024-04-17T05:46:32.565869+0000","last_deep_scrub":"264'299","last_deep_scrub_stamp":"2024-04-17T05:46:32.565869+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.565869+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:01:45.736052+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28561,"num_omap_keys":113,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'437","reported_seq":1152,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.595499+0000","last_change":"2024-04-17T05:46:50.595639+0000","last_active":"2024-04-17T05:46:50.595499+0000","last_peered":"2024-04-17T05:46:50.595499+0000","last_clean":"2024-04-17T05:46:50.595499+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:46:50.595499+0000","last_undegraded":"2024-04-17T05:46:50.595499+0000","last_fullsized":"2024-04-17T05:46:50.595499+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'437","last_scrub_stamp":"2024-04-17T05:46:50.595303+0000","last_deep_scrub":"264'437","last_deep_scrub_stamp":"2024-04-17T05:46:50.595303+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.595303+0000","objects_scrubbed":6,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:17:13.320357+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":515218,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":663,"num_read_kb":71297,"num_write":1686,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"264'8","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.766334+0000","last_change":"2024-04-17T05:46:20.766334+0000","last_active":"2024-04-17T05:46:20.766334+0000","last_peered":"2024-04-17T05:46:20.766334+0000","last_clean":"2024-04-17T05:46:20.766334+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:46:20.766334+0000","last_undegraded":"2024-04-17T05:46:20.766334+0000","last_fullsized":"2024-04-17T05:46:20.766334+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.766051+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:22.853924+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":501,"num_read_kb":814,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'324","reported_seq":944,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.796472+0000","last_change":"2024-04-17T05:46:25.796472+0000","last_active":"2024-04-17T05:46:25.796472+0000","last_peered":"2024-04-17T05:46:25.796472+0000","last_clean":"2024-04-17T05:46:25.796472+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:46:25.796472+0000","last_undegraded":"2024-04-17T05:46:25.796472+0000","last_fullsized":"2024-04-17T05:46:25.796472+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'324","last_scrub_stamp":"2024-04-17T05:46:25.796196+0000","last_deep_scrub":"264'324","last_deep_scrub_stamp":"2024-04-17T05:46:25.796196+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.796196+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:16.415248+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46192,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":530,"num_read_kb":1178,"num_write":420,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.616502+0000","last_change":"2024-04-17T05:46:29.616502+0000","last_active":"2024-04-17T05:46:29.616502+0000","last_peered":"2024-04-17T05:46:29.616502+0000","last_clean":"2024-04-17T05:46:29.616502+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:46:29.616502+0000","last_undegraded":"2024-04-17T05:46:29.616502+0000","last_fullsized":"2024-04-17T05:46:29.616502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.616255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.616255+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.616255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:25:32.388074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'141","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.802101+0000","last_change":"2024-04-17T05:46:10.802101+0000","last_active":"2024-04-17T05:46:10.802101+0000","last_peered":"2024-04-17T05:46:10.802101+0000","last_clean":"2024-04-17T05:46:10.802101+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:46:10.802101+0000","last_undegraded":"2024-04-17T05:46:10.802101+0000","last_fullsized":"2024-04-17T05:46:10.802101+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.801687+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.852555+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'155","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.741104+0000","last_change":"2024-04-17T05:46:21.741243+0000","last_active":"2024-04-17T05:46:21.741104+0000","last_peered":"2024-04-17T05:46:21.741104+0000","last_clean":"2024-04-17T05:46:21.741104+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:46:21.741104+0000","last_undegraded":"2024-04-17T05:46:21.741104+0000","last_fullsized":"2024-04-17T05:46:21.741104+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.740847+0000","objects_scrubbed":1,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:03.471838+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'420","reported_seq":1147,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.811870+0000","last_change":"2024-04-17T05:46:34.812010+0000","last_active":"2024-04-17T05:46:34.811870+0000","last_peered":"2024-04-17T05:46:34.811870+0000","last_clean":"2024-04-17T05:46:34.811870+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:46:34.811870+0000","last_undegraded":"2024-04-17T05:46:34.811870+0000","last_fullsized":"2024-04-17T05:46:34.811870+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'420","last_scrub_stamp":"2024-04-17T05:46:34.811680+0000","last_deep_scrub":"264'420","last_deep_scrub_stamp":"2024-04-17T05:46:34.811680+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.811680+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:57.763900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262268,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":835,"num_read_kb":6303,"num_write":1376,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":658,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.183156+0000","last_change":"2024-04-17T05:46:10.183156+0000","last_active":"2024-04-17T05:46:10.183156+0000","last_peered":"2024-04-17T05:46:10.183156+0000","last_clean":"2024-04-17T05:46:10.183156+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:46:10.183156+0000","last_undegraded":"2024-04-17T05:46:10.183156+0000","last_fullsized":"2024-04-17T05:46:10.183156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'90","last_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_deep_scrub":"264'90","last_deep_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.182909+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:58:10.036518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'365","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.440689+0000","last_change":"2024-04-17T05:46:26.440875+0000","last_active":"2024-04-17T05:46:26.440689+0000","last_peered":"2024-04-17T05:46:26.440689+0000","last_clean":"2024-04-17T05:46:26.440689+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:46:26.440689+0000","last_undegraded":"2024-04-17T05:46:26.440689+0000","last_fullsized":"2024-04-17T05:46:26.440689+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'365","last_scrub_stamp":"2024-04-17T05:46:26.440480+0000","last_deep_scrub":"264'365","last_deep_scrub_stamp":"2024-04-17T05:46:26.440480+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.440480+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:58.716606+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6192676,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":568,"num_read_kb":39832,"num_write":1198,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'149","reported_seq":772,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.243871+0000","last_change":"2024-04-17T05:46:27.243940+0000","last_active":"2024-04-17T05:46:27.243871+0000","last_peered":"2024-04-17T05:46:27.243871+0000","last_clean":"2024-04-17T05:46:27.243871+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:46:27.243871+0000","last_undegraded":"2024-04-17T05:46:27.243871+0000","last_fullsized":"2024-04-17T05:46:27.243871+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'149","last_scrub_stamp":"2024-04-17T05:46:27.243689+0000","last_deep_scrub":"264'149","last_deep_scrub_stamp":"2024-04-17T05:46:27.243689+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.243689+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:03.394294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'307","reported_seq":953,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.272331+0000","last_change":"2024-04-17T05:46:41.272639+0000","last_active":"2024-04-17T05:46:41.272331+0000","last_peered":"2024-04-17T05:46:41.272331+0000","last_clean":"2024-04-17T05:46:41.272331+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:46:41.272331+0000","last_undegraded":"2024-04-17T05:46:41.272331+0000","last_fullsized":"2024-04-17T05:46:41.272331+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'307","last_scrub_stamp":"2024-04-17T05:46:41.272065+0000","last_deep_scrub":"264'307","last_deep_scrub_stamp":"2024-04-17T05:46:41.272065+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.272065+0000","objects_scrubbed":10,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:14.652269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4341641,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":436,"num_read_kb":10784,"num_write":1165,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'95","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.829126+0000","last_change":"2024-04-17T05:46:31.829195+0000","last_active":"2024-04-17T05:46:31.829126+0000","last_peered":"2024-04-17T05:46:31.829126+0000","last_clean":"2024-04-17T05:46:31.829126+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:46:31.829126+0000","last_undegraded":"2024-04-17T05:46:31.829126+0000","last_fullsized":"2024-04-17T05:46:31.829126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'95","last_scrub_stamp":"2024-04-17T05:46:31.828927+0000","last_deep_scrub":"264'95","last_deep_scrub_stamp":"2024-04-17T05:46:31.828927+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.828927+0000","objects_scrubbed":1,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:59.765753+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'308","reported_seq":1043,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:59.554713+0000","last_change":"2024-04-17T05:46:59.554915+0000","last_active":"2024-04-17T05:46:59.554713+0000","last_peered":"2024-04-17T05:46:59.554713+0000","last_clean":"2024-04-17T05:46:59.554713+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:46:59.554713+0000","last_undegraded":"2024-04-17T05:46:59.554713+0000","last_fullsized":"2024-04-17T05:46:59.554713+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'308","last_scrub_stamp":"2024-04-17T05:46:59.554444+0000","last_deep_scrub":"264'308","last_deep_scrub_stamp":"2024-04-17T05:46:59.554444+0000","last_clean_scrub_stamp":"2024-04-17T05:46:59.554444+0000","objects_scrubbed":5,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:39.077454+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":709,"num_read_kb":32424,"num_write":1121,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":644,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.725516+0000","last_change":"2024-04-17T05:46:44.725575+0000","last_active":"2024-04-17T05:46:44.725516+0000","last_peered":"2024-04-17T05:46:44.725516+0000","last_clean":"2024-04-17T05:46:44.725516+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:46:44.725516+0000","last_undegraded":"2024-04-17T05:46:44.725516+0000","last_fullsized":"2024-04-17T05:46:44.725516+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'58","last_scrub_stamp":"2024-04-17T05:46:44.725324+0000","last_deep_scrub":"264'58","last_deep_scrub_stamp":"2024-04-17T05:46:44.725324+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.725324+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:00.884116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'281","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.814149+0000","last_change":"2024-04-17T05:46:16.814273+0000","last_active":"2024-04-17T05:46:16.814149+0000","last_peered":"2024-04-17T05:46:16.814149+0000","last_clean":"2024-04-17T05:46:16.814149+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:46:16.814149+0000","last_undegraded":"2024-04-17T05:46:16.814149+0000","last_fullsized":"2024-04-17T05:46:16.814149+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.813947+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:56.482239+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5047006,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":359,"num_read_kb":5921,"num_write":992,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":641,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.664684+0000","last_change":"2024-04-17T05:46:28.664753+0000","last_active":"2024-04-17T05:46:28.664684+0000","last_peered":"2024-04-17T05:46:28.664684+0000","last_clean":"2024-04-17T05:46:28.664684+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:46:28.664684+0000","last_undegraded":"2024-04-17T05:46:28.664684+0000","last_fullsized":"2024-04-17T05:46:28.664684+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'54","last_scrub_stamp":"2024-04-17T05:46:28.664477+0000","last_deep_scrub":"264'54","last_deep_scrub_stamp":"2024-04-17T05:46:28.664477+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.664477+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:11.472041+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'319","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.820625+0000","last_change":"2024-04-17T05:46:36.820896+0000","last_active":"2024-04-17T05:46:36.820625+0000","last_peered":"2024-04-17T05:46:36.820625+0000","last_clean":"2024-04-17T05:46:36.820625+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:46:36.820625+0000","last_undegraded":"2024-04-17T05:46:36.820625+0000","last_fullsized":"2024-04-17T05:46:36.820625+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:36.820373+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:36.820373+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.820373+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:09.390555+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5067983,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":492,"num_read_kb":71000,"num_write":1186,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'216","reported_seq":825,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.756140+0000","last_change":"2024-04-17T05:46:45.756202+0000","last_active":"2024-04-17T05:46:45.756140+0000","last_peered":"2024-04-17T05:46:45.756140+0000","last_clean":"2024-04-17T05:46:45.756140+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:46:45.756140+0000","last_undegraded":"2024-04-17T05:46:45.756140+0000","last_fullsized":"2024-04-17T05:46:45.756140+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'216","last_scrub_stamp":"2024-04-17T05:46:45.755911+0000","last_deep_scrub":"264'216","last_deep_scrub_stamp":"2024-04-17T05:46:45.755911+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.755911+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:01.103569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8079,"num_omap_keys":32,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'371","reported_seq":1061,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.914547+0000","last_change":"2024-04-17T05:46:38.914661+0000","last_active":"2024-04-17T05:46:38.914547+0000","last_peered":"2024-04-17T05:46:38.914547+0000","last_clean":"2024-04-17T05:46:38.914547+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:46:38.914547+0000","last_undegraded":"2024-04-17T05:46:38.914547+0000","last_fullsized":"2024-04-17T05:46:38.914547+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'371","last_scrub_stamp":"2024-04-17T05:46:38.914310+0000","last_deep_scrub":"264'371","last_deep_scrub_stamp":"2024-04-17T05:46:38.914310+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.914310+0000","objects_scrubbed":6,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:28.802337+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2189648,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":654,"num_read_kb":34085,"num_write":1471,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":757,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.000521+0000","last_change":"2024-04-17T05:46:27.000580+0000","last_active":"2024-04-17T05:46:27.000521+0000","last_peered":"2024-04-17T05:46:27.000521+0000","last_clean":"2024-04-17T05:46:27.000521+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:46:27.000521+0000","last_undegraded":"2024-04-17T05:46:27.000521+0000","last_fullsized":"2024-04-17T05:46:27.000521+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'135","last_scrub_stamp":"2024-04-17T05:46:27.000319+0000","last_deep_scrub":"264'135","last_deep_scrub_stamp":"2024-04-17T05:46:27.000319+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.000319+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:15.623855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'335","reported_seq":1015,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.675343+0000","last_change":"2024-04-17T05:46:53.675677+0000","last_active":"2024-04-17T05:46:53.675343+0000","last_peered":"2024-04-17T05:46:53.675343+0000","last_clean":"2024-04-17T05:46:53.675343+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:46:53.675343+0000","last_undegraded":"2024-04-17T05:46:53.675343+0000","last_fullsized":"2024-04-17T05:46:53.675343+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'335","last_scrub_stamp":"2024-04-17T05:46:53.675099+0000","last_deep_scrub":"264'335","last_deep_scrub_stamp":"2024-04-17T05:46:53.675099+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.675099+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:38:40.955125+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534609,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":546,"num_read_kb":42711,"num_write":1290,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.655975+0000","last_change":"2024-04-17T05:46:42.656040+0000","last_active":"2024-04-17T05:46:42.655975+0000","last_peered":"2024-04-17T05:46:42.655975+0000","last_clean":"2024-04-17T05:46:42.655975+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:46:42.655975+0000","last_undegraded":"2024-04-17T05:46:42.655975+0000","last_fullsized":"2024-04-17T05:46:42.655975+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:42.655782+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:42.655782+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.655782+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:25.917924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'300","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.230338+0000","last_change":"2024-04-17T05:46:48.230447+0000","last_active":"2024-04-17T05:46:48.230338+0000","last_peered":"2024-04-17T05:46:48.230338+0000","last_clean":"2024-04-17T05:46:48.230338+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:46:48.230338+0000","last_undegraded":"2024-04-17T05:46:48.230338+0000","last_fullsized":"2024-04-17T05:46:48.230338+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'300","last_scrub_stamp":"2024-04-17T05:46:48.230130+0000","last_deep_scrub":"264'300","last_deep_scrub_stamp":"2024-04-17T05:46:48.230130+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.230130+0000","objects_scrubbed":6,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:19.387099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262289,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":548,"num_write":1115,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":764,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.512660+0000","last_change":"2024-04-17T05:46:48.512719+0000","last_active":"2024-04-17T05:46:48.512660+0000","last_peered":"2024-04-17T05:46:48.512660+0000","last_clean":"2024-04-17T05:46:48.512660+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:46:48.512660+0000","last_undegraded":"2024-04-17T05:46:48.512660+0000","last_fullsized":"2024-04-17T05:46:48.512660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'110","last_scrub_stamp":"2024-04-17T05:46:48.512459+0000","last_deep_scrub":"264'110","last_deep_scrub_stamp":"2024-04-17T05:46:48.512459+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.512459+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:19.000329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'302","reported_seq":961,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:59.670232+0000","last_change":"2024-04-17T05:46:59.670336+0000","last_active":"2024-04-17T05:46:59.670232+0000","last_peered":"2024-04-17T05:46:59.670232+0000","last_clean":"2024-04-17T05:46:59.670232+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:46:59.670232+0000","last_undegraded":"2024-04-17T05:46:59.670232+0000","last_fullsized":"2024-04-17T05:46:59.670232+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'302","last_scrub_stamp":"2024-04-17T05:46:59.670021+0000","last_deep_scrub":"264'302","last_deep_scrub_stamp":"2024-04-17T05:46:59.670021+0000","last_clean_scrub_stamp":"2024-04-17T05:46:59.670021+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:52.914846+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171414,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":4352,"num_write":1098,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":847,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.463713+0000","last_change":"2024-04-17T05:46:31.463772+0000","last_active":"2024-04-17T05:46:31.463713+0000","last_peered":"2024-04-17T05:46:31.463713+0000","last_clean":"2024-04-17T05:46:31.463713+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:46:31.463713+0000","last_undegraded":"2024-04-17T05:46:31.463713+0000","last_fullsized":"2024-04-17T05:46:31.463713+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'202","last_scrub_stamp":"2024-04-17T05:46:31.463487+0000","last_deep_scrub":"264'202","last_deep_scrub_stamp":"2024-04-17T05:46:31.463487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.463487+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:24.787961+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'317","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.663532+0000","last_change":"2024-04-17T05:46:10.663532+0000","last_active":"2024-04-17T05:46:10.663532+0000","last_peered":"2024-04-17T05:46:10.663532+0000","last_clean":"2024-04-17T05:46:10.663532+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:46:10.663532+0000","last_undegraded":"2024-04-17T05:46:10.663532+0000","last_fullsized":"2024-04-17T05:46:10.663532+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'317","last_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_deep_scrub":"264'317","last_deep_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.663219+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.001315+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":443,"num_read_kb":16015,"num_write":1067,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'96","reported_seq":694,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.604944+0000","last_change":"2024-04-17T05:46:34.605089+0000","last_active":"2024-04-17T05:46:34.604944+0000","last_peered":"2024-04-17T05:46:34.604944+0000","last_clean":"2024-04-17T05:46:34.604944+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:46:34.604944+0000","last_undegraded":"2024-04-17T05:46:34.604944+0000","last_fullsized":"2024-04-17T05:46:34.604944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'96","last_scrub_stamp":"2024-04-17T05:46:34.604692+0000","last_deep_scrub":"264'96","last_deep_scrub_stamp":"2024-04-17T05:46:34.604692+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.604692+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:59.288725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'360","reported_seq":1087,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.647400+0000","last_change":"2024-04-17T05:46:55.647491+0000","last_active":"2024-04-17T05:46:55.647400+0000","last_peered":"2024-04-17T05:46:55.647400+0000","last_clean":"2024-04-17T05:46:55.647400+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:46:55.647400+0000","last_undegraded":"2024-04-17T05:46:55.647400+0000","last_fullsized":"2024-04-17T05:46:55.647400+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'360","last_scrub_stamp":"2024-04-17T05:46:55.647216+0000","last_deep_scrub":"264'360","last_deep_scrub_stamp":"2024-04-17T05:46:55.647216+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.647216+0000","objects_scrubbed":4,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:13.323803+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":746,"num_read_kb":12134,"num_write":1256,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":696,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.414667+0000","last_change":"2024-04-17T05:46:37.414725+0000","last_active":"2024-04-17T05:46:37.414667+0000","last_peered":"2024-04-17T05:46:37.414667+0000","last_clean":"2024-04-17T05:46:37.414667+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:46:37.414667+0000","last_undegraded":"2024-04-17T05:46:37.414667+0000","last_fullsized":"2024-04-17T05:46:37.414667+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:37.414492+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:37.414492+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.414492+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:18.807121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'357","reported_seq":1003,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.855024+0000","last_change":"2024-04-17T05:46:33.855258+0000","last_active":"2024-04-17T05:46:33.855024+0000","last_peered":"2024-04-17T05:46:33.855024+0000","last_clean":"2024-04-17T05:46:33.855024+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:46:33.855024+0000","last_undegraded":"2024-04-17T05:46:33.855024+0000","last_fullsized":"2024-04-17T05:46:33.855024+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'357","last_scrub_stamp":"2024-04-17T05:46:33.854698+0000","last_deep_scrub":"264'357","last_deep_scrub_stamp":"2024-04-17T05:46:33.854698+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.854698+0000","objects_scrubbed":3,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:38.333183+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":501,"num_read_kb":657,"num_write":1252,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'222","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.665956+0000","last_change":"2024-04-17T05:46:44.666020+0000","last_active":"2024-04-17T05:46:44.665956+0000","last_peered":"2024-04-17T05:46:44.665956+0000","last_clean":"2024-04-17T05:46:44.665956+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:46:44.665956+0000","last_undegraded":"2024-04-17T05:46:44.665956+0000","last_fullsized":"2024-04-17T05:46:44.665956+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'222","last_scrub_stamp":"2024-04-17T05:46:44.665753+0000","last_deep_scrub":"264'222","last_deep_scrub_stamp":"2024-04-17T05:46:44.665753+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.665753+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:05.646503+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'420","reported_seq":1168,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.230796+0000","last_change":"2024-04-17T05:46:45.230919+0000","last_active":"2024-04-17T05:46:45.230796+0000","last_peered":"2024-04-17T05:46:45.230796+0000","last_clean":"2024-04-17T05:46:45.230796+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:46:45.230796+0000","last_undegraded":"2024-04-17T05:46:45.230796+0000","last_fullsized":"2024-04-17T05:46:45.230796+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'420","last_scrub_stamp":"2024-04-17T05:46:45.230572+0000","last_deep_scrub":"264'420","last_deep_scrub_stamp":"2024-04-17T05:46:45.230572+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.230572+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:12.917895+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058993,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":894,"num_read_kb":32480,"num_write":1416,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.744514+0000","last_change":"2024-04-17T05:46:41.744578+0000","last_active":"2024-04-17T05:46:41.744514+0000","last_peered":"2024-04-17T05:46:41.744514+0000","last_clean":"2024-04-17T05:46:41.744514+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:46:41.744514+0000","last_undegraded":"2024-04-17T05:46:41.744514+0000","last_fullsized":"2024-04-17T05:46:41.744514+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'352","last_scrub_stamp":"2024-04-17T05:46:41.744313+0000","last_deep_scrub":"264'352","last_deep_scrub_stamp":"2024-04-17T05:46:41.744313+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.744313+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:33.007856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'314","reported_seq":1000,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.458853+0000","last_change":"2024-04-17T05:46:25.458952+0000","last_active":"2024-04-17T05:46:25.458853+0000","last_peered":"2024-04-17T05:46:25.458853+0000","last_clean":"2024-04-17T05:46:25.458853+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:46:25.458853+0000","last_undegraded":"2024-04-17T05:46:25.458853+0000","last_fullsized":"2024-04-17T05:46:25.458853+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'314","last_scrub_stamp":"2024-04-17T05:46:25.458647+0000","last_deep_scrub":"264'314","last_deep_scrub_stamp":"2024-04-17T05:46:25.458647+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.458647+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:02.295132+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7281800,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":526,"num_read_kb":46303,"num_write":1296,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'322","reported_seq":1053,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.325081+0000","last_change":"2024-04-17T05:46:23.325152+0000","last_active":"2024-04-17T05:46:23.325081+0000","last_peered":"2024-04-17T05:46:23.325081+0000","last_clean":"2024-04-17T05:46:23.325081+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:46:23.325081+0000","last_undegraded":"2024-04-17T05:46:23.325081+0000","last_fullsized":"2024-04-17T05:46:23.325081+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.324895+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:02.515429+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'328","reported_seq":971,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.276500+0000","last_change":"2024-04-17T05:46:49.276594+0000","last_active":"2024-04-17T05:46:49.276500+0000","last_peered":"2024-04-17T05:46:49.276500+0000","last_clean":"2024-04-17T05:46:49.276500+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:46:49.276500+0000","last_undegraded":"2024-04-17T05:46:49.276500+0000","last_fullsized":"2024-04-17T05:46:49.276500+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'328","last_scrub_stamp":"2024-04-17T05:46:49.276270+0000","last_deep_scrub":"264'328","last_deep_scrub_stamp":"2024-04-17T05:46:49.276270+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.276270+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:32.984779+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":466,"num_read_kb":49597,"num_write":1231,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'297","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.292701+0000","last_change":"2024-04-17T05:46:45.292805+0000","last_active":"2024-04-17T05:46:45.292701+0000","last_peered":"2024-04-17T05:46:45.292701+0000","last_clean":"2024-04-17T05:46:45.292701+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:46:45.292701+0000","last_undegraded":"2024-04-17T05:46:45.292701+0000","last_fullsized":"2024-04-17T05:46:45.292701+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'297","last_scrub_stamp":"2024-04-17T05:46:45.292505+0000","last_deep_scrub":"264'297","last_deep_scrub_stamp":"2024-04-17T05:46:45.292505+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.292505+0000","objects_scrubbed":8,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:54.845561+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129194,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":413,"num_read_kb":7661,"num_write":1087,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'244","reported_seq":853,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.670960+0000","last_change":"2024-04-17T05:46:37.670960+0000","last_active":"2024-04-17T05:46:37.670960+0000","last_peered":"2024-04-17T05:46:37.670960+0000","last_clean":"2024-04-17T05:46:37.670960+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:46:37.670960+0000","last_undegraded":"2024-04-17T05:46:37.670960+0000","last_fullsized":"2024-04-17T05:46:37.670960+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'244","last_scrub_stamp":"2024-04-17T05:46:37.670697+0000","last_deep_scrub":"264'244","last_deep_scrub_stamp":"2024-04-17T05:46:37.670697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.670697+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":200,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:58.583622+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'277","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.851613+0000","last_change":"2024-04-17T05:46:41.851831+0000","last_active":"2024-04-17T05:46:41.851613+0000","last_peered":"2024-04-17T05:46:41.851613+0000","last_clean":"2024-04-17T05:46:41.851613+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:46:41.851613+0000","last_undegraded":"2024-04-17T05:46:41.851613+0000","last_fullsized":"2024-04-17T05:46:41.851613+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'277","last_scrub_stamp":"2024-04-17T05:46:41.851309+0000","last_deep_scrub":"264'277","last_deep_scrub_stamp":"2024-04-17T05:46:41.851309+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.851309+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:22:42.210177+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":447,"num_read_kb":20933,"num_write":1123,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":829,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.314199+0000","last_change":"2024-04-17T05:46:29.314199+0000","last_active":"2024-04-17T05:46:29.314199+0000","last_peered":"2024-04-17T05:46:29.314199+0000","last_clean":"2024-04-17T05:46:29.314199+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:46:29.314199+0000","last_undegraded":"2024-04-17T05:46:29.314199+0000","last_fullsized":"2024-04-17T05:46:29.314199+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'198","last_scrub_stamp":"2024-04-17T05:46:29.313797+0000","last_deep_scrub":"264'198","last_deep_scrub_stamp":"2024-04-17T05:46:29.313797+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.313797+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:17.398694+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6748,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1282,"ondisk_log_size":1282,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4334,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1780,"ondisk_log_size":1780,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1458963,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":18010,"num_read_kb":37614,"num_write":15043,"num_write_kb":11218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":628236,"num_omap_keys":1518,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2038,"ondisk_log_size":2038,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1652,"num_read_kb":1963,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43882,"num_omap_keys":181,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3970,"ondisk_log_size":3970,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187841250,"num_objects":443,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":34833,"num_read_kb":1864137,"num_write":77494,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253607936,"data_stored":253607936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3747,"ondisk_log_size":3747,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":221363,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":145,"num_read_kb":210051,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575704,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349100,"kb_used_data":38884,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":94022740,"statfs":{"total":96636764160,"available":96279285760,"internally_reserved":0,"allocated":39817216,"data_stored":37809825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":317651402},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575705,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359832,"kb_used_data":39888,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94012008,"statfs":{"total":96636764160,"available":96268296192,"internally_reserved":0,"allocated":40845312,"data_stored":38885655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":327612871},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575705,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206104,"kb_used_data":58384,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94165736,"statfs":{"total":96636764160,"available":96425713664,"internally_reserved":0,"allocated":59785216,"data_stored":57737844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575707,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148100,"kb_used_data":36412,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94223740,"statfs":{"total":96636764160,"available":96485109760,"internally_reserved":0,"allocated":37285888,"data_stored":35330686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575705,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298456,"kb_used_data":42448,"kb_used_omap":1,"kb_used_meta":255998,"kb_avail":94073384,"statfs":{"total":96636764160,"available":96331145216,"internally_reserved":0,"allocated":43466752,"data_stored":41436434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":262142407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575709,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159304,"kb_used_data":56000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94212536,"statfs":{"total":96636764160,"available":96473636864,"internally_reserved":0,"allocated":57344000,"data_stored":55396653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575707,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183784,"kb_used_data":36064,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94188056,"statfs":{"total":96636764160,"available":96448569344,"internally_reserved":0,"allocated":36929536,"data_stored":34946873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575705,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327488,"kb_used_data":39544,"kb_used_omap":1,"kb_used_meta":287934,"kb_avail":94044352,"statfs":{"total":96636764160,"available":96301416448,"internally_reserved":0,"allocated":40493056,"data_stored":38470704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294844869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35614720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26390528,"data_stored":26390528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36356096,"data_stored":36356096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33927168,"data_stored":33927168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22581248,"data_stored":22581248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":38694912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31379456,"data_stored":31379456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28663808,"data_stored":28663808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:47:11.928 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:47:11.929 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-17T05:07:44.680118+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=7, tm_sec=44, tm_wday=2, tm_yday=108, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=17, tm_hour=5, tm_min=46, tm_sec=5, tm_wday=2, tm_yday=108, tm_isdst=0) 2024-04-17T05:47:11.933 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-17T05:47:31.935 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T05:47:32.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-17T05:47:32.281 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-17T05:47:32.285 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":2397,"stamp":"2024-04-17T05:47:31.339579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223466608,"num_objects":822,"num_object_clones":0,"num_object_copies":2530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":822,"num_whiteouts":0,"num_read":160833,"num_read_kb":2423241,"num_write":146364,"num_write_kb":2463359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":69,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684328,"num_omap_keys":1751,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13339,"ondisk_log_size":13339,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2032168,"kb_used_data":347624,"kb_used_omap":12,"kb_used_meta":1684467,"kb_avail":752942552,"statfs":{"total":773094113280,"available":771013173248,"internally_reserved":0,"allocated":355966976,"data_stored":340014674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":1724894769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007677"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.209007+0000","last_change":"2024-04-17T05:46:55.209062+0000","last_active":"2024-04-17T05:46:55.209007+0000","last_peered":"2024-04-17T05:46:55.209007+0000","last_clean":"2024-04-17T05:46:55.209007+0000","last_became_active":"2024-04-17T05:07:45.741907+0000","last_became_peered":"2024-04-17T05:07:45.741907+0000","last_unstale":"2024-04-17T05:46:55.209007+0000","last_undegraded":"2024-04-17T05:46:55.209007+0000","last_fullsized":"2024-04-17T05:46:55.209007+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:55.208798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:55.208798+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.208798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:05:07.820451+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.237389+0000","last_change":"2024-04-17T05:46:54.237445+0000","last_active":"2024-04-17T05:46:54.237389+0000","last_peered":"2024-04-17T05:46:54.237389+0000","last_clean":"2024-04-17T05:46:54.237389+0000","last_became_active":"2024-04-17T04:51:43.681357+0000","last_became_peered":"2024-04-17T04:51:43.681357+0000","last_unstale":"2024-04-17T05:46:54.237389+0000","last_undegraded":"2024-04-17T05:46:54.237389+0000","last_fullsized":"2024-04-17T05:46:54.237389+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:54.237198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:54.237198+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.237198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:22:01.742139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"261'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.830930+0000","last_change":"2024-04-17T05:46:14.830930+0000","last_active":"2024-04-17T05:46:14.830930+0000","last_peered":"2024-04-17T05:46:14.830930+0000","last_clean":"2024-04-17T05:46:14.830930+0000","last_became_active":"2024-04-17T04:51:39.630640+0000","last_became_peered":"2024-04-17T04:51:39.630640+0000","last_unstale":"2024-04-17T05:46:14.830930+0000","last_undegraded":"2024-04-17T05:46:14.830930+0000","last_fullsized":"2024-04-17T05:46:14.830930+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:14.830571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.830571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:08.876068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"264'38","reported_seq":158,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.685785+0000","last_change":"2024-04-17T05:46:08.685785+0000","last_active":"2024-04-17T05:46:08.685785+0000","last_peered":"2024-04-17T05:46:08.685785+0000","last_clean":"2024-04-17T05:46:08.685785+0000","last_became_active":"2024-04-17T04:47:56.967315+0000","last_became_peered":"2024-04-17T04:47:56.967315+0000","last_unstale":"2024-04-17T05:46:08.685785+0000","last_undegraded":"2024-04-17T05:46:08.685785+0000","last_fullsized":"2024-04-17T05:46:08.685785+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:08.685550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.685550+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:05.034233+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":62,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.544197+0000","last_change":"2024-04-17T05:46:39.544342+0000","last_active":"2024-04-17T05:46:39.544197+0000","last_peered":"2024-04-17T05:46:39.544197+0000","last_clean":"2024-04-17T05:46:39.544197+0000","last_became_active":"2024-04-17T05:07:45.745847+0000","last_became_peered":"2024-04-17T05:07:45.745847+0000","last_unstale":"2024-04-17T05:46:39.544197+0000","last_undegraded":"2024-04-17T05:46:39.544197+0000","last_fullsized":"2024-04-17T05:46:39.544197+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:39.543940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:39.543940+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.543940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:06.130204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"264'3","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.619326+0000","last_change":"2024-04-17T05:46:55.619394+0000","last_active":"2024-04-17T05:46:55.619326+0000","last_peered":"2024-04-17T05:46:55.619326+0000","last_clean":"2024-04-17T05:46:55.619326+0000","last_became_active":"2024-04-17T04:51:43.656081+0000","last_became_peered":"2024-04-17T04:51:43.656081+0000","last_unstale":"2024-04-17T05:46:55.619326+0000","last_undegraded":"2024-04-17T05:46:55.619326+0000","last_fullsized":"2024-04-17T05:46:55.619326+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:55.619100+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:55.619100+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.619100+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:45:48.858607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":59,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.692660+0000","last_change":"2024-04-17T05:46:12.692660+0000","last_active":"2024-04-17T05:46:12.692660+0000","last_peered":"2024-04-17T05:46:12.692660+0000","last_clean":"2024-04-17T05:46:12.692660+0000","last_became_active":"2024-04-17T04:51:39.639995+0000","last_became_peered":"2024-04-17T04:51:39.639995+0000","last_unstale":"2024-04-17T05:46:12.692660+0000","last_undegraded":"2024-04-17T05:46:12.692660+0000","last_fullsized":"2024-04-17T05:46:12.692660+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:12.692409+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.692409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:20.713002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"264'97","reported_seq":361,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.504742+0000","last_change":"2024-04-17T05:46:20.504828+0000","last_active":"2024-04-17T05:46:20.504742+0000","last_peered":"2024-04-17T05:46:20.504742+0000","last_clean":"2024-04-17T05:46:20.504742+0000","last_became_active":"2024-04-17T04:47:56.937137+0000","last_became_peered":"2024-04-17T04:47:56.937137+0000","last_unstale":"2024-04-17T05:46:20.504742+0000","last_undegraded":"2024-04-17T05:46:20.504742+0000","last_fullsized":"2024-04-17T05:46:20.504742+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:20.504541+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.504541+0000","objects_scrubbed":1,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:36:15.125809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":554,"num_read_kb":399,"num_write":131,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3493,"num_omap_keys":7,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.887667+0000","last_change":"2024-04-17T05:46:45.887724+0000","last_active":"2024-04-17T05:46:45.887667+0000","last_peered":"2024-04-17T05:46:45.887667+0000","last_clean":"2024-04-17T05:46:45.887667+0000","last_became_active":"2024-04-17T05:07:45.734299+0000","last_became_peered":"2024-04-17T05:07:45.734299+0000","last_unstale":"2024-04-17T05:46:45.887667+0000","last_undegraded":"2024-04-17T05:46:45.887667+0000","last_fullsized":"2024-04-17T05:46:45.887667+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:45.887431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:45.887431+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.887431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:34.879198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"261'3","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.677240+0000","last_change":"2024-04-17T05:46:35.677355+0000","last_active":"2024-04-17T05:46:35.677240+0000","last_peered":"2024-04-17T05:46:35.677240+0000","last_clean":"2024-04-17T05:46:35.677240+0000","last_became_active":"2024-04-17T04:51:43.672808+0000","last_became_peered":"2024-04-17T04:51:43.672808+0000","last_unstale":"2024-04-17T05:46:35.677240+0000","last_undegraded":"2024-04-17T05:46:35.677240+0000","last_fullsized":"2024-04-17T05:46:35.677240+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:35.676927+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:35.676927+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.676927+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:48:40.289079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"264'3","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.493151+0000","last_change":"2024-04-17T05:46:32.493211+0000","last_active":"2024-04-17T05:46:32.493151+0000","last_peered":"2024-04-17T05:46:32.493151+0000","last_clean":"2024-04-17T05:46:32.493151+0000","last_became_active":"2024-04-17T04:51:39.604193+0000","last_became_peered":"2024-04-17T04:51:39.604193+0000","last_unstale":"2024-04-17T05:46:32.493151+0000","last_undegraded":"2024-04-17T05:46:32.493151+0000","last_fullsized":"2024-04-17T05:46:32.493151+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:32.492942+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:32.492942+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.492942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:21:29.955893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"264'25","reported_seq":130,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.592867+0000","last_change":"2024-04-17T05:46:16.593030+0000","last_active":"2024-04-17T05:46:16.592867+0000","last_peered":"2024-04-17T05:46:16.592867+0000","last_clean":"2024-04-17T05:46:16.592867+0000","last_became_active":"2024-04-17T04:47:56.952515+0000","last_became_peered":"2024-04-17T04:47:56.952515+0000","last_unstale":"2024-04-17T05:46:16.592867+0000","last_undegraded":"2024-04-17T05:46:16.592867+0000","last_fullsized":"2024-04-17T05:46:16.592867+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'25","last_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_deep_scrub":"264'25","last_deep_scrub_stamp":"2024-04-17T05:46:16.592614+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.592614+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:44:38.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":83,"num_write":37,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:10.587646+0000","last_change":"2024-04-17T05:47:10.587702+0000","last_active":"2024-04-17T05:47:10.587646+0000","last_peered":"2024-04-17T05:47:10.587646+0000","last_clean":"2024-04-17T05:47:10.587646+0000","last_became_active":"2024-04-17T05:07:45.746182+0000","last_became_peered":"2024-04-17T05:07:45.746182+0000","last_unstale":"2024-04-17T05:47:10.587646+0000","last_undegraded":"2024-04-17T05:47:10.587646+0000","last_fullsized":"2024-04-17T05:47:10.587646+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:10.587441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:10.587441+0000","last_clean_scrub_stamp":"2024-04-17T05:47:10.587441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:59.900713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"0'0","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.760387+0000","last_change":"2024-04-17T05:46:26.760496+0000","last_active":"2024-04-17T05:46:26.760387+0000","last_peered":"2024-04-17T05:46:26.760387+0000","last_clean":"2024-04-17T05:46:26.760387+0000","last_became_active":"2024-04-17T04:51:43.675979+0000","last_became_peered":"2024-04-17T04:51:43.675979+0000","last_unstale":"2024-04-17T05:46:26.760387+0000","last_undegraded":"2024-04-17T05:46:26.760387+0000","last_fullsized":"2024-04-17T05:46:26.760387+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:26.760176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:26.760176+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.760176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:38.567211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.770225+0000","last_change":"2024-04-17T05:46:20.770279+0000","last_active":"2024-04-17T05:46:20.770225+0000","last_peered":"2024-04-17T05:46:20.770225+0000","last_clean":"2024-04-17T05:46:20.770225+0000","last_became_active":"2024-04-17T04:51:39.614505+0000","last_became_peered":"2024-04-17T04:51:39.614505+0000","last_unstale":"2024-04-17T05:46:20.770225+0000","last_undegraded":"2024-04-17T05:46:20.770225+0000","last_fullsized":"2024-04-17T05:46:20.770225+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.770027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.770027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:10.097683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"264'42","reported_seq":169,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.814048+0000","last_change":"2024-04-17T05:46:08.814070+0000","last_active":"2024-04-17T05:46:08.814048+0000","last_peered":"2024-04-17T05:46:08.814048+0000","last_clean":"2024-04-17T05:46:08.814048+0000","last_became_active":"2024-04-17T04:47:56.968596+0000","last_became_peered":"2024-04-17T04:47:56.968596+0000","last_unstale":"2024-04-17T05:46:08.814048+0000","last_undegraded":"2024-04-17T05:46:08.814048+0000","last_fullsized":"2024-04-17T05:46:08.814048+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'42","last_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_deep_scrub":"264'42","last_deep_scrub_stamp":"2024-04-17T05:46:08.813969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.813969+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:41.013929+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":129,"num_write":86,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:09.609567+0000","last_change":"2024-04-17T05:47:09.609620+0000","last_active":"2024-04-17T05:47:09.609567+0000","last_peered":"2024-04-17T05:47:09.609567+0000","last_clean":"2024-04-17T05:47:09.609567+0000","last_became_active":"2024-04-17T05:07:45.744373+0000","last_became_peered":"2024-04-17T05:07:45.744373+0000","last_unstale":"2024-04-17T05:47:09.609567+0000","last_undegraded":"2024-04-17T05:47:09.609567+0000","last_fullsized":"2024-04-17T05:47:09.609567+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:09.609399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:09.609399+0000","last_clean_scrub_stamp":"2024-04-17T05:47:09.609399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:57:29.808680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"264'2","reported_seq":58,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.245387+0000","last_change":"2024-04-17T05:46:53.245387+0000","last_active":"2024-04-17T05:46:53.245387+0000","last_peered":"2024-04-17T05:46:53.245387+0000","last_clean":"2024-04-17T05:46:53.245387+0000","last_became_active":"2024-04-17T04:51:43.665618+0000","last_became_peered":"2024-04-17T04:51:43.665618+0000","last_unstale":"2024-04-17T05:46:53.245387+0000","last_undegraded":"2024-04-17T05:46:53.245387+0000","last_fullsized":"2024-04-17T05:46:53.245387+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:53.245100+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:53.245100+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.245100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:56:30.267498+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.720653+0000","last_change":"2024-04-17T05:46:07.720653+0000","last_active":"2024-04-17T05:46:07.720653+0000","last_peered":"2024-04-17T05:46:07.720653+0000","last_clean":"2024-04-17T05:46:07.720653+0000","last_became_active":"2024-04-17T04:51:39.624890+0000","last_became_peered":"2024-04-17T04:51:39.624890+0000","last_unstale":"2024-04-17T05:46:07.720653+0000","last_undegraded":"2024-04-17T05:46:07.720653+0000","last_fullsized":"2024-04-17T05:46:07.720653+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:07.720559+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.720559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:04.229433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"264'62","reported_seq":209,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.804847+0000","last_change":"2024-04-17T05:46:09.804897+0000","last_active":"2024-04-17T05:46:09.804847+0000","last_peered":"2024-04-17T05:46:09.804847+0000","last_clean":"2024-04-17T05:46:09.804847+0000","last_became_active":"2024-04-17T04:47:56.966335+0000","last_became_peered":"2024-04-17T04:47:56.966335+0000","last_unstale":"2024-04-17T05:46:09.804847+0000","last_undegraded":"2024-04-17T05:46:09.804847+0000","last_fullsized":"2024-04-17T05:46:09.804847+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:09.804697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.804697+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:40.852058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":249,"num_read_kb":175,"num_write":111,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.905791+0000","last_change":"2024-04-17T05:46:49.905791+0000","last_active":"2024-04-17T05:46:49.905791+0000","last_peered":"2024-04-17T05:46:49.905791+0000","last_clean":"2024-04-17T05:46:49.905791+0000","last_became_active":"2024-04-17T05:07:45.753537+0000","last_became_peered":"2024-04-17T05:07:45.753537+0000","last_unstale":"2024-04-17T05:46:49.905791+0000","last_undegraded":"2024-04-17T05:46:49.905791+0000","last_fullsized":"2024-04-17T05:46:49.905791+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:49.905522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:49.905522+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.905522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:33.396236+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"264'6","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.924541+0000","last_change":"2024-04-17T05:46:41.924541+0000","last_active":"2024-04-17T05:46:41.924541+0000","last_peered":"2024-04-17T05:46:41.924541+0000","last_clean":"2024-04-17T05:46:41.924541+0000","last_became_active":"2024-04-17T04:51:43.658476+0000","last_became_peered":"2024-04-17T04:51:43.658476+0000","last_unstale":"2024-04-17T05:46:41.924541+0000","last_undegraded":"2024-04-17T05:46:41.924541+0000","last_fullsized":"2024-04-17T05:46:41.924541+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:41.924290+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:41.924290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.924290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:42.756248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":22528,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"264'6","reported_seq":76,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.770216+0000","last_change":"2024-04-17T05:46:12.770216+0000","last_active":"2024-04-17T05:46:12.770216+0000","last_peered":"2024-04-17T05:46:12.770216+0000","last_clean":"2024-04-17T05:46:12.770216+0000","last_became_active":"2024-04-17T04:51:39.610549+0000","last_became_peered":"2024-04-17T04:51:39.610549+0000","last_unstale":"2024-04-17T05:46:12.770216+0000","last_undegraded":"2024-04-17T05:46:12.770216+0000","last_fullsized":"2024-04-17T05:46:12.770216+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.769907+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.769907+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:23.154229+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"264'75","reported_seq":293,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.023607+0000","last_change":"2024-04-17T05:46:12.023675+0000","last_active":"2024-04-17T05:46:12.023607+0000","last_peered":"2024-04-17T05:46:12.023607+0000","last_clean":"2024-04-17T05:46:12.023607+0000","last_became_active":"2024-04-17T04:47:56.951278+0000","last_became_peered":"2024-04-17T04:47:56.951278+0000","last_unstale":"2024-04-17T05:46:12.023607+0000","last_undegraded":"2024-04-17T05:46:12.023607+0000","last_fullsized":"2024-04-17T05:46:12.023607+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'75","last_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_deep_scrub":"264'75","last_deep_scrub_stamp":"2024-04-17T05:46:12.023402+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.023402+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:01:08.097898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":431,"num_read_kb":318,"num_write":115,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3255,"num_omap_keys":7,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"264'2","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:02.337385+0000","last_change":"2024-04-17T05:47:02.337449+0000","last_active":"2024-04-17T05:47:02.337385+0000","last_peered":"2024-04-17T05:47:02.337385+0000","last_clean":"2024-04-17T05:47:02.337385+0000","last_became_active":"2024-04-17T05:07:45.751693+0000","last_became_peered":"2024-04-17T05:07:45.751693+0000","last_unstale":"2024-04-17T05:47:02.337385+0000","last_undegraded":"2024-04-17T05:47:02.337385+0000","last_fullsized":"2024-04-17T05:47:02.337385+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:02.337195+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:02.337195+0000","last_clean_scrub_stamp":"2024-04-17T05:47:02.337195+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:34.622453+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.909074+0000","last_change":"2024-04-17T05:46:42.909074+0000","last_active":"2024-04-17T05:46:42.909074+0000","last_peered":"2024-04-17T05:46:42.909074+0000","last_clean":"2024-04-17T05:46:42.909074+0000","last_became_active":"2024-04-17T04:51:43.657233+0000","last_became_peered":"2024-04-17T04:51:43.657233+0000","last_unstale":"2024-04-17T05:46:42.909074+0000","last_undegraded":"2024-04-17T05:46:42.909074+0000","last_fullsized":"2024-04-17T05:46:42.909074+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.908818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.908818+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.908818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:58:22.365739+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"264'9","reported_seq":77,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.545960+0000","last_change":"2024-04-17T05:46:51.546023+0000","last_active":"2024-04-17T05:46:51.545960+0000","last_peered":"2024-04-17T05:46:51.545960+0000","last_clean":"2024-04-17T05:46:51.545960+0000","last_became_active":"2024-04-17T04:51:39.620530+0000","last_became_peered":"2024-04-17T04:51:39.620530+0000","last_unstale":"2024-04-17T05:46:51.545960+0000","last_undegraded":"2024-04-17T05:46:51.545960+0000","last_fullsized":"2024-04-17T05:46:51.545960+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'9","last_scrub_stamp":"2024-04-17T05:46:51.545764+0000","last_deep_scrub":"264'9","last_deep_scrub_stamp":"2024-04-17T05:46:51.545764+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.545764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:40.874309+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":16384,"num_write":24,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'8","reported_seq":84,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.595282+0000","last_change":"2024-04-17T05:46:14.595378+0000","last_active":"2024-04-17T05:46:14.595282+0000","last_peered":"2024-04-17T05:46:14.595282+0000","last_clean":"2024-04-17T05:46:14.595282+0000","last_became_active":"2024-04-17T04:47:56.954864+0000","last_became_peered":"2024-04-17T04:47:56.954864+0000","last_unstale":"2024-04-17T05:46:14.595282+0000","last_undegraded":"2024-04-17T05:46:14.595282+0000","last_fullsized":"2024-04-17T05:46:14.595282+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-17T05:46:14.594970+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.594970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:05:26.805897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":20,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":36,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:05.383248+0000","last_change":"2024-04-17T05:47:05.383302+0000","last_active":"2024-04-17T05:47:05.383248+0000","last_peered":"2024-04-17T05:47:05.383248+0000","last_clean":"2024-04-17T05:47:05.383248+0000","last_became_active":"2024-04-17T05:07:45.756821+0000","last_became_peered":"2024-04-17T05:07:45.756821+0000","last_unstale":"2024-04-17T05:47:05.383248+0000","last_undegraded":"2024-04-17T05:47:05.383248+0000","last_fullsized":"2024-04-17T05:47:05.383248+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:05.383047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:05.383047+0000","last_clean_scrub_stamp":"2024-04-17T05:47:05.383047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:15.434278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":53,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:05.709366+0000","last_change":"2024-04-17T05:47:05.709418+0000","last_active":"2024-04-17T05:47:05.709366+0000","last_peered":"2024-04-17T05:47:05.709366+0000","last_clean":"2024-04-17T05:47:05.709366+0000","last_became_active":"2024-04-17T04:51:43.685936+0000","last_became_peered":"2024-04-17T04:51:43.685936+0000","last_unstale":"2024-04-17T05:47:05.709366+0000","last_undegraded":"2024-04-17T05:47:05.709366+0000","last_fullsized":"2024-04-17T05:47:05.709366+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:05.709173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:05.709173+0000","last_clean_scrub_stamp":"2024-04-17T05:47:05.709173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:18.346966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.616272+0000","last_change":"2024-04-17T05:46:27.616272+0000","last_active":"2024-04-17T05:46:27.616272+0000","last_peered":"2024-04-17T05:46:27.616272+0000","last_clean":"2024-04-17T05:46:27.616272+0000","last_became_active":"2024-04-17T04:51:39.642769+0000","last_became_peered":"2024-04-17T04:51:39.642769+0000","last_unstale":"2024-04-17T05:46:27.616272+0000","last_undegraded":"2024-04-17T05:46:27.616272+0000","last_fullsized":"2024-04-17T05:46:27.616272+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-17T05:46:27.615997+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-17T05:46:27.615997+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.615997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:21:10.518740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"264'13","reported_seq":99,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.712592+0000","last_change":"2024-04-17T05:46:11.712658+0000","last_active":"2024-04-17T05:46:11.712592+0000","last_peered":"2024-04-17T05:46:11.712592+0000","last_clean":"2024-04-17T05:46:11.712592+0000","last_became_active":"2024-04-17T04:47:56.932013+0000","last_became_peered":"2024-04-17T04:47:56.932013+0000","last_unstale":"2024-04-17T05:46:11.712592+0000","last_undegraded":"2024-04-17T05:46:11.712592+0000","last_fullsized":"2024-04-17T05:46:11.712592+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'13","last_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_deep_scrub":"264'13","last_deep_scrub_stamp":"2024-04-17T05:46:11.712398+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.712398+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:12.832777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.918919+0000","last_change":"2024-04-17T05:46:48.918919+0000","last_active":"2024-04-17T05:46:48.918919+0000","last_peered":"2024-04-17T05:46:48.918919+0000","last_clean":"2024-04-17T05:46:48.918919+0000","last_became_active":"2024-04-17T05:07:45.742263+0000","last_became_peered":"2024-04-17T05:07:45.742263+0000","last_unstale":"2024-04-17T05:46:48.918919+0000","last_undegraded":"2024-04-17T05:46:48.918919+0000","last_fullsized":"2024-04-17T05:46:48.918919+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:48.918658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:48.918658+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.918658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:35:22.598547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.794764+0000","last_change":"2024-04-17T05:46:18.794764+0000","last_active":"2024-04-17T05:46:18.794764+0000","last_peered":"2024-04-17T05:46:18.794764+0000","last_clean":"2024-04-17T05:46:18.794764+0000","last_became_active":"2024-04-17T04:51:43.663968+0000","last_became_peered":"2024-04-17T04:51:43.663968+0000","last_unstale":"2024-04-17T05:46:18.794764+0000","last_undegraded":"2024-04-17T05:46:18.794764+0000","last_fullsized":"2024-04-17T05:46:18.794764+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.794517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.794517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:29:12.950953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.225008+0000","last_change":"2024-04-17T05:46:24.225059+0000","last_active":"2024-04-17T05:46:24.225008+0000","last_peered":"2024-04-17T05:46:24.225008+0000","last_clean":"2024-04-17T05:46:24.225008+0000","last_became_active":"2024-04-17T04:51:39.625252+0000","last_became_peered":"2024-04-17T04:51:39.625252+0000","last_unstale":"2024-04-17T05:46:24.225008+0000","last_undegraded":"2024-04-17T05:46:24.225008+0000","last_fullsized":"2024-04-17T05:46:24.225008+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.224809+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.224809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:16.454241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'9","reported_seq":89,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.623954+0000","last_change":"2024-04-17T05:46:12.624024+0000","last_active":"2024-04-17T05:46:12.623954+0000","last_peered":"2024-04-17T05:46:12.623954+0000","last_clean":"2024-04-17T05:46:12.623954+0000","last_became_active":"2024-04-17T04:47:56.955402+0000","last_became_peered":"2024-04-17T04:47:56.955402+0000","last_unstale":"2024-04-17T05:46:12.623954+0000","last_undegraded":"2024-04-17T05:46:12.623954+0000","last_fullsized":"2024-04-17T05:46:12.623954+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-17T05:46:12.623741+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.623741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:56:20.441738+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":30,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"264'2","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.529348+0000","last_change":"2024-04-17T05:46:36.529348+0000","last_active":"2024-04-17T05:46:36.529348+0000","last_peered":"2024-04-17T05:46:36.529348+0000","last_clean":"2024-04-17T05:46:36.529348+0000","last_became_active":"2024-04-17T05:07:45.737195+0000","last_became_peered":"2024-04-17T05:07:45.737195+0000","last_unstale":"2024-04-17T05:46:36.529348+0000","last_undegraded":"2024-04-17T05:46:36.529348+0000","last_fullsized":"2024-04-17T05:46:36.529348+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:36.529078+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:36.529078+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.529078+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:48.549685+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"261'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.829356+0000","last_change":"2024-04-17T05:46:34.829509+0000","last_active":"2024-04-17T05:46:34.829356+0000","last_peered":"2024-04-17T05:46:34.829356+0000","last_clean":"2024-04-17T05:46:34.829356+0000","last_became_active":"2024-04-17T04:51:43.676489+0000","last_became_peered":"2024-04-17T04:51:43.676489+0000","last_unstale":"2024-04-17T05:46:34.829356+0000","last_undegraded":"2024-04-17T05:46:34.829356+0000","last_fullsized":"2024-04-17T05:46:34.829356+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:34.829120+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:34.829120+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.829120+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:26:21.304145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"264'2","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.129787+0000","last_change":"2024-04-17T05:46:11.129787+0000","last_active":"2024-04-17T05:46:11.129787+0000","last_peered":"2024-04-17T05:46:11.129787+0000","last_clean":"2024-04-17T05:46:11.129787+0000","last_became_active":"2024-04-17T04:51:39.638706+0000","last_became_peered":"2024-04-17T04:51:39.638706+0000","last_unstale":"2024-04-17T05:46:11.129787+0000","last_undegraded":"2024-04-17T05:46:11.129787+0000","last_fullsized":"2024-04-17T05:46:11.129787+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:11.129393+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.129393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:13:45.300258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"264'29","reported_seq":125,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.852514+0000","last_change":"2024-04-17T05:46:11.852573+0000","last_active":"2024-04-17T05:46:11.852514+0000","last_peered":"2024-04-17T05:46:11.852514+0000","last_clean":"2024-04-17T05:46:11.852514+0000","last_became_active":"2024-04-17T04:47:56.966121+0000","last_became_peered":"2024-04-17T04:47:56.966121+0000","last_unstale":"2024-04-17T05:46:11.852514+0000","last_undegraded":"2024-04-17T05:46:11.852514+0000","last_fullsized":"2024-04-17T05:46:11.852514+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.852327+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.852327+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:22:08.854562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":64,"num_write":60,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.319899+0000","last_change":"2024-04-17T05:46:32.319899+0000","last_active":"2024-04-17T05:46:32.319899+0000","last_peered":"2024-04-17T05:46:32.319899+0000","last_clean":"2024-04-17T05:46:32.319899+0000","last_became_active":"2024-04-17T05:07:45.746408+0000","last_became_peered":"2024-04-17T05:07:45.746408+0000","last_unstale":"2024-04-17T05:46:32.319899+0000","last_undegraded":"2024-04-17T05:46:32.319899+0000","last_fullsized":"2024-04-17T05:46:32.319899+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.319638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.319638+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.319638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:39:10.301900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"264'5","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.734137+0000","last_change":"2024-04-17T05:46:22.734137+0000","last_active":"2024-04-17T05:46:22.734137+0000","last_peered":"2024-04-17T05:46:22.734137+0000","last_clean":"2024-04-17T05:46:22.734137+0000","last_became_active":"2024-04-17T04:51:43.670940+0000","last_became_peered":"2024-04-17T04:51:43.670940+0000","last_unstale":"2024-04-17T05:46:22.734137+0000","last_undegraded":"2024-04-17T05:46:22.734137+0000","last_fullsized":"2024-04-17T05:46:22.734137+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:22.733832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.733832+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:04.087293+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":48,"num_write":10,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"264'14","reported_seq":92,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.511896+0000","last_change":"2024-04-17T05:46:33.511896+0000","last_active":"2024-04-17T05:46:33.511896+0000","last_peered":"2024-04-17T05:46:33.511896+0000","last_clean":"2024-04-17T05:46:33.511896+0000","last_became_active":"2024-04-17T04:51:39.611929+0000","last_became_peered":"2024-04-17T04:51:39.611929+0000","last_unstale":"2024-04-17T05:46:33.511896+0000","last_undegraded":"2024-04-17T05:46:33.511896+0000","last_fullsized":"2024-04-17T05:46:33.511896+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'14","last_scrub_stamp":"2024-04-17T05:46:33.511695+0000","last_deep_scrub":"264'14","last_deep_scrub_stamp":"2024-04-17T05:46:33.511695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.511695+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:13.607331+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":46082,"num_write":17,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'18","reported_seq":107,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.547405+0000","last_change":"2024-04-17T05:46:18.547463+0000","last_active":"2024-04-17T05:46:18.547405+0000","last_peered":"2024-04-17T05:46:18.547405+0000","last_clean":"2024-04-17T05:46:18.547405+0000","last_became_active":"2024-04-17T04:47:56.936314+0000","last_became_peered":"2024-04-17T04:47:56.936314+0000","last_unstale":"2024-04-17T05:46:18.547405+0000","last_undegraded":"2024-04-17T05:46:18.547405+0000","last_fullsized":"2024-04-17T05:46:18.547405+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'18","last_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_deep_scrub":"261'18","last_deep_scrub_stamp":"2024-04-17T05:46:18.547198+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.547198+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:22.594305+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":49,"num_write":30,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.186371+0000","last_change":"2024-04-17T05:46:56.186371+0000","last_active":"2024-04-17T05:46:56.186371+0000","last_peered":"2024-04-17T05:46:56.186371+0000","last_clean":"2024-04-17T05:46:56.186371+0000","last_became_active":"2024-04-17T05:07:45.743458+0000","last_became_peered":"2024-04-17T05:07:45.743458+0000","last_unstale":"2024-04-17T05:46:56.186371+0000","last_undegraded":"2024-04-17T05:46:56.186371+0000","last_fullsized":"2024-04-17T05:46:56.186371+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:56.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:56.186078+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.186078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:51:49.151800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"264'12","reported_seq":83,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.644776+0000","last_change":"2024-04-17T05:46:56.644776+0000","last_active":"2024-04-17T05:46:56.644776+0000","last_peered":"2024-04-17T05:46:56.644776+0000","last_clean":"2024-04-17T05:46:56.644776+0000","last_became_active":"2024-04-17T04:51:43.666622+0000","last_became_peered":"2024-04-17T04:51:43.666622+0000","last_unstale":"2024-04-17T05:46:56.644776+0000","last_undegraded":"2024-04-17T05:46:56.644776+0000","last_fullsized":"2024-04-17T05:46:56.644776+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-04-17T05:46:56.644490+0000","last_deep_scrub":"264'12","last_deep_scrub_stamp":"2024-04-17T05:46:56.644490+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.644490+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:25:56.759217+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":37890,"num_write":20,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.607603+0000","last_change":"2024-04-17T05:46:23.607603+0000","last_active":"2024-04-17T05:46:23.607603+0000","last_peered":"2024-04-17T05:46:23.607603+0000","last_clean":"2024-04-17T05:46:23.607603+0000","last_became_active":"2024-04-17T04:51:39.636829+0000","last_became_peered":"2024-04-17T04:51:39.636829+0000","last_unstale":"2024-04-17T05:46:23.607603+0000","last_undegraded":"2024-04-17T05:46:23.607603+0000","last_fullsized":"2024-04-17T05:46:23.607603+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:23.607294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.607294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:20.141964+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"264'65","reported_seq":221,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.194774+0000","last_change":"2024-04-17T05:46:18.194774+0000","last_active":"2024-04-17T05:46:18.194774+0000","last_peered":"2024-04-17T05:46:18.194774+0000","last_clean":"2024-04-17T05:46:18.194774+0000","last_became_active":"2024-04-17T04:47:56.963417+0000","last_became_peered":"2024-04-17T04:47:56.963417+0000","last_unstale":"2024-04-17T05:46:18.194774+0000","last_undegraded":"2024-04-17T05:46:18.194774+0000","last_fullsized":"2024-04-17T05:46:18.194774+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'65","last_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_deep_scrub":"264'65","last_deep_scrub_stamp":"2024-04-17T05:46:18.194519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.194519+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:52:31.405244+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":268,"num_read_kb":190,"num_write":107,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:11.592778+0000","last_change":"2024-04-17T05:47:11.592810+0000","last_active":"2024-04-17T05:47:11.592778+0000","last_peered":"2024-04-17T05:47:11.592778+0000","last_clean":"2024-04-17T05:47:11.592778+0000","last_became_active":"2024-04-17T05:07:45.748692+0000","last_became_peered":"2024-04-17T05:07:45.748692+0000","last_unstale":"2024-04-17T05:47:11.592778+0000","last_undegraded":"2024-04-17T05:47:11.592778+0000","last_fullsized":"2024-04-17T05:47:11.592778+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:11.592624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:11.592624+0000","last_clean_scrub_stamp":"2024-04-17T05:47:11.592624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:05:29.321949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"264'6","reported_seq":70,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.915407+0000","last_change":"2024-04-17T05:46:45.915482+0000","last_active":"2024-04-17T05:46:45.915407+0000","last_peered":"2024-04-17T05:46:45.915407+0000","last_clean":"2024-04-17T05:46:45.915407+0000","last_became_active":"2024-04-17T04:51:43.667551+0000","last_became_peered":"2024-04-17T04:51:43.667551+0000","last_unstale":"2024-04-17T05:46:45.915407+0000","last_undegraded":"2024-04-17T05:46:45.915407+0000","last_fullsized":"2024-04-17T05:46:45.915407+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:45.915212+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:45.915212+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.915212+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:02.071375+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.807390+0000","last_change":"2024-04-17T05:46:16.807574+0000","last_active":"2024-04-17T05:46:16.807390+0000","last_peered":"2024-04-17T05:46:16.807390+0000","last_clean":"2024-04-17T05:46:16.807390+0000","last_became_active":"2024-04-17T04:51:39.612237+0000","last_became_peered":"2024-04-17T04:51:39.612237+0000","last_unstale":"2024-04-17T05:46:16.807390+0000","last_undegraded":"2024-04-17T05:46:16.807390+0000","last_fullsized":"2024-04-17T05:46:16.807390+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:16.807094+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.807094+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:53:57.818718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"264'37","reported_seq":157,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.309760+0000","last_change":"2024-04-17T05:46:22.309837+0000","last_active":"2024-04-17T05:46:22.309760+0000","last_peered":"2024-04-17T05:46:22.309760+0000","last_clean":"2024-04-17T05:46:22.309760+0000","last_became_active":"2024-04-17T04:47:56.961371+0000","last_became_peered":"2024-04-17T04:47:56.961371+0000","last_unstale":"2024-04-17T05:46:22.309760+0000","last_undegraded":"2024-04-17T05:46:22.309760+0000","last_fullsized":"2024-04-17T05:46:22.309760+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:22.309548+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.309548+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:24:29.733950+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":115,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":37,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:07.562430+0000","last_change":"2024-04-17T05:47:07.562483+0000","last_active":"2024-04-17T05:47:07.562430+0000","last_peered":"2024-04-17T05:47:07.562430+0000","last_clean":"2024-04-17T05:47:07.562430+0000","last_became_active":"2024-04-17T05:07:45.749102+0000","last_became_peered":"2024-04-17T05:07:45.749102+0000","last_unstale":"2024-04-17T05:47:07.562430+0000","last_undegraded":"2024-04-17T05:47:07.562430+0000","last_fullsized":"2024-04-17T05:47:07.562430+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:07.562220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:07.562220+0000","last_clean_scrub_stamp":"2024-04-17T05:47:07.562220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:16:08.871059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"264'2","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.726875+0000","last_change":"2024-04-17T05:46:16.727020+0000","last_active":"2024-04-17T05:46:16.726875+0000","last_peered":"2024-04-17T05:46:16.726875+0000","last_clean":"2024-04-17T05:46:16.726875+0000","last_became_active":"2024-04-17T04:51:43.649590+0000","last_became_peered":"2024-04-17T04:51:43.649590+0000","last_unstale":"2024-04-17T05:46:16.726875+0000","last_undegraded":"2024-04-17T05:46:16.726875+0000","last_fullsized":"2024-04-17T05:46:16.726875+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:16.726644+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.726644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:09.817384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.831759+0000","last_change":"2024-04-17T05:46:10.831811+0000","last_active":"2024-04-17T05:46:10.831759+0000","last_peered":"2024-04-17T05:46:10.831759+0000","last_clean":"2024-04-17T05:46:10.831759+0000","last_became_active":"2024-04-17T04:51:39.631169+0000","last_became_peered":"2024-04-17T04:51:39.631169+0000","last_unstale":"2024-04-17T05:46:10.831759+0000","last_undegraded":"2024-04-17T05:46:10.831759+0000","last_fullsized":"2024-04-17T05:46:10.831759+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:10.831558+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.831558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:16:46.336010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"264'33","reported_seq":142,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.670845+0000","last_change":"2024-04-17T05:46:09.670845+0000","last_active":"2024-04-17T05:46:09.670845+0000","last_peered":"2024-04-17T05:46:09.670845+0000","last_clean":"2024-04-17T05:46:09.670845+0000","last_became_active":"2024-04-17T04:47:56.966825+0000","last_became_peered":"2024-04-17T04:47:56.966825+0000","last_unstale":"2024-04-17T05:46:09.670845+0000","last_undegraded":"2024-04-17T05:46:09.670845+0000","last_fullsized":"2024-04-17T05:46:09.670845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:09.670593+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.670593+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:59.152267+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":88,"num_write":55,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"264'2","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:06.590495+0000","last_change":"2024-04-17T05:47:06.590561+0000","last_active":"2024-04-17T05:47:06.590495+0000","last_peered":"2024-04-17T05:47:06.590495+0000","last_clean":"2024-04-17T05:47:06.590495+0000","last_became_active":"2024-04-17T05:07:45.745714+0000","last_became_peered":"2024-04-17T05:07:45.745714+0000","last_unstale":"2024-04-17T05:47:06.590495+0000","last_undegraded":"2024-04-17T05:47:06.590495+0000","last_fullsized":"2024-04-17T05:47:06.590495+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:06.590290+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:06.590290+0000","last_clean_scrub_stamp":"2024-04-17T05:47:06.590290+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:12.967635+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"264'3","reported_seq":60,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.712424+0000","last_change":"2024-04-17T05:46:38.712582+0000","last_active":"2024-04-17T05:46:38.712424+0000","last_peered":"2024-04-17T05:46:38.712424+0000","last_clean":"2024-04-17T05:46:38.712424+0000","last_became_active":"2024-04-17T04:51:43.683372+0000","last_became_peered":"2024-04-17T04:51:43.683372+0000","last_unstale":"2024-04-17T05:46:38.712424+0000","last_undegraded":"2024-04-17T05:46:38.712424+0000","last_fullsized":"2024-04-17T05:46:38.712424+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:38.712195+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:38.712195+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.712195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:58.037440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"264'5","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.032735+0000","last_change":"2024-04-17T05:46:14.032796+0000","last_active":"2024-04-17T05:46:14.032735+0000","last_peered":"2024-04-17T05:46:14.032735+0000","last_clean":"2024-04-17T05:46:14.032735+0000","last_became_active":"2024-04-17T04:51:39.626157+0000","last_became_peered":"2024-04-17T04:51:39.626157+0000","last_unstale":"2024-04-17T05:46:14.032735+0000","last_undegraded":"2024-04-17T05:46:14.032735+0000","last_fullsized":"2024-04-17T05:46:14.032735+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:14.032549+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.032549+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:53:46.453310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"264'19","reported_seq":111,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.793130+0000","last_change":"2024-04-17T05:46:09.793130+0000","last_active":"2024-04-17T05:46:09.793130+0000","last_peered":"2024-04-17T05:46:09.793130+0000","last_clean":"2024-04-17T05:46:09.793130+0000","last_became_active":"2024-04-17T04:47:56.943954+0000","last_became_peered":"2024-04-17T04:47:56.943954+0000","last_unstale":"2024-04-17T05:46:09.793130+0000","last_undegraded":"2024-04-17T05:46:09.793130+0000","last_fullsized":"2024-04-17T05:46:09.793130+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:09.792865+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.792865+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:12:51.707714+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":33,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"264'2","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:07.664030+0000","last_change":"2024-04-17T05:47:07.664098+0000","last_active":"2024-04-17T05:47:07.664030+0000","last_peered":"2024-04-17T05:47:07.664030+0000","last_clean":"2024-04-17T05:47:07.664030+0000","last_became_active":"2024-04-17T05:07:45.740467+0000","last_became_peered":"2024-04-17T05:07:45.740467+0000","last_unstale":"2024-04-17T05:47:07.664030+0000","last_undegraded":"2024-04-17T05:47:07.664030+0000","last_fullsized":"2024-04-17T05:47:07.664030+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:07.663845+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:07.663845+0000","last_clean_scrub_stamp":"2024-04-17T05:47:07.663845+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:39:01.427821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"0'0","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.871034+0000","last_change":"2024-04-17T05:46:43.871089+0000","last_active":"2024-04-17T05:46:43.871034+0000","last_peered":"2024-04-17T05:46:43.871034+0000","last_clean":"2024-04-17T05:46:43.871034+0000","last_became_active":"2024-04-17T04:51:43.672962+0000","last_became_peered":"2024-04-17T04:51:43.672962+0000","last_unstale":"2024-04-17T05:46:43.871034+0000","last_undegraded":"2024-04-17T05:46:43.871034+0000","last_fullsized":"2024-04-17T05:46:43.871034+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.870838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.870838+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.870838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:26:23.777410+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"261'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.134683+0000","last_change":"2024-04-17T05:46:20.134745+0000","last_active":"2024-04-17T05:46:20.134683+0000","last_peered":"2024-04-17T05:46:20.134683+0000","last_clean":"2024-04-17T05:46:20.134683+0000","last_became_active":"2024-04-17T04:51:39.625185+0000","last_became_peered":"2024-04-17T04:51:39.625185+0000","last_unstale":"2024-04-17T05:46:20.134683+0000","last_undegraded":"2024-04-17T05:46:20.134683+0000","last_fullsized":"2024-04-17T05:46:20.134683+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:20.134488+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.134488+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:24:48.392260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"264'55","reported_seq":205,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.861464+0000","last_change":"2024-04-17T05:46:13.861464+0000","last_active":"2024-04-17T05:46:13.861464+0000","last_peered":"2024-04-17T05:46:13.861464+0000","last_clean":"2024-04-17T05:46:13.861464+0000","last_became_active":"2024-04-17T04:47:56.960129+0000","last_became_peered":"2024-04-17T04:47:56.960129+0000","last_unstale":"2024-04-17T05:46:13.861464+0000","last_undegraded":"2024-04-17T05:46:13.861464+0000","last_fullsized":"2024-04-17T05:46:13.861464+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'55","last_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_deep_scrub":"264'55","last_deep_scrub_stamp":"2024-04-17T05:46:13.861056+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.861056+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:57.160789+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":178,"num_write":90,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.568563+0000","last_change":"2024-04-17T05:46:37.568563+0000","last_active":"2024-04-17T05:46:37.568563+0000","last_peered":"2024-04-17T05:46:37.568563+0000","last_clean":"2024-04-17T05:46:37.568563+0000","last_became_active":"2024-04-17T05:07:45.738588+0000","last_became_peered":"2024-04-17T05:07:45.738588+0000","last_unstale":"2024-04-17T05:46:37.568563+0000","last_undegraded":"2024-04-17T05:46:37.568563+0000","last_fullsized":"2024-04-17T05:46:37.568563+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:37.568302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:37.568302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.568302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:25.004821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"264'6","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.841237+0000","last_change":"2024-04-17T05:46:30.841297+0000","last_active":"2024-04-17T05:46:30.841237+0000","last_peered":"2024-04-17T05:46:30.841237+0000","last_clean":"2024-04-17T05:46:30.841237+0000","last_became_active":"2024-04-17T04:51:43.681622+0000","last_became_peered":"2024-04-17T04:51:43.681622+0000","last_unstale":"2024-04-17T05:46:30.841237+0000","last_undegraded":"2024-04-17T05:46:30.841237+0000","last_fullsized":"2024-04-17T05:46:30.841237+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:30.841048+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:30.841048+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.841048+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:11.067215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":11,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.170964+0000","last_change":"2024-04-17T05:46:10.171028+0000","last_active":"2024-04-17T05:46:10.170964+0000","last_peered":"2024-04-17T05:46:10.170964+0000","last_clean":"2024-04-17T05:46:10.170964+0000","last_became_active":"2024-04-17T04:51:39.644950+0000","last_became_peered":"2024-04-17T04:51:39.644950+0000","last_unstale":"2024-04-17T05:46:10.170964+0000","last_undegraded":"2024-04-17T05:46:10.170964+0000","last_fullsized":"2024-04-17T05:46:10.170964+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:10.170766+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.170766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:36.494972+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"264'33","reported_seq":143,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.881133+0000","last_change":"2024-04-17T05:46:12.881133+0000","last_active":"2024-04-17T05:46:12.881133+0000","last_peered":"2024-04-17T05:46:12.881133+0000","last_clean":"2024-04-17T05:46:12.881133+0000","last_became_active":"2024-04-17T04:47:56.961165+0000","last_became_peered":"2024-04-17T04:47:56.961165+0000","last_unstale":"2024-04-17T05:46:12.881133+0000","last_undegraded":"2024-04-17T05:46:12.881133+0000","last_fullsized":"2024-04-17T05:46:12.881133+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'33","last_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_deep_scrub":"264'33","last_deep_scrub_stamp":"2024-04-17T05:46:12.880856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.880856+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:56.211415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":56,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:59.267104+0000","last_change":"2024-04-17T05:46:59.267210+0000","last_active":"2024-04-17T05:46:59.267104+0000","last_peered":"2024-04-17T05:46:59.267104+0000","last_clean":"2024-04-17T05:46:59.267104+0000","last_became_active":"2024-04-17T05:07:45.749664+0000","last_became_peered":"2024-04-17T05:07:45.749664+0000","last_unstale":"2024-04-17T05:46:59.267104+0000","last_undegraded":"2024-04-17T05:46:59.267104+0000","last_fullsized":"2024-04-17T05:46:59.267104+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:59.266869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:59.266869+0000","last_clean_scrub_stamp":"2024-04-17T05:46:59.266869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:28:30.878390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"264'5","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:06.670567+0000","last_change":"2024-04-17T05:47:06.670625+0000","last_active":"2024-04-17T05:47:06.670567+0000","last_peered":"2024-04-17T05:47:06.670567+0000","last_clean":"2024-04-17T05:47:06.670567+0000","last_became_active":"2024-04-17T04:51:43.688362+0000","last_became_peered":"2024-04-17T04:51:43.688362+0000","last_unstale":"2024-04-17T05:47:06.670567+0000","last_undegraded":"2024-04-17T05:47:06.670567+0000","last_fullsized":"2024-04-17T05:47:06.670567+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:47:06.670372+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:47:06.670372+0000","last_clean_scrub_stamp":"2024-04-17T05:47:06.670372+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:47:47.401249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"264'5","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.505904+0000","last_change":"2024-04-17T05:46:34.506036+0000","last_active":"2024-04-17T05:46:34.505904+0000","last_peered":"2024-04-17T05:46:34.505904+0000","last_clean":"2024-04-17T05:46:34.505904+0000","last_became_active":"2024-04-17T04:51:39.633988+0000","last_became_peered":"2024-04-17T04:51:39.633988+0000","last_unstale":"2024-04-17T05:46:34.505904+0000","last_undegraded":"2024-04-17T05:46:34.505904+0000","last_fullsized":"2024-04-17T05:46:34.505904+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:34.505613+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:34.505613+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.505613+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:12:13.411372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":11264,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"264'37","reported_seq":163,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.131285+0000","last_change":"2024-04-17T05:46:16.131285+0000","last_active":"2024-04-17T05:46:16.131285+0000","last_peered":"2024-04-17T05:46:16.131285+0000","last_clean":"2024-04-17T05:46:16.131285+0000","last_became_active":"2024-04-17T04:47:56.953605+0000","last_became_peered":"2024-04-17T04:47:56.953605+0000","last_unstale":"2024-04-17T05:46:16.131285+0000","last_undegraded":"2024-04-17T05:46:16.131285+0000","last_fullsized":"2024-04-17T05:46:16.131285+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'37","last_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_deep_scrub":"264'37","last_deep_scrub_stamp":"2024-04-17T05:46:16.131038+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.131038+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:32:33.132984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":121,"num_write":62,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.896222+0000","last_change":"2024-04-17T05:46:46.896279+0000","last_active":"2024-04-17T05:46:46.896222+0000","last_peered":"2024-04-17T05:46:46.896222+0000","last_clean":"2024-04-17T05:46:46.896222+0000","last_became_active":"2024-04-17T05:07:45.740431+0000","last_became_peered":"2024-04-17T05:07:45.740431+0000","last_unstale":"2024-04-17T05:46:46.896222+0000","last_undegraded":"2024-04-17T05:46:46.896222+0000","last_fullsized":"2024-04-17T05:46:46.896222+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:46.896013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:46.896013+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.896013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:52.872792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"261'3","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.908271+0000","last_change":"2024-04-17T05:46:44.908338+0000","last_active":"2024-04-17T05:46:44.908271+0000","last_peered":"2024-04-17T05:46:44.908271+0000","last_clean":"2024-04-17T05:46:44.908271+0000","last_became_active":"2024-04-17T04:51:43.656097+0000","last_became_peered":"2024-04-17T04:51:43.656097+0000","last_unstale":"2024-04-17T05:46:44.908271+0000","last_undegraded":"2024-04-17T05:46:44.908271+0000","last_fullsized":"2024-04-17T05:46:44.908271+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:44.908063+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:44.908063+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.908063+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:55:38.209073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"264'3","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.674045+0000","last_change":"2024-04-17T05:46:19.674109+0000","last_active":"2024-04-17T05:46:19.674045+0000","last_peered":"2024-04-17T05:46:19.674045+0000","last_clean":"2024-04-17T05:46:19.674045+0000","last_became_active":"2024-04-17T04:51:39.633754+0000","last_became_peered":"2024-04-17T04:51:39.633754+0000","last_unstale":"2024-04-17T05:46:19.674045+0000","last_undegraded":"2024-04-17T05:46:19.674045+0000","last_fullsized":"2024-04-17T05:46:19.674045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:19.673832+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.673832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:20:55.342567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"264'29","reported_seq":133,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.057017+0000","last_change":"2024-04-17T05:46:11.057017+0000","last_active":"2024-04-17T05:46:11.057017+0000","last_peered":"2024-04-17T05:46:11.057017+0000","last_clean":"2024-04-17T05:46:11.057017+0000","last_became_active":"2024-04-17T04:47:56.963286+0000","last_became_peered":"2024-04-17T04:47:56.963286+0000","last_unstale":"2024-04-17T05:46:11.057017+0000","last_undegraded":"2024-04-17T05:46:11.057017+0000","last_fullsized":"2024-04-17T05:46:11.057017+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'29","last_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_deep_scrub":"264'29","last_deep_scrub_stamp":"2024-04-17T05:46:11.056723+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.056723+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:04.980611+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":53,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:03.336353+0000","last_change":"2024-04-17T05:47:03.336408+0000","last_active":"2024-04-17T05:47:03.336353+0000","last_peered":"2024-04-17T05:47:03.336353+0000","last_clean":"2024-04-17T05:47:03.336353+0000","last_became_active":"2024-04-17T05:07:45.756471+0000","last_became_peered":"2024-04-17T05:07:45.756471+0000","last_unstale":"2024-04-17T05:47:03.336353+0000","last_undegraded":"2024-04-17T05:47:03.336353+0000","last_fullsized":"2024-04-17T05:47:03.336353+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:03.336113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:03.336113+0000","last_clean_scrub_stamp":"2024-04-17T05:47:03.336113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:21.079049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":54,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:03.576340+0000","last_change":"2024-04-17T05:47:03.576404+0000","last_active":"2024-04-17T05:47:03.576340+0000","last_peered":"2024-04-17T05:47:03.576340+0000","last_clean":"2024-04-17T05:47:03.576340+0000","last_became_active":"2024-04-17T04:52:04.673806+0000","last_became_peered":"2024-04-17T04:52:04.673806+0000","last_unstale":"2024-04-17T05:47:03.576340+0000","last_undegraded":"2024-04-17T05:47:03.576340+0000","last_fullsized":"2024-04-17T05:47:03.576340+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:47:03.576128+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:47:03.576128+0000","last_clean_scrub_stamp":"2024-04-17T05:47:03.576128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:11:34.896558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.680401+0000","last_change":"2024-04-17T05:46:08.680438+0000","last_active":"2024-04-17T05:46:08.680401+0000","last_peered":"2024-04-17T05:46:08.680401+0000","last_clean":"2024-04-17T05:46:08.680401+0000","last_became_active":"2024-04-17T04:51:39.617475+0000","last_became_peered":"2024-04-17T04:51:39.617475+0000","last_unstale":"2024-04-17T05:46:08.680401+0000","last_undegraded":"2024-04-17T05:46:08.680401+0000","last_fullsized":"2024-04-17T05:46:08.680401+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.680268+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.680268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:01:03.743450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"264'30","reported_seq":138,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.494346+0000","last_change":"2024-04-17T05:46:22.494346+0000","last_active":"2024-04-17T05:46:22.494346+0000","last_peered":"2024-04-17T05:46:22.494346+0000","last_clean":"2024-04-17T05:46:22.494346+0000","last_became_active":"2024-04-17T04:47:56.961124+0000","last_became_peered":"2024-04-17T04:47:56.961124+0000","last_unstale":"2024-04-17T05:46:22.494346+0000","last_undegraded":"2024-04-17T05:46:22.494346+0000","last_fullsized":"2024-04-17T05:46:22.494346+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'30","last_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_deep_scrub":"264'30","last_deep_scrub_stamp":"2024-04-17T05:46:22.494058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.494058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:08.498830+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":87,"num_write":52,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:08.637373+0000","last_change":"2024-04-17T05:47:08.637431+0000","last_active":"2024-04-17T05:47:08.637373+0000","last_peered":"2024-04-17T05:47:08.637373+0000","last_clean":"2024-04-17T05:47:08.637373+0000","last_became_active":"2024-04-17T05:07:45.743441+0000","last_became_peered":"2024-04-17T05:07:45.743441+0000","last_unstale":"2024-04-17T05:47:08.637373+0000","last_undegraded":"2024-04-17T05:47:08.637373+0000","last_fullsized":"2024-04-17T05:47:08.637373+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:08.637164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:08.637164+0000","last_clean_scrub_stamp":"2024-04-17T05:47:08.637164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:58.059260+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"264'7","reported_seq":69,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.601250+0000","last_change":"2024-04-17T05:46:54.601315+0000","last_active":"2024-04-17T05:46:54.601250+0000","last_peered":"2024-04-17T05:46:54.601250+0000","last_clean":"2024-04-17T05:46:54.601250+0000","last_became_active":"2024-04-17T04:51:43.660371+0000","last_became_peered":"2024-04-17T04:51:43.660371+0000","last_unstale":"2024-04-17T05:46:54.601250+0000","last_undegraded":"2024-04-17T05:46:54.601250+0000","last_fullsized":"2024-04-17T05:46:54.601250+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'7","last_scrub_stamp":"2024-04-17T05:46:54.601024+0000","last_deep_scrub":"264'7","last_deep_scrub_stamp":"2024-04-17T05:46:54.601024+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.601024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:01.709982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":5216,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.623849+0000","last_change":"2024-04-17T05:46:11.623908+0000","last_active":"2024-04-17T05:46:11.623849+0000","last_peered":"2024-04-17T05:46:11.623849+0000","last_clean":"2024-04-17T05:46:11.623849+0000","last_became_active":"2024-04-17T04:51:39.632499+0000","last_became_peered":"2024-04-17T04:51:39.632499+0000","last_unstale":"2024-04-17T05:46:11.623849+0000","last_undegraded":"2024-04-17T05:46:11.623849+0000","last_fullsized":"2024-04-17T05:46:11.623849+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'3","last_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_deep_scrub":"261'3","last_deep_scrub_stamp":"2024-04-17T05:46:11.623642+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.623642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:23.552160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"264'19","reported_seq":112,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.642068+0000","last_change":"2024-04-17T05:46:10.642068+0000","last_active":"2024-04-17T05:46:10.642068+0000","last_peered":"2024-04-17T05:46:10.642068+0000","last_clean":"2024-04-17T05:46:10.642068+0000","last_became_active":"2024-04-17T04:47:56.962574+0000","last_became_peered":"2024-04-17T04:47:56.962574+0000","last_unstale":"2024-04-17T05:46:10.642068+0000","last_undegraded":"2024-04-17T05:46:10.642068+0000","last_fullsized":"2024-04-17T05:46:10.642068+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'19","last_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_deep_scrub":"264'19","last_deep_scrub_stamp":"2024-04-17T05:46:10.641783+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.641783+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:09.385082+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":55,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:04.367142+0000","last_change":"2024-04-17T05:47:04.367201+0000","last_active":"2024-04-17T05:47:04.367142+0000","last_peered":"2024-04-17T05:47:04.367142+0000","last_clean":"2024-04-17T05:47:04.367142+0000","last_became_active":"2024-04-17T05:07:45.756500+0000","last_became_peered":"2024-04-17T05:07:45.756500+0000","last_unstale":"2024-04-17T05:47:04.367142+0000","last_undegraded":"2024-04-17T05:47:04.367142+0000","last_fullsized":"2024-04-17T05:47:04.367142+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:04.366926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:04.366926+0000","last_clean_scrub_stamp":"2024-04-17T05:47:04.366926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:49.317729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"264'5","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.167379+0000","last_change":"2024-04-17T05:46:16.167438+0000","last_active":"2024-04-17T05:46:16.167379+0000","last_peered":"2024-04-17T05:46:16.167379+0000","last_clean":"2024-04-17T05:46:16.167379+0000","last_became_active":"2024-04-17T04:51:43.669311+0000","last_became_peered":"2024-04-17T04:51:43.669311+0000","last_unstale":"2024-04-17T05:46:16.167379+0000","last_undegraded":"2024-04-17T05:46:16.167379+0000","last_fullsized":"2024-04-17T05:46:16.167379+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:16.167190+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.167190+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:00.500071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8192,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:07.682742+0000","last_change":"2024-04-17T05:46:07.682778+0000","last_active":"2024-04-17T05:46:07.682742+0000","last_peered":"2024-04-17T05:46:07.682742+0000","last_clean":"2024-04-17T05:46:07.682742+0000","last_became_active":"2024-04-17T04:51:39.617302+0000","last_became_peered":"2024-04-17T04:51:39.617302+0000","last_unstale":"2024-04-17T05:46:07.682742+0000","last_undegraded":"2024-04-17T05:46:07.682742+0000","last_fullsized":"2024-04-17T05:46:07.682742+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:07.682661+0000","last_clean_scrub_stamp":"2024-04-17T05:46:07.682661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:49:07.614824+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"264'93","reported_seq":296,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.265908+0000","last_change":"2024-04-17T05:46:21.265908+0000","last_active":"2024-04-17T05:46:21.265908+0000","last_peered":"2024-04-17T05:46:21.265908+0000","last_clean":"2024-04-17T05:46:21.265908+0000","last_became_active":"2024-04-17T04:47:56.962451+0000","last_became_peered":"2024-04-17T04:47:56.962451+0000","last_unstale":"2024-04-17T05:46:21.265908+0000","last_undegraded":"2024-04-17T05:46:21.265908+0000","last_fullsized":"2024-04-17T05:46:21.265908+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'93","last_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_deep_scrub":"264'93","last_deep_scrub_stamp":"2024-04-17T05:46:21.265637+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.265637+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:31.736633+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":404,"num_read_kb":288,"num_write":152,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"264'2","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:01.316689+0000","last_change":"2024-04-17T05:47:01.316763+0000","last_active":"2024-04-17T05:47:01.316689+0000","last_peered":"2024-04-17T05:47:01.316689+0000","last_clean":"2024-04-17T05:47:01.316689+0000","last_became_active":"2024-04-17T05:07:45.751521+0000","last_became_peered":"2024-04-17T05:07:45.751521+0000","last_unstale":"2024-04-17T05:47:01.316689+0000","last_undegraded":"2024-04-17T05:47:01.316689+0000","last_fullsized":"2024-04-17T05:47:01.316689+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:47:01.316494+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:47:01.316494+0000","last_clean_scrub_stamp":"2024-04-17T05:47:01.316494+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:37:46.857428+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"0'0","reported_seq":55,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.884070+0000","last_change":"2024-04-17T05:46:43.884127+0000","last_active":"2024-04-17T05:46:43.884070+0000","last_peered":"2024-04-17T05:46:43.884070+0000","last_clean":"2024-04-17T05:46:43.884070+0000","last_became_active":"2024-04-17T04:51:43.651814+0000","last_became_peered":"2024-04-17T04:51:43.651814+0000","last_unstale":"2024-04-17T05:46:43.884070+0000","last_undegraded":"2024-04-17T05:46:43.884070+0000","last_fullsized":"2024-04-17T05:46:43.884070+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.883876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.883876+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.883876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:50:29.135926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":62,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.584946+0000","last_change":"2024-04-17T05:46:15.585004+0000","last_active":"2024-04-17T05:46:15.584946+0000","last_peered":"2024-04-17T05:46:15.584946+0000","last_clean":"2024-04-17T05:46:15.584946+0000","last_became_active":"2024-04-17T04:51:39.632566+0000","last_became_peered":"2024-04-17T04:51:39.632566+0000","last_unstale":"2024-04-17T05:46:15.584946+0000","last_undegraded":"2024-04-17T05:46:15.584946+0000","last_fullsized":"2024-04-17T05:46:15.584946+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.584695+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.584695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:51:18.890015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"264'22","reported_seq":116,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.833951+0000","last_change":"2024-04-17T05:46:15.833951+0000","last_active":"2024-04-17T05:46:15.833951+0000","last_peered":"2024-04-17T05:46:15.833951+0000","last_clean":"2024-04-17T05:46:15.833951+0000","last_became_active":"2024-04-17T04:47:56.951777+0000","last_became_peered":"2024-04-17T04:47:56.951777+0000","last_unstale":"2024-04-17T05:46:15.833951+0000","last_undegraded":"2024-04-17T05:46:15.833951+0000","last_fullsized":"2024-04-17T05:46:15.833951+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'22","last_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_deep_scrub":"264'22","last_deep_scrub_stamp":"2024-04-17T05:46:15.833697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.833697+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:34:19.233513+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":52,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":38,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:05.601118+0000","last_change":"2024-04-17T05:47:05.601170+0000","last_active":"2024-04-17T05:47:05.601118+0000","last_peered":"2024-04-17T05:47:05.601118+0000","last_clean":"2024-04-17T05:47:05.601118+0000","last_became_active":"2024-04-17T05:07:45.738872+0000","last_became_peered":"2024-04-17T05:07:45.738872+0000","last_unstale":"2024-04-17T05:47:05.601118+0000","last_undegraded":"2024-04-17T05:47:05.601118+0000","last_fullsized":"2024-04-17T05:47:05.601118+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:05.600917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:05.600917+0000","last_clean_scrub_stamp":"2024-04-17T05:47:05.600917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:15:37.604006+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.268742+0000","last_change":"2024-04-17T05:46:20.268796+0000","last_active":"2024-04-17T05:46:20.268742+0000","last_peered":"2024-04-17T05:46:20.268742+0000","last_clean":"2024-04-17T05:46:20.268742+0000","last_became_active":"2024-04-17T04:51:43.674983+0000","last_became_peered":"2024-04-17T04:51:43.674983+0000","last_unstale":"2024-04-17T05:46:20.268742+0000","last_undegraded":"2024-04-17T05:46:20.268742+0000","last_fullsized":"2024-04-17T05:46:20.268742+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.268538+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.268538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:35.180807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"264'3","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.745329+0000","last_change":"2024-04-17T05:46:24.745393+0000","last_active":"2024-04-17T05:46:24.745329+0000","last_peered":"2024-04-17T05:46:24.745329+0000","last_clean":"2024-04-17T05:46:24.745329+0000","last_became_active":"2024-04-17T04:51:39.620385+0000","last_became_peered":"2024-04-17T04:51:39.620385+0000","last_unstale":"2024-04-17T05:46:24.745329+0000","last_undegraded":"2024-04-17T05:46:24.745329+0000","last_fullsized":"2024-04-17T05:46:24.745329+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:24.745126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.745126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:13.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"264'52","reported_seq":198,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.022550+0000","last_change":"2024-04-17T05:46:10.022550+0000","last_active":"2024-04-17T05:46:10.022550+0000","last_peered":"2024-04-17T05:46:10.022550+0000","last_clean":"2024-04-17T05:46:10.022550+0000","last_became_active":"2024-04-17T04:47:56.965495+0000","last_became_peered":"2024-04-17T04:47:56.965495+0000","last_unstale":"2024-04-17T05:46:10.022550+0000","last_undegraded":"2024-04-17T05:46:10.022550+0000","last_fullsized":"2024-04-17T05:46:10.022550+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'52","last_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_deep_scrub":"264'52","last_deep_scrub_stamp":"2024-04-17T05:46:10.022290+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.022290+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.993887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":228,"num_read_kb":165,"num_write":85,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.273717+0000","last_change":"2024-04-17T05:46:28.273717+0000","last_active":"2024-04-17T05:46:28.273717+0000","last_peered":"2024-04-17T05:46:28.273717+0000","last_clean":"2024-04-17T05:46:28.273717+0000","last_became_active":"2024-04-17T05:07:45.746514+0000","last_became_peered":"2024-04-17T05:07:45.746514+0000","last_unstale":"2024-04-17T05:46:28.273717+0000","last_undegraded":"2024-04-17T05:46:28.273717+0000","last_fullsized":"2024-04-17T05:46:28.273717+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.273435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.273435+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.273435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:53.897551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'2","reported_seq":61,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:02.564850+0000","last_change":"2024-04-17T05:47:02.565018+0000","last_active":"2024-04-17T05:47:02.564850+0000","last_peered":"2024-04-17T05:47:02.564850+0000","last_clean":"2024-04-17T05:47:02.564850+0000","last_became_active":"2024-04-17T04:51:43.673620+0000","last_became_peered":"2024-04-17T04:51:43.673620+0000","last_unstale":"2024-04-17T05:47:02.564850+0000","last_undegraded":"2024-04-17T05:47:02.564850+0000","last_fullsized":"2024-04-17T05:47:02.564850+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-17T05:47:02.564560+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-17T05:47:02.564560+0000","last_clean_scrub_stamp":"2024-04-17T05:47:02.564560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:33.650712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":27,"num_write":2,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"264'8","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.163258+0000","last_change":"2024-04-17T05:46:15.163324+0000","last_active":"2024-04-17T05:46:15.163258+0000","last_peered":"2024-04-17T05:46:15.163258+0000","last_clean":"2024-04-17T05:46:15.163258+0000","last_became_active":"2024-04-17T04:51:39.635090+0000","last_became_peered":"2024-04-17T04:51:39.635090+0000","last_unstale":"2024-04-17T05:46:15.163258+0000","last_undegraded":"2024-04-17T05:46:15.163258+0000","last_fullsized":"2024-04-17T05:46:15.163258+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:15.163041+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.163041+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:02:18.146680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":11291,"num_write":13,"num_write_kb":11291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"264'39","reported_seq":159,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.612312+0000","last_change":"2024-04-17T05:46:13.612312+0000","last_active":"2024-04-17T05:46:13.612312+0000","last_peered":"2024-04-17T05:46:13.612312+0000","last_clean":"2024-04-17T05:46:13.612312+0000","last_became_active":"2024-04-17T04:47:56.954310+0000","last_became_peered":"2024-04-17T04:47:56.954310+0000","last_unstale":"2024-04-17T05:46:13.612312+0000","last_undegraded":"2024-04-17T05:46:13.612312+0000","last_fullsized":"2024-04-17T05:46:13.612312+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'39","last_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_deep_scrub":"264'39","last_deep_scrub_stamp":"2024-04-17T05:46:13.612027+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.612027+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:48:28.378268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":107,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.189351+0000","last_change":"2024-04-17T05:46:57.189351+0000","last_active":"2024-04-17T05:46:57.189351+0000","last_peered":"2024-04-17T05:46:57.189351+0000","last_clean":"2024-04-17T05:46:57.189351+0000","last_became_active":"2024-04-17T05:07:45.749005+0000","last_became_peered":"2024-04-17T05:07:45.749005+0000","last_unstale":"2024-04-17T05:46:57.189351+0000","last_undegraded":"2024-04-17T05:46:57.189351+0000","last_fullsized":"2024-04-17T05:46:57.189351+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:57.189086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:57.189086+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.189086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:05:22.548179+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.614461+0000","last_change":"2024-04-17T05:46:31.614516+0000","last_active":"2024-04-17T05:46:31.614461+0000","last_peered":"2024-04-17T05:46:31.614461+0000","last_clean":"2024-04-17T05:46:31.614461+0000","last_became_active":"2024-04-17T04:51:43.672396+0000","last_became_peered":"2024-04-17T04:51:43.672396+0000","last_unstale":"2024-04-17T05:46:31.614461+0000","last_undegraded":"2024-04-17T05:46:31.614461+0000","last_fullsized":"2024-04-17T05:46:31.614461+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.614267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.614267+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.614267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:54:03.860590+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"264'3","reported_seq":68,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.554323+0000","last_change":"2024-04-17T05:46:35.554396+0000","last_active":"2024-04-17T05:46:35.554323+0000","last_peered":"2024-04-17T05:46:35.554323+0000","last_clean":"2024-04-17T05:46:35.554323+0000","last_became_active":"2024-04-17T04:51:39.634115+0000","last_became_peered":"2024-04-17T04:51:39.634115+0000","last_unstale":"2024-04-17T05:46:35.554323+0000","last_undegraded":"2024-04-17T05:46:35.554323+0000","last_fullsized":"2024-04-17T05:46:35.554323+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:35.554116+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:35.554116+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.554116+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:59:28.363602+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"264'60","reported_seq":206,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.995853+0000","last_change":"2024-04-17T05:46:14.995853+0000","last_active":"2024-04-17T05:46:14.995853+0000","last_peered":"2024-04-17T05:46:14.995853+0000","last_clean":"2024-04-17T05:46:14.995853+0000","last_became_active":"2024-04-17T04:47:56.944777+0000","last_became_peered":"2024-04-17T04:47:56.944777+0000","last_unstale":"2024-04-17T05:46:14.995853+0000","last_undegraded":"2024-04-17T05:46:14.995853+0000","last_fullsized":"2024-04-17T05:46:14.995853+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'60","last_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_deep_scrub":"264'60","last_deep_scrub_stamp":"2024-04-17T05:46:14.995600+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.995600+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:08.616032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":239,"num_read_kb":169,"num_write":109,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.964095+0000","last_change":"2024-04-17T05:46:47.964149+0000","last_active":"2024-04-17T05:46:47.964095+0000","last_peered":"2024-04-17T05:46:47.964095+0000","last_clean":"2024-04-17T05:46:47.964095+0000","last_became_active":"2024-04-17T05:07:45.735474+0000","last_became_peered":"2024-04-17T05:07:45.735474+0000","last_unstale":"2024-04-17T05:46:47.964095+0000","last_undegraded":"2024-04-17T05:46:47.964095+0000","last_fullsized":"2024-04-17T05:46:47.964095+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:47.963889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:47.963889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.963889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:33.636904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"264'21","reported_seq":101,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.298945+0000","last_change":"2024-04-17T05:46:24.299018+0000","last_active":"2024-04-17T05:46:24.298945+0000","last_peered":"2024-04-17T05:46:24.298945+0000","last_clean":"2024-04-17T05:46:24.298945+0000","last_became_active":"2024-04-17T04:51:43.675069+0000","last_became_peered":"2024-04-17T04:51:43.675069+0000","last_unstale":"2024-04-17T05:46:24.298945+0000","last_undegraded":"2024-04-17T05:46:24.298945+0000","last_fullsized":"2024-04-17T05:46:24.298945+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'21","last_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_deep_scrub":"264'21","last_deep_scrub_stamp":"2024-04-17T05:46:24.298719+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.298719+0000","objects_scrubbed":1,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:44:55.830264+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":54274,"num_write":19,"num_write_kb":33792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"264'12","reported_seq":90,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.319932+0000","last_change":"2024-04-17T05:46:51.320013+0000","last_active":"2024-04-17T05:46:51.319932+0000","last_peered":"2024-04-17T05:46:51.319932+0000","last_clean":"2024-04-17T05:46:51.319932+0000","last_became_active":"2024-04-17T04:51:39.634349+0000","last_became_peered":"2024-04-17T04:51:39.634349+0000","last_unstale":"2024-04-17T05:46:51.319932+0000","last_undegraded":"2024-04-17T05:46:51.319932+0000","last_fullsized":"2024-04-17T05:46:51.319932+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'12","last_scrub_stamp":"2024-04-17T05:46:51.319740+0000","last_deep_scrub":"264'12","last_deep_scrub_stamp":"2024-04-17T05:46:51.319740+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.319740+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:47.902459+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":24580,"num_write":13,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"264'62","reported_seq":233,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.186314+0000","last_change":"2024-04-17T05:46:17.186314+0000","last_active":"2024-04-17T05:46:17.186314+0000","last_peered":"2024-04-17T05:46:17.186314+0000","last_clean":"2024-04-17T05:46:17.186314+0000","last_became_active":"2024-04-17T04:47:56.967403+0000","last_became_peered":"2024-04-17T04:47:56.967403+0000","last_unstale":"2024-04-17T05:46:17.186314+0000","last_undegraded":"2024-04-17T05:46:17.186314+0000","last_fullsized":"2024-04-17T05:46:17.186314+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'62","last_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_deep_scrub":"264'62","last_deep_scrub_stamp":"2024-04-17T05:46:17.186068+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.186068+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:38:32.734141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":301,"num_read_kb":218,"num_write":94,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.652989+0000","last_change":"2024-04-17T05:46:57.652989+0000","last_active":"2024-04-17T05:46:57.652989+0000","last_peered":"2024-04-17T05:46:57.652989+0000","last_clean":"2024-04-17T05:46:57.652989+0000","last_became_active":"2024-04-17T05:07:45.748536+0000","last_became_peered":"2024-04-17T05:07:45.748536+0000","last_unstale":"2024-04-17T05:46:57.652989+0000","last_undegraded":"2024-04-17T05:46:57.652989+0000","last_fullsized":"2024-04-17T05:46:57.652989+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:57.652734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:57.652734+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.652734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:47:27.873329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":56,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.527408+0000","last_change":"2024-04-17T05:46:52.527462+0000","last_active":"2024-04-17T05:46:52.527408+0000","last_peered":"2024-04-17T05:46:52.527408+0000","last_clean":"2024-04-17T05:46:52.527408+0000","last_became_active":"2024-04-17T04:51:43.647900+0000","last_became_peered":"2024-04-17T04:51:43.647900+0000","last_unstale":"2024-04-17T05:46:52.527408+0000","last_undegraded":"2024-04-17T05:46:52.527408+0000","last_fullsized":"2024-04-17T05:46:52.527408+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:52.527195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:52.527195+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.527195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:45.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"264'6","reported_seq":73,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.099801+0000","last_change":"2024-04-17T05:46:12.099865+0000","last_active":"2024-04-17T05:46:12.099801+0000","last_peered":"2024-04-17T05:46:12.099801+0000","last_clean":"2024-04-17T05:46:12.099801+0000","last_became_active":"2024-04-17T04:51:39.645523+0000","last_became_peered":"2024-04-17T04:51:39.645523+0000","last_unstale":"2024-04-17T05:46:12.099801+0000","last_undegraded":"2024-04-17T05:46:12.099801+0000","last_fullsized":"2024-04-17T05:46:12.099801+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'6","last_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_deep_scrub":"264'6","last_deep_scrub_stamp":"2024-04-17T05:46:12.099605+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.099605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:49.682658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":8192,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'21","reported_seq":120,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.665206+0000","last_change":"2024-04-17T05:46:10.665206+0000","last_active":"2024-04-17T05:46:10.665206+0000","last_peered":"2024-04-17T05:46:10.665206+0000","last_clean":"2024-04-17T05:46:10.665206+0000","last_became_active":"2024-04-17T04:47:56.946139+0000","last_became_peered":"2024-04-17T04:47:56.946139+0000","last_unstale":"2024-04-17T05:46:10.665206+0000","last_undegraded":"2024-04-17T05:46:10.665206+0000","last_fullsized":"2024-04-17T05:46:10.665206+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'21","last_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_deep_scrub":"261'21","last_deep_scrub_stamp":"2024-04-17T05:46:10.664937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.664937+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:18.989139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":63,"num_write":35,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"264'2","reported_seq":41,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.564484+0000","last_change":"2024-04-17T05:46:38.564484+0000","last_active":"2024-04-17T05:46:38.564484+0000","last_peered":"2024-04-17T05:46:38.564484+0000","last_clean":"2024-04-17T05:46:38.564484+0000","last_became_active":"2024-04-17T05:07:45.739295+0000","last_became_peered":"2024-04-17T05:07:45.739295+0000","last_unstale":"2024-04-17T05:46:38.564484+0000","last_undegraded":"2024-04-17T05:46:38.564484+0000","last_fullsized":"2024-04-17T05:46:38.564484+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:38.564228+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:38.564228+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.564228+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:54:51.812713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"264'9","reported_seq":72,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.926385+0000","last_change":"2024-04-17T05:46:46.926459+0000","last_active":"2024-04-17T05:46:46.926385+0000","last_peered":"2024-04-17T05:46:46.926385+0000","last_clean":"2024-04-17T05:46:46.926385+0000","last_became_active":"2024-04-17T04:51:43.674650+0000","last_became_peered":"2024-04-17T04:51:43.674650+0000","last_unstale":"2024-04-17T05:46:46.926385+0000","last_undegraded":"2024-04-17T05:46:46.926385+0000","last_fullsized":"2024-04-17T05:46:46.926385+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'9","last_scrub_stamp":"2024-04-17T05:46:46.926173+0000","last_deep_scrub":"264'9","last_deep_scrub_stamp":"2024-04-17T05:46:46.926173+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.926173+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:21.554513+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":4098,"num_write":17,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":64,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.130174+0000","last_change":"2024-04-17T05:46:13.130289+0000","last_active":"2024-04-17T05:46:13.130174+0000","last_peered":"2024-04-17T05:46:13.130174+0000","last_clean":"2024-04-17T05:46:13.130174+0000","last_became_active":"2024-04-17T04:51:39.636456+0000","last_became_peered":"2024-04-17T04:51:39.636456+0000","last_unstale":"2024-04-17T05:46:13.130174+0000","last_undegraded":"2024-04-17T05:46:13.130174+0000","last_fullsized":"2024-04-17T05:46:13.130174+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:13.129923+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.129923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:48.339568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"253'32","reported_seq":153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.106637+0000","last_change":"2024-04-17T05:46:12.106637+0000","last_active":"2024-04-17T05:46:12.106637+0000","last_peered":"2024-04-17T05:46:12.106637+0000","last_clean":"2024-04-17T05:46:12.106637+0000","last_became_active":"2024-04-17T04:47:56.948811+0000","last_became_peered":"2024-04-17T04:47:56.948811+0000","last_unstale":"2024-04-17T05:46:12.106637+0000","last_undegraded":"2024-04-17T05:46:12.106637+0000","last_fullsized":"2024-04-17T05:46:12.106637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'32","last_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_deep_scrub":"253'32","last_deep_scrub_stamp":"2024-04-17T05:46:12.106369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.106369+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:23.602749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":113,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":39,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:04.610139+0000","last_change":"2024-04-17T05:47:04.610193+0000","last_active":"2024-04-17T05:47:04.610139+0000","last_peered":"2024-04-17T05:47:04.610139+0000","last_clean":"2024-04-17T05:47:04.610139+0000","last_became_active":"2024-04-17T05:07:45.726113+0000","last_became_peered":"2024-04-17T05:07:45.726113+0000","last_unstale":"2024-04-17T05:47:04.610139+0000","last_undegraded":"2024-04-17T05:47:04.610139+0000","last_fullsized":"2024-04-17T05:47:04.610139+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:04.609938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:04.609938+0000","last_clean_scrub_stamp":"2024-04-17T05:47:04.609938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:54:57.801405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"264'3","reported_seq":65,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.259328+0000","last_change":"2024-04-17T05:46:52.259392+0000","last_active":"2024-04-17T05:46:52.259328+0000","last_peered":"2024-04-17T05:46:52.259328+0000","last_clean":"2024-04-17T05:46:52.259328+0000","last_became_active":"2024-04-17T04:51:43.647892+0000","last_became_peered":"2024-04-17T04:51:43.647892+0000","last_unstale":"2024-04-17T05:46:52.259328+0000","last_undegraded":"2024-04-17T05:46:52.259328+0000","last_fullsized":"2024-04-17T05:46:52.259328+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'3","last_scrub_stamp":"2024-04-17T05:46:52.259111+0000","last_deep_scrub":"264'3","last_deep_scrub_stamp":"2024-04-17T05:46:52.259111+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.259111+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:09.465653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":63,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.127295+0000","last_change":"2024-04-17T05:46:14.127359+0000","last_active":"2024-04-17T05:46:14.127295+0000","last_peered":"2024-04-17T05:46:14.127295+0000","last_clean":"2024-04-17T05:46:14.127295+0000","last_became_active":"2024-04-17T04:51:39.636202+0000","last_became_peered":"2024-04-17T04:51:39.636202+0000","last_unstale":"2024-04-17T05:46:14.127295+0000","last_undegraded":"2024-04-17T05:46:14.127295+0000","last_fullsized":"2024-04-17T05:46:14.127295+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:14.127097+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.127097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:17:00.931818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"264'32","reported_seq":151,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.750372+0000","last_change":"2024-04-17T05:46:08.750372+0000","last_active":"2024-04-17T05:46:08.750372+0000","last_peered":"2024-04-17T05:46:08.750372+0000","last_clean":"2024-04-17T05:46:08.750372+0000","last_became_active":"2024-04-17T04:47:56.968384+0000","last_became_peered":"2024-04-17T04:47:56.968384+0000","last_unstale":"2024-04-17T05:46:08.750372+0000","last_undegraded":"2024-04-17T05:46:08.750372+0000","last_fullsized":"2024-04-17T05:46:08.750372+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'32","last_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_deep_scrub":"264'32","last_deep_scrub_stamp":"2024-04-17T05:46:08.750109+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.750109+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:42:20.266910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":102,"num_write":52,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:08.530076+0000","last_change":"2024-04-17T05:47:08.530076+0000","last_active":"2024-04-17T05:47:08.530076+0000","last_peered":"2024-04-17T05:47:08.530076+0000","last_clean":"2024-04-17T05:47:08.530076+0000","last_became_active":"2024-04-17T05:07:45.755764+0000","last_became_peered":"2024-04-17T05:07:45.755764+0000","last_unstale":"2024-04-17T05:47:08.530076+0000","last_undegraded":"2024-04-17T05:47:08.530076+0000","last_fullsized":"2024-04-17T05:47:08.530076+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:08.529819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:08.529819+0000","last_clean_scrub_stamp":"2024-04-17T05:47:08.529819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:56:32.267749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.858323+0000","last_change":"2024-04-17T05:46:44.858377+0000","last_active":"2024-04-17T05:46:44.858323+0000","last_peered":"2024-04-17T05:46:44.858323+0000","last_clean":"2024-04-17T05:46:44.858323+0000","last_became_active":"2024-04-17T04:51:43.681654+0000","last_became_peered":"2024-04-17T04:51:43.681654+0000","last_unstale":"2024-04-17T05:46:44.858323+0000","last_undegraded":"2024-04-17T05:46:44.858323+0000","last_fullsized":"2024-04-17T05:46:44.858323+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:44.858126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:44.858126+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.858126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:47:56.173235+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"264'2","reported_seq":66,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.643349+0000","last_change":"2024-04-17T05:46:09.643497+0000","last_active":"2024-04-17T05:46:09.643349+0000","last_peered":"2024-04-17T05:46:09.643349+0000","last_clean":"2024-04-17T05:46:09.643349+0000","last_became_active":"2024-04-17T04:51:39.616673+0000","last_became_peered":"2024-04-17T04:51:39.616673+0000","last_unstale":"2024-04-17T05:46:09.643349+0000","last_undegraded":"2024-04-17T05:46:09.643349+0000","last_fullsized":"2024-04-17T05:46:09.643349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:09.643135+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.643135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:35:49.245538+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"264'69","reported_seq":223,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.049363+0000","last_change":"2024-04-17T05:46:13.049363+0000","last_active":"2024-04-17T05:46:13.049363+0000","last_peered":"2024-04-17T05:46:13.049363+0000","last_clean":"2024-04-17T05:46:13.049363+0000","last_became_active":"2024-04-17T04:47:56.951212+0000","last_became_peered":"2024-04-17T04:47:56.951212+0000","last_unstale":"2024-04-17T05:46:13.049363+0000","last_undegraded":"2024-04-17T05:46:13.049363+0000","last_fullsized":"2024-04-17T05:46:13.049363+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'69","last_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_deep_scrub":"264'69","last_deep_scrub_stamp":"2024-04-17T05:46:13.048938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.048938+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:33:49.209093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":265,"num_read_kb":186,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":40,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:00.299410+0000","last_change":"2024-04-17T05:47:00.299410+0000","last_active":"2024-04-17T05:47:00.299410+0000","last_peered":"2024-04-17T05:47:00.299410+0000","last_clean":"2024-04-17T05:47:00.299410+0000","last_became_active":"2024-04-17T05:07:45.750154+0000","last_became_peered":"2024-04-17T05:07:45.750154+0000","last_unstale":"2024-04-17T05:47:00.299410+0000","last_undegraded":"2024-04-17T05:47:00.299410+0000","last_fullsized":"2024-04-17T05:47:00.299410+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:47:00.299142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:47:00.299142+0000","last_clean_scrub_stamp":"2024-04-17T05:47:00.299142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:48:49.331319+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":57,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.556697+0000","last_change":"2024-04-17T05:46:53.556758+0000","last_active":"2024-04-17T05:46:53.556697+0000","last_peered":"2024-04-17T05:46:53.556697+0000","last_clean":"2024-04-17T05:46:53.556697+0000","last_became_active":"2024-04-17T04:51:43.650433+0000","last_became_peered":"2024-04-17T04:51:43.650433+0000","last_unstale":"2024-04-17T05:46:53.556697+0000","last_undegraded":"2024-04-17T05:46:53.556697+0000","last_fullsized":"2024-04-17T05:46:53.556697+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":257,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:53.556487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:53.556487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.556487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:39.303780+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"264'5","reported_seq":71,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.936901+0000","last_change":"2024-04-17T05:46:40.936964+0000","last_active":"2024-04-17T05:46:40.936901+0000","last_peered":"2024-04-17T05:46:40.936901+0000","last_clean":"2024-04-17T05:46:40.936901+0000","last_became_active":"2024-04-17T04:51:39.629521+0000","last_became_peered":"2024-04-17T04:51:39.629521+0000","last_unstale":"2024-04-17T05:46:40.936901+0000","last_undegraded":"2024-04-17T05:46:40.936901+0000","last_fullsized":"2024-04-17T05:46:40.936901+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'5","last_scrub_stamp":"2024-04-17T05:46:40.936715+0000","last_deep_scrub":"264'5","last_deep_scrub_stamp":"2024-04-17T05:46:40.936715+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.936715+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:52:06.846607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6192,"num_write":5,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"264'27","reported_seq":137,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.225148+0000","last_change":"2024-04-17T05:46:19.225296+0000","last_active":"2024-04-17T05:46:19.225148+0000","last_peered":"2024-04-17T05:46:19.225148+0000","last_clean":"2024-04-17T05:46:19.225148+0000","last_became_active":"2024-04-17T04:47:56.962480+0000","last_became_peered":"2024-04-17T04:47:56.962480+0000","last_unstale":"2024-04-17T05:46:19.225148+0000","last_undegraded":"2024-04-17T05:46:19.225148+0000","last_fullsized":"2024-04-17T05:46:19.225148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'27","last_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_deep_scrub":"264'27","last_deep_scrub_stamp":"2024-04-17T05:46:19.224892+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.224892+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:29.117161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":83,"num_write":58,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"264'349","reported_seq":1020,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.239424+0000","last_change":"2024-04-17T05:46:57.239424+0000","last_active":"2024-04-17T05:46:57.239424+0000","last_peered":"2024-04-17T05:46:57.239424+0000","last_clean":"2024-04-17T05:46:57.239424+0000","last_became_active":"2024-04-17T04:36:25.376459+0000","last_became_peered":"2024-04-17T04:36:25.376459+0000","last_unstale":"2024-04-17T05:46:57.239424+0000","last_undegraded":"2024-04-17T05:46:57.239424+0000","last_fullsized":"2024-04-17T05:46:57.239424+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'349","last_scrub_stamp":"2024-04-17T05:46:57.239101+0000","last_deep_scrub":"264'349","last_deep_scrub_stamp":"2024-04-17T05:46:57.239101+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.239101+0000","objects_scrubbed":3,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:02:55.639650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":725,"num_write":1314,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0,5],"acting":[7,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"264'80","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.269919+0000","last_change":"2024-04-17T05:46:28.269919+0000","last_active":"2024-04-17T05:46:28.269919+0000","last_peered":"2024-04-17T05:46:28.269919+0000","last_clean":"2024-04-17T05:46:28.269919+0000","last_became_active":"2024-04-17T04:36:27.389203+0000","last_became_peered":"2024-04-17T04:36:27.389203+0000","last_unstale":"2024-04-17T05:46:28.269919+0000","last_undegraded":"2024-04-17T05:46:28.269919+0000","last_fullsized":"2024-04-17T05:46:28.269919+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'80","last_scrub_stamp":"2024-04-17T05:46:28.269667+0000","last_deep_scrub":"264'80","last_deep_scrub_stamp":"2024-04-17T05:46:28.269667+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.269667+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:21:37.474212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":305,"num_write":115,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"264'382","reported_seq":1064,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.830336+0000","last_change":"2024-04-17T05:46:39.830336+0000","last_active":"2024-04-17T05:46:39.830336+0000","last_peered":"2024-04-17T05:46:39.830336+0000","last_clean":"2024-04-17T05:46:39.830336+0000","last_became_active":"2024-04-17T04:36:25.353193+0000","last_became_peered":"2024-04-17T04:36:25.353193+0000","last_unstale":"2024-04-17T05:46:39.830336+0000","last_undegraded":"2024-04-17T05:46:39.830336+0000","last_fullsized":"2024-04-17T05:46:39.830336+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'382","last_scrub_stamp":"2024-04-17T05:46:39.830021+0000","last_deep_scrub":"264'382","last_deep_scrub_stamp":"2024-04-17T05:46:39.830021+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.830021+0000","objects_scrubbed":6,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:40.048724+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":600,"num_read_kb":10797,"num_write":1506,"num_write_kb":40235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"264'102","reported_seq":716,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.391956+0000","last_change":"2024-04-17T05:46:39.392102+0000","last_active":"2024-04-17T05:46:39.391956+0000","last_peered":"2024-04-17T05:46:39.391956+0000","last_clean":"2024-04-17T05:46:39.391956+0000","last_became_active":"2024-04-17T04:36:27.381165+0000","last_became_peered":"2024-04-17T04:36:27.381165+0000","last_unstale":"2024-04-17T05:46:39.391956+0000","last_undegraded":"2024-04-17T05:46:39.391956+0000","last_fullsized":"2024-04-17T05:46:39.391956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:39.391733+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:39.391733+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.391733+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:03:24.653144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":144,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"264'312","reported_seq":988,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.651417+0000","last_change":"2024-04-17T05:46:54.651417+0000","last_active":"2024-04-17T05:46:54.651417+0000","last_peered":"2024-04-17T05:46:54.651417+0000","last_clean":"2024-04-17T05:46:54.651417+0000","last_became_active":"2024-04-17T04:36:25.328930+0000","last_became_peered":"2024-04-17T04:36:25.328930+0000","last_unstale":"2024-04-17T05:46:54.651417+0000","last_undegraded":"2024-04-17T05:46:54.651417+0000","last_fullsized":"2024-04-17T05:46:54.651417+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'312","last_scrub_stamp":"2024-04-17T05:46:54.650922+0000","last_deep_scrub":"264'312","last_deep_scrub_stamp":"2024-04-17T05:46:54.650922+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.650922+0000","objects_scrubbed":12,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.389913+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11583546,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":475,"num_read_kb":58403,"num_write":1188,"num_write_kb":52720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"264'525","reported_seq":1215,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.608673+0000","last_change":"2024-04-17T05:46:37.608807+0000","last_active":"2024-04-17T05:46:37.608673+0000","last_peered":"2024-04-17T05:46:37.608673+0000","last_clean":"2024-04-17T05:46:37.608673+0000","last_became_active":"2024-04-17T04:36:27.384441+0000","last_became_peered":"2024-04-17T04:36:27.384441+0000","last_unstale":"2024-04-17T05:46:37.608673+0000","last_undegraded":"2024-04-17T05:46:37.608673+0000","last_fullsized":"2024-04-17T05:46:37.608673+0000","mapping_epoch":25,"log_start":"264'500","ondisk_log_start":"264'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'525","last_scrub_stamp":"2024-04-17T05:46:37.608425+0000","last_deep_scrub":"264'525","last_deep_scrub_stamp":"2024-04-17T05:46:37.608425+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.608425+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":500,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:33.078323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1963,"num_read_kb":1516,"num_write":798,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"264'284","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.266538+0000","last_change":"2024-04-17T05:46:47.266902+0000","last_active":"2024-04-17T05:46:47.266538+0000","last_peered":"2024-04-17T05:46:47.266538+0000","last_clean":"2024-04-17T05:46:47.266538+0000","last_became_active":"2024-04-17T04:36:25.355257+0000","last_became_peered":"2024-04-17T04:36:25.355257+0000","last_unstale":"2024-04-17T05:46:47.266538+0000","last_undegraded":"2024-04-17T05:46:47.266538+0000","last_fullsized":"2024-04-17T05:46:47.266538+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'284","last_scrub_stamp":"2024-04-17T05:46:47.266303+0000","last_deep_scrub":"264'284","last_deep_scrub_stamp":"2024-04-17T05:46:47.266303+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.266303+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:17:04.388187+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174737,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":460,"num_read_kb":37884,"num_write":1128,"num_write_kb":52095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"264'305","reported_seq":908,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.703858+0000","last_change":"2024-04-17T05:46:39.703930+0000","last_active":"2024-04-17T05:46:39.703858+0000","last_peered":"2024-04-17T05:46:39.703858+0000","last_clean":"2024-04-17T05:46:39.703858+0000","last_became_active":"2024-04-17T04:36:27.385118+0000","last_became_peered":"2024-04-17T04:36:27.385118+0000","last_unstale":"2024-04-17T05:46:39.703858+0000","last_undegraded":"2024-04-17T05:46:39.703858+0000","last_fullsized":"2024-04-17T05:46:39.703858+0000","mapping_epoch":25,"log_start":"261'200","ondisk_log_start":"261'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'305","last_scrub_stamp":"2024-04-17T05:46:39.703631+0000","last_deep_scrub":"264'305","last_deep_scrub_stamp":"2024-04-17T05:46:39.703631+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.703631+0000","objects_scrubbed":1,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:28:54.709159+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1176,"num_read_kb":903,"num_write":503,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"264'336","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.112424+0000","last_change":"2024-04-17T05:46:49.112596+0000","last_active":"2024-04-17T05:46:49.112424+0000","last_peered":"2024-04-17T05:46:49.112424+0000","last_clean":"2024-04-17T05:46:49.112424+0000","last_became_active":"2024-04-17T04:36:25.376631+0000","last_became_peered":"2024-04-17T04:36:25.376631+0000","last_unstale":"2024-04-17T05:46:49.112424+0000","last_undegraded":"2024-04-17T05:46:49.112424+0000","last_fullsized":"2024-04-17T05:46:49.112424+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'336","last_scrub_stamp":"2024-04-17T05:46:49.112208+0000","last_deep_scrub":"264'336","last_deep_scrub_stamp":"2024-04-17T05:46:49.112208+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.112208+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:03:27.854631+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":528,"num_read_kb":13359,"num_write":1173,"num_write_kb":48268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"264'4413","reported_seq":5189,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.926781+0000","last_change":"2024-04-17T05:46:28.926781+0000","last_active":"2024-04-17T05:46:28.926781+0000","last_peered":"2024-04-17T05:46:28.926781+0000","last_clean":"2024-04-17T05:46:28.926781+0000","last_became_active":"2024-04-17T04:36:27.378061+0000","last_became_peered":"2024-04-17T04:36:27.378061+0000","last_unstale":"2024-04-17T05:46:28.926781+0000","last_undegraded":"2024-04-17T05:46:28.926781+0000","last_fullsized":"2024-04-17T05:46:28.926781+0000","mapping_epoch":25,"log_start":"264'4400","ondisk_log_start":"264'4400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4413","last_scrub_stamp":"2024-04-17T05:46:28.926530+0000","last_deep_scrub":"264'4413","last_deep_scrub_stamp":"2024-04-17T05:46:28.926530+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.926530+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":3000,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:55:24.685918+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15768,"num_read_kb":12925,"num_write":6729,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"264'301","reported_seq":1024,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.312758+0000","last_change":"2024-04-17T05:46:46.312861+0000","last_active":"2024-04-17T05:46:46.312758+0000","last_peered":"2024-04-17T05:46:46.312758+0000","last_clean":"2024-04-17T05:46:46.312758+0000","last_became_active":"2024-04-17T04:36:25.350807+0000","last_became_peered":"2024-04-17T04:36:25.350807+0000","last_unstale":"2024-04-17T05:46:46.312758+0000","last_undegraded":"2024-04-17T05:46:46.312758+0000","last_fullsized":"2024-04-17T05:46:46.312758+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'301","last_scrub_stamp":"2024-04-17T05:46:46.312563+0000","last_deep_scrub":"264'301","last_deep_scrub_stamp":"2024-04-17T05:46:46.312563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.312563+0000","objects_scrubbed":9,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:18.182664+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206995,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":584,"num_read_kb":64592,"num_write":977,"num_write_kb":37773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"264'157","reported_seq":784,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.560872+0000","last_change":"2024-04-17T05:46:33.560872+0000","last_active":"2024-04-17T05:46:33.560872+0000","last_peered":"2024-04-17T05:46:33.560872+0000","last_clean":"2024-04-17T05:46:33.560872+0000","last_became_active":"2024-04-17T04:36:27.384704+0000","last_became_peered":"2024-04-17T04:36:27.384704+0000","last_unstale":"2024-04-17T05:46:33.560872+0000","last_undegraded":"2024-04-17T05:46:33.560872+0000","last_fullsized":"2024-04-17T05:46:33.560872+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'157","last_scrub_stamp":"2024-04-17T05:46:33.560601+0000","last_deep_scrub":"264'157","last_deep_scrub_stamp":"2024-04-17T05:46:33.560601+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.560601+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:27.508256+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":631,"num_read_kb":505,"num_write":253,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"264'324","reported_seq":956,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.236524+0000","last_change":"2024-04-17T05:46:56.236634+0000","last_active":"2024-04-17T05:46:56.236524+0000","last_peered":"2024-04-17T05:46:56.236524+0000","last_clean":"2024-04-17T05:46:56.236524+0000","last_became_active":"2024-04-17T04:36:25.376059+0000","last_became_peered":"2024-04-17T04:36:25.376059+0000","last_unstale":"2024-04-17T05:46:56.236524+0000","last_undegraded":"2024-04-17T05:46:56.236524+0000","last_fullsized":"2024-04-17T05:46:56.236524+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'324","last_scrub_stamp":"2024-04-17T05:46:56.236320+0000","last_deep_scrub":"264'324","last_deep_scrub_stamp":"2024-04-17T05:46:56.236320+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.236320+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:29:21.306325+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2269430,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":458,"num_read_kb":9424,"num_write":1128,"num_write_kb":36577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"264'165","reported_seq":818,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.313738+0000","last_change":"2024-04-17T05:46:31.313738+0000","last_active":"2024-04-17T05:46:31.313738+0000","last_peered":"2024-04-17T05:46:31.313738+0000","last_clean":"2024-04-17T05:46:31.313738+0000","last_became_active":"2024-04-17T04:36:27.376116+0000","last_became_peered":"2024-04-17T04:36:27.376116+0000","last_unstale":"2024-04-17T05:46:31.313738+0000","last_undegraded":"2024-04-17T05:46:31.313738+0000","last_fullsized":"2024-04-17T05:46:31.313738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'165","last_scrub_stamp":"2024-04-17T05:46:31.313464+0000","last_deep_scrub":"264'165","last_deep_scrub_stamp":"2024-04-17T05:46:31.313464+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.313464+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":100,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:46.132113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":679,"num_read_kb":559,"num_write":243,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"264'318","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.627848+0000","last_change":"2024-04-17T05:46:57.628017+0000","last_active":"2024-04-17T05:46:57.627848+0000","last_peered":"2024-04-17T05:46:57.627848+0000","last_clean":"2024-04-17T05:46:57.627848+0000","last_became_active":"2024-04-17T04:36:25.343496+0000","last_became_peered":"2024-04-17T04:36:25.343496+0000","last_unstale":"2024-04-17T05:46:57.627848+0000","last_undegraded":"2024-04-17T05:46:57.627848+0000","last_fullsized":"2024-04-17T05:46:57.627848+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'318","last_scrub_stamp":"2024-04-17T05:46:57.627640+0000","last_deep_scrub":"264'318","last_deep_scrub_stamp":"2024-04-17T05:46:57.627640+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.627640+0000","objects_scrubbed":10,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:24.431177+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076711,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":427,"num_read_kb":14992,"num_write":1164,"num_write_kb":55003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"264'166","reported_seq":780,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.467227+0000","last_change":"2024-04-17T05:46:28.467227+0000","last_active":"2024-04-17T05:46:28.467227+0000","last_peered":"2024-04-17T05:46:28.467227+0000","last_clean":"2024-04-17T05:46:28.467227+0000","last_became_active":"2024-04-17T04:36:27.378415+0000","last_became_peered":"2024-04-17T04:36:27.378415+0000","last_unstale":"2024-04-17T05:46:28.467227+0000","last_undegraded":"2024-04-17T05:46:28.467227+0000","last_fullsized":"2024-04-17T05:46:28.467227+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'166","last_scrub_stamp":"2024-04-17T05:46:28.466958+0000","last_deep_scrub":"264'166","last_deep_scrub_stamp":"2024-04-17T05:46:28.466958+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.466958+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:50:48.423737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":551,"num_write":270,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"264'128","reported_seq":739,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.590578+0000","last_change":"2024-04-17T05:46:31.590640+0000","last_active":"2024-04-17T05:46:31.590578+0000","last_peered":"2024-04-17T05:46:31.590578+0000","last_clean":"2024-04-17T05:46:31.590578+0000","last_became_active":"2024-04-17T04:36:27.384798+0000","last_became_peered":"2024-04-17T04:36:27.384798+0000","last_unstale":"2024-04-17T05:46:31.590578+0000","last_undegraded":"2024-04-17T05:46:31.590578+0000","last_fullsized":"2024-04-17T05:46:31.590578+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'128","last_scrub_stamp":"2024-04-17T05:46:31.590386+0000","last_deep_scrub":"264'128","last_deep_scrub_stamp":"2024-04-17T05:46:31.590386+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.590386+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:05:21.620204+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":497,"num_read_kb":409,"num_write":177,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"264'322","reported_seq":939,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.544710+0000","last_change":"2024-04-17T05:46:16.544710+0000","last_active":"2024-04-17T05:46:16.544710+0000","last_peered":"2024-04-17T05:46:16.544710+0000","last_clean":"2024-04-17T05:46:16.544710+0000","last_became_active":"2024-04-17T04:37:03.647722+0000","last_became_peered":"2024-04-17T04:37:03.647722+0000","last_unstale":"2024-04-17T05:46:16.544710+0000","last_undegraded":"2024-04-17T05:46:16.544710+0000","last_fullsized":"2024-04-17T05:46:16.544710+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:16.544168+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.544168+0000","objects_scrubbed":4,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:59.343831+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8704691,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":453,"num_read_kb":14105,"num_write":1178,"num_write_kb":25630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.31","version":"264'256","reported_seq":900,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.357473+0000","last_change":"2024-04-17T05:46:41.357550+0000","last_active":"2024-04-17T05:46:41.357473+0000","last_peered":"2024-04-17T05:46:41.357473+0000","last_clean":"2024-04-17T05:46:41.357473+0000","last_became_active":"2024-04-17T04:36:27.382105+0000","last_became_peered":"2024-04-17T04:36:27.382105+0000","last_unstale":"2024-04-17T05:46:41.357473+0000","last_undegraded":"2024-04-17T05:46:41.357473+0000","last_fullsized":"2024-04-17T05:46:41.357473+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'256","last_scrub_stamp":"2024-04-17T05:46:41.357278+0000","last_deep_scrub":"264'256","last_deep_scrub_stamp":"2024-04-17T05:46:41.357278+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.357278+0000","objects_scrubbed":1,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:04:53.727788+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1033,"num_read_kb":817,"num_write":406,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":506,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"264'270","reported_seq":898,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.886876+0000","last_change":"2024-04-17T05:46:36.886994+0000","last_active":"2024-04-17T05:46:36.886876+0000","last_peered":"2024-04-17T05:46:36.886876+0000","last_clean":"2024-04-17T05:46:36.886876+0000","last_became_active":"2024-04-17T04:36:25.384788+0000","last_became_peered":"2024-04-17T04:36:25.384788+0000","last_unstale":"2024-04-17T05:46:36.886876+0000","last_undegraded":"2024-04-17T05:46:36.886876+0000","last_fullsized":"2024-04-17T05:46:36.886876+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'270","last_scrub_stamp":"2024-04-17T05:46:36.886656+0000","last_deep_scrub":"264'270","last_deep_scrub_stamp":"2024-04-17T05:46:36.886656+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.886656+0000","objects_scrubbed":8,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:17:11.179807+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1730173,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":405,"num_read_kb":10551,"num_write":968,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"264'141","reported_seq":733,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.307726+0000","last_change":"2024-04-17T05:46:33.307726+0000","last_active":"2024-04-17T05:46:33.307726+0000","last_peered":"2024-04-17T05:46:33.307726+0000","last_clean":"2024-04-17T05:46:33.307726+0000","last_became_active":"2024-04-17T04:36:27.375609+0000","last_became_peered":"2024-04-17T04:36:27.375609+0000","last_unstale":"2024-04-17T05:46:33.307726+0000","last_undegraded":"2024-04-17T05:46:33.307726+0000","last_fullsized":"2024-04-17T05:46:33.307726+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:33.307469+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:33.307469+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.307469+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:11:08.248068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":548,"num_read_kb":436,"num_write":226,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"264'302","reported_seq":994,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.891762+0000","last_change":"2024-04-17T05:46:37.892036+0000","last_active":"2024-04-17T05:46:37.891762+0000","last_peered":"2024-04-17T05:46:37.891762+0000","last_clean":"2024-04-17T05:46:37.891762+0000","last_became_active":"2024-04-17T04:36:25.384819+0000","last_became_peered":"2024-04-17T04:36:25.384819+0000","last_unstale":"2024-04-17T05:46:37.891762+0000","last_undegraded":"2024-04-17T05:46:37.891762+0000","last_fullsized":"2024-04-17T05:46:37.891762+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'302","last_scrub_stamp":"2024-04-17T05:46:37.891555+0000","last_deep_scrub":"264'302","last_deep_scrub_stamp":"2024-04-17T05:46:37.891555+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.891555+0000","objects_scrubbed":9,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:33:00.393928+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044776,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":456,"num_read_kb":160096,"num_write":1126,"num_write_kb":72677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"264'289","reported_seq":909,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.420737+0000","last_change":"2024-04-17T05:46:30.420737+0000","last_active":"2024-04-17T05:46:30.420737+0000","last_peered":"2024-04-17T05:46:30.420737+0000","last_clean":"2024-04-17T05:46:30.420737+0000","last_became_active":"2024-04-17T04:36:27.383152+0000","last_became_peered":"2024-04-17T04:36:27.383152+0000","last_unstale":"2024-04-17T05:46:30.420737+0000","last_undegraded":"2024-04-17T05:46:30.420737+0000","last_fullsized":"2024-04-17T05:46:30.420737+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'289","last_scrub_stamp":"2024-04-17T05:46:30.420473+0000","last_deep_scrub":"264'289","last_deep_scrub_stamp":"2024-04-17T05:46:30.420473+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.420473+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:34.221983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1098,"num_read_kb":846,"num_write":466,"num_write_kb":227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"264'306","reported_seq":1021,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.545633+0000","last_change":"2024-04-17T05:46:46.545721+0000","last_active":"2024-04-17T05:46:46.545633+0000","last_peered":"2024-04-17T05:46:46.545633+0000","last_clean":"2024-04-17T05:46:46.545633+0000","last_became_active":"2024-04-17T04:36:25.360996+0000","last_became_peered":"2024-04-17T04:36:25.360996+0000","last_unstale":"2024-04-17T05:46:46.545633+0000","last_undegraded":"2024-04-17T05:46:46.545633+0000","last_fullsized":"2024-04-17T05:46:46.545633+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'306","last_scrub_stamp":"2024-04-17T05:46:46.545412+0000","last_deep_scrub":"264'306","last_deep_scrub_stamp":"2024-04-17T05:46:46.545412+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.545412+0000","objects_scrubbed":7,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:21:16.960386+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399248,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":586,"num_read_kb":64730,"num_write":1112,"num_write_kb":35047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"264'4181","reported_seq":4814,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.442968+0000","last_change":"2024-04-17T05:46:29.442968+0000","last_active":"2024-04-17T05:46:29.442968+0000","last_peered":"2024-04-17T05:46:29.442968+0000","last_clean":"2024-04-17T05:46:29.442968+0000","last_became_active":"2024-04-17T04:36:27.379179+0000","last_became_peered":"2024-04-17T04:36:27.379179+0000","last_unstale":"2024-04-17T05:46:29.442968+0000","last_undegraded":"2024-04-17T05:46:29.442968+0000","last_fullsized":"2024-04-17T05:46:29.442968+0000","mapping_epoch":25,"log_start":"264'4100","ondisk_log_start":"264'4100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4181","last_scrub_stamp":"2024-04-17T05:46:29.442703+0000","last_deep_scrub":"264'4181","last_deep_scrub_stamp":"2024-04-17T05:46:29.442703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.442703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":3000,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:07:51.847043+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14765,"num_read_kb":12039,"num_write":6298,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"264'318","reported_seq":1098,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:02.696238+0000","last_change":"2024-04-17T05:47:02.696493+0000","last_active":"2024-04-17T05:47:02.696238+0000","last_peered":"2024-04-17T05:47:02.696238+0000","last_clean":"2024-04-17T05:47:02.696238+0000","last_became_active":"2024-04-17T04:36:25.373990+0000","last_became_peered":"2024-04-17T04:36:25.373990+0000","last_unstale":"2024-04-17T05:47:02.696238+0000","last_undegraded":"2024-04-17T05:47:02.696238+0000","last_fullsized":"2024-04-17T05:47:02.696238+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'318","last_scrub_stamp":"2024-04-17T05:47:02.695933+0000","last_deep_scrub":"264'318","last_deep_scrub_stamp":"2024-04-17T05:47:02.695933+0000","last_clean_scrub_stamp":"2024-04-17T05:47:02.695933+0000","objects_scrubbed":7,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:00.943737+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170354,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":810,"num_read_kb":57781,"num_write":1265,"num_write_kb":55129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"264'174","reported_seq":825,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.625599+0000","last_change":"2024-04-17T05:46:35.625793+0000","last_active":"2024-04-17T05:46:35.625599+0000","last_peered":"2024-04-17T05:46:35.625599+0000","last_clean":"2024-04-17T05:46:35.625599+0000","last_became_active":"2024-04-17T04:36:27.384407+0000","last_became_peered":"2024-04-17T04:36:27.384407+0000","last_unstale":"2024-04-17T05:46:35.625599+0000","last_undegraded":"2024-04-17T05:46:35.625599+0000","last_fullsized":"2024-04-17T05:46:35.625599+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:35.625369+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:35.625369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.625369+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:22:17.124001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":715,"num_read_kb":584,"num_write":252,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"264'346","reported_seq":990,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:00.518075+0000","last_change":"2024-04-17T05:47:00.518281+0000","last_active":"2024-04-17T05:47:00.518075+0000","last_peered":"2024-04-17T05:47:00.518075+0000","last_clean":"2024-04-17T05:47:00.518075+0000","last_became_active":"2024-04-17T04:36:25.354015+0000","last_became_peered":"2024-04-17T04:36:25.354015+0000","last_unstale":"2024-04-17T05:47:00.518075+0000","last_undegraded":"2024-04-17T05:47:00.518075+0000","last_fullsized":"2024-04-17T05:47:00.518075+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'346","last_scrub_stamp":"2024-04-17T05:47:00.517862+0000","last_deep_scrub":"264'346","last_deep_scrub_stamp":"2024-04-17T05:47:00.517862+0000","last_clean_scrub_stamp":"2024-04-17T05:47:00.517862+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:07:17.380395+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244513,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":463,"num_read_kb":836,"num_write":1174,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"264'155","reported_seq":681,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.797345+0000","last_change":"2024-04-17T05:46:13.797345+0000","last_active":"2024-04-17T05:46:13.797345+0000","last_peered":"2024-04-17T05:46:13.797345+0000","last_clean":"2024-04-17T05:46:13.797345+0000","last_became_active":"2024-04-17T04:36:35.417684+0000","last_became_peered":"2024-04-17T04:36:35.417684+0000","last_unstale":"2024-04-17T05:46:13.797345+0000","last_undegraded":"2024-04-17T05:46:13.797345+0000","last_fullsized":"2024-04-17T05:46:13.797345+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:13.796936+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.796936+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:06:42.445551+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":175,"num_write":378,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"264'38","reported_seq":608,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.733949+0000","last_change":"2024-04-17T05:46:43.733949+0000","last_active":"2024-04-17T05:46:43.733949+0000","last_peered":"2024-04-17T05:46:43.733949+0000","last_clean":"2024-04-17T05:46:43.733949+0000","last_became_active":"2024-04-17T04:36:27.384438+0000","last_became_peered":"2024-04-17T04:36:27.384438+0000","last_unstale":"2024-04-17T05:46:43.733949+0000","last_undegraded":"2024-04-17T05:46:43.733949+0000","last_fullsized":"2024-04-17T05:46:43.733949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'38","last_scrub_stamp":"2024-04-17T05:46:43.733678+0000","last_deep_scrub":"264'38","last_deep_scrub_stamp":"2024-04-17T05:46:43.733678+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.733678+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:13:08.252101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":144,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"264'314","reported_seq":948,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.175131+0000","last_change":"2024-04-17T05:46:53.175403+0000","last_active":"2024-04-17T05:46:53.175131+0000","last_peered":"2024-04-17T05:46:53.175131+0000","last_clean":"2024-04-17T05:46:53.175131+0000","last_became_active":"2024-04-17T04:36:25.367775+0000","last_became_peered":"2024-04-17T04:36:25.367775+0000","last_unstale":"2024-04-17T05:46:53.175131+0000","last_undegraded":"2024-04-17T05:46:53.175131+0000","last_fullsized":"2024-04-17T05:46:53.175131+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'314","last_scrub_stamp":"2024-04-17T05:46:53.174917+0000","last_deep_scrub":"264'314","last_deep_scrub_stamp":"2024-04-17T05:46:53.174917+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.174917+0000","objects_scrubbed":11,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:22:45.964725+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7313223,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":429,"num_read_kb":2527,"num_write":1169,"num_write_kb":35276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.338579+0000","last_change":"2024-04-17T05:46:32.338579+0000","last_active":"2024-04-17T05:46:32.338579+0000","last_peered":"2024-04-17T05:46:32.338579+0000","last_clean":"2024-04-17T05:46:32.338579+0000","last_became_active":"2024-04-17T04:36:29.380112+0000","last_became_peered":"2024-04-17T04:36:29.380112+0000","last_unstale":"2024-04-17T05:46:32.338579+0000","last_undegraded":"2024-04-17T05:46:32.338579+0000","last_fullsized":"2024-04-17T05:46:32.338579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.338322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.338322+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.338322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:55:19.679591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"264'373","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.677388+0000","last_change":"2024-04-17T05:46:36.677388+0000","last_active":"2024-04-17T05:46:36.677388+0000","last_peered":"2024-04-17T05:46:36.677388+0000","last_clean":"2024-04-17T05:46:36.677388+0000","last_became_active":"2024-04-17T04:36:31.405604+0000","last_became_peered":"2024-04-17T04:36:31.405604+0000","last_unstale":"2024-04-17T05:46:36.677388+0000","last_undegraded":"2024-04-17T05:46:36.677388+0000","last_fullsized":"2024-04-17T05:46:36.677388+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:36.677124+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:36.677124+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.677124+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:16:01.400465+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45299,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":596,"num_read_kb":1277,"num_write":472,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.304719+0000","last_change":"2024-04-17T05:46:29.304719+0000","last_active":"2024-04-17T05:46:29.304719+0000","last_peered":"2024-04-17T05:46:29.304719+0000","last_clean":"2024-04-17T05:46:29.304719+0000","last_became_active":"2024-04-17T04:36:33.424132+0000","last_became_peered":"2024-04-17T04:36:33.424132+0000","last_unstale":"2024-04-17T05:46:29.304719+0000","last_undegraded":"2024-04-17T05:46:29.304719+0000","last_fullsized":"2024-04-17T05:46:29.304719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.304289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.304289+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.304289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:21:58.667090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"264'172","reported_seq":735,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.112010+0000","last_change":"2024-04-17T05:46:15.112010+0000","last_active":"2024-04-17T05:46:15.112010+0000","last_peered":"2024-04-17T05:46:15.112010+0000","last_clean":"2024-04-17T05:46:15.112010+0000","last_became_active":"2024-04-17T04:36:35.411626+0000","last_became_peered":"2024-04-17T04:36:35.411626+0000","last_unstale":"2024-04-17T05:46:15.112010+0000","last_undegraded":"2024-04-17T05:46:15.112010+0000","last_fullsized":"2024-04-17T05:46:15.112010+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'172","last_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_deep_scrub":"264'172","last_deep_scrub_stamp":"2024-04-17T05:46:15.111703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.111703+0000","objects_scrubbed":3,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:36:19.615360+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":201,"num_write":333,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"264'97","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.266652+0000","last_change":"2024-04-17T05:46:26.266652+0000","last_active":"2024-04-17T05:46:26.266652+0000","last_peered":"2024-04-17T05:46:26.266652+0000","last_clean":"2024-04-17T05:46:26.266652+0000","last_became_active":"2024-04-17T04:36:27.389110+0000","last_became_peered":"2024-04-17T04:36:27.389110+0000","last_unstale":"2024-04-17T05:46:26.266652+0000","last_undegraded":"2024-04-17T05:46:26.266652+0000","last_fullsized":"2024-04-17T05:46:26.266652+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'97","last_scrub_stamp":"2024-04-17T05:46:26.266405+0000","last_deep_scrub":"264'97","last_deep_scrub_stamp":"2024-04-17T05:46:26.266405+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.266405+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:34:32.444534+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":393,"num_read_kb":321,"num_write":149,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"264'339","reported_seq":1120,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.618752+0000","last_change":"2024-04-17T05:46:53.618981+0000","last_active":"2024-04-17T05:46:53.618752+0000","last_peered":"2024-04-17T05:46:53.618752+0000","last_clean":"2024-04-17T05:46:53.618752+0000","last_became_active":"2024-04-17T04:36:25.327615+0000","last_became_peered":"2024-04-17T04:36:25.327615+0000","last_unstale":"2024-04-17T05:46:53.618752+0000","last_undegraded":"2024-04-17T05:46:53.618752+0000","last_fullsized":"2024-04-17T05:46:53.618752+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'339","last_scrub_stamp":"2024-04-17T05:46:53.618500+0000","last_deep_scrub":"264'339","last_deep_scrub_stamp":"2024-04-17T05:46:53.618500+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.618500+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:34:27.991022+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1328,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":754,"num_read_kb":61435,"num_write":1363,"num_write_kb":38462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.624575+0000","last_change":"2024-04-17T05:46:54.624676+0000","last_active":"2024-04-17T05:46:54.624575+0000","last_peered":"2024-04-17T05:46:54.624575+0000","last_clean":"2024-04-17T05:46:54.624575+0000","last_became_active":"2024-04-17T04:36:29.373452+0000","last_became_peered":"2024-04-17T04:36:29.373452+0000","last_unstale":"2024-04-17T05:46:54.624575+0000","last_undegraded":"2024-04-17T05:46:54.624575+0000","last_fullsized":"2024-04-17T05:46:54.624575+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:54.624351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:54.624351+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.624351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:29:59.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"264'103","reported_seq":656,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.736490+0000","last_change":"2024-04-17T05:46:22.736490+0000","last_active":"2024-04-17T05:46:22.736490+0000","last_peered":"2024-04-17T05:46:22.736490+0000","last_clean":"2024-04-17T05:46:22.736490+0000","last_became_active":"2024-04-17T04:36:31.394260+0000","last_became_peered":"2024-04-17T04:36:31.394260+0000","last_unstale":"2024-04-17T05:46:22.736490+0000","last_undegraded":"2024-04-17T05:46:22.736490+0000","last_fullsized":"2024-04-17T05:46:22.736490+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'103","last_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_deep_scrub":"264'103","last_deep_scrub_stamp":"2024-04-17T05:46:22.736227+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.736227+0000","objects_scrubbed":4,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:38:14.605978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":112,"num_write":76,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.786406+0000","last_change":"2024-04-17T05:46:15.786406+0000","last_active":"2024-04-17T05:46:15.786406+0000","last_peered":"2024-04-17T05:46:15.786406+0000","last_clean":"2024-04-17T05:46:15.786406+0000","last_became_active":"2024-04-17T04:36:33.413173+0000","last_became_peered":"2024-04-17T04:36:33.413173+0000","last_unstale":"2024-04-17T05:46:15.786406+0000","last_undegraded":"2024-04-17T05:46:15.786406+0000","last_fullsized":"2024-04-17T05:46:15.786406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:15.786152+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.786152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:05:10.717154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"264'129","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.649727+0000","last_change":"2024-04-17T05:46:22.649727+0000","last_active":"2024-04-17T05:46:22.649727+0000","last_peered":"2024-04-17T05:46:22.649727+0000","last_clean":"2024-04-17T05:46:22.649727+0000","last_became_active":"2024-04-17T04:36:35.426899+0000","last_became_peered":"2024-04-17T04:36:35.426899+0000","last_unstale":"2024-04-17T05:46:22.649727+0000","last_undegraded":"2024-04-17T05:46:22.649727+0000","last_fullsized":"2024-04-17T05:46:22.649727+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'129","last_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_deep_scrub":"264'129","last_deep_scrub_stamp":"2024-04-17T05:46:22.649467+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.649467+0000","objects_scrubbed":6,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:10:06.914685+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1098,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":124,"num_write":305,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"264'85","reported_seq":669,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.727502+0000","last_change":"2024-04-17T05:46:40.727563+0000","last_active":"2024-04-17T05:46:40.727502+0000","last_peered":"2024-04-17T05:46:40.727502+0000","last_clean":"2024-04-17T05:46:40.727502+0000","last_became_active":"2024-04-17T04:36:27.375561+0000","last_became_peered":"2024-04-17T04:36:27.375561+0000","last_unstale":"2024-04-17T05:46:40.727502+0000","last_undegraded":"2024-04-17T05:46:40.727502+0000","last_fullsized":"2024-04-17T05:46:40.727502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'85","last_scrub_stamp":"2024-04-17T05:46:40.727294+0000","last_deep_scrub":"264'85","last_deep_scrub_stamp":"2024-04-17T05:46:40.727294+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.727294+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:27.135914+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":283,"num_write":118,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"264'333","reported_seq":1122,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.183587+0000","last_change":"2024-04-17T05:46:47.183732+0000","last_active":"2024-04-17T05:46:47.183587+0000","last_peered":"2024-04-17T05:46:47.183587+0000","last_clean":"2024-04-17T05:46:47.183587+0000","last_became_active":"2024-04-17T04:36:25.380836+0000","last_became_peered":"2024-04-17T04:36:25.380836+0000","last_unstale":"2024-04-17T05:46:47.183587+0000","last_undegraded":"2024-04-17T05:46:47.183587+0000","last_fullsized":"2024-04-17T05:46:47.183587+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'333","last_scrub_stamp":"2024-04-17T05:46:47.183363+0000","last_deep_scrub":"264'333","last_deep_scrub_stamp":"2024-04-17T05:46:47.183363+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.183363+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:44:14.756516+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":672,"num_read_kb":75397,"num_write":1317,"num_write_kb":50737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.776716+0000","last_change":"2024-04-17T05:46:48.776790+0000","last_active":"2024-04-17T05:46:48.776716+0000","last_peered":"2024-04-17T05:46:48.776716+0000","last_clean":"2024-04-17T05:46:48.776716+0000","last_became_active":"2024-04-17T04:36:29.359060+0000","last_became_peered":"2024-04-17T04:36:29.359060+0000","last_unstale":"2024-04-17T05:46:48.776716+0000","last_undegraded":"2024-04-17T05:46:48.776716+0000","last_fullsized":"2024-04-17T05:46:48.776716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:48.776517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:48.776517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.776517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:34:15.117730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"264'144","reported_seq":721,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.644625+0000","last_change":"2024-04-17T05:46:13.644625+0000","last_active":"2024-04-17T05:46:13.644625+0000","last_peered":"2024-04-17T05:46:13.644625+0000","last_clean":"2024-04-17T05:46:13.644625+0000","last_became_active":"2024-04-17T04:36:31.388084+0000","last_became_peered":"2024-04-17T04:36:31.388084+0000","last_unstale":"2024-04-17T05:46:13.644625+0000","last_undegraded":"2024-04-17T05:46:13.644625+0000","last_fullsized":"2024-04-17T05:46:13.644625+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'144","last_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_deep_scrub":"264'144","last_deep_scrub_stamp":"2024-04-17T05:46:13.644329+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.644329+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:14.646427+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":110,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"264'519","reported_seq":2426,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.368300+0000","last_change":"2024-04-17T05:46:31.368300+0000","last_active":"2024-04-17T05:46:31.368300+0000","last_peered":"2024-04-17T05:46:31.368300+0000","last_clean":"2024-04-17T05:46:31.368300+0000","last_became_active":"2024-04-17T04:36:33.419987+0000","last_became_peered":"2024-04-17T04:36:33.419987+0000","last_unstale":"2024-04-17T05:46:31.368300+0000","last_undegraded":"2024-04-17T05:46:31.368300+0000","last_fullsized":"2024-04-17T05:46:31.368300+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'519","last_scrub_stamp":"2024-04-17T05:46:31.367883+0000","last_deep_scrub":"264'519","last_deep_scrub_stamp":"2024-04-17T05:46:31.367883+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.367883+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:42.078252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"264'199","reported_seq":1022,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.828863+0000","last_change":"2024-04-17T05:46:14.828863+0000","last_active":"2024-04-17T05:46:14.828863+0000","last_peered":"2024-04-17T05:46:14.828863+0000","last_clean":"2024-04-17T05:46:14.828863+0000","last_became_active":"2024-04-17T04:36:35.402784+0000","last_became_peered":"2024-04-17T04:36:35.402784+0000","last_unstale":"2024-04-17T05:46:14.828863+0000","last_undegraded":"2024-04-17T05:46:14.828863+0000","last_fullsized":"2024-04-17T05:46:14.828863+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'199","last_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_deep_scrub":"264'199","last_deep_scrub_stamp":"2024-04-17T05:46:14.828583+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.828583+0000","objects_scrubbed":4,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:49:40.862007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":820,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":560,"num_read_kb":557,"num_write":428,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"264'234","reported_seq":935,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.741170+0000","last_change":"2024-04-17T05:46:42.741232+0000","last_active":"2024-04-17T05:46:42.741170+0000","last_peered":"2024-04-17T05:46:42.741170+0000","last_clean":"2024-04-17T05:46:42.741170+0000","last_became_active":"2024-04-17T04:36:27.395239+0000","last_became_peered":"2024-04-17T04:36:27.395239+0000","last_unstale":"2024-04-17T05:46:42.741170+0000","last_undegraded":"2024-04-17T05:46:42.741170+0000","last_fullsized":"2024-04-17T05:46:42.741170+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'234","last_scrub_stamp":"2024-04-17T05:46:42.740966+0000","last_deep_scrub":"264'234","last_deep_scrub_stamp":"2024-04-17T05:46:42.740966+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.740966+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:49:41.989989+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1045,"num_read_kb":872,"num_write":393,"num_write_kb":221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"264'293","reported_seq":949,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.858878+0000","last_change":"2024-04-17T05:46:35.858991+0000","last_active":"2024-04-17T05:46:35.858878+0000","last_peered":"2024-04-17T05:46:35.858878+0000","last_clean":"2024-04-17T05:46:35.858878+0000","last_became_active":"2024-04-17T04:36:25.384095+0000","last_became_peered":"2024-04-17T04:36:25.384095+0000","last_unstale":"2024-04-17T05:46:35.858878+0000","last_undegraded":"2024-04-17T05:46:35.858878+0000","last_fullsized":"2024-04-17T05:46:35.858878+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'293","last_scrub_stamp":"2024-04-17T05:46:35.858673+0000","last_deep_scrub":"264'293","last_deep_scrub_stamp":"2024-04-17T05:46:35.858673+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.858673+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:47.864378+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3722445,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":48921,"num_write":1143,"num_write_kb":21529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.344025+0000","last_change":"2024-04-17T05:46:36.344082+0000","last_active":"2024-04-17T05:46:36.344025+0000","last_peered":"2024-04-17T05:46:36.344025+0000","last_clean":"2024-04-17T05:46:36.344025+0000","last_became_active":"2024-04-17T04:36:29.385870+0000","last_became_peered":"2024-04-17T04:36:29.385870+0000","last_unstale":"2024-04-17T05:46:36.344025+0000","last_undegraded":"2024-04-17T05:46:36.344025+0000","last_fullsized":"2024-04-17T05:46:36.344025+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:36.343820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:36.343820+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.343820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:58:48.938245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"264'284","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.144810+0000","last_change":"2024-04-17T05:46:48.144886+0000","last_active":"2024-04-17T05:46:48.144810+0000","last_peered":"2024-04-17T05:46:48.144810+0000","last_clean":"2024-04-17T05:46:48.144810+0000","last_became_active":"2024-04-17T04:36:31.421949+0000","last_became_peered":"2024-04-17T04:36:31.421949+0000","last_unstale":"2024-04-17T05:46:48.144810+0000","last_undegraded":"2024-04-17T05:46:48.144810+0000","last_fullsized":"2024-04-17T05:46:48.144810+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'284","last_scrub_stamp":"2024-04-17T05:46:48.144597+0000","last_deep_scrub":"264'284","last_deep_scrub_stamp":"2024-04-17T05:46:48.144597+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.144597+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:30:23.030105+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51171,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1108,"num_write":376,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"264'534","reported_seq":2312,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.727792+0000","last_change":"2024-04-17T05:46:25.727792+0000","last_active":"2024-04-17T05:46:25.727792+0000","last_peered":"2024-04-17T05:46:25.727792+0000","last_clean":"2024-04-17T05:46:25.727792+0000","last_became_active":"2024-04-17T04:36:33.421833+0000","last_became_peered":"2024-04-17T04:36:33.421833+0000","last_unstale":"2024-04-17T05:46:25.727792+0000","last_undegraded":"2024-04-17T05:46:25.727792+0000","last_fullsized":"2024-04-17T05:46:25.727792+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'534","last_scrub_stamp":"2024-04-17T05:46:25.727540+0000","last_deep_scrub":"264'534","last_deep_scrub_stamp":"2024-04-17T05:46:25.727540+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.727540+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:47.622101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"264'122","reported_seq":730,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.284199+0000","last_change":"2024-04-17T05:46:30.284267+0000","last_active":"2024-04-17T05:46:30.284199+0000","last_peered":"2024-04-17T05:46:30.284199+0000","last_clean":"2024-04-17T05:46:30.284199+0000","last_became_active":"2024-04-17T04:36:27.397045+0000","last_became_peered":"2024-04-17T04:36:27.397045+0000","last_unstale":"2024-04-17T05:46:30.284199+0000","last_undegraded":"2024-04-17T05:46:30.284199+0000","last_fullsized":"2024-04-17T05:46:30.284199+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'122","last_scrub_stamp":"2024-04-17T05:46:30.283976+0000","last_deep_scrub":"264'122","last_deep_scrub_stamp":"2024-04-17T05:46:30.283976+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.283976+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:33:13.769030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":419,"num_write":178,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"264'281","reported_seq":924,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:54.203978+0000","last_change":"2024-04-17T05:46:54.204083+0000","last_active":"2024-04-17T05:46:54.203978+0000","last_peered":"2024-04-17T05:46:54.203978+0000","last_clean":"2024-04-17T05:46:54.203978+0000","last_became_active":"2024-04-17T04:36:25.370036+0000","last_became_peered":"2024-04-17T04:36:25.370036+0000","last_unstale":"2024-04-17T05:46:54.203978+0000","last_undegraded":"2024-04-17T05:46:54.203978+0000","last_fullsized":"2024-04-17T05:46:54.203978+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:54.203767+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:54.203767+0000","last_clean_scrub_stamp":"2024-04-17T05:46:54.203767+0000","objects_scrubbed":6,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:03.849010+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6974633,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":437,"num_read_kb":24084,"num_write":1121,"num_write_kb":47040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.373791+0000","last_change":"2024-04-17T05:46:38.373891+0000","last_active":"2024-04-17T05:46:38.373791+0000","last_peered":"2024-04-17T05:46:38.373791+0000","last_clean":"2024-04-17T05:46:38.373791+0000","last_became_active":"2024-04-17T04:36:29.388683+0000","last_became_peered":"2024-04-17T04:36:29.388683+0000","last_unstale":"2024-04-17T05:46:38.373791+0000","last_undegraded":"2024-04-17T05:46:38.373791+0000","last_fullsized":"2024-04-17T05:46:38.373791+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.373516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.373516+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.373516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:46:40.616887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"264'391","reported_seq":1038,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.206831+0000","last_change":"2024-04-17T05:46:22.206940+0000","last_active":"2024-04-17T05:46:22.206831+0000","last_peered":"2024-04-17T05:46:22.206831+0000","last_clean":"2024-04-17T05:46:22.206831+0000","last_became_active":"2024-04-17T04:36:31.422163+0000","last_became_peered":"2024-04-17T04:36:31.422163+0000","last_unstale":"2024-04-17T05:46:22.206831+0000","last_undegraded":"2024-04-17T05:46:22.206831+0000","last_fullsized":"2024-04-17T05:46:22.206831+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'391","last_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_deep_scrub":"264'391","last_deep_scrub_stamp":"2024-04-17T05:46:22.206623+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.206623+0000","objects_scrubbed":10,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:28:39.352908+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44506,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":619,"num_read_kb":1253,"num_write":476,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.860898+0000","last_change":"2024-04-17T05:46:30.860950+0000","last_active":"2024-04-17T05:46:30.860898+0000","last_peered":"2024-04-17T05:46:30.860898+0000","last_clean":"2024-04-17T05:46:30.860898+0000","last_became_active":"2024-04-17T04:36:33.407421+0000","last_became_peered":"2024-04-17T04:36:33.407421+0000","last_unstale":"2024-04-17T05:46:30.860898+0000","last_undegraded":"2024-04-17T05:46:30.860898+0000","last_fullsized":"2024-04-17T05:46:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.860694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.860694+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.860694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:43:48.164981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"264'564","reported_seq":1398,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.160792+0000","last_change":"2024-04-17T05:46:18.160792+0000","last_active":"2024-04-17T05:46:18.160792+0000","last_peered":"2024-04-17T05:46:18.160792+0000","last_clean":"2024-04-17T05:46:18.160792+0000","last_became_active":"2024-04-17T04:36:35.408359+0000","last_became_peered":"2024-04-17T04:36:35.408359+0000","last_unstale":"2024-04-17T05:46:18.160792+0000","last_undegraded":"2024-04-17T05:46:18.160792+0000","last_fullsized":"2024-04-17T05:46:18.160792+0000","mapping_epoch":33,"log_start":"264'500","ondisk_log_start":"264'500","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'564","last_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_deep_scrub":"264'564","last_deep_scrub_stamp":"2024-04-17T05:46:18.160537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.160537+0000","objects_scrubbed":4,"log_size":64,"log_dups_size":500,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:23.558644+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1323,"num_read_kb":1311,"num_write":1181,"num_write_kb":399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"264'259","reported_seq":874,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.594125+0000","last_change":"2024-04-17T05:46:41.594194+0000","last_active":"2024-04-17T05:46:41.594125+0000","last_peered":"2024-04-17T05:46:41.594125+0000","last_clean":"2024-04-17T05:46:41.594125+0000","last_became_active":"2024-04-17T04:36:27.378982+0000","last_became_peered":"2024-04-17T04:36:27.378982+0000","last_unstale":"2024-04-17T05:46:41.594125+0000","last_undegraded":"2024-04-17T05:46:41.594125+0000","last_fullsized":"2024-04-17T05:46:41.594125+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'259","last_scrub_stamp":"2024-04-17T05:46:41.593897+0000","last_deep_scrub":"264'259","last_deep_scrub_stamp":"2024-04-17T05:46:41.593897+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.593897+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:15:27.355644+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":995,"num_read_kb":771,"num_write":419,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"264'395","reported_seq":1065,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.215535+0000","last_change":"2024-04-17T05:46:55.215740+0000","last_active":"2024-04-17T05:46:55.215535+0000","last_peered":"2024-04-17T05:46:55.215535+0000","last_clean":"2024-04-17T05:46:55.215535+0000","last_became_active":"2024-04-17T04:36:25.370399+0000","last_became_peered":"2024-04-17T04:36:25.370399+0000","last_unstale":"2024-04-17T05:46:55.215535+0000","last_undegraded":"2024-04-17T05:46:55.215535+0000","last_fullsized":"2024-04-17T05:46:55.215535+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'395","last_scrub_stamp":"2024-04-17T05:46:55.215336+0000","last_deep_scrub":"264'395","last_deep_scrub_stamp":"2024-04-17T05:46:55.215336+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.215336+0000","objects_scrubbed":8,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:44:01.768966+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396022,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":571,"num_read_kb":14718,"num_write":1316,"num_write_kb":30606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.469546+0000","last_change":"2024-04-17T05:46:23.469644+0000","last_active":"2024-04-17T05:46:23.469546+0000","last_peered":"2024-04-17T05:46:23.469546+0000","last_clean":"2024-04-17T05:46:23.469546+0000","last_became_active":"2024-04-17T04:36:29.371466+0000","last_became_peered":"2024-04-17T04:36:29.371466+0000","last_unstale":"2024-04-17T05:46:23.469546+0000","last_undegraded":"2024-04-17T05:46:23.469546+0000","last_fullsized":"2024-04-17T05:46:23.469546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.469283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.469283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:59.151653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"264'274","reported_seq":873,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:04.736724+0000","last_change":"2024-04-17T05:47:04.736825+0000","last_active":"2024-04-17T05:47:04.736724+0000","last_peered":"2024-04-17T05:47:04.736724+0000","last_clean":"2024-04-17T05:47:04.736724+0000","last_became_active":"2024-04-17T04:36:31.391341+0000","last_became_peered":"2024-04-17T04:36:31.391341+0000","last_unstale":"2024-04-17T05:47:04.736724+0000","last_undegraded":"2024-04-17T05:47:04.736724+0000","last_fullsized":"2024-04-17T05:47:04.736724+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'274","last_scrub_stamp":"2024-04-17T05:47:04.736535+0000","last_deep_scrub":"264'274","last_deep_scrub_stamp":"2024-04-17T05:47:04.736535+0000","last_clean_scrub_stamp":"2024-04-17T05:47:04.736535+0000","objects_scrubbed":5,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:24.653815+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42027,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":427,"num_read_kb":943,"num_write":336,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.833829+0000","last_change":"2024-04-17T05:46:29.833879+0000","last_active":"2024-04-17T05:46:29.833829+0000","last_peered":"2024-04-17T05:46:29.833829+0000","last_clean":"2024-04-17T05:46:29.833829+0000","last_became_active":"2024-04-17T04:36:33.414308+0000","last_became_peered":"2024-04-17T04:36:33.414308+0000","last_unstale":"2024-04-17T05:46:29.833829+0000","last_undegraded":"2024-04-17T05:46:29.833829+0000","last_fullsized":"2024-04-17T05:46:29.833829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.833621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.833621+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.833621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:23.380092+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"264'2375","reported_seq":4186,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.963343+0000","last_change":"2024-04-17T05:46:27.963419+0000","last_active":"2024-04-17T05:46:27.963343+0000","last_peered":"2024-04-17T05:46:27.963343+0000","last_clean":"2024-04-17T05:46:27.963343+0000","last_became_active":"2024-04-17T04:36:35.417490+0000","last_became_peered":"2024-04-17T04:36:35.417490+0000","last_unstale":"2024-04-17T05:46:27.963343+0000","last_undegraded":"2024-04-17T05:46:27.963343+0000","last_fullsized":"2024-04-17T05:46:27.963343+0000","mapping_epoch":33,"log_start":"264'2300","ondisk_log_start":"264'2300","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2375","last_scrub_stamp":"2024-04-17T05:46:27.963136+0000","last_deep_scrub":"264'2375","last_deep_scrub_stamp":"2024-04-17T05:46:27.963136+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.963136+0000","objects_scrubbed":4,"log_size":75,"log_dups_size":2300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:13:51.672615+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1043,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5902,"num_read_kb":5905,"num_write":4808,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"264'154","reported_seq":708,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.910107+0000","last_change":"2024-04-17T05:46:30.910177+0000","last_active":"2024-04-17T05:46:30.910107+0000","last_peered":"2024-04-17T05:46:30.910107+0000","last_clean":"2024-04-17T05:46:30.910107+0000","last_became_active":"2024-04-17T04:36:27.375655+0000","last_became_peered":"2024-04-17T04:36:27.375655+0000","last_unstale":"2024-04-17T05:46:30.910107+0000","last_undegraded":"2024-04-17T05:46:30.910107+0000","last_fullsized":"2024-04-17T05:46:30.910107+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'154","last_scrub_stamp":"2024-04-17T05:46:30.909913+0000","last_deep_scrub":"264'154","last_deep_scrub_stamp":"2024-04-17T05:46:30.909913+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.909913+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:48:00.648226+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":585,"num_read_kb":491,"num_write":243,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"264'356","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.297884+0000","last_change":"2024-04-17T05:46:43.298040+0000","last_active":"2024-04-17T05:46:43.297884+0000","last_peered":"2024-04-17T05:46:43.297884+0000","last_clean":"2024-04-17T05:46:43.297884+0000","last_became_active":"2024-04-17T04:36:25.385794+0000","last_became_peered":"2024-04-17T04:36:25.385794+0000","last_unstale":"2024-04-17T05:46:43.297884+0000","last_undegraded":"2024-04-17T05:46:43.297884+0000","last_fullsized":"2024-04-17T05:46:43.297884+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:43.297672+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:43.297672+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.297672+0000","objects_scrubbed":5,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:52:50.015406+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524369,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":604,"num_read_kb":65373,"num_write":1319,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.457184+0000","last_change":"2024-04-17T05:46:21.457242+0000","last_active":"2024-04-17T05:46:21.457184+0000","last_peered":"2024-04-17T05:46:21.457184+0000","last_clean":"2024-04-17T05:46:21.457184+0000","last_became_active":"2024-04-17T04:36:29.371471+0000","last_became_peered":"2024-04-17T04:36:29.371471+0000","last_unstale":"2024-04-17T05:46:21.457184+0000","last_undegraded":"2024-04-17T05:46:21.457184+0000","last_fullsized":"2024-04-17T05:46:21.457184+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.456977+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.456977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:16:03.328209+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"264'344","reported_seq":969,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.334603+0000","last_change":"2024-04-17T05:46:42.334679+0000","last_active":"2024-04-17T05:46:42.334603+0000","last_peered":"2024-04-17T05:46:42.334603+0000","last_clean":"2024-04-17T05:46:42.334603+0000","last_became_active":"2024-04-17T04:36:31.379779+0000","last_became_peered":"2024-04-17T04:36:31.379779+0000","last_unstale":"2024-04-17T05:46:42.334603+0000","last_undegraded":"2024-04-17T05:46:42.334603+0000","last_fullsized":"2024-04-17T05:46:42.334603+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'344","last_scrub_stamp":"2024-04-17T05:46:42.334476+0000","last_deep_scrub":"264'344","last_deep_scrub_stamp":"2024-04-17T05:46:42.334476+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.334476+0000","objects_scrubbed":9,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:22.249150+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48038,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":543,"num_read_kb":1167,"num_write":428,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"264'539","reported_seq":4624,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.621904+0000","last_change":"2024-04-17T05:46:33.621973+0000","last_active":"2024-04-17T05:46:33.621904+0000","last_peered":"2024-04-17T05:46:33.621904+0000","last_clean":"2024-04-17T05:46:33.621904+0000","last_became_active":"2024-04-17T04:36:33.410779+0000","last_became_peered":"2024-04-17T04:36:33.410779+0000","last_unstale":"2024-04-17T05:46:33.621904+0000","last_undegraded":"2024-04-17T05:46:33.621904+0000","last_fullsized":"2024-04-17T05:46:33.621904+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:33.621706+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:33.621706+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.621706+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:58:44.561807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"264'119","reported_seq":630,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.594579+0000","last_change":"2024-04-17T05:46:21.594642+0000","last_active":"2024-04-17T05:46:21.594579+0000","last_peered":"2024-04-17T05:46:21.594579+0000","last_clean":"2024-04-17T05:46:21.594579+0000","last_became_active":"2024-04-17T04:36:35.426795+0000","last_became_peered":"2024-04-17T04:36:35.426795+0000","last_unstale":"2024-04-17T05:46:21.594579+0000","last_undegraded":"2024-04-17T05:46:21.594579+0000","last_fullsized":"2024-04-17T05:46:21.594579+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:21.594379+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.594379+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:06.802802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":104,"num_write":262,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"264'96","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.599050+0000","last_change":"2024-04-17T05:46:46.599114+0000","last_active":"2024-04-17T05:46:46.599050+0000","last_peered":"2024-04-17T05:46:46.599050+0000","last_clean":"2024-04-17T05:46:46.599050+0000","last_became_active":"2024-04-17T04:36:27.367750+0000","last_became_peered":"2024-04-17T04:36:27.367750+0000","last_unstale":"2024-04-17T05:46:46.599050+0000","last_undegraded":"2024-04-17T05:46:46.599050+0000","last_fullsized":"2024-04-17T05:46:46.599050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'96","last_scrub_stamp":"2024-04-17T05:46:46.598859+0000","last_deep_scrub":"264'96","last_deep_scrub_stamp":"2024-04-17T05:46:46.598859+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.598859+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:09:35.086138+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":383,"num_read_kb":317,"num_write":135,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"264'339","reported_seq":1048,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.599664+0000","last_change":"2024-04-17T05:46:14.599664+0000","last_active":"2024-04-17T05:46:14.599664+0000","last_peered":"2024-04-17T05:46:14.599664+0000","last_clean":"2024-04-17T05:46:14.599664+0000","last_became_active":"2024-04-17T04:37:03.647710+0000","last_became_peered":"2024-04-17T04:37:03.647710+0000","last_unstale":"2024-04-17T05:46:14.599664+0000","last_undegraded":"2024-04-17T05:46:14.599664+0000","last_fullsized":"2024-04-17T05:46:14.599664+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'339","last_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_deep_scrub":"264'339","last_deep_scrub_stamp":"2024-04-17T05:46:14.599211+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.599211+0000","objects_scrubbed":6,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:59.675254+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":662,"num_read_kb":24375,"num_write":1169,"num_write_kb":21806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6,1],"acting":[2,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":520,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.517281+0000","last_change":"2024-04-17T05:46:17.517338+0000","last_active":"2024-04-17T05:46:17.517281+0000","last_peered":"2024-04-17T05:46:17.517281+0000","last_clean":"2024-04-17T05:46:17.517281+0000","last_became_active":"2024-04-17T04:36:29.375301+0000","last_became_peered":"2024-04-17T04:36:29.375301+0000","last_unstale":"2024-04-17T05:46:17.517281+0000","last_undegraded":"2024-04-17T05:46:17.517281+0000","last_fullsized":"2024-04-17T05:46:17.517281+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.517077+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.517077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:40:41.667573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"264'479","reported_seq":1145,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:01.532888+0000","last_change":"2024-04-17T05:47:01.532969+0000","last_active":"2024-04-17T05:47:01.532888+0000","last_peered":"2024-04-17T05:47:01.532888+0000","last_clean":"2024-04-17T05:47:01.532888+0000","last_became_active":"2024-04-17T04:36:31.405631+0000","last_became_peered":"2024-04-17T04:36:31.405631+0000","last_unstale":"2024-04-17T05:47:01.532888+0000","last_undegraded":"2024-04-17T05:47:01.532888+0000","last_fullsized":"2024-04-17T05:47:01.532888+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'479","last_scrub_stamp":"2024-04-17T05:47:01.532657+0000","last_deep_scrub":"264'479","last_deep_scrub_stamp":"2024-04-17T05:47:01.532657+0000","last_clean_scrub_stamp":"2024-04-17T05:47:01.532657+0000","objects_scrubbed":7,"log_size":79,"log_dups_size":400,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:21.356823+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87769,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":831,"num_read_kb":1997,"num_write":666,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:08.734179+0000","last_change":"2024-04-17T05:46:08.734235+0000","last_active":"2024-04-17T05:46:08.734179+0000","last_peered":"2024-04-17T05:46:08.734179+0000","last_clean":"2024-04-17T05:46:08.734179+0000","last_became_active":"2024-04-17T04:36:33.419707+0000","last_became_peered":"2024-04-17T04:36:33.419707+0000","last_unstale":"2024-04-17T05:46:08.734179+0000","last_undegraded":"2024-04-17T05:46:08.734179+0000","last_fullsized":"2024-04-17T05:46:08.734179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:08.733969+0000","last_clean_scrub_stamp":"2024-04-17T05:46:08.733969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:20.733334+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"264'150","reported_seq":700,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.732921+0000","last_change":"2024-04-17T05:46:14.733009+0000","last_active":"2024-04-17T05:46:14.732921+0000","last_peered":"2024-04-17T05:46:14.732921+0000","last_clean":"2024-04-17T05:46:14.732921+0000","last_became_active":"2024-04-17T04:36:35.416537+0000","last_became_peered":"2024-04-17T04:36:35.416537+0000","last_unstale":"2024-04-17T05:46:14.732921+0000","last_undegraded":"2024-04-17T05:46:14.732921+0000","last_fullsized":"2024-04-17T05:46:14.732921+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'150","last_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_deep_scrub":"264'150","last_deep_scrub_stamp":"2024-04-17T05:46:14.732679+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.732679+0000","objects_scrubbed":2,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:23:10.487087+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":918,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":222,"num_write":367,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"264'157","reported_seq":780,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.706858+0000","last_change":"2024-04-17T05:46:27.706921+0000","last_active":"2024-04-17T05:46:27.706858+0000","last_peered":"2024-04-17T05:46:27.706858+0000","last_clean":"2024-04-17T05:46:27.706858+0000","last_became_active":"2024-04-17T04:36:27.393175+0000","last_became_peered":"2024-04-17T04:36:27.393175+0000","last_unstale":"2024-04-17T05:46:27.706858+0000","last_undegraded":"2024-04-17T05:46:27.706858+0000","last_fullsized":"2024-04-17T05:46:27.706858+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'157","last_scrub_stamp":"2024-04-17T05:46:27.706654+0000","last_deep_scrub":"264'157","last_deep_scrub_stamp":"2024-04-17T05:46:27.706654+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.706654+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:06.380118+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":601,"num_read_kb":485,"num_write":255,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"264'350","reported_seq":1028,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.610408+0000","last_change":"2024-04-17T05:46:55.610535+0000","last_active":"2024-04-17T05:46:55.610408+0000","last_peered":"2024-04-17T05:46:55.610408+0000","last_clean":"2024-04-17T05:46:55.610408+0000","last_became_active":"2024-04-17T04:36:25.379198+0000","last_became_peered":"2024-04-17T04:36:25.379198+0000","last_unstale":"2024-04-17T05:46:55.610408+0000","last_undegraded":"2024-04-17T05:46:55.610408+0000","last_fullsized":"2024-04-17T05:46:55.610408+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'350","last_scrub_stamp":"2024-04-17T05:46:55.610204+0000","last_deep_scrub":"264'350","last_deep_scrub_stamp":"2024-04-17T05:46:55.610204+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.610204+0000","objects_scrubbed":9,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:29.485147+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1985843,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":564,"num_read_kb":9569,"num_write":1227,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.531942+0000","last_change":"2024-04-17T05:46:19.531995+0000","last_active":"2024-04-17T05:46:19.531942+0000","last_peered":"2024-04-17T05:46:19.531942+0000","last_clean":"2024-04-17T05:46:19.531942+0000","last_became_active":"2024-04-17T04:36:29.373388+0000","last_became_peered":"2024-04-17T04:36:29.373388+0000","last_unstale":"2024-04-17T05:46:19.531942+0000","last_undegraded":"2024-04-17T05:46:19.531942+0000","last_fullsized":"2024-04-17T05:46:19.531942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.531732+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.531732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:32.490714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"264'319","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.744134+0000","last_change":"2024-04-17T05:46:23.744249+0000","last_active":"2024-04-17T05:46:23.744134+0000","last_peered":"2024-04-17T05:46:23.744134+0000","last_clean":"2024-04-17T05:46:23.744134+0000","last_became_active":"2024-04-17T04:36:31.392127+0000","last_became_peered":"2024-04-17T04:36:31.392127+0000","last_unstale":"2024-04-17T05:46:23.744134+0000","last_undegraded":"2024-04-17T05:46:23.744134+0000","last_fullsized":"2024-04-17T05:46:23.744134+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:23.743937+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.743937+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:03.777566+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44250,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":535,"num_read_kb":1188,"num_write":472,"num_write_kb":375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15128,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":504,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.643838+0000","last_change":"2024-04-17T05:46:25.643838+0000","last_active":"2024-04-17T05:46:25.643838+0000","last_peered":"2024-04-17T05:46:25.643838+0000","last_clean":"2024-04-17T05:46:25.643838+0000","last_became_active":"2024-04-17T04:36:33.426189+0000","last_became_peered":"2024-04-17T04:36:33.426189+0000","last_unstale":"2024-04-17T05:46:25.643838+0000","last_undegraded":"2024-04-17T05:46:25.643838+0000","last_fullsized":"2024-04-17T05:46:25.643838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:25.643431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:25.643431+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.643431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:15:10.449552+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"264'133","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.789793+0000","last_change":"2024-04-17T05:46:19.789793+0000","last_active":"2024-04-17T05:46:19.789793+0000","last_peered":"2024-04-17T05:46:19.789793+0000","last_clean":"2024-04-17T05:46:19.789793+0000","last_became_active":"2024-04-17T04:36:35.422243+0000","last_became_peered":"2024-04-17T04:36:35.422243+0000","last_unstale":"2024-04-17T05:46:19.789793+0000","last_undegraded":"2024-04-17T05:46:19.789793+0000","last_fullsized":"2024-04-17T05:46:19.789793+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:19.789529+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.789529+0000","objects_scrubbed":4,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:00:17.259142+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":130,"num_read_kb":118,"num_write":287,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"264'149","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.626107+0000","last_change":"2024-04-17T05:46:39.626172+0000","last_active":"2024-04-17T05:46:39.626107+0000","last_peered":"2024-04-17T05:46:39.626107+0000","last_clean":"2024-04-17T05:46:39.626107+0000","last_became_active":"2024-04-17T04:36:27.380660+0000","last_became_peered":"2024-04-17T04:36:27.380660+0000","last_unstale":"2024-04-17T05:46:39.626107+0000","last_undegraded":"2024-04-17T05:46:39.626107+0000","last_fullsized":"2024-04-17T05:46:39.626107+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'149","last_scrub_stamp":"2024-04-17T05:46:39.625903+0000","last_deep_scrub":"264'149","last_deep_scrub_stamp":"2024-04-17T05:46:39.625903+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.625903+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:29.931185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":582,"num_read_kb":464,"num_write":222,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"264'365","reported_seq":1040,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.843059+0000","last_change":"2024-04-17T05:46:32.843361+0000","last_active":"2024-04-17T05:46:32.843059+0000","last_peered":"2024-04-17T05:46:32.843059+0000","last_clean":"2024-04-17T05:46:32.843059+0000","last_became_active":"2024-04-17T04:36:25.371111+0000","last_became_peered":"2024-04-17T04:36:25.371111+0000","last_unstale":"2024-04-17T05:46:32.843059+0000","last_undegraded":"2024-04-17T05:46:32.843059+0000","last_fullsized":"2024-04-17T05:46:32.843059+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'365","last_scrub_stamp":"2024-04-17T05:46:32.842802+0000","last_deep_scrub":"264'365","last_deep_scrub_stamp":"2024-04-17T05:46:32.842802+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.842802+0000","objects_scrubbed":7,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:41:42.238535+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524382,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":609,"num_read_kb":15072,"num_write":1258,"num_write_kb":28594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":517,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.751463+0000","last_change":"2024-04-17T05:46:19.751517+0000","last_active":"2024-04-17T05:46:19.751463+0000","last_peered":"2024-04-17T05:46:19.751463+0000","last_clean":"2024-04-17T05:46:19.751463+0000","last_became_active":"2024-04-17T04:36:29.358887+0000","last_became_peered":"2024-04-17T04:36:29.358887+0000","last_unstale":"2024-04-17T05:46:19.751463+0000","last_undegraded":"2024-04-17T05:46:19.751463+0000","last_fullsized":"2024-04-17T05:46:19.751463+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.751263+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.751263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:23:08.609702+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"264'298","reported_seq":917,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.671356+0000","last_change":"2024-04-17T05:46:11.671356+0000","last_active":"2024-04-17T05:46:11.671356+0000","last_peered":"2024-04-17T05:46:11.671356+0000","last_clean":"2024-04-17T05:46:11.671356+0000","last_became_active":"2024-04-17T04:36:31.397091+0000","last_became_peered":"2024-04-17T04:36:31.397091+0000","last_unstale":"2024-04-17T05:46:11.671356+0000","last_undegraded":"2024-04-17T05:46:11.671356+0000","last_fullsized":"2024-04-17T05:46:11.671356+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:11.670938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.670938+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:15.760804+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47207,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":490,"num_read_kb":1044,"num_write":380,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"264'512","reported_seq":2453,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:09.695035+0000","last_change":"2024-04-17T05:46:09.695035+0000","last_active":"2024-04-17T05:46:09.695035+0000","last_peered":"2024-04-17T05:46:09.695035+0000","last_clean":"2024-04-17T05:46:09.695035+0000","last_became_active":"2024-04-17T04:36:33.407765+0000","last_became_peered":"2024-04-17T04:36:33.407765+0000","last_unstale":"2024-04-17T05:46:09.695035+0000","last_undegraded":"2024-04-17T05:46:09.695035+0000","last_fullsized":"2024-04-17T05:46:09.695035+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'512","last_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_deep_scrub":"264'512","last_deep_scrub_stamp":"2024-04-17T05:46:09.694925+0000","last_clean_scrub_stamp":"2024-04-17T05:46:09.694925+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":500,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.559083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"264'102","reported_seq":618,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.732374+0000","last_change":"2024-04-17T05:46:13.732374+0000","last_active":"2024-04-17T05:46:13.732374+0000","last_peered":"2024-04-17T05:46:13.732374+0000","last_clean":"2024-04-17T05:46:13.732374+0000","last_became_active":"2024-04-17T04:36:35.427773+0000","last_became_peered":"2024-04-17T04:36:35.427773+0000","last_unstale":"2024-04-17T05:46:13.732374+0000","last_undegraded":"2024-04-17T05:46:13.732374+0000","last_fullsized":"2024-04-17T05:46:13.732374+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:13.732113+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.732113+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:45:34.325042+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":87,"num_write":234,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"264'133","reported_seq":740,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.636438+0000","last_change":"2024-04-17T05:46:29.636597+0000","last_active":"2024-04-17T05:46:29.636438+0000","last_peered":"2024-04-17T05:46:29.636438+0000","last_clean":"2024-04-17T05:46:29.636438+0000","last_became_active":"2024-04-17T04:36:27.392316+0000","last_became_peered":"2024-04-17T04:36:27.392316+0000","last_unstale":"2024-04-17T05:46:29.636438+0000","last_undegraded":"2024-04-17T05:46:29.636438+0000","last_fullsized":"2024-04-17T05:46:29.636438+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:29.636101+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:29.636101+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.636101+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:14.328596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":539,"num_read_kb":442,"num_write":192,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"264'295","reported_seq":994,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.600236+0000","last_change":"2024-04-17T05:46:42.600435+0000","last_active":"2024-04-17T05:46:42.600236+0000","last_peered":"2024-04-17T05:46:42.600236+0000","last_clean":"2024-04-17T05:46:42.600236+0000","last_became_active":"2024-04-17T04:36:25.355182+0000","last_became_peered":"2024-04-17T04:36:25.355182+0000","last_unstale":"2024-04-17T05:46:42.600236+0000","last_undegraded":"2024-04-17T05:46:42.600236+0000","last_fullsized":"2024-04-17T05:46:42.600236+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'295","last_scrub_stamp":"2024-04-17T05:46:42.600013+0000","last_deep_scrub":"264'295","last_deep_scrub_stamp":"2024-04-17T05:46:42.600013+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.600013+0000","objects_scrubbed":8,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:40.877675+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243143,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":575,"num_read_kb":5671,"num_write":1192,"num_write_kb":35737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"264'4","reported_seq":1166,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.330528+0000","last_change":"2024-04-17T05:46:34.330595+0000","last_active":"2024-04-17T05:46:34.330528+0000","last_peered":"2024-04-17T05:46:34.330528+0000","last_clean":"2024-04-17T05:46:34.330528+0000","last_became_active":"2024-04-17T04:36:29.385901+0000","last_became_peered":"2024-04-17T04:36:29.385901+0000","last_unstale":"2024-04-17T05:46:34.330528+0000","last_undegraded":"2024-04-17T05:46:34.330528+0000","last_fullsized":"2024-04-17T05:46:34.330528+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4","last_scrub_stamp":"2024-04-17T05:46:34.330337+0000","last_deep_scrub":"264'4","last_deep_scrub_stamp":"2024-04-17T05:46:34.330337+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.330337+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:21.974891+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":652,"num_read_kb":652,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"264'102","reported_seq":647,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.863661+0000","last_change":"2024-04-17T05:46:33.863738+0000","last_active":"2024-04-17T05:46:33.863661+0000","last_peered":"2024-04-17T05:46:33.863661+0000","last_clean":"2024-04-17T05:46:33.863661+0000","last_became_active":"2024-04-17T04:36:31.420405+0000","last_became_peered":"2024-04-17T04:36:31.420405+0000","last_unstale":"2024-04-17T05:46:33.863661+0000","last_undegraded":"2024-04-17T05:46:33.863661+0000","last_fullsized":"2024-04-17T05:46:33.863661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'102","last_scrub_stamp":"2024-04-17T05:46:33.863448+0000","last_deep_scrub":"264'102","last_deep_scrub_stamp":"2024-04-17T05:46:33.863448+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.863448+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:04.150489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":70,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"264'539","reported_seq":2257,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.997502+0000","last_change":"2024-04-17T05:46:25.997576+0000","last_active":"2024-04-17T05:46:25.997502+0000","last_peered":"2024-04-17T05:46:25.997502+0000","last_clean":"2024-04-17T05:46:25.997502+0000","last_became_active":"2024-04-17T04:36:33.420461+0000","last_became_peered":"2024-04-17T04:36:33.420461+0000","last_unstale":"2024-04-17T05:46:25.997502+0000","last_undegraded":"2024-04-17T05:46:25.997502+0000","last_fullsized":"2024-04-17T05:46:25.997502+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:25.997283+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:25.997283+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.997283+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:16.675494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"264'98","reported_seq":611,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.796311+0000","last_change":"2024-04-17T05:46:11.796311+0000","last_active":"2024-04-17T05:46:11.796311+0000","last_peered":"2024-04-17T05:46:11.796311+0000","last_clean":"2024-04-17T05:46:11.796311+0000","last_became_active":"2024-04-17T04:36:35.426590+0000","last_became_peered":"2024-04-17T04:36:35.426590+0000","last_unstale":"2024-04-17T05:46:11.796311+0000","last_undegraded":"2024-04-17T05:46:11.796311+0000","last_fullsized":"2024-04-17T05:46:11.796311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'98","last_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_deep_scrub":"264'98","last_deep_scrub_stamp":"2024-04-17T05:46:11.796022+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.796022+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:25.138191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":223,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"264'134","reported_seq":766,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.742441+0000","last_change":"2024-04-17T05:46:25.742441+0000","last_active":"2024-04-17T05:46:25.742441+0000","last_peered":"2024-04-17T05:46:25.742441+0000","last_clean":"2024-04-17T05:46:25.742441+0000","last_became_active":"2024-04-17T04:36:27.377711+0000","last_became_peered":"2024-04-17T04:36:27.377711+0000","last_unstale":"2024-04-17T05:46:25.742441+0000","last_undegraded":"2024-04-17T05:46:25.742441+0000","last_fullsized":"2024-04-17T05:46:25.742441+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:25.742055+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:25.742055+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.742055+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:17:19.477880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":567,"num_read_kb":468,"num_write":191,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"264'296","reported_seq":941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:00.706078+0000","last_change":"2024-04-17T05:47:00.706286+0000","last_active":"2024-04-17T05:47:00.706078+0000","last_peered":"2024-04-17T05:47:00.706078+0000","last_clean":"2024-04-17T05:47:00.706078+0000","last_became_active":"2024-04-17T04:36:25.353130+0000","last_became_peered":"2024-04-17T04:36:25.353130+0000","last_unstale":"2024-04-17T05:47:00.706078+0000","last_undegraded":"2024-04-17T05:47:00.706078+0000","last_fullsized":"2024-04-17T05:47:00.706078+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'296","last_scrub_stamp":"2024-04-17T05:47:00.705786+0000","last_deep_scrub":"264'296","last_deep_scrub_stamp":"2024-04-17T05:47:00.705786+0000","last_clean_scrub_stamp":"2024-04-17T05:47:00.705786+0000","objects_scrubbed":7,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:18:27.453238+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170337,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":410,"num_read_kb":36760,"num_write":993,"num_write_kb":39586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.522731+0000","last_change":"2024-04-17T05:46:50.522786+0000","last_active":"2024-04-17T05:46:50.522731+0000","last_peered":"2024-04-17T05:46:50.522731+0000","last_clean":"2024-04-17T05:46:50.522731+0000","last_became_active":"2024-04-17T04:36:29.388673+0000","last_became_peered":"2024-04-17T04:36:29.388673+0000","last_unstale":"2024-04-17T05:46:50.522731+0000","last_undegraded":"2024-04-17T05:46:50.522731+0000","last_fullsized":"2024-04-17T05:46:50.522731+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:50.522512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:50.522512+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.522512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:05.669837+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"264'359","reported_seq":992,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.232070+0000","last_change":"2024-04-17T05:46:25.232126+0000","last_active":"2024-04-17T05:46:25.232070+0000","last_peered":"2024-04-17T05:46:25.232070+0000","last_clean":"2024-04-17T05:46:25.232070+0000","last_became_active":"2024-04-17T04:36:31.396142+0000","last_became_peered":"2024-04-17T04:36:31.396142+0000","last_unstale":"2024-04-17T05:46:25.232070+0000","last_undegraded":"2024-04-17T05:46:25.232070+0000","last_fullsized":"2024-04-17T05:46:25.232070+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'359","last_scrub_stamp":"2024-04-17T05:46:25.231966+0000","last_deep_scrub":"264'359","last_deep_scrub_stamp":"2024-04-17T05:46:25.231966+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.231966+0000","objects_scrubbed":8,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:47:43.934129+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45806,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1167,"num_write":448,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.992307+0000","last_change":"2024-04-17T05:46:21.992363+0000","last_active":"2024-04-17T05:46:21.992307+0000","last_peered":"2024-04-17T05:46:21.992307+0000","last_clean":"2024-04-17T05:46:21.992307+0000","last_became_active":"2024-04-17T04:36:33.418872+0000","last_became_peered":"2024-04-17T04:36:33.418872+0000","last_unstale":"2024-04-17T05:46:21.992307+0000","last_undegraded":"2024-04-17T05:46:21.992307+0000","last_fullsized":"2024-04-17T05:46:21.992307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:21.992088+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.992088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:40.290560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"264'161","reported_seq":677,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.736060+0000","last_change":"2024-04-17T05:46:15.736060+0000","last_active":"2024-04-17T05:46:15.736060+0000","last_peered":"2024-04-17T05:46:15.736060+0000","last_clean":"2024-04-17T05:46:15.736060+0000","last_became_active":"2024-04-17T04:36:35.407471+0000","last_became_peered":"2024-04-17T04:36:35.407471+0000","last_unstale":"2024-04-17T05:46:15.736060+0000","last_undegraded":"2024-04-17T05:46:15.736060+0000","last_fullsized":"2024-04-17T05:46:15.736060+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:15.735662+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.735662+0000","objects_scrubbed":4,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:46:51.803225+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":108,"num_read_kb":103,"num_write":337,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"264'101","reported_seq":705,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.006216+0000","last_change":"2024-04-17T05:46:25.006216+0000","last_active":"2024-04-17T05:46:25.006216+0000","last_peered":"2024-04-17T05:46:25.006216+0000","last_clean":"2024-04-17T05:46:25.006216+0000","last_became_active":"2024-04-17T04:36:27.386566+0000","last_became_peered":"2024-04-17T04:36:27.386566+0000","last_unstale":"2024-04-17T05:46:25.006216+0000","last_undegraded":"2024-04-17T05:46:25.006216+0000","last_fullsized":"2024-04-17T05:46:25.006216+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:25.005814+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.005814+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:23:49.763097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":433,"num_read_kb":358,"num_write":152,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"264'119","reported_seq":692,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.064201+0000","last_change":"2024-04-17T05:46:50.064279+0000","last_active":"2024-04-17T05:46:50.064201+0000","last_peered":"2024-04-17T05:46:50.064201+0000","last_clean":"2024-04-17T05:46:50.064201+0000","last_became_active":"2024-04-17T04:36:12.828936+0000","last_became_peered":"2024-04-17T04:36:12.828936+0000","last_unstale":"2024-04-17T05:46:50.064201+0000","last_undegraded":"2024-04-17T05:46:50.064201+0000","last_fullsized":"2024-04-17T05:46:50.064201+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'119","last_scrub_stamp":"2024-04-17T05:46:50.063983+0000","last_deep_scrub":"264'119","last_deep_scrub_stamp":"2024-04-17T05:46:50.063983+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.063983+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:28.852792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":531,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.225234+0000","last_change":"2024-04-17T05:46:43.225234+0000","last_active":"2024-04-17T05:46:43.225234+0000","last_peered":"2024-04-17T05:46:43.225234+0000","last_clean":"2024-04-17T05:46:43.225234+0000","last_became_active":"2024-04-17T04:36:17.215105+0000","last_became_peered":"2024-04-17T04:36:17.215105+0000","last_unstale":"2024-04-17T05:46:43.225234+0000","last_undegraded":"2024-04-17T05:46:43.225234+0000","last_fullsized":"2024-04-17T05:46:43.225234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.224981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.224981+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.224981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:19:16.644730+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"264'327","reported_seq":988,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:01.735526+0000","last_change":"2024-04-17T05:47:01.735526+0000","last_active":"2024-04-17T05:47:01.735526+0000","last_peered":"2024-04-17T05:47:01.735526+0000","last_clean":"2024-04-17T05:47:01.735526+0000","last_became_active":"2024-04-17T04:36:25.363351+0000","last_became_peered":"2024-04-17T04:36:25.363351+0000","last_unstale":"2024-04-17T05:47:01.735526+0000","last_undegraded":"2024-04-17T05:47:01.735526+0000","last_fullsized":"2024-04-17T05:47:01.735526+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'327","last_scrub_stamp":"2024-04-17T05:47:01.735208+0000","last_deep_scrub":"264'327","last_deep_scrub_stamp":"2024-04-17T05:47:01.735208+0000","last_clean_scrub_stamp":"2024-04-17T05:47:01.735208+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:19:19.437082+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344976,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":547,"num_read_kb":2736,"num_write":1215,"num_write_kb":6327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6,4],"acting":[3,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":515,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.441319+0000","last_change":"2024-04-17T05:46:36.441319+0000","last_active":"2024-04-17T05:46:36.441319+0000","last_peered":"2024-04-17T05:46:36.441319+0000","last_clean":"2024-04-17T05:46:36.441319+0000","last_became_active":"2024-04-17T04:36:29.380693+0000","last_became_peered":"2024-04-17T04:36:29.380693+0000","last_unstale":"2024-04-17T05:46:36.441319+0000","last_undegraded":"2024-04-17T05:46:36.441319+0000","last_fullsized":"2024-04-17T05:46:36.441319+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:36.441066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:36.441066+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.441066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:15.277478+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"264'728","reported_seq":1458,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.227559+0000","last_change":"2024-04-17T05:46:23.227642+0000","last_active":"2024-04-17T05:46:23.227559+0000","last_peered":"2024-04-17T05:46:23.227559+0000","last_clean":"2024-04-17T05:46:23.227559+0000","last_became_active":"2024-04-17T04:36:31.399356+0000","last_became_peered":"2024-04-17T04:36:31.399356+0000","last_unstale":"2024-04-17T05:46:23.227559+0000","last_undegraded":"2024-04-17T05:46:23.227559+0000","last_fullsized":"2024-04-17T05:46:23.227559+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'728","last_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_deep_scrub":"264'728","last_deep_scrub_stamp":"2024-04-17T05:46:23.227369+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.227369+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":700,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:26.750099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1411,"num_read_kb":3198,"num_write":1348,"num_write_kb":1175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":95136,"num_omap_keys":246,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.647366+0000","last_change":"2024-04-17T05:46:28.647420+0000","last_active":"2024-04-17T05:46:28.647366+0000","last_peered":"2024-04-17T05:46:28.647366+0000","last_clean":"2024-04-17T05:46:28.647366+0000","last_became_active":"2024-04-17T04:36:33.426550+0000","last_became_peered":"2024-04-17T04:36:33.426550+0000","last_unstale":"2024-04-17T05:46:28.647366+0000","last_undegraded":"2024-04-17T05:46:28.647366+0000","last_fullsized":"2024-04-17T05:46:28.647366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.647167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.647167+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.647167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:05:59.157329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"264'223","reported_seq":925,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.911300+0000","last_change":"2024-04-17T05:46:29.911389+0000","last_active":"2024-04-17T05:46:29.911300+0000","last_peered":"2024-04-17T05:46:29.911300+0000","last_clean":"2024-04-17T05:46:29.911300+0000","last_became_active":"2024-04-17T04:36:35.419538+0000","last_became_peered":"2024-04-17T04:36:35.419538+0000","last_unstale":"2024-04-17T05:46:29.911300+0000","last_undegraded":"2024-04-17T05:46:29.911300+0000","last_fullsized":"2024-04-17T05:46:29.911300+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'223","last_scrub_stamp":"2024-04-17T05:46:29.911091+0000","last_deep_scrub":"264'223","last_deep_scrub_stamp":"2024-04-17T05:46:29.911091+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.911091+0000","objects_scrubbed":3,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:25.435946+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":425,"num_read_kb":418,"num_write":461,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"264'84","reported_seq":677,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.627128+0000","last_change":"2024-04-17T05:46:48.627245+0000","last_active":"2024-04-17T05:46:48.627128+0000","last_peered":"2024-04-17T05:46:48.627128+0000","last_clean":"2024-04-17T05:46:48.627128+0000","last_became_active":"2024-04-17T04:36:27.364890+0000","last_became_peered":"2024-04-17T04:36:27.364890+0000","last_unstale":"2024-04-17T05:46:48.627128+0000","last_undegraded":"2024-04-17T05:46:48.627128+0000","last_fullsized":"2024-04-17T05:46:48.627128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'84","last_scrub_stamp":"2024-04-17T05:46:48.626891+0000","last_deep_scrub":"264'84","last_deep_scrub_stamp":"2024-04-17T05:46:48.626891+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.626891+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:55.642870+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":309,"num_write":135,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":534,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.100977+0000","last_change":"2024-04-17T05:46:51.101031+0000","last_active":"2024-04-17T05:46:51.100977+0000","last_peered":"2024-04-17T05:46:51.100977+0000","last_clean":"2024-04-17T05:46:51.100977+0000","last_became_active":"2024-04-17T04:36:17.215129+0000","last_became_peered":"2024-04-17T04:36:17.215129+0000","last_unstale":"2024-04-17T05:46:51.100977+0000","last_undegraded":"2024-04-17T05:46:51.100977+0000","last_fullsized":"2024-04-17T05:46:51.100977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:51.100774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:51.100774+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.100774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:20:01.756488+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"264'343","reported_seq":1005,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.587177+0000","last_change":"2024-04-17T05:46:52.587177+0000","last_active":"2024-04-17T05:46:52.587177+0000","last_peered":"2024-04-17T05:46:52.587177+0000","last_clean":"2024-04-17T05:46:52.587177+0000","last_became_active":"2024-04-17T04:36:25.355145+0000","last_became_peered":"2024-04-17T04:36:25.355145+0000","last_unstale":"2024-04-17T05:46:52.587177+0000","last_undegraded":"2024-04-17T05:46:52.587177+0000","last_fullsized":"2024-04-17T05:46:52.587177+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'343","last_scrub_stamp":"2024-04-17T05:46:52.586788+0000","last_deep_scrub":"264'343","last_deep_scrub_stamp":"2024-04-17T05:46:52.586788+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.586788+0000","objects_scrubbed":11,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:57:32.792717+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2309902,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":570,"num_read_kb":3714,"num_write":1237,"num_write_kb":20373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.294868+0000","last_change":"2024-04-17T05:46:40.294921+0000","last_active":"2024-04-17T05:46:40.294868+0000","last_peered":"2024-04-17T05:46:40.294868+0000","last_clean":"2024-04-17T05:46:40.294868+0000","last_became_active":"2024-04-17T04:36:29.392468+0000","last_became_peered":"2024-04-17T04:36:29.392468+0000","last_unstale":"2024-04-17T05:46:40.294868+0000","last_undegraded":"2024-04-17T05:46:40.294868+0000","last_fullsized":"2024-04-17T05:46:40.294868+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:40.294666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:40.294666+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.294666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:50:08.051052+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"264'1109","reported_seq":2837,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.770431+0000","last_change":"2024-04-17T05:46:26.770431+0000","last_active":"2024-04-17T05:46:26.770431+0000","last_peered":"2024-04-17T05:46:26.770431+0000","last_clean":"2024-04-17T05:46:26.770431+0000","last_became_active":"2024-04-17T04:36:31.395945+0000","last_became_peered":"2024-04-17T04:36:31.395945+0000","last_unstale":"2024-04-17T05:46:26.770431+0000","last_undegraded":"2024-04-17T05:46:26.770431+0000","last_fullsized":"2024-04-17T05:46:26.770431+0000","mapping_epoch":29,"log_start":"264'1000","ondisk_log_start":"264'1000","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'1109","last_scrub_stamp":"2024-04-17T05:46:26.770091+0000","last_deep_scrub":"264'1109","last_deep_scrub_stamp":"2024-04-17T05:46:26.770091+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.770091+0000","objects_scrubbed":12,"log_size":109,"log_dups_size":1000,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:23.849377+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1819,"num_read_kb":3719,"num_write":1122,"num_write_kb":937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5370,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.327552+0000","last_change":"2024-04-17T05:46:30.327657+0000","last_active":"2024-04-17T05:46:30.327552+0000","last_peered":"2024-04-17T05:46:30.327552+0000","last_clean":"2024-04-17T05:46:30.327552+0000","last_became_active":"2024-04-17T04:36:33.421739+0000","last_became_peered":"2024-04-17T04:36:33.421739+0000","last_unstale":"2024-04-17T05:46:30.327552+0000","last_undegraded":"2024-04-17T05:46:30.327552+0000","last_fullsized":"2024-04-17T05:46:30.327552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.327323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.327323+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.327323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:13:37.291684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"264'114","reported_seq":633,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.750777+0000","last_change":"2024-04-17T05:46:17.750850+0000","last_active":"2024-04-17T05:46:17.750777+0000","last_peered":"2024-04-17T05:46:17.750777+0000","last_clean":"2024-04-17T05:46:17.750777+0000","last_became_active":"2024-04-17T04:36:35.401935+0000","last_became_peered":"2024-04-17T04:36:35.401935+0000","last_unstale":"2024-04-17T05:46:17.750777+0000","last_undegraded":"2024-04-17T05:46:17.750777+0000","last_fullsized":"2024-04-17T05:46:17.750777+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'114","last_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_deep_scrub":"264'114","last_deep_scrub_stamp":"2024-04-17T05:46:17.750563+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.750563+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:00.237624+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":118,"num_write":282,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"264'343","reported_seq":1026,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.816940+0000","last_change":"2024-04-17T05:46:29.817060+0000","last_active":"2024-04-17T05:46:29.816940+0000","last_peered":"2024-04-17T05:46:29.816940+0000","last_clean":"2024-04-17T05:46:29.816940+0000","last_became_active":"2024-04-17T04:36:27.375847+0000","last_became_peered":"2024-04-17T04:36:27.375847+0000","last_unstale":"2024-04-17T05:46:29.816940+0000","last_undegraded":"2024-04-17T05:46:29.816940+0000","last_fullsized":"2024-04-17T05:46:29.816940+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'343","last_scrub_stamp":"2024-04-17T05:46:29.816699+0000","last_deep_scrub":"264'343","last_deep_scrub_stamp":"2024-04-17T05:46:29.816699+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.816699+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:49.710831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1346,"num_read_kb":1069,"num_write":539,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":540,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.432548+0000","last_change":"2024-04-17T05:46:24.432548+0000","last_active":"2024-04-17T05:46:24.432548+0000","last_peered":"2024-04-17T05:46:24.432548+0000","last_clean":"2024-04-17T05:46:24.432548+0000","last_became_active":"2024-04-17T04:36:17.751876+0000","last_became_peered":"2024-04-17T04:36:17.751876+0000","last_unstale":"2024-04-17T05:46:24.432548+0000","last_undegraded":"2024-04-17T05:46:24.432548+0000","last_fullsized":"2024-04-17T05:46:24.432548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:24.432302+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.432302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:04:49.304795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"264'298","reported_seq":964,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.846776+0000","last_change":"2024-04-17T05:46:37.846776+0000","last_active":"2024-04-17T05:46:37.846776+0000","last_peered":"2024-04-17T05:46:37.846776+0000","last_clean":"2024-04-17T05:46:37.846776+0000","last_became_active":"2024-04-17T04:36:25.343177+0000","last_became_peered":"2024-04-17T04:36:25.343177+0000","last_unstale":"2024-04-17T05:46:37.846776+0000","last_undegraded":"2024-04-17T05:46:37.846776+0000","last_fullsized":"2024-04-17T05:46:37.846776+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'298","last_scrub_stamp":"2024-04-17T05:46:37.846228+0000","last_deep_scrub":"264'298","last_deep_scrub_stamp":"2024-04-17T05:46:37.846228+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.846228+0000","objects_scrubbed":9,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:57:41.146058+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5679739,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":479,"num_read_kb":21975,"num_write":1114,"num_write_kb":21037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"264'2","reported_seq":522,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.548976+0000","last_change":"2024-04-17T05:46:15.549042+0000","last_active":"2024-04-17T05:46:15.548976+0000","last_peered":"2024-04-17T05:46:15.548976+0000","last_clean":"2024-04-17T05:46:15.548976+0000","last_became_active":"2024-04-17T04:36:29.392525+0000","last_became_peered":"2024-04-17T04:36:29.392525+0000","last_unstale":"2024-04-17T05:46:15.548976+0000","last_undegraded":"2024-04-17T05:46:15.548976+0000","last_fullsized":"2024-04-17T05:46:15.548976+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'2","last_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_deep_scrub":"264'2","last_deep_scrub_stamp":"2024-04-17T05:46:15.548772+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.548772+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:16:50.031612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"264'243","reported_seq":1636,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:58.265285+0000","last_change":"2024-04-17T05:46:58.265370+0000","last_active":"2024-04-17T05:46:58.265285+0000","last_peered":"2024-04-17T05:46:58.265285+0000","last_clean":"2024-04-17T05:46:58.265285+0000","last_became_active":"2024-04-17T04:36:31.426306+0000","last_became_peered":"2024-04-17T04:36:31.426306+0000","last_unstale":"2024-04-17T05:46:58.265285+0000","last_undegraded":"2024-04-17T05:46:58.265285+0000","last_fullsized":"2024-04-17T05:46:58.265285+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'243","last_scrub_stamp":"2024-04-17T05:46:58.265097+0000","last_deep_scrub":"264'243","last_deep_scrub_stamp":"2024-04-17T05:46:58.265097+0000","last_clean_scrub_stamp":"2024-04-17T05:46:58.265097+0000","objects_scrubbed":9,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:35:38.453917+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":216,"num_read_kb":207,"num_write":173,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5136,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.628630+0000","last_change":"2024-04-17T05:46:32.628630+0000","last_active":"2024-04-17T05:46:32.628630+0000","last_peered":"2024-04-17T05:46:32.628630+0000","last_clean":"2024-04-17T05:46:32.628630+0000","last_became_active":"2024-04-17T04:36:33.425473+0000","last_became_peered":"2024-04-17T04:36:33.425473+0000","last_unstale":"2024-04-17T05:46:32.628630+0000","last_undegraded":"2024-04-17T05:46:32.628630+0000","last_fullsized":"2024-04-17T05:46:32.628630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.628381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.628381+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.628381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:52.474027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"264'265","reported_seq":879,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.770655+0000","last_change":"2024-04-17T05:46:24.770806+0000","last_active":"2024-04-17T05:46:24.770655+0000","last_peered":"2024-04-17T05:46:24.770655+0000","last_clean":"2024-04-17T05:46:24.770655+0000","last_became_active":"2024-04-17T04:36:35.407280+0000","last_became_peered":"2024-04-17T04:36:35.407280+0000","last_unstale":"2024-04-17T05:46:24.770655+0000","last_undegraded":"2024-04-17T05:46:24.770655+0000","last_fullsized":"2024-04-17T05:46:24.770655+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'265","last_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_deep_scrub":"264'265","last_deep_scrub_stamp":"2024-04-17T05:46:24.770438+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.770438+0000","objects_scrubbed":3,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:40:52.656005+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":493,"num_read_kb":515,"num_write":574,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"264'311","reported_seq":959,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.751218+0000","last_change":"2024-04-17T05:46:33.751218+0000","last_active":"2024-04-17T05:46:33.751218+0000","last_peered":"2024-04-17T05:46:33.751218+0000","last_clean":"2024-04-17T05:46:33.751218+0000","last_became_active":"2024-04-17T04:36:27.361304+0000","last_became_peered":"2024-04-17T04:36:27.361304+0000","last_unstale":"2024-04-17T05:46:33.751218+0000","last_undegraded":"2024-04-17T05:46:33.751218+0000","last_fullsized":"2024-04-17T05:46:33.751218+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'311","last_scrub_stamp":"2024-04-17T05:46:33.750961+0000","last_deep_scrub":"264'311","last_deep_scrub_stamp":"2024-04-17T05:46:33.750961+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.750961+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:04:15.653327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1233,"num_read_kb":961,"num_write":518,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":531,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.265009+0000","last_change":"2024-04-17T05:46:42.265009+0000","last_active":"2024-04-17T05:46:42.265009+0000","last_peered":"2024-04-17T05:46:42.265009+0000","last_clean":"2024-04-17T05:46:42.265009+0000","last_became_active":"2024-04-17T04:36:17.216134+0000","last_became_peered":"2024-04-17T04:36:17.216134+0000","last_unstale":"2024-04-17T05:46:42.265009+0000","last_undegraded":"2024-04-17T05:46:42.265009+0000","last_fullsized":"2024-04-17T05:46:42.265009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.264768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.264768+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.264768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:12:20.993345+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"264'341","reported_seq":1108,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.616921+0000","last_change":"2024-04-17T05:46:43.616921+0000","last_active":"2024-04-17T05:46:43.616921+0000","last_peered":"2024-04-17T05:46:43.616921+0000","last_clean":"2024-04-17T05:46:43.616921+0000","last_became_active":"2024-04-17T04:36:25.355631+0000","last_became_peered":"2024-04-17T04:36:25.355631+0000","last_unstale":"2024-04-17T05:46:43.616921+0000","last_undegraded":"2024-04-17T05:46:43.616921+0000","last_fullsized":"2024-04-17T05:46:43.616921+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'341","last_scrub_stamp":"2024-04-17T05:46:43.616551+0000","last_deep_scrub":"264'341","last_deep_scrub_stamp":"2024-04-17T05:46:43.616551+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.616551+0000","objects_scrubbed":12,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:41.283471+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048803,"num_objects":12,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":626,"num_read_kb":36553,"num_write":1237,"num_write_kb":43390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":515,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:43.635780+0000","last_change":"2024-04-17T05:46:43.635780+0000","last_active":"2024-04-17T05:46:43.635780+0000","last_peered":"2024-04-17T05:46:43.635780+0000","last_clean":"2024-04-17T05:46:43.635780+0000","last_became_active":"2024-04-17T04:36:29.371457+0000","last_became_peered":"2024-04-17T04:36:29.371457+0000","last_unstale":"2024-04-17T05:46:43.635780+0000","last_undegraded":"2024-04-17T05:46:43.635780+0000","last_fullsized":"2024-04-17T05:46:43.635780+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:43.635515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:43.635515+0000","last_clean_scrub_stamp":"2024-04-17T05:46:43.635515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:46:56.243388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"264'146","reported_seq":715,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.761478+0000","last_change":"2024-04-17T05:46:37.761563+0000","last_active":"2024-04-17T05:46:37.761478+0000","last_peered":"2024-04-17T05:46:37.761478+0000","last_clean":"2024-04-17T05:46:37.761478+0000","last_became_active":"2024-04-17T04:36:31.398848+0000","last_became_peered":"2024-04-17T04:36:31.398848+0000","last_unstale":"2024-04-17T05:46:37.761478+0000","last_undegraded":"2024-04-17T05:46:37.761478+0000","last_fullsized":"2024-04-17T05:46:37.761478+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'146","last_scrub_stamp":"2024-04-17T05:46:37.761274+0000","last_deep_scrub":"264'146","last_deep_scrub_stamp":"2024-04-17T05:46:37.761274+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.761274+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:03.176459+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":176,"num_read_kb":170,"num_write":124,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1416,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.968621+0000","last_change":"2024-04-17T05:46:19.968684+0000","last_active":"2024-04-17T05:46:19.968621+0000","last_peered":"2024-04-17T05:46:19.968621+0000","last_clean":"2024-04-17T05:46:19.968621+0000","last_became_active":"2024-04-17T04:36:33.414097+0000","last_became_peered":"2024-04-17T04:36:33.414097+0000","last_unstale":"2024-04-17T05:46:19.968621+0000","last_undegraded":"2024-04-17T05:46:19.968621+0000","last_fullsized":"2024-04-17T05:46:19.968621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.968394+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.968394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:13:02.908417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"264'134","reported_seq":663,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.184018+0000","last_change":"2024-04-17T05:46:17.184135+0000","last_active":"2024-04-17T05:46:17.184018+0000","last_peered":"2024-04-17T05:46:17.184018+0000","last_clean":"2024-04-17T05:46:17.184018+0000","last_became_active":"2024-04-17T04:36:35.410389+0000","last_became_peered":"2024-04-17T04:36:35.410389+0000","last_unstale":"2024-04-17T05:46:17.184018+0000","last_undegraded":"2024-04-17T05:46:17.184018+0000","last_fullsized":"2024-04-17T05:46:17.184018+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'134","last_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_deep_scrub":"264'134","last_deep_scrub_stamp":"2024-04-17T05:46:17.183823+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.183823+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:54.008088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":823,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":160,"num_write":310,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"264'145","reported_seq":761,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.771954+0000","last_change":"2024-04-17T05:46:27.771954+0000","last_active":"2024-04-17T05:46:27.771954+0000","last_peered":"2024-04-17T05:46:27.771954+0000","last_clean":"2024-04-17T05:46:27.771954+0000","last_became_active":"2024-04-17T04:36:27.375967+0000","last_became_peered":"2024-04-17T04:36:27.375967+0000","last_unstale":"2024-04-17T05:46:27.771954+0000","last_undegraded":"2024-04-17T05:46:27.771954+0000","last_fullsized":"2024-04-17T05:46:27.771954+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'145","last_scrub_stamp":"2024-04-17T05:46:27.771685+0000","last_deep_scrub":"264'145","last_deep_scrub_stamp":"2024-04-17T05:46:27.771685+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.771685+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:13.790569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":612,"num_read_kb":491,"num_write":258,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"264'300","reported_seq":972,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.370505+0000","last_change":"2024-04-17T05:46:37.370505+0000","last_active":"2024-04-17T05:46:37.370505+0000","last_peered":"2024-04-17T05:46:37.370505+0000","last_clean":"2024-04-17T05:46:37.370505+0000","last_became_active":"2024-04-17T04:36:25.362773+0000","last_became_peered":"2024-04-17T04:36:25.362773+0000","last_unstale":"2024-04-17T05:46:37.370505+0000","last_undegraded":"2024-04-17T05:46:37.370505+0000","last_fullsized":"2024-04-17T05:46:37.370505+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'300","last_scrub_stamp":"2024-04-17T05:46:37.370221+0000","last_deep_scrub":"264'300","last_deep_scrub_stamp":"2024-04-17T05:46:37.370221+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.370221+0000","objects_scrubbed":2,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:46:56.727712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":475,"num_read_kb":1448,"num_write":1131,"num_write_kb":6295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":538,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.782544+0000","last_change":"2024-04-17T05:46:35.782544+0000","last_active":"2024-04-17T05:46:35.782544+0000","last_peered":"2024-04-17T05:46:35.782544+0000","last_clean":"2024-04-17T05:46:35.782544+0000","last_became_active":"2024-04-17T04:36:17.872188+0000","last_became_peered":"2024-04-17T04:36:17.872188+0000","last_unstale":"2024-04-17T05:46:35.782544+0000","last_undegraded":"2024-04-17T05:46:35.782544+0000","last_fullsized":"2024-04-17T05:46:35.782544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:35.782279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:35.782279+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.782279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:01:00.464190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":515,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.605511+0000","last_change":"2024-04-17T05:46:47.605511+0000","last_active":"2024-04-17T05:46:47.605511+0000","last_peered":"2024-04-17T05:46:47.605511+0000","last_clean":"2024-04-17T05:46:47.605511+0000","last_became_active":"2024-04-17T04:36:29.384568+0000","last_became_peered":"2024-04-17T04:36:29.384568+0000","last_unstale":"2024-04-17T05:46:47.605511+0000","last_undegraded":"2024-04-17T05:46:47.605511+0000","last_fullsized":"2024-04-17T05:46:47.605511+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:47.605143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:47.605143+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.605143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:45:36.604088+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"264'631","reported_seq":1337,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.682650+0000","last_change":"2024-04-17T05:46:34.682650+0000","last_active":"2024-04-17T05:46:34.682650+0000","last_peered":"2024-04-17T05:46:34.682650+0000","last_clean":"2024-04-17T05:46:34.682650+0000","last_became_active":"2024-04-17T04:36:31.410763+0000","last_became_peered":"2024-04-17T04:36:31.410763+0000","last_unstale":"2024-04-17T05:46:34.682650+0000","last_undegraded":"2024-04-17T05:46:34.682650+0000","last_fullsized":"2024-04-17T05:46:34.682650+0000","mapping_epoch":29,"log_start":"264'600","ondisk_log_start":"264'600","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'631","last_scrub_stamp":"2024-04-17T05:46:34.682357+0000","last_deep_scrub":"264'631","last_deep_scrub_stamp":"2024-04-17T05:46:34.682357+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.682357+0000","objects_scrubbed":11,"log_size":31,"log_dups_size":600,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:41.613666+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86860,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1659,"num_read_kb":2834,"num_write":2174,"num_write_kb":1973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":470034,"num_omap_keys":1124,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.791757+0000","last_change":"2024-04-17T05:46:17.791757+0000","last_active":"2024-04-17T05:46:17.791757+0000","last_peered":"2024-04-17T05:46:17.791757+0000","last_clean":"2024-04-17T05:46:17.791757+0000","last_became_active":"2024-04-17T04:36:33.406111+0000","last_became_peered":"2024-04-17T04:36:33.406111+0000","last_unstale":"2024-04-17T05:46:17.791757+0000","last_undegraded":"2024-04-17T05:46:17.791757+0000","last_fullsized":"2024-04-17T05:46:17.791757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.791493+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.791493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:18:50.092156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"264'86","reported_seq":598,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.643211+0000","last_change":"2024-04-17T05:46:18.643211+0000","last_active":"2024-04-17T05:46:18.643211+0000","last_peered":"2024-04-17T05:46:18.643211+0000","last_clean":"2024-04-17T05:46:18.643211+0000","last_became_active":"2024-04-17T04:36:35.428301+0000","last_became_peered":"2024-04-17T04:36:35.428301+0000","last_unstale":"2024-04-17T05:46:18.643211+0000","last_undegraded":"2024-04-17T05:46:18.643211+0000","last_fullsized":"2024-04-17T05:46:18.643211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'86","last_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_deep_scrub":"264'86","last_deep_scrub_stamp":"2024-04-17T05:46:18.642880+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.642880+0000","objects_scrubbed":4,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:33.629614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":711,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":68,"num_write":179,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"264'34","reported_seq":602,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.677786+0000","last_change":"2024-04-17T05:46:38.677786+0000","last_active":"2024-04-17T05:46:38.677786+0000","last_peered":"2024-04-17T05:46:38.677786+0000","last_clean":"2024-04-17T05:46:38.677786+0000","last_became_active":"2024-04-17T04:36:27.375686+0000","last_became_peered":"2024-04-17T04:36:27.375686+0000","last_unstale":"2024-04-17T05:46:38.677786+0000","last_undegraded":"2024-04-17T05:46:38.677786+0000","last_fullsized":"2024-04-17T05:46:38.677786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'34","last_scrub_stamp":"2024-04-17T05:46:38.677518+0000","last_deep_scrub":"264'34","last_deep_scrub_stamp":"2024-04-17T05:46:38.677518+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.677518+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:22.863769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"264'269","reported_seq":900,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.836824+0000","last_change":"2024-04-17T05:46:38.836824+0000","last_active":"2024-04-17T05:46:38.836824+0000","last_peered":"2024-04-17T05:46:38.836824+0000","last_clean":"2024-04-17T05:46:38.836824+0000","last_became_active":"2024-04-17T04:36:25.344601+0000","last_became_peered":"2024-04-17T04:36:25.344601+0000","last_unstale":"2024-04-17T05:46:38.836824+0000","last_undegraded":"2024-04-17T05:46:38.836824+0000","last_fullsized":"2024-04-17T05:46:38.836824+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'269","last_scrub_stamp":"2024-04-17T05:46:38.836321+0000","last_deep_scrub":"264'269","last_deep_scrub_stamp":"2024-04-17T05:46:38.836321+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.836321+0000","objects_scrubbed":4,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:25.285046+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194313,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":396,"num_read_kb":35213,"num_write":840,"num_write_kb":20993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":533,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.148749+0000","last_change":"2024-04-17T05:46:52.148749+0000","last_active":"2024-04-17T05:46:52.148749+0000","last_peered":"2024-04-17T05:46:52.148749+0000","last_clean":"2024-04-17T05:46:52.148749+0000","last_became_active":"2024-04-17T04:36:17.874207+0000","last_became_peered":"2024-04-17T04:36:17.874207+0000","last_unstale":"2024-04-17T05:46:52.148749+0000","last_undegraded":"2024-04-17T05:46:52.148749+0000","last_fullsized":"2024-04-17T05:46:52.148749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:52.148489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:52.148489+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.148489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:00:21.186802+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.958830+0000","last_change":"2024-04-17T05:46:20.958830+0000","last_active":"2024-04-17T05:46:20.958830+0000","last_peered":"2024-04-17T05:46:20.958830+0000","last_clean":"2024-04-17T05:46:20.958830+0000","last_became_active":"2024-04-17T04:36:29.374581+0000","last_became_peered":"2024-04-17T04:36:29.374581+0000","last_unstale":"2024-04-17T05:46:20.958830+0000","last_undegraded":"2024-04-17T05:46:20.958830+0000","last_fullsized":"2024-04-17T05:46:20.958830+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:20.958569+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.958569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:21:07.756009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"264'207","reported_seq":810,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.813826+0000","last_change":"2024-04-17T05:46:28.813826+0000","last_active":"2024-04-17T05:46:28.813826+0000","last_peered":"2024-04-17T05:46:28.813826+0000","last_clean":"2024-04-17T05:46:28.813826+0000","last_became_active":"2024-04-17T04:36:31.420275+0000","last_became_peered":"2024-04-17T04:36:31.420275+0000","last_unstale":"2024-04-17T05:46:28.813826+0000","last_undegraded":"2024-04-17T05:46:28.813826+0000","last_fullsized":"2024-04-17T05:46:28.813826+0000","mapping_epoch":29,"log_start":"261'100","ondisk_log_start":"261'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'207","last_scrub_stamp":"2024-04-17T05:46:28.813550+0000","last_deep_scrub":"264'207","last_deep_scrub_stamp":"2024-04-17T05:46:28.813550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.813550+0000","objects_scrubbed":8,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:10.411851+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":248,"num_read_kb":240,"num_write":166,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"264'540","reported_seq":4368,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.162167+0000","last_change":"2024-04-17T05:46:21.162167+0000","last_active":"2024-04-17T05:46:21.162167+0000","last_peered":"2024-04-17T05:46:21.162167+0000","last_clean":"2024-04-17T05:46:21.162167+0000","last_became_active":"2024-04-17T04:36:33.407081+0000","last_became_peered":"2024-04-17T04:36:33.407081+0000","last_unstale":"2024-04-17T05:46:21.162167+0000","last_undegraded":"2024-04-17T05:46:21.162167+0000","last_fullsized":"2024-04-17T05:46:21.162167+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'540","last_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_deep_scrub":"264'540","last_deep_scrub_stamp":"2024-04-17T05:46:21.161902+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.161902+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":500,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:05:14.043415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"264'161","reported_seq":698,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.780606+0000","last_change":"2024-04-17T05:46:23.780606+0000","last_active":"2024-04-17T05:46:23.780606+0000","last_peered":"2024-04-17T05:46:23.780606+0000","last_clean":"2024-04-17T05:46:23.780606+0000","last_became_active":"2024-04-17T04:36:35.417728+0000","last_became_peered":"2024-04-17T04:36:35.417728+0000","last_unstale":"2024-04-17T05:46:23.780606+0000","last_undegraded":"2024-04-17T05:46:23.780606+0000","last_fullsized":"2024-04-17T05:46:23.780606+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'161","last_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_deep_scrub":"264'161","last_deep_scrub_stamp":"2024-04-17T05:46:23.780338+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.780338+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:57.240957+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1068,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":186,"num_write":274,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"264'168","reported_seq":809,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.800965+0000","last_change":"2024-04-17T05:46:46.801029+0000","last_active":"2024-04-17T05:46:46.800965+0000","last_peered":"2024-04-17T05:46:46.800965+0000","last_clean":"2024-04-17T05:46:46.800965+0000","last_became_active":"2024-04-17T04:36:27.395230+0000","last_became_peered":"2024-04-17T04:36:27.395230+0000","last_unstale":"2024-04-17T05:46:46.800965+0000","last_undegraded":"2024-04-17T05:46:46.800965+0000","last_fullsized":"2024-04-17T05:46:46.800965+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'168","last_scrub_stamp":"2024-04-17T05:46:46.800742+0000","last_deep_scrub":"264'168","last_deep_scrub_stamp":"2024-04-17T05:46:46.800742+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.800742+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:35.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":699,"num_read_kb":565,"num_write":262,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"264'379","reported_seq":1045,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:58.653020+0000","last_change":"2024-04-17T05:46:58.653101+0000","last_active":"2024-04-17T05:46:58.653020+0000","last_peered":"2024-04-17T05:46:58.653020+0000","last_clean":"2024-04-17T05:46:58.653020+0000","last_became_active":"2024-04-17T04:36:25.347063+0000","last_became_peered":"2024-04-17T04:36:25.347063+0000","last_unstale":"2024-04-17T05:46:58.653020+0000","last_undegraded":"2024-04-17T05:46:58.653020+0000","last_fullsized":"2024-04-17T05:46:58.653020+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'379","last_scrub_stamp":"2024-04-17T05:46:58.652814+0000","last_deep_scrub":"264'379","last_deep_scrub_stamp":"2024-04-17T05:46:58.652814+0000","last_clean_scrub_stamp":"2024-04-17T05:46:58.652814+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:17:02.434927+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262239,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":591,"num_read_kb":5844,"num_write":1385,"num_write_kb":23871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":536,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.778199+0000","last_change":"2024-04-17T05:46:34.778260+0000","last_active":"2024-04-17T05:46:34.778199+0000","last_peered":"2024-04-17T05:46:34.778199+0000","last_clean":"2024-04-17T05:46:34.778199+0000","last_became_active":"2024-04-17T04:36:17.212154+0000","last_became_peered":"2024-04-17T04:36:17.212154+0000","last_unstale":"2024-04-17T05:46:34.778199+0000","last_undegraded":"2024-04-17T05:46:34.778199+0000","last_fullsized":"2024-04-17T05:46:34.778199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:34.778008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:34.778008+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.778008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:15:33.294550+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":519,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.587078+0000","last_change":"2024-04-17T05:46:38.587078+0000","last_active":"2024-04-17T05:46:38.587078+0000","last_peered":"2024-04-17T05:46:38.587078+0000","last_clean":"2024-04-17T05:46:38.587078+0000","last_became_active":"2024-04-17T04:36:29.374834+0000","last_became_peered":"2024-04-17T04:36:29.374834+0000","last_unstale":"2024-04-17T05:46:38.587078+0000","last_undegraded":"2024-04-17T05:46:38.587078+0000","last_fullsized":"2024-04-17T05:46:38.587078+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.586815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.586815+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.586815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:35:51.535527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"264'142","reported_seq":714,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.761940+0000","last_change":"2024-04-17T05:46:21.761940+0000","last_active":"2024-04-17T05:46:21.761940+0000","last_peered":"2024-04-17T05:46:21.761940+0000","last_clean":"2024-04-17T05:46:21.761940+0000","last_became_active":"2024-04-17T04:36:31.398981+0000","last_became_peered":"2024-04-17T04:36:31.398981+0000","last_unstale":"2024-04-17T05:46:21.761940+0000","last_undegraded":"2024-04-17T05:46:21.761940+0000","last_fullsized":"2024-04-17T05:46:21.761940+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'142","last_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_deep_scrub":"264'142","last_deep_scrub_stamp":"2024-04-17T05:46:21.761664+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.761664+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:07:22.526188+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":162,"num_read_kb":156,"num_write":110,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"264'533","reported_seq":2314,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.803031+0000","last_change":"2024-04-17T05:46:27.803031+0000","last_active":"2024-04-17T05:46:27.803031+0000","last_peered":"2024-04-17T05:46:27.803031+0000","last_clean":"2024-04-17T05:46:27.803031+0000","last_became_active":"2024-04-17T04:36:33.420252+0000","last_became_peered":"2024-04-17T04:36:33.420252+0000","last_unstale":"2024-04-17T05:46:27.803031+0000","last_undegraded":"2024-04-17T05:46:27.803031+0000","last_fullsized":"2024-04-17T05:46:27.803031+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'533","last_scrub_stamp":"2024-04-17T05:46:27.802603+0000","last_deep_scrub":"264'533","last_deep_scrub_stamp":"2024-04-17T05:46:27.802603+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.802603+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":500,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:56:54.996107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"264'224","reported_seq":750,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.621850+0000","last_change":"2024-04-17T05:46:17.621850+0000","last_active":"2024-04-17T05:46:17.621850+0000","last_peered":"2024-04-17T05:46:17.621850+0000","last_clean":"2024-04-17T05:46:17.621850+0000","last_became_active":"2024-04-17T04:36:35.431024+0000","last_became_peered":"2024-04-17T04:36:35.431024+0000","last_unstale":"2024-04-17T05:46:17.621850+0000","last_undegraded":"2024-04-17T05:46:17.621850+0000","last_fullsized":"2024-04-17T05:46:17.621850+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'224","last_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_deep_scrub":"264'224","last_deep_scrub_stamp":"2024-04-17T05:46:17.621550+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.621550+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:22:57.559950+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":207,"num_write":474,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"264'4094","reported_seq":4735,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.498067+0000","last_change":"2024-04-17T05:46:49.498067+0000","last_active":"2024-04-17T05:46:49.498067+0000","last_peered":"2024-04-17T05:46:49.498067+0000","last_clean":"2024-04-17T05:46:49.498067+0000","last_became_active":"2024-04-17T04:36:27.377547+0000","last_became_peered":"2024-04-17T04:36:27.377547+0000","last_unstale":"2024-04-17T05:46:49.498067+0000","last_undegraded":"2024-04-17T05:46:49.498067+0000","last_fullsized":"2024-04-17T05:46:49.498067+0000","mapping_epoch":25,"log_start":"264'4000","ondisk_log_start":"264'4000","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4094","last_scrub_stamp":"2024-04-17T05:46:49.497699+0000","last_deep_scrub":"264'4094","last_deep_scrub_stamp":"2024-04-17T05:46:49.497699+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.497699+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:26:37.910397+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14437,"num_read_kb":11804,"num_write":6125,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"264'356","reported_seq":982,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:58.582947+0000","last_change":"2024-04-17T05:46:58.583154+0000","last_active":"2024-04-17T05:46:58.582947+0000","last_peered":"2024-04-17T05:46:58.582947+0000","last_clean":"2024-04-17T05:46:58.582947+0000","last_became_active":"2024-04-17T04:36:25.354272+0000","last_became_peered":"2024-04-17T04:36:25.354272+0000","last_unstale":"2024-04-17T05:46:58.582947+0000","last_undegraded":"2024-04-17T05:46:58.582947+0000","last_fullsized":"2024-04-17T05:46:58.582947+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:58.582703+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:58.582703+0000","last_clean_scrub_stamp":"2024-04-17T05:46:58.582703+0000","objects_scrubbed":7,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:52:04.025146+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145969,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":466,"num_read_kb":7519,"num_write":1284,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":528,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.882687+0000","last_change":"2024-04-17T05:46:31.882747+0000","last_active":"2024-04-17T05:46:31.882687+0000","last_peered":"2024-04-17T05:46:31.882687+0000","last_clean":"2024-04-17T05:46:31.882687+0000","last_became_active":"2024-04-17T04:36:17.217019+0000","last_became_peered":"2024-04-17T04:36:17.217019+0000","last_unstale":"2024-04-17T05:46:31.882687+0000","last_undegraded":"2024-04-17T05:46:31.882687+0000","last_fullsized":"2024-04-17T05:46:31.882687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.882477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.882477+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.882477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:06:43.637695+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":518,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.758092+0000","last_change":"2024-04-17T05:46:50.758190+0000","last_active":"2024-04-17T05:46:50.758092+0000","last_peered":"2024-04-17T05:46:50.758092+0000","last_clean":"2024-04-17T05:46:50.758092+0000","last_became_active":"2024-04-17T04:36:29.362345+0000","last_became_peered":"2024-04-17T04:36:29.362345+0000","last_unstale":"2024-04-17T05:46:50.758092+0000","last_undegraded":"2024-04-17T05:46:50.758092+0000","last_fullsized":"2024-04-17T05:46:50.758092+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:50.757743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:50.757743+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.757743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:15.565878+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"264'739","reported_seq":1493,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:46.240287+0000","last_change":"2024-04-17T05:46:46.240287+0000","last_active":"2024-04-17T05:46:46.240287+0000","last_peered":"2024-04-17T05:46:46.240287+0000","last_clean":"2024-04-17T05:46:46.240287+0000","last_became_active":"2024-04-17T04:36:31.421826+0000","last_became_peered":"2024-04-17T04:36:31.421826+0000","last_unstale":"2024-04-17T05:46:46.240287+0000","last_undegraded":"2024-04-17T05:46:46.240287+0000","last_fullsized":"2024-04-17T05:46:46.240287+0000","mapping_epoch":29,"log_start":"264'700","ondisk_log_start":"264'700","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'739","last_scrub_stamp":"2024-04-17T05:46:46.239975+0000","last_deep_scrub":"264'739","last_deep_scrub_stamp":"2024-04-17T05:46:46.239975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:46.239975+0000","objects_scrubbed":9,"log_size":39,"log_dups_size":700,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:16:48.081136+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138854,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1296,"num_read_kb":3065,"num_write":1038,"num_write_kb":823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.854181+0000","last_change":"2024-04-17T05:46:31.854181+0000","last_active":"2024-04-17T05:46:31.854181+0000","last_peered":"2024-04-17T05:46:31.854181+0000","last_clean":"2024-04-17T05:46:31.854181+0000","last_became_active":"2024-04-17T04:36:33.413510+0000","last_became_peered":"2024-04-17T04:36:33.413510+0000","last_unstale":"2024-04-17T05:46:31.854181+0000","last_undegraded":"2024-04-17T05:46:31.854181+0000","last_fullsized":"2024-04-17T05:46:31.854181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:31.853942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:31.853942+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.853942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:11.976239+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"264'105","reported_seq":604,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:13.069586+0000","last_change":"2024-04-17T05:46:13.069586+0000","last_active":"2024-04-17T05:46:13.069586+0000","last_peered":"2024-04-17T05:46:13.069586+0000","last_clean":"2024-04-17T05:46:13.069586+0000","last_became_active":"2024-04-17T04:36:35.424019+0000","last_became_peered":"2024-04-17T04:36:35.424019+0000","last_unstale":"2024-04-17T05:46:13.069586+0000","last_undegraded":"2024-04-17T05:46:13.069586+0000","last_fullsized":"2024-04-17T05:46:13.069586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:13.069188+0000","last_clean_scrub_stamp":"2024-04-17T05:46:13.069188+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:53:06.656617+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":85,"num_write":247,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"264'151","reported_seq":728,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.648424+0000","last_change":"2024-04-17T05:46:24.648424+0000","last_active":"2024-04-17T05:46:24.648424+0000","last_peered":"2024-04-17T05:46:24.648424+0000","last_clean":"2024-04-17T05:46:24.648424+0000","last_became_active":"2024-04-17T04:36:35.424867+0000","last_became_peered":"2024-04-17T04:36:35.424867+0000","last_unstale":"2024-04-17T05:46:24.648424+0000","last_undegraded":"2024-04-17T05:46:24.648424+0000","last_fullsized":"2024-04-17T05:46:24.648424+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'151","last_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_deep_scrub":"264'151","last_deep_scrub_stamp":"2024-04-17T05:46:24.648141+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.648141+0000","objects_scrubbed":2,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:48.144202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":254,"num_read_kb":242,"num_write":376,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"264'74","reported_seq":685,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.642292+0000","last_change":"2024-04-17T05:46:30.642444+0000","last_active":"2024-04-17T05:46:30.642292+0000","last_peered":"2024-04-17T05:46:30.642292+0000","last_clean":"2024-04-17T05:46:30.642292+0000","last_became_active":"2024-04-17T04:36:27.399758+0000","last_became_peered":"2024-04-17T04:36:27.399758+0000","last_unstale":"2024-04-17T05:46:30.642292+0000","last_undegraded":"2024-04-17T05:46:30.642292+0000","last_fullsized":"2024-04-17T05:46:30.642292+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'74","last_scrub_stamp":"2024-04-17T05:46:30.642053+0000","last_deep_scrub":"264'74","last_deep_scrub_stamp":"2024-04-17T05:46:30.642053+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.642053+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:04.769101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":277,"num_write":100,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"264'281","reported_seq":929,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.540711+0000","last_change":"2024-04-17T05:46:45.540862+0000","last_active":"2024-04-17T05:46:45.540711+0000","last_peered":"2024-04-17T05:46:45.540711+0000","last_clean":"2024-04-17T05:46:45.540711+0000","last_became_active":"2024-04-17T04:36:25.347223+0000","last_became_peered":"2024-04-17T04:36:25.347223+0000","last_unstale":"2024-04-17T05:46:45.540711+0000","last_undegraded":"2024-04-17T05:46:45.540711+0000","last_fullsized":"2024-04-17T05:46:45.540711+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:45.540487+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:45.540487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.540487+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:58.652080+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1017984,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":431,"num_read_kb":11105,"num_write":979,"num_write_kb":17177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:22.773169+0000","last_change":"2024-04-17T05:46:22.773169+0000","last_active":"2024-04-17T05:46:22.773169+0000","last_peered":"2024-04-17T05:46:22.773169+0000","last_clean":"2024-04-17T05:46:22.773169+0000","last_became_active":"2024-04-17T04:36:29.391036+0000","last_became_peered":"2024-04-17T04:36:29.391036+0000","last_unstale":"2024-04-17T05:46:22.773169+0000","last_undegraded":"2024-04-17T05:46:22.773169+0000","last_fullsized":"2024-04-17T05:46:22.773169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:22.772889+0000","last_clean_scrub_stamp":"2024-04-17T05:46:22.772889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:47:13.164944+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"264'307","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.772343+0000","last_change":"2024-04-17T05:46:36.772420+0000","last_active":"2024-04-17T05:46:36.772343+0000","last_peered":"2024-04-17T05:46:36.772343+0000","last_clean":"2024-04-17T05:46:36.772343+0000","last_became_active":"2024-04-17T04:36:31.406068+0000","last_became_peered":"2024-04-17T04:36:31.406068+0000","last_unstale":"2024-04-17T05:46:36.772343+0000","last_undegraded":"2024-04-17T05:46:36.772343+0000","last_fullsized":"2024-04-17T05:46:36.772343+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'307","last_scrub_stamp":"2024-04-17T05:46:36.772164+0000","last_deep_scrub":"264'307","last_deep_scrub_stamp":"2024-04-17T05:46:36.772164+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.772164+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:26:09.701358+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44026,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":524,"num_read_kb":1245,"num_write":424,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.845796+0000","last_change":"2024-04-17T05:46:28.845796+0000","last_active":"2024-04-17T05:46:28.845796+0000","last_peered":"2024-04-17T05:46:28.845796+0000","last_clean":"2024-04-17T05:46:28.845796+0000","last_became_active":"2024-04-17T04:36:33.418652+0000","last_became_peered":"2024-04-17T04:36:33.418652+0000","last_unstale":"2024-04-17T05:46:28.845796+0000","last_undegraded":"2024-04-17T05:46:28.845796+0000","last_fullsized":"2024-04-17T05:46:28.845796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:28.845537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:28.845537+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.845537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:25:28.366296+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"264'99","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.795125+0000","last_change":"2024-04-17T05:46:17.795125+0000","last_active":"2024-04-17T05:46:17.795125+0000","last_peered":"2024-04-17T05:46:17.795125+0000","last_clean":"2024-04-17T05:46:17.795125+0000","last_became_active":"2024-04-17T04:36:35.429528+0000","last_became_peered":"2024-04-17T04:36:35.429528+0000","last_unstale":"2024-04-17T05:46:17.795125+0000","last_undegraded":"2024-04-17T05:46:17.795125+0000","last_fullsized":"2024-04-17T05:46:17.795125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'99","last_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_deep_scrub":"264'99","last_deep_scrub_stamp":"2024-04-17T05:46:17.794856+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.794856+0000","objects_scrubbed":1,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:12:18.080437+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":93,"num_write":226,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"264'127","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.804194+0000","last_change":"2024-04-17T05:46:23.804194+0000","last_active":"2024-04-17T05:46:23.804194+0000","last_peered":"2024-04-17T05:46:23.804194+0000","last_clean":"2024-04-17T05:46:23.804194+0000","last_became_active":"2024-04-17T04:36:27.388806+0000","last_became_peered":"2024-04-17T04:36:27.388806+0000","last_unstale":"2024-04-17T05:46:23.804194+0000","last_undegraded":"2024-04-17T05:46:23.804194+0000","last_fullsized":"2024-04-17T05:46:23.804194+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'127","last_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_deep_scrub":"264'127","last_deep_scrub_stamp":"2024-04-17T05:46:23.803934+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.803934+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:58:22.971464+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":191,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"264'373","reported_seq":1035,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:12.684055+0000","last_change":"2024-04-17T05:46:12.684055+0000","last_active":"2024-04-17T05:46:12.684055+0000","last_peered":"2024-04-17T05:46:12.684055+0000","last_clean":"2024-04-17T05:46:12.684055+0000","last_became_active":"2024-04-17T04:37:03.635084+0000","last_became_peered":"2024-04-17T04:37:03.635084+0000","last_unstale":"2024-04-17T05:46:12.684055+0000","last_undegraded":"2024-04-17T05:46:12.684055+0000","last_fullsized":"2024-04-17T05:46:12.684055+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'373","last_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_deep_scrub":"264'373","last_deep_scrub_stamp":"2024-04-17T05:46:12.683727+0000","last_clean_scrub_stamp":"2024-04-17T05:46:12.683727+0000","objects_scrubbed":6,"log_size":73,"log_dups_size":300,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:21:58.133653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":625,"num_read_kb":15946,"num_write":1543,"num_write_kb":26713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,4],"acting":[2,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:52.697466+0000","last_change":"2024-04-17T05:46:52.697520+0000","last_active":"2024-04-17T05:46:52.697466+0000","last_peered":"2024-04-17T05:46:52.697466+0000","last_clean":"2024-04-17T05:46:52.697466+0000","last_became_active":"2024-04-17T04:36:29.370936+0000","last_became_peered":"2024-04-17T04:36:29.370936+0000","last_unstale":"2024-04-17T05:46:52.697466+0000","last_undegraded":"2024-04-17T05:46:52.697466+0000","last_fullsized":"2024-04-17T05:46:52.697466+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:52.697265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:52.697265+0000","last_clean_scrub_stamp":"2024-04-17T05:46:52.697265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:56.004700+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"264'59","reported_seq":580,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.558443+0000","last_change":"2024-04-17T05:46:51.558538+0000","last_active":"2024-04-17T05:46:51.558443+0000","last_peered":"2024-04-17T05:46:51.558443+0000","last_clean":"2024-04-17T05:46:51.558443+0000","last_became_active":"2024-04-17T04:36:31.403577+0000","last_became_peered":"2024-04-17T04:36:31.403577+0000","last_unstale":"2024-04-17T05:46:51.558443+0000","last_undegraded":"2024-04-17T05:46:51.558443+0000","last_fullsized":"2024-04-17T05:46:51.558443+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'59","last_scrub_stamp":"2024-04-17T05:46:51.558239+0000","last_deep_scrub":"264'59","last_deep_scrub_stamp":"2024-04-17T05:46:51.558239+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.558239+0000","objects_scrubbed":3,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:04:08.889728+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":42,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":500,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.325328+0000","last_change":"2024-04-17T05:46:33.325328+0000","last_active":"2024-04-17T05:46:33.325328+0000","last_peered":"2024-04-17T05:46:33.325328+0000","last_clean":"2024-04-17T05:46:33.325328+0000","last_became_active":"2024-04-17T04:36:33.415067+0000","last_became_peered":"2024-04-17T04:36:33.415067+0000","last_unstale":"2024-04-17T05:46:33.325328+0000","last_undegraded":"2024-04-17T05:46:33.325328+0000","last_fullsized":"2024-04-17T05:46:33.325328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:33.325058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:33.325058+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.325058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:24.544642+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"264'219","reported_seq":751,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.647715+0000","last_change":"2024-04-17T05:46:20.647715+0000","last_active":"2024-04-17T05:46:20.647715+0000","last_peered":"2024-04-17T05:46:20.647715+0000","last_clean":"2024-04-17T05:46:20.647715+0000","last_became_active":"2024-04-17T04:36:35.427673+0000","last_became_peered":"2024-04-17T04:36:35.427673+0000","last_unstale":"2024-04-17T05:46:20.647715+0000","last_undegraded":"2024-04-17T05:46:20.647715+0000","last_fullsized":"2024-04-17T05:46:20.647715+0000","mapping_epoch":33,"log_start":"264'200","ondisk_log_start":"264'200","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'219","last_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_deep_scrub":"264'219","last_deep_scrub_stamp":"2024-04-17T05:46:20.647446+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.647446+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:41:57.324481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":306,"num_read_kb":298,"num_write":481,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"264'196","reported_seq":808,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.733738+0000","last_change":"2024-04-17T05:46:40.733801+0000","last_active":"2024-04-17T05:46:40.733738+0000","last_peered":"2024-04-17T05:46:40.733738+0000","last_clean":"2024-04-17T05:46:40.733738+0000","last_became_active":"2024-04-17T04:36:27.395076+0000","last_became_peered":"2024-04-17T04:36:27.395076+0000","last_unstale":"2024-04-17T05:46:40.733738+0000","last_undegraded":"2024-04-17T05:46:40.733738+0000","last_fullsized":"2024-04-17T05:46:40.733738+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'196","last_scrub_stamp":"2024-04-17T05:46:40.733533+0000","last_deep_scrub":"264'196","last_deep_scrub_stamp":"2024-04-17T05:46:40.733533+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.733533+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:56:04.953287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":767,"num_read_kb":603,"num_write":297,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"264'356","reported_seq":1045,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.248025+0000","last_change":"2024-04-17T05:46:44.248263+0000","last_active":"2024-04-17T05:46:44.248025+0000","last_peered":"2024-04-17T05:46:44.248025+0000","last_clean":"2024-04-17T05:46:44.248025+0000","last_became_active":"2024-04-17T04:36:25.337833+0000","last_became_peered":"2024-04-17T04:36:25.337833+0000","last_unstale":"2024-04-17T05:46:44.248025+0000","last_undegraded":"2024-04-17T05:46:44.248025+0000","last_fullsized":"2024-04-17T05:46:44.248025+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'356","last_scrub_stamp":"2024-04-17T05:46:44.247646+0000","last_deep_scrub":"264'356","last_deep_scrub_stamp":"2024-04-17T05:46:44.247646+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.247646+0000","objects_scrubbed":6,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:23:11.550314+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":588,"num_read_kb":11343,"num_write":1303,"num_write_kb":31806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":511,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.358946+0000","last_change":"2024-04-17T05:46:34.359052+0000","last_active":"2024-04-17T05:46:34.358946+0000","last_peered":"2024-04-17T05:46:34.358946+0000","last_clean":"2024-04-17T05:46:34.358946+0000","last_became_active":"2024-04-17T04:36:29.380408+0000","last_became_peered":"2024-04-17T04:36:29.380408+0000","last_unstale":"2024-04-17T05:46:34.358946+0000","last_undegraded":"2024-04-17T05:46:34.358946+0000","last_fullsized":"2024-04-17T05:46:34.358946+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:34.358700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:34.358700+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.358700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:56:23.819994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"264'345","reported_seq":960,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.793714+0000","last_change":"2024-04-17T05:46:35.793792+0000","last_active":"2024-04-17T05:46:35.793714+0000","last_peered":"2024-04-17T05:46:35.793714+0000","last_clean":"2024-04-17T05:46:35.793714+0000","last_became_active":"2024-04-17T04:36:31.406262+0000","last_became_peered":"2024-04-17T04:36:31.406262+0000","last_unstale":"2024-04-17T05:46:35.793714+0000","last_undegraded":"2024-04-17T05:46:35.793714+0000","last_fullsized":"2024-04-17T05:46:35.793714+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'345","last_scrub_stamp":"2024-04-17T05:46:35.793519+0000","last_deep_scrub":"264'345","last_deep_scrub_stamp":"2024-04-17T05:46:35.793519+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.793519+0000","objects_scrubbed":5,"log_size":45,"log_dups_size":300,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:11:19.755889+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48000,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":552,"num_read_kb":1237,"num_write":440,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":500,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:17.983452+0000","last_change":"2024-04-17T05:46:17.983452+0000","last_active":"2024-04-17T05:46:17.983452+0000","last_peered":"2024-04-17T05:46:17.983452+0000","last_clean":"2024-04-17T05:46:17.983452+0000","last_became_active":"2024-04-17T04:36:33.412139+0000","last_became_peered":"2024-04-17T04:36:33.412139+0000","last_unstale":"2024-04-17T05:46:17.983452+0000","last_undegraded":"2024-04-17T05:46:17.983452+0000","last_fullsized":"2024-04-17T05:46:17.983452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:17.983070+0000","last_clean_scrub_stamp":"2024-04-17T05:46:17.983070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:29.782925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"264'113","reported_seq":610,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:14.111907+0000","last_change":"2024-04-17T05:46:14.111907+0000","last_active":"2024-04-17T05:46:14.111907+0000","last_peered":"2024-04-17T05:46:14.111907+0000","last_clean":"2024-04-17T05:46:14.111907+0000","last_became_active":"2024-04-17T04:36:35.423815+0000","last_became_peered":"2024-04-17T04:36:35.423815+0000","last_unstale":"2024-04-17T05:46:14.111907+0000","last_undegraded":"2024-04-17T05:46:14.111907+0000","last_fullsized":"2024-04-17T05:46:14.111907+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'113","last_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_deep_scrub":"264'113","last_deep_scrub_stamp":"2024-04-17T05:46:14.111624+0000","last_clean_scrub_stamp":"2024-04-17T05:46:14.111624+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:33:35.374698+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":88,"num_read_kb":85,"num_write":258,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"264'132","reported_seq":769,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.750706+0000","last_change":"2024-04-17T05:46:49.750765+0000","last_active":"2024-04-17T05:46:49.750706+0000","last_peered":"2024-04-17T05:46:49.750706+0000","last_clean":"2024-04-17T05:46:49.750706+0000","last_became_active":"2024-04-17T04:36:27.372636+0000","last_became_peered":"2024-04-17T04:36:27.372636+0000","last_unstale":"2024-04-17T05:46:49.750706+0000","last_undegraded":"2024-04-17T05:46:49.750706+0000","last_fullsized":"2024-04-17T05:46:49.750706+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'132","last_scrub_stamp":"2024-04-17T05:46:49.750514+0000","last_deep_scrub":"264'132","last_deep_scrub_stamp":"2024-04-17T05:46:49.750514+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.750514+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:01:14.989520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":471,"num_write":195,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"264'310","reported_seq":968,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.608799+0000","last_change":"2024-04-17T05:46:56.608932+0000","last_active":"2024-04-17T05:46:56.608799+0000","last_peered":"2024-04-17T05:46:56.608799+0000","last_clean":"2024-04-17T05:46:56.608799+0000","last_became_active":"2024-04-17T04:36:25.344049+0000","last_became_peered":"2024-04-17T04:36:25.344049+0000","last_unstale":"2024-04-17T05:46:56.608799+0000","last_undegraded":"2024-04-17T05:46:56.608799+0000","last_fullsized":"2024-04-17T05:46:56.608799+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'310","last_scrub_stamp":"2024-04-17T05:46:56.608557+0000","last_deep_scrub":"264'310","last_deep_scrub_stamp":"2024-04-17T05:46:56.608557+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.608557+0000","objects_scrubbed":9,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:29.339994+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12846794,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":445,"num_read_kb":84514,"num_write":990,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":508,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.942372+0000","last_change":"2024-04-17T05:46:16.942372+0000","last_active":"2024-04-17T05:46:16.942372+0000","last_peered":"2024-04-17T05:46:16.942372+0000","last_clean":"2024-04-17T05:46:16.942372+0000","last_became_active":"2024-04-17T04:36:29.377198+0000","last_became_peered":"2024-04-17T04:36:29.377198+0000","last_unstale":"2024-04-17T05:46:16.942372+0000","last_undegraded":"2024-04-17T05:46:16.942372+0000","last_fullsized":"2024-04-17T05:46:16.942372+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:16.942121+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.942121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:06:13.925928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"264'262","reported_seq":852,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.356250+0000","last_change":"2024-04-17T05:46:40.356321+0000","last_active":"2024-04-17T05:46:40.356250+0000","last_peered":"2024-04-17T05:46:40.356250+0000","last_clean":"2024-04-17T05:46:40.356250+0000","last_became_active":"2024-04-17T04:36:31.379568+0000","last_became_peered":"2024-04-17T04:36:31.379568+0000","last_unstale":"2024-04-17T05:46:40.356250+0000","last_undegraded":"2024-04-17T05:46:40.356250+0000","last_fullsized":"2024-04-17T05:46:40.356250+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'262","last_scrub_stamp":"2024-04-17T05:46:40.356034+0000","last_deep_scrub":"264'262","last_deep_scrub_stamp":"2024-04-17T05:46:40.356034+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.356034+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:11.830162+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47325,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":432,"num_read_kb":996,"num_write":342,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":503,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:30.631115+0000","last_change":"2024-04-17T05:46:30.631115+0000","last_active":"2024-04-17T05:46:30.631115+0000","last_peered":"2024-04-17T05:46:30.631115+0000","last_clean":"2024-04-17T05:46:30.631115+0000","last_became_active":"2024-04-17T04:36:33.426817+0000","last_became_peered":"2024-04-17T04:36:33.426817+0000","last_unstale":"2024-04-17T05:46:30.631115+0000","last_undegraded":"2024-04-17T05:46:30.631115+0000","last_fullsized":"2024-04-17T05:46:30.631115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:30.630868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:30.630868+0000","last_clean_scrub_stamp":"2024-04-17T05:46:30.630868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:07:08.526024+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"264'290","reported_seq":927,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.020277+0000","last_change":"2024-04-17T05:46:23.020277+0000","last_active":"2024-04-17T05:46:23.020277+0000","last_peered":"2024-04-17T05:46:23.020277+0000","last_clean":"2024-04-17T05:46:23.020277+0000","last_became_active":"2024-04-17T04:36:27.386819+0000","last_became_peered":"2024-04-17T04:36:27.386819+0000","last_unstale":"2024-04-17T05:46:23.020277+0000","last_undegraded":"2024-04-17T05:46:23.020277+0000","last_fullsized":"2024-04-17T05:46:23.020277+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'290","last_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_deep_scrub":"264'290","last_deep_scrub_stamp":"2024-04-17T05:46:23.019975+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.019975+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:18:21.304396+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1179,"num_read_kb":983,"num_write":467,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"264'313","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:56.620802+0000","last_change":"2024-04-17T05:46:56.620978+0000","last_active":"2024-04-17T05:46:56.620802+0000","last_peered":"2024-04-17T05:46:56.620802+0000","last_clean":"2024-04-17T05:46:56.620802+0000","last_became_active":"2024-04-17T04:36:25.326154+0000","last_became_peered":"2024-04-17T04:36:25.326154+0000","last_unstale":"2024-04-17T05:46:56.620802+0000","last_undegraded":"2024-04-17T05:46:56.620802+0000","last_fullsized":"2024-04-17T05:46:56.620802+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'313","last_scrub_stamp":"2024-04-17T05:46:56.620571+0000","last_deep_scrub":"264'313","last_deep_scrub_stamp":"2024-04-17T05:46:56.620571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:56.620571+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:43.525135+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752753,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":542,"num_read_kb":33053,"num_write":1174,"num_write_kb":24154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":516,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.615906+0000","last_change":"2024-04-17T05:46:45.615960+0000","last_active":"2024-04-17T05:46:45.615906+0000","last_peered":"2024-04-17T05:46:45.615906+0000","last_clean":"2024-04-17T05:46:45.615906+0000","last_became_active":"2024-04-17T04:36:29.372490+0000","last_became_peered":"2024-04-17T04:36:29.372490+0000","last_unstale":"2024-04-17T05:46:45.615906+0000","last_undegraded":"2024-04-17T05:46:45.615906+0000","last_fullsized":"2024-04-17T05:46:45.615906+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:45.615722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:45.615722+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.615722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:59:21.102882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"264'480","reported_seq":1141,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:24.798222+0000","last_change":"2024-04-17T05:46:24.798222+0000","last_active":"2024-04-17T05:46:24.798222+0000","last_peered":"2024-04-17T05:46:24.798222+0000","last_clean":"2024-04-17T05:46:24.798222+0000","last_became_active":"2024-04-17T04:36:31.396973+0000","last_became_peered":"2024-04-17T04:36:31.396973+0000","last_unstale":"2024-04-17T05:46:24.798222+0000","last_undegraded":"2024-04-17T05:46:24.798222+0000","last_fullsized":"2024-04-17T05:46:24.798222+0000","mapping_epoch":29,"log_start":"264'400","ondisk_log_start":"264'400","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'480","last_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_deep_scrub":"264'480","last_deep_scrub_stamp":"2024-04-17T05:46:24.797947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:24.797947+0000","objects_scrubbed":7,"log_size":80,"log_dups_size":400,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:33:12.511479+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87094,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":846,"num_read_kb":2027,"num_write":720,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22936,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.147623+0000","last_change":"2024-04-17T05:46:19.147623+0000","last_active":"2024-04-17T05:46:19.147623+0000","last_peered":"2024-04-17T05:46:19.147623+0000","last_clean":"2024-04-17T05:46:19.147623+0000","last_became_active":"2024-04-17T04:36:33.418057+0000","last_became_peered":"2024-04-17T04:36:33.418057+0000","last_unstale":"2024-04-17T05:46:19.147623+0000","last_undegraded":"2024-04-17T05:46:19.147623+0000","last_fullsized":"2024-04-17T05:46:19.147623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.147370+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.147370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:09:18.842204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"264'174","reported_seq":691,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.787327+0000","last_change":"2024-04-17T05:46:21.787327+0000","last_active":"2024-04-17T05:46:21.787327+0000","last_peered":"2024-04-17T05:46:21.787327+0000","last_clean":"2024-04-17T05:46:21.787327+0000","last_became_active":"2024-04-17T04:36:35.418211+0000","last_became_peered":"2024-04-17T04:36:35.418211+0000","last_unstale":"2024-04-17T05:46:21.787327+0000","last_undegraded":"2024-04-17T05:46:21.787327+0000","last_fullsized":"2024-04-17T05:46:21.787327+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'174","last_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_deep_scrub":"264'174","last_deep_scrub_stamp":"2024-04-17T05:46:21.787059+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.787059+0000","objects_scrubbed":3,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:11.126921+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":214,"num_write":394,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"264'104","reported_seq":719,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.695888+0000","last_change":"2024-04-17T05:46:41.696009+0000","last_active":"2024-04-17T05:46:41.695888+0000","last_peered":"2024-04-17T05:46:41.695888+0000","last_clean":"2024-04-17T05:46:41.695888+0000","last_became_active":"2024-04-17T04:36:27.384950+0000","last_became_peered":"2024-04-17T04:36:27.384950+0000","last_unstale":"2024-04-17T05:46:41.695888+0000","last_undegraded":"2024-04-17T05:46:41.695888+0000","last_fullsized":"2024-04-17T05:46:41.695888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'104","last_scrub_stamp":"2024-04-17T05:46:41.695676+0000","last_deep_scrub":"264'104","last_deep_scrub_stamp":"2024-04-17T05:46:41.695676+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.695676+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:56:52.077069+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":467,"num_read_kb":388,"num_write":153,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"264'315","reported_seq":1006,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.805432+0000","last_change":"2024-04-17T05:46:40.805432+0000","last_active":"2024-04-17T05:46:40.805432+0000","last_peered":"2024-04-17T05:46:40.805432+0000","last_clean":"2024-04-17T05:46:40.805432+0000","last_became_active":"2024-04-17T04:36:25.353212+0000","last_became_peered":"2024-04-17T04:36:25.353212+0000","last_unstale":"2024-04-17T05:46:40.805432+0000","last_undegraded":"2024-04-17T05:46:40.805432+0000","last_fullsized":"2024-04-17T05:46:40.805432+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'315","last_scrub_stamp":"2024-04-17T05:46:40.805067+0000","last_deep_scrub":"264'315","last_deep_scrub_stamp":"2024-04-17T05:46:40.805067+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.805067+0000","objects_scrubbed":9,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:14:53.427777+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122783,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":510,"num_read_kb":112390,"num_write":1223,"num_write_kb":25855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,6],"acting":[1,3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":512,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.012353+0000","last_change":"2024-04-17T05:46:19.012353+0000","last_active":"2024-04-17T05:46:19.012353+0000","last_peered":"2024-04-17T05:46:19.012353+0000","last_clean":"2024-04-17T05:46:19.012353+0000","last_became_active":"2024-04-17T04:36:29.375124+0000","last_became_peered":"2024-04-17T04:36:29.375124+0000","last_unstale":"2024-04-17T05:46:19.012353+0000","last_undegraded":"2024-04-17T05:46:19.012353+0000","last_fullsized":"2024-04-17T05:46:19.012353+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:19.012071+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.012071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:04:43.685517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"264'323","reported_seq":932,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.261454+0000","last_change":"2024-04-17T05:46:44.261542+0000","last_active":"2024-04-17T05:46:44.261454+0000","last_peered":"2024-04-17T05:46:44.261454+0000","last_clean":"2024-04-17T05:46:44.261454+0000","last_became_active":"2024-04-17T04:36:31.402071+0000","last_became_peered":"2024-04-17T04:36:31.402071+0000","last_unstale":"2024-04-17T05:46:44.261454+0000","last_undegraded":"2024-04-17T05:46:44.261454+0000","last_fullsized":"2024-04-17T05:46:44.261454+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'323","last_scrub_stamp":"2024-04-17T05:46:44.261253+0000","last_deep_scrub":"264'323","last_deep_scrub_stamp":"2024-04-17T05:46:44.261253+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.261253+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:46:24.010153+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40265,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":1168,"num_write":442,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10932,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":506,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.856669+0000","last_change":"2024-04-17T05:46:42.856727+0000","last_active":"2024-04-17T05:46:42.856669+0000","last_peered":"2024-04-17T05:46:42.856669+0000","last_clean":"2024-04-17T05:46:42.856669+0000","last_became_active":"2024-04-17T04:36:33.413567+0000","last_became_peered":"2024-04-17T04:36:33.413567+0000","last_unstale":"2024-04-17T05:46:42.856669+0000","last_undegraded":"2024-04-17T05:46:42.856669+0000","last_fullsized":"2024-04-17T05:46:42.856669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:42.856474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:42.856474+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.856474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:20:08.791141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"264'180","reported_seq":726,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.904163+0000","last_change":"2024-04-17T05:46:39.904297+0000","last_active":"2024-04-17T05:46:39.904163+0000","last_peered":"2024-04-17T05:46:39.904163+0000","last_clean":"2024-04-17T05:46:39.904163+0000","last_became_active":"2024-04-17T04:36:35.423475+0000","last_became_peered":"2024-04-17T04:36:35.423475+0000","last_unstale":"2024-04-17T05:46:39.904163+0000","last_undegraded":"2024-04-17T05:46:39.904163+0000","last_fullsized":"2024-04-17T05:46:39.904163+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'180","last_scrub_stamp":"2024-04-17T05:46:39.903938+0000","last_deep_scrub":"264'180","last_deep_scrub_stamp":"2024-04-17T05:46:39.903938+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.903938+0000","objects_scrubbed":5,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:08:36.200762+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1619,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":321,"num_read_kb":289,"num_write":496,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"264'115","reported_seq":695,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:51.716735+0000","last_change":"2024-04-17T05:46:51.716821+0000","last_active":"2024-04-17T05:46:51.716735+0000","last_peered":"2024-04-17T05:46:51.716735+0000","last_clean":"2024-04-17T05:46:51.716735+0000","last_became_active":"2024-04-17T04:36:27.377493+0000","last_became_peered":"2024-04-17T04:36:27.377493+0000","last_unstale":"2024-04-17T05:46:51.716735+0000","last_undegraded":"2024-04-17T05:46:51.716735+0000","last_fullsized":"2024-04-17T05:46:51.716735+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'115","last_scrub_stamp":"2024-04-17T05:46:51.716517+0000","last_deep_scrub":"264'115","last_deep_scrub_stamp":"2024-04-17T05:46:51.716517+0000","last_clean_scrub_stamp":"2024-04-17T05:46:51.716517+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":100,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:00:30.417919+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":445,"num_read_kb":356,"num_write":161,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"264'358","reported_seq":1052,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.580780+0000","last_change":"2024-04-17T05:46:44.580927+0000","last_active":"2024-04-17T05:46:44.580780+0000","last_peered":"2024-04-17T05:46:44.580780+0000","last_clean":"2024-04-17T05:46:44.580780+0000","last_became_active":"2024-04-17T04:36:25.355355+0000","last_became_peered":"2024-04-17T04:36:25.355355+0000","last_unstale":"2024-04-17T05:46:44.580780+0000","last_undegraded":"2024-04-17T05:46:44.580780+0000","last_fullsized":"2024-04-17T05:46:44.580780+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'358","last_scrub_stamp":"2024-04-17T05:46:44.580574+0000","last_deep_scrub":"264'358","last_deep_scrub_stamp":"2024-04-17T05:46:44.580574+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.580574+0000","objects_scrubbed":8,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:46:15.221508+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568179,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":576,"num_read_kb":10795,"num_write":1350,"num_write_kb":24962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"264'4","reported_seq":844,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.630354+0000","last_change":"2024-04-17T05:46:36.630422+0000","last_active":"2024-04-17T05:46:36.630354+0000","last_peered":"2024-04-17T05:46:36.630354+0000","last_clean":"2024-04-17T05:46:36.630354+0000","last_became_active":"2024-04-17T04:36:29.369449+0000","last_became_peered":"2024-04-17T04:36:29.369449+0000","last_unstale":"2024-04-17T05:46:36.630354+0000","last_undegraded":"2024-04-17T05:46:36.630354+0000","last_fullsized":"2024-04-17T05:46:36.630354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'4","last_scrub_stamp":"2024-04-17T05:46:36.630154+0000","last_deep_scrub":"264'4","last_deep_scrub_stamp":"2024-04-17T05:46:36.630154+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.630154+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:28.761949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":499,"num_read_kb":497,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"264'304","reported_seq":913,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.834435+0000","last_change":"2024-04-17T05:46:32.834435+0000","last_active":"2024-04-17T05:46:32.834435+0000","last_peered":"2024-04-17T05:46:32.834435+0000","last_clean":"2024-04-17T05:46:32.834435+0000","last_became_active":"2024-04-17T04:36:31.420461+0000","last_became_peered":"2024-04-17T04:36:31.420461+0000","last_unstale":"2024-04-17T05:46:32.834435+0000","last_undegraded":"2024-04-17T05:46:32.834435+0000","last_fullsized":"2024-04-17T05:46:32.834435+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'304","last_scrub_stamp":"2024-04-17T05:46:32.834034+0000","last_deep_scrub":"264'304","last_deep_scrub_stamp":"2024-04-17T05:46:32.834034+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.834034+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:51:37.797882+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45015,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":503,"num_read_kb":1173,"num_write":404,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":513,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.782872+0000","last_change":"2024-04-17T05:46:32.782872+0000","last_active":"2024-04-17T05:46:32.782872+0000","last_peered":"2024-04-17T05:46:32.782872+0000","last_clean":"2024-04-17T05:46:32.782872+0000","last_became_active":"2024-04-17T04:36:33.414384+0000","last_became_peered":"2024-04-17T04:36:33.414384+0000","last_unstale":"2024-04-17T05:46:32.782872+0000","last_undegraded":"2024-04-17T05:46:32.782872+0000","last_fullsized":"2024-04-17T05:46:32.782872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:32.782613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:32.782613+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.782613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:22:08.388670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"264'105","reported_seq":614,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:11.146585+0000","last_change":"2024-04-17T05:46:11.146585+0000","last_active":"2024-04-17T05:46:11.146585+0000","last_peered":"2024-04-17T05:46:11.146585+0000","last_clean":"2024-04-17T05:46:11.146585+0000","last_became_active":"2024-04-17T04:36:35.423857+0000","last_became_peered":"2024-04-17T04:36:35.423857+0000","last_unstale":"2024-04-17T05:46:11.146585+0000","last_undegraded":"2024-04-17T05:46:11.146585+0000","last_fullsized":"2024-04-17T05:46:11.146585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'105","last_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_deep_scrub":"264'105","last_deep_scrub_stamp":"2024-04-17T05:46:11.146298+0000","last_clean_scrub_stamp":"2024-04-17T05:46:11.146298+0000","objects_scrubbed":3,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:24.914414+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":119,"num_write":258,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"264'84","reported_seq":684,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.440967+0000","last_change":"2024-04-17T05:46:27.440967+0000","last_active":"2024-04-17T05:46:27.440967+0000","last_peered":"2024-04-17T05:46:27.440967+0000","last_clean":"2024-04-17T05:46:27.440967+0000","last_became_active":"2024-04-17T04:36:27.355733+0000","last_became_peered":"2024-04-17T04:36:27.355733+0000","last_unstale":"2024-04-17T05:46:27.440967+0000","last_undegraded":"2024-04-17T05:46:27.440967+0000","last_fullsized":"2024-04-17T05:46:27.440967+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'84","last_scrub_stamp":"2024-04-17T05:46:27.440660+0000","last_deep_scrub":"264'84","last_deep_scrub_stamp":"2024-04-17T05:46:27.440660+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.440660+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:02:26.099591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":293,"num_write":115,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"264'322","reported_seq":1157,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:57.598893+0000","last_change":"2024-04-17T05:46:57.599081+0000","last_active":"2024-04-17T05:46:57.598893+0000","last_peered":"2024-04-17T05:46:57.598893+0000","last_clean":"2024-04-17T05:46:57.598893+0000","last_became_active":"2024-04-17T04:36:25.353437+0000","last_became_peered":"2024-04-17T04:36:25.353437+0000","last_unstale":"2024-04-17T05:46:57.598893+0000","last_undegraded":"2024-04-17T05:46:57.598893+0000","last_fullsized":"2024-04-17T05:46:57.598893+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:57.598571+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:57.598571+0000","last_clean_scrub_stamp":"2024-04-17T05:46:57.598571+0000","objects_scrubbed":3,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:51.547796+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263230,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":844,"num_read_kb":11279,"num_write":1205,"num_write_kb":49509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.722456+0000","last_change":"2024-04-17T05:46:18.722456+0000","last_active":"2024-04-17T05:46:18.722456+0000","last_peered":"2024-04-17T05:46:18.722456+0000","last_clean":"2024-04-17T05:46:18.722456+0000","last_became_active":"2024-04-17T04:36:29.365155+0000","last_became_peered":"2024-04-17T04:36:29.365155+0000","last_unstale":"2024-04-17T05:46:18.722456+0000","last_undegraded":"2024-04-17T05:46:18.722456+0000","last_fullsized":"2024-04-17T05:46:18.722456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.722193+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.722193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:13:41.699090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"264'126","reported_seq":683,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:19.818501+0000","last_change":"2024-04-17T05:46:19.818501+0000","last_active":"2024-04-17T05:46:19.818501+0000","last_peered":"2024-04-17T05:46:19.818501+0000","last_clean":"2024-04-17T05:46:19.818501+0000","last_became_active":"2024-04-17T04:36:31.421719+0000","last_became_peered":"2024-04-17T04:36:31.421719+0000","last_unstale":"2024-04-17T05:46:19.818501+0000","last_undegraded":"2024-04-17T05:46:19.818501+0000","last_fullsized":"2024-04-17T05:46:19.818501+0000","mapping_epoch":29,"log_start":"264'100","ondisk_log_start":"264'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'126","last_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_deep_scrub":"264'126","last_deep_scrub_stamp":"2024-04-17T05:46:19.818199+0000","last_clean_scrub_stamp":"2024-04-17T05:46:19.818199+0000","objects_scrubbed":7,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:31.924099+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":127,"num_read_kb":117,"num_write":94,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1648,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"264'539","reported_seq":2324,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:15.961007+0000","last_change":"2024-04-17T05:46:15.961007+0000","last_active":"2024-04-17T05:46:15.961007+0000","last_peered":"2024-04-17T05:46:15.961007+0000","last_clean":"2024-04-17T05:46:15.961007+0000","last_became_active":"2024-04-17T04:36:33.403454+0000","last_became_peered":"2024-04-17T04:36:33.403454+0000","last_unstale":"2024-04-17T05:46:15.961007+0000","last_undegraded":"2024-04-17T05:46:15.961007+0000","last_fullsized":"2024-04-17T05:46:15.961007+0000","mapping_epoch":31,"log_start":"264'500","ondisk_log_start":"264'500","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'539","last_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_deep_scrub":"264'539","last_deep_scrub_stamp":"2024-04-17T05:46:15.960604+0000","last_clean_scrub_stamp":"2024-04-17T05:46:15.960604+0000","objects_scrubbed":1,"log_size":39,"log_dups_size":500,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:37:11.021591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"264'116","reported_seq":626,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.290621+0000","last_change":"2024-04-17T05:46:27.290621+0000","last_active":"2024-04-17T05:46:27.290621+0000","last_peered":"2024-04-17T05:46:27.290621+0000","last_clean":"2024-04-17T05:46:27.290621+0000","last_became_active":"2024-04-17T04:36:35.412875+0000","last_became_peered":"2024-04-17T04:36:35.412875+0000","last_unstale":"2024-04-17T05:46:27.290621+0000","last_undegraded":"2024-04-17T05:46:27.290621+0000","last_fullsized":"2024-04-17T05:46:27.290621+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'116","last_scrub_stamp":"2024-04-17T05:46:27.290353+0000","last_deep_scrub":"264'116","last_deep_scrub_stamp":"2024-04-17T05:46:27.290353+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.290353+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:29.531238+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":111,"num_write":283,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"264'231","reported_seq":887,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.753757+0000","last_change":"2024-04-17T05:46:26.753757+0000","last_active":"2024-04-17T05:46:26.753757+0000","last_peered":"2024-04-17T05:46:26.753757+0000","last_clean":"2024-04-17T05:46:26.753757+0000","last_became_active":"2024-04-17T04:36:27.394100+0000","last_became_peered":"2024-04-17T04:36:27.394100+0000","last_unstale":"2024-04-17T05:46:26.753757+0000","last_undegraded":"2024-04-17T05:46:26.753757+0000","last_fullsized":"2024-04-17T05:46:26.753757+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'231","last_scrub_stamp":"2024-04-17T05:46:26.753483+0000","last_deep_scrub":"264'231","last_deep_scrub_stamp":"2024-04-17T05:46:26.753483+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.753483+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:03:44.040635+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":932,"num_read_kb":753,"num_write":350,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":397,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"264'330","reported_seq":939,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.366081+0000","last_change":"2024-04-17T05:46:35.366211+0000","last_active":"2024-04-17T05:46:35.366081+0000","last_peered":"2024-04-17T05:46:35.366081+0000","last_clean":"2024-04-17T05:46:35.366081+0000","last_became_active":"2024-04-17T04:36:25.367851+0000","last_became_peered":"2024-04-17T04:36:25.367851+0000","last_unstale":"2024-04-17T05:46:35.366081+0000","last_undegraded":"2024-04-17T05:46:35.366081+0000","last_fullsized":"2024-04-17T05:46:35.366081+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'330","last_scrub_stamp":"2024-04-17T05:46:35.365868+0000","last_deep_scrub":"264'330","last_deep_scrub_stamp":"2024-04-17T05:46:35.365868+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.365868+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:27.660066+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5244482,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":438,"num_read_kb":414,"num_write":1194,"num_write_kb":21262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":509,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:35.406987+0000","last_change":"2024-04-17T05:46:35.407172+0000","last_active":"2024-04-17T05:46:35.406987+0000","last_peered":"2024-04-17T05:46:35.406987+0000","last_clean":"2024-04-17T05:46:35.406987+0000","last_became_active":"2024-04-17T04:36:29.378429+0000","last_became_peered":"2024-04-17T04:36:29.378429+0000","last_unstale":"2024-04-17T05:46:35.406987+0000","last_undegraded":"2024-04-17T05:46:35.406987+0000","last_fullsized":"2024-04-17T05:46:35.406987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:35.406755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:35.406755+0000","last_clean_scrub_stamp":"2024-04-17T05:46:35.406755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:18.935388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"264'352","reported_seq":985,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.303937+0000","last_change":"2024-04-17T05:46:50.303937+0000","last_active":"2024-04-17T05:46:50.303937+0000","last_peered":"2024-04-17T05:46:50.303937+0000","last_clean":"2024-04-17T05:46:50.303937+0000","last_became_active":"2024-04-17T04:36:31.395455+0000","last_became_peered":"2024-04-17T04:36:31.395455+0000","last_unstale":"2024-04-17T05:46:50.303937+0000","last_undegraded":"2024-04-17T05:46:50.303937+0000","last_fullsized":"2024-04-17T05:46:50.303937+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'352","last_scrub_stamp":"2024-04-17T05:46:50.303664+0000","last_deep_scrub":"264'352","last_deep_scrub_stamp":"2024-04-17T05:46:50.303664+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.303664+0000","objects_scrubbed":11,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:10:50.032317+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44869,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":547,"num_read_kb":1136,"num_write":426,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":502,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.659892+0000","last_change":"2024-04-17T05:46:26.659892+0000","last_active":"2024-04-17T05:46:26.659892+0000","last_peered":"2024-04-17T05:46:26.659892+0000","last_clean":"2024-04-17T05:46:26.659892+0000","last_became_active":"2024-04-17T04:36:33.429842+0000","last_became_peered":"2024-04-17T04:36:33.429842+0000","last_unstale":"2024-04-17T05:46:26.659892+0000","last_undegraded":"2024-04-17T05:46:26.659892+0000","last_fullsized":"2024-04-17T05:46:26.659892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:26.659492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:26.659492+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.659492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:45.664443+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"264'131","reported_seq":643,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.296344+0000","last_change":"2024-04-17T05:46:25.296344+0000","last_active":"2024-04-17T05:46:25.296344+0000","last_peered":"2024-04-17T05:46:25.296344+0000","last_clean":"2024-04-17T05:46:25.296344+0000","last_became_active":"2024-04-17T04:36:35.435468+0000","last_became_peered":"2024-04-17T04:36:35.435468+0000","last_unstale":"2024-04-17T05:46:25.296344+0000","last_undegraded":"2024-04-17T05:46:25.296344+0000","last_fullsized":"2024-04-17T05:46:25.296344+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'131","last_scrub_stamp":"2024-04-17T05:46:25.295943+0000","last_deep_scrub":"264'131","last_deep_scrub_stamp":"2024-04-17T05:46:25.295943+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.295943+0000","objects_scrubbed":4,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:07:33.340705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":132,"num_read_kb":124,"num_write":300,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"264'162","reported_seq":814,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.510682+0000","last_change":"2024-04-17T05:46:47.510744+0000","last_active":"2024-04-17T05:46:47.510682+0000","last_peered":"2024-04-17T05:46:47.510682+0000","last_clean":"2024-04-17T05:46:47.510682+0000","last_became_active":"2024-04-17T04:36:27.377265+0000","last_became_peered":"2024-04-17T04:36:27.377265+0000","last_unstale":"2024-04-17T05:46:47.510682+0000","last_undegraded":"2024-04-17T05:46:47.510682+0000","last_fullsized":"2024-04-17T05:46:47.510682+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'162","last_scrub_stamp":"2024-04-17T05:46:47.510484+0000","last_deep_scrub":"264'162","last_deep_scrub_stamp":"2024-04-17T05:46:47.510484+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.510484+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:58:06.834002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":682,"num_read_kb":577,"num_write":240,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"264'349","reported_seq":1016,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.338098+0000","last_change":"2024-04-17T05:46:39.338196+0000","last_active":"2024-04-17T05:46:39.338098+0000","last_peered":"2024-04-17T05:46:39.338098+0000","last_clean":"2024-04-17T05:46:39.338098+0000","last_became_active":"2024-04-17T04:36:25.365192+0000","last_became_peered":"2024-04-17T04:36:25.365192+0000","last_unstale":"2024-04-17T05:46:39.338098+0000","last_undegraded":"2024-04-17T05:46:39.338098+0000","last_fullsized":"2024-04-17T05:46:39.338098+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'349","last_scrub_stamp":"2024-04-17T05:46:39.337896+0000","last_deep_scrub":"264'349","last_deep_scrub_stamp":"2024-04-17T05:46:39.337896+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.337896+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":300,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:30.714630+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":558,"num_read_kb":31239,"num_write":1274,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,6],"acting":[5,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":507,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.337367+0000","last_change":"2024-04-17T05:46:38.337476+0000","last_active":"2024-04-17T05:46:38.337367+0000","last_peered":"2024-04-17T05:46:38.337367+0000","last_clean":"2024-04-17T05:46:38.337367+0000","last_became_active":"2024-04-17T04:36:29.392155+0000","last_became_peered":"2024-04-17T04:36:29.392155+0000","last_unstale":"2024-04-17T05:46:38.337367+0000","last_undegraded":"2024-04-17T05:46:38.337367+0000","last_fullsized":"2024-04-17T05:46:38.337367+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:38.337093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:38.337093+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.337093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:47:42.557669+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"264'278","reported_seq":873,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.603830+0000","last_change":"2024-04-17T05:46:49.603906+0000","last_active":"2024-04-17T05:46:49.603830+0000","last_peered":"2024-04-17T05:46:49.603830+0000","last_clean":"2024-04-17T05:46:49.603830+0000","last_became_active":"2024-04-17T04:36:31.402486+0000","last_became_peered":"2024-04-17T04:36:31.402486+0000","last_unstale":"2024-04-17T05:46:49.603830+0000","last_undegraded":"2024-04-17T05:46:49.603830+0000","last_fullsized":"2024-04-17T05:46:49.603830+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'278","last_scrub_stamp":"2024-04-17T05:46:49.603630+0000","last_deep_scrub":"264'278","last_deep_scrub_stamp":"2024-04-17T05:46:49.603630+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.603630+0000","objects_scrubbed":5,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:12:15.688909+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46669,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":461,"num_read_kb":1085,"num_write":368,"num_write_kb":285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":499,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.983269+0000","last_change":"2024-04-17T05:46:23.983269+0000","last_active":"2024-04-17T05:46:23.983269+0000","last_peered":"2024-04-17T05:46:23.983269+0000","last_clean":"2024-04-17T05:46:23.983269+0000","last_became_active":"2024-04-17T04:36:33.418142+0000","last_became_peered":"2024-04-17T04:36:33.418142+0000","last_unstale":"2024-04-17T05:46:23.983269+0000","last_undegraded":"2024-04-17T05:46:23.983269+0000","last_fullsized":"2024-04-17T05:46:23.983269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:23.983009+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.983009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:24:08.626906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"264'136","reported_seq":667,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.796701+0000","last_change":"2024-04-17T05:46:20.796701+0000","last_active":"2024-04-17T05:46:20.796701+0000","last_peered":"2024-04-17T05:46:20.796701+0000","last_clean":"2024-04-17T05:46:20.796701+0000","last_became_active":"2024-04-17T04:36:35.421978+0000","last_became_peered":"2024-04-17T04:36:35.421978+0000","last_unstale":"2024-04-17T05:46:20.796701+0000","last_undegraded":"2024-04-17T05:46:20.796701+0000","last_fullsized":"2024-04-17T05:46:20.796701+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'136","last_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_deep_scrub":"264'136","last_deep_scrub_stamp":"2024-04-17T05:46:20.796340+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.796340+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:26:15.845948+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":171,"num_write":323,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"264'204","reported_seq":821,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:47.764924+0000","last_change":"2024-04-17T05:46:47.764924+0000","last_active":"2024-04-17T05:46:47.764924+0000","last_peered":"2024-04-17T05:46:47.764924+0000","last_clean":"2024-04-17T05:46:47.764924+0000","last_became_active":"2024-04-17T04:36:27.380838+0000","last_became_peered":"2024-04-17T04:36:27.380838+0000","last_unstale":"2024-04-17T05:46:47.764924+0000","last_undegraded":"2024-04-17T05:46:47.764924+0000","last_fullsized":"2024-04-17T05:46:47.764924+0000","mapping_epoch":25,"log_start":"261'100","ondisk_log_start":"261'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'204","last_scrub_stamp":"2024-04-17T05:46:47.764628+0000","last_deep_scrub":"264'204","last_deep_scrub_stamp":"2024-04-17T05:46:47.764628+0000","last_clean_scrub_stamp":"2024-04-17T05:46:47.764628+0000","objects_scrubbed":1,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:15:52.071861+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":838,"num_read_kb":659,"num_write":307,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3065,"num_omap_keys":18,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"264'386","reported_seq":1059,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:47:03.718544+0000","last_change":"2024-04-17T05:47:03.718673+0000","last_active":"2024-04-17T05:47:03.718544+0000","last_peered":"2024-04-17T05:47:03.718544+0000","last_clean":"2024-04-17T05:47:03.718544+0000","last_became_active":"2024-04-17T04:36:25.374108+0000","last_became_peered":"2024-04-17T04:36:25.374108+0000","last_unstale":"2024-04-17T05:47:03.718544+0000","last_undegraded":"2024-04-17T05:47:03.718544+0000","last_fullsized":"2024-04-17T05:47:03.718544+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'386","last_scrub_stamp":"2024-04-17T05:47:03.718335+0000","last_deep_scrub":"264'386","last_deep_scrub_stamp":"2024-04-17T05:47:03.718335+0000","last_clean_scrub_stamp":"2024-04-17T05:47:03.718335+0000","objects_scrubbed":11,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:07.484945+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1731334,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":524,"num_read_kb":19843,"num_write":1402,"num_write_kb":53130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":514,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:40.586905+0000","last_change":"2024-04-17T05:46:40.586958+0000","last_active":"2024-04-17T05:46:40.586905+0000","last_peered":"2024-04-17T05:46:40.586905+0000","last_clean":"2024-04-17T05:46:40.586905+0000","last_became_active":"2024-04-17T04:36:29.376204+0000","last_became_peered":"2024-04-17T04:36:29.376204+0000","last_unstale":"2024-04-17T05:46:40.586905+0000","last_undegraded":"2024-04-17T05:46:40.586905+0000","last_fullsized":"2024-04-17T05:46:40.586905+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:40.586716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:40.586716+0000","last_clean_scrub_stamp":"2024-04-17T05:46:40.586716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:45:35.339099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"264'263","reported_seq":845,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:39.699498+0000","last_change":"2024-04-17T05:46:39.699578+0000","last_active":"2024-04-17T05:46:39.699498+0000","last_peered":"2024-04-17T05:46:39.699498+0000","last_clean":"2024-04-17T05:46:39.699498+0000","last_became_active":"2024-04-17T04:36:31.397449+0000","last_became_peered":"2024-04-17T04:36:31.397449+0000","last_unstale":"2024-04-17T05:46:39.699498+0000","last_undegraded":"2024-04-17T05:46:39.699498+0000","last_fullsized":"2024-04-17T05:46:39.699498+0000","mapping_epoch":29,"log_start":"264'200","ondisk_log_start":"264'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'263","last_scrub_stamp":"2024-04-17T05:46:39.699314+0000","last_deep_scrub":"264'263","last_deep_scrub_stamp":"2024-04-17T05:46:39.699314+0000","last_clean_scrub_stamp":"2024-04-17T05:46:39.699314+0000","objects_scrubbed":3,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:22:28.949715+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44294,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":445,"num_read_kb":1105,"num_write":366,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":505,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:18.762692+0000","last_change":"2024-04-17T05:46:18.762692+0000","last_active":"2024-04-17T05:46:18.762692+0000","last_peered":"2024-04-17T05:46:18.762692+0000","last_clean":"2024-04-17T05:46:18.762692+0000","last_became_active":"2024-04-17T04:36:33.399610+0000","last_became_peered":"2024-04-17T04:36:33.399610+0000","last_unstale":"2024-04-17T05:46:18.762692+0000","last_undegraded":"2024-04-17T05:46:18.762692+0000","last_fullsized":"2024-04-17T05:46:18.762692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:18.762296+0000","last_clean_scrub_stamp":"2024-04-17T05:46:18.762296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:44.671414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"264'146","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.248471+0000","last_change":"2024-04-17T05:46:26.248471+0000","last_active":"2024-04-17T05:46:26.248471+0000","last_peered":"2024-04-17T05:46:26.248471+0000","last_clean":"2024-04-17T05:46:26.248471+0000","last_became_active":"2024-04-17T04:36:35.422950+0000","last_became_peered":"2024-04-17T04:36:35.422950+0000","last_unstale":"2024-04-17T05:46:26.248471+0000","last_undegraded":"2024-04-17T05:46:26.248471+0000","last_fullsized":"2024-04-17T05:46:26.248471+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'146","last_scrub_stamp":"2024-04-17T05:46:26.248201+0000","last_deep_scrub":"264'146","last_deep_scrub_stamp":"2024-04-17T05:46:26.248201+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.248201+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:46:59.437725+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":480,"num_write":333,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"264'299","reported_seq":987,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:32.566140+0000","last_change":"2024-04-17T05:46:32.566140+0000","last_active":"2024-04-17T05:46:32.566140+0000","last_peered":"2024-04-17T05:46:32.566140+0000","last_clean":"2024-04-17T05:46:32.566140+0000","last_became_active":"2024-04-17T04:36:27.399920+0000","last_became_peered":"2024-04-17T04:36:27.399920+0000","last_unstale":"2024-04-17T05:46:32.566140+0000","last_undegraded":"2024-04-17T05:46:32.566140+0000","last_fullsized":"2024-04-17T05:46:32.566140+0000","mapping_epoch":25,"log_start":"250'200","ondisk_log_start":"250'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'299","last_scrub_stamp":"2024-04-17T05:46:32.565869+0000","last_deep_scrub":"264'299","last_deep_scrub_stamp":"2024-04-17T05:46:32.565869+0000","last_clean_scrub_stamp":"2024-04-17T05:46:32.565869+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:01:45.736052+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1309,"num_read_kb":1144,"num_write":451,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":28561,"num_omap_keys":113,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"264'437","reported_seq":1153,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:50.595639+0000","last_change":"2024-04-17T05:46:50.595639+0000","last_active":"2024-04-17T05:46:50.595639+0000","last_peered":"2024-04-17T05:46:50.595639+0000","last_clean":"2024-04-17T05:46:50.595639+0000","last_became_active":"2024-04-17T04:36:25.356058+0000","last_became_peered":"2024-04-17T04:36:25.356058+0000","last_unstale":"2024-04-17T05:46:50.595639+0000","last_undegraded":"2024-04-17T05:46:50.595639+0000","last_fullsized":"2024-04-17T05:46:50.595639+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'437","last_scrub_stamp":"2024-04-17T05:46:50.595303+0000","last_deep_scrub":"264'437","last_deep_scrub_stamp":"2024-04-17T05:46:50.595303+0000","last_clean_scrub_stamp":"2024-04-17T05:46:50.595303+0000","objects_scrubbed":6,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:17:13.320357+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":515218,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":663,"num_read_kb":71297,"num_write":1686,"num_write_kb":67073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"264'8","reported_seq":848,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:20.766334+0000","last_change":"2024-04-17T05:46:20.766334+0000","last_active":"2024-04-17T05:46:20.766334+0000","last_peered":"2024-04-17T05:46:20.766334+0000","last_clean":"2024-04-17T05:46:20.766334+0000","last_became_active":"2024-04-17T04:36:29.376415+0000","last_became_peered":"2024-04-17T04:36:29.376415+0000","last_unstale":"2024-04-17T05:46:20.766334+0000","last_undegraded":"2024-04-17T05:46:20.766334+0000","last_fullsized":"2024-04-17T05:46:20.766334+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'8","last_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_deep_scrub":"264'8","last_deep_scrub_stamp":"2024-04-17T05:46:20.766051+0000","last_clean_scrub_stamp":"2024-04-17T05:46:20.766051+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:42:22.853924+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1185,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":501,"num_read_kb":814,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"264'324","reported_seq":944,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.796472+0000","last_change":"2024-04-17T05:46:25.796472+0000","last_active":"2024-04-17T05:46:25.796472+0000","last_peered":"2024-04-17T05:46:25.796472+0000","last_clean":"2024-04-17T05:46:25.796472+0000","last_became_active":"2024-04-17T04:36:31.390404+0000","last_became_peered":"2024-04-17T04:36:31.390404+0000","last_unstale":"2024-04-17T05:46:25.796472+0000","last_undegraded":"2024-04-17T05:46:25.796472+0000","last_fullsized":"2024-04-17T05:46:25.796472+0000","mapping_epoch":29,"log_start":"264'300","ondisk_log_start":"264'300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'324","last_scrub_stamp":"2024-04-17T05:46:25.796196+0000","last_deep_scrub":"264'324","last_deep_scrub_stamp":"2024-04-17T05:46:25.796196+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.796196+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:16.415248+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46192,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":530,"num_read_kb":1178,"num_write":420,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":501,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.616502+0000","last_change":"2024-04-17T05:46:29.616502+0000","last_active":"2024-04-17T05:46:29.616502+0000","last_peered":"2024-04-17T05:46:29.616502+0000","last_clean":"2024-04-17T05:46:29.616502+0000","last_became_active":"2024-04-17T04:36:33.427385+0000","last_became_peered":"2024-04-17T04:36:33.427385+0000","last_unstale":"2024-04-17T05:46:29.616502+0000","last_undegraded":"2024-04-17T05:46:29.616502+0000","last_fullsized":"2024-04-17T05:46:29.616502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T05:46:29.616255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T05:46:29.616255+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.616255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:25:32.388074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"264'141","reported_seq":650,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.802101+0000","last_change":"2024-04-17T05:46:10.802101+0000","last_active":"2024-04-17T05:46:10.802101+0000","last_peered":"2024-04-17T05:46:10.802101+0000","last_clean":"2024-04-17T05:46:10.802101+0000","last_became_active":"2024-04-17T04:36:35.426973+0000","last_became_peered":"2024-04-17T04:36:35.426973+0000","last_unstale":"2024-04-17T05:46:10.802101+0000","last_undegraded":"2024-04-17T05:46:10.802101+0000","last_fullsized":"2024-04-17T05:46:10.802101+0000","mapping_epoch":33,"log_start":"264'100","ondisk_log_start":"264'100","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'141","last_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_deep_scrub":"264'141","last_deep_scrub_stamp":"2024-04-17T05:46:10.801687+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.801687+0000","objects_scrubbed":3,"log_size":41,"log_dups_size":100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:52:16.852555+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":111,"num_write":335,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"264'155","reported_seq":798,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:21.741104+0000","last_change":"2024-04-17T05:46:21.741243+0000","last_active":"2024-04-17T05:46:21.741104+0000","last_peered":"2024-04-17T05:46:21.741104+0000","last_clean":"2024-04-17T05:46:21.741104+0000","last_became_active":"2024-04-17T04:36:27.397173+0000","last_became_peered":"2024-04-17T04:36:27.397173+0000","last_unstale":"2024-04-17T05:46:21.741104+0000","last_undegraded":"2024-04-17T05:46:21.741104+0000","last_fullsized":"2024-04-17T05:46:21.741104+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'155","last_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_deep_scrub":"264'155","last_deep_scrub_stamp":"2024-04-17T05:46:21.740847+0000","last_clean_scrub_stamp":"2024-04-17T05:46:21.740847+0000","objects_scrubbed":1,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:03.471838+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":548,"num_write":260,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"264'420","reported_seq":1148,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.812010+0000","last_change":"2024-04-17T05:46:34.812010+0000","last_active":"2024-04-17T05:46:34.812010+0000","last_peered":"2024-04-17T05:46:34.812010+0000","last_clean":"2024-04-17T05:46:34.812010+0000","last_became_active":"2024-04-17T04:36:25.383101+0000","last_became_peered":"2024-04-17T04:36:25.383101+0000","last_unstale":"2024-04-17T05:46:34.812010+0000","last_undegraded":"2024-04-17T05:46:34.812010+0000","last_fullsized":"2024-04-17T05:46:34.812010+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'420","last_scrub_stamp":"2024-04-17T05:46:34.811680+0000","last_deep_scrub":"264'420","last_deep_scrub_stamp":"2024-04-17T05:46:34.811680+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.811680+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:35:57.763900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262268,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":835,"num_read_kb":6303,"num_write":1376,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"264'90","reported_seq":658,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.183156+0000","last_change":"2024-04-17T05:46:10.183156+0000","last_active":"2024-04-17T05:46:10.183156+0000","last_peered":"2024-04-17T05:46:10.183156+0000","last_clean":"2024-04-17T05:46:10.183156+0000","last_became_active":"2024-04-17T04:37:03.629229+0000","last_became_peered":"2024-04-17T04:37:03.629229+0000","last_unstale":"2024-04-17T05:46:10.183156+0000","last_undegraded":"2024-04-17T05:46:10.183156+0000","last_fullsized":"2024-04-17T05:46:10.183156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'90","last_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_deep_scrub":"264'90","last_deep_scrub_stamp":"2024-04-17T05:46:10.182909+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.182909+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T05:58:10.036518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":375,"num_read_kb":307,"num_write":125,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"264'365","reported_seq":1041,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:26.440875+0000","last_change":"2024-04-17T05:46:26.440875+0000","last_active":"2024-04-17T05:46:26.440875+0000","last_peered":"2024-04-17T05:46:26.440875+0000","last_clean":"2024-04-17T05:46:26.440875+0000","last_became_active":"2024-04-17T04:36:25.354038+0000","last_became_peered":"2024-04-17T04:36:25.354038+0000","last_unstale":"2024-04-17T05:46:26.440875+0000","last_undegraded":"2024-04-17T05:46:26.440875+0000","last_fullsized":"2024-04-17T05:46:26.440875+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'365","last_scrub_stamp":"2024-04-17T05:46:26.440480+0000","last_deep_scrub":"264'365","last_deep_scrub_stamp":"2024-04-17T05:46:26.440480+0000","last_clean_scrub_stamp":"2024-04-17T05:46:26.440480+0000","objects_scrubbed":10,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:23:58.716606+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6192676,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":568,"num_read_kb":39832,"num_write":1198,"num_write_kb":31112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"264'149","reported_seq":772,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.243871+0000","last_change":"2024-04-17T05:46:27.243940+0000","last_active":"2024-04-17T05:46:27.243871+0000","last_peered":"2024-04-17T05:46:27.243871+0000","last_clean":"2024-04-17T05:46:27.243871+0000","last_became_active":"2024-04-17T04:36:27.387831+0000","last_became_peered":"2024-04-17T04:36:27.387831+0000","last_unstale":"2024-04-17T05:46:27.243871+0000","last_undegraded":"2024-04-17T05:46:27.243871+0000","last_fullsized":"2024-04-17T05:46:27.243871+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'149","last_scrub_stamp":"2024-04-17T05:46:27.243689+0000","last_deep_scrub":"264'149","last_deep_scrub_stamp":"2024-04-17T05:46:27.243689+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.243689+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:03.394294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":608,"num_read_kb":494,"num_write":236,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"264'307","reported_seq":954,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.272639+0000","last_change":"2024-04-17T05:46:41.272639+0000","last_active":"2024-04-17T05:46:41.272639+0000","last_peered":"2024-04-17T05:46:41.272639+0000","last_clean":"2024-04-17T05:46:41.272639+0000","last_became_active":"2024-04-17T04:36:25.364089+0000","last_became_peered":"2024-04-17T04:36:25.364089+0000","last_unstale":"2024-04-17T05:46:41.272639+0000","last_undegraded":"2024-04-17T05:46:41.272639+0000","last_fullsized":"2024-04-17T05:46:41.272639+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'307","last_scrub_stamp":"2024-04-17T05:46:41.272065+0000","last_deep_scrub":"264'307","last_deep_scrub_stamp":"2024-04-17T05:46:41.272065+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.272065+0000","objects_scrubbed":10,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:27:14.652269+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4341641,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":436,"num_read_kb":10784,"num_write":1165,"num_write_kb":20954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"264'95","reported_seq":729,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.829126+0000","last_change":"2024-04-17T05:46:31.829195+0000","last_active":"2024-04-17T05:46:31.829126+0000","last_peered":"2024-04-17T05:46:31.829126+0000","last_clean":"2024-04-17T05:46:31.829126+0000","last_became_active":"2024-04-17T04:36:27.361992+0000","last_became_peered":"2024-04-17T04:36:27.361992+0000","last_unstale":"2024-04-17T05:46:31.829126+0000","last_undegraded":"2024-04-17T05:46:31.829126+0000","last_fullsized":"2024-04-17T05:46:31.829126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'95","last_scrub_stamp":"2024-04-17T05:46:31.828927+0000","last_deep_scrub":"264'95","last_deep_scrub_stamp":"2024-04-17T05:46:31.828927+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.828927+0000","objects_scrubbed":1,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:49:59.765753+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":438,"num_read_kb":372,"num_write":141,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"264'308","reported_seq":1044,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:59.554915+0000","last_change":"2024-04-17T05:46:59.554915+0000","last_active":"2024-04-17T05:46:59.554915+0000","last_peered":"2024-04-17T05:46:59.554915+0000","last_clean":"2024-04-17T05:46:59.554915+0000","last_became_active":"2024-04-17T04:36:25.354089+0000","last_became_peered":"2024-04-17T04:36:25.354089+0000","last_unstale":"2024-04-17T05:46:59.554915+0000","last_undegraded":"2024-04-17T05:46:59.554915+0000","last_fullsized":"2024-04-17T05:46:59.554915+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'308","last_scrub_stamp":"2024-04-17T05:46:59.554444+0000","last_deep_scrub":"264'308","last_deep_scrub_stamp":"2024-04-17T05:46:59.554444+0000","last_clean_scrub_stamp":"2024-04-17T05:46:59.554444+0000","objects_scrubbed":5,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:33:39.077454+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":709,"num_read_kb":32424,"num_write":1121,"num_write_kb":32753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"264'58","reported_seq":644,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.725516+0000","last_change":"2024-04-17T05:46:44.725575+0000","last_active":"2024-04-17T05:46:44.725516+0000","last_peered":"2024-04-17T05:46:44.725516+0000","last_clean":"2024-04-17T05:46:44.725516+0000","last_became_active":"2024-04-17T04:36:27.395102+0000","last_became_peered":"2024-04-17T04:36:27.395102+0000","last_unstale":"2024-04-17T05:46:44.725516+0000","last_undegraded":"2024-04-17T05:46:44.725516+0000","last_fullsized":"2024-04-17T05:46:44.725516+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'58","last_scrub_stamp":"2024-04-17T05:46:44.725324+0000","last_deep_scrub":"264'58","last_deep_scrub_stamp":"2024-04-17T05:46:44.725324+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.725324+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:28:00.884116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":219,"num_write":83,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"264'281","reported_seq":902,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:16.814149+0000","last_change":"2024-04-17T05:46:16.814273+0000","last_active":"2024-04-17T05:46:16.814149+0000","last_peered":"2024-04-17T05:46:16.814149+0000","last_clean":"2024-04-17T05:46:16.814149+0000","last_became_active":"2024-04-17T04:37:03.635734+0000","last_became_peered":"2024-04-17T04:37:03.635734+0000","last_unstale":"2024-04-17T05:46:16.814149+0000","last_undegraded":"2024-04-17T05:46:16.814149+0000","last_fullsized":"2024-04-17T05:46:16.814149+0000","mapping_epoch":36,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'281","last_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_deep_scrub":"264'281","last_deep_scrub_stamp":"2024-04-17T05:46:16.813947+0000","last_clean_scrub_stamp":"2024-04-17T05:46:16.813947+0000","objects_scrubbed":8,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:56.482239+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5047006,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":359,"num_read_kb":5921,"num_write":992,"num_write_kb":39266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,7],"acting":[3,2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"264'54","reported_seq":642,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:28.664753+0000","last_change":"2024-04-17T05:46:28.664753+0000","last_active":"2024-04-17T05:46:28.664753+0000","last_peered":"2024-04-17T05:46:28.664753+0000","last_clean":"2024-04-17T05:46:28.664753+0000","last_became_active":"2024-04-17T04:36:27.394630+0000","last_became_peered":"2024-04-17T04:36:27.394630+0000","last_unstale":"2024-04-17T05:46:28.664753+0000","last_undegraded":"2024-04-17T05:46:28.664753+0000","last_fullsized":"2024-04-17T05:46:28.664753+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'54","last_scrub_stamp":"2024-04-17T05:46:28.664477+0000","last_deep_scrub":"264'54","last_deep_scrub_stamp":"2024-04-17T05:46:28.664477+0000","last_clean_scrub_stamp":"2024-04-17T05:46:28.664477+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:31:11.472041+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":209,"num_write":76,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"264'319","reported_seq":1008,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:36.820625+0000","last_change":"2024-04-17T05:46:36.820896+0000","last_active":"2024-04-17T05:46:36.820625+0000","last_peered":"2024-04-17T05:46:36.820625+0000","last_clean":"2024-04-17T05:46:36.820625+0000","last_became_active":"2024-04-17T04:36:25.342244+0000","last_became_peered":"2024-04-17T04:36:25.342244+0000","last_unstale":"2024-04-17T05:46:36.820625+0000","last_undegraded":"2024-04-17T05:46:36.820625+0000","last_fullsized":"2024-04-17T05:46:36.820625+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'319","last_scrub_stamp":"2024-04-17T05:46:36.820373+0000","last_deep_scrub":"264'319","last_deep_scrub_stamp":"2024-04-17T05:46:36.820373+0000","last_clean_scrub_stamp":"2024-04-17T05:46:36.820373+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:47:09.390555+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5067983,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":492,"num_read_kb":71000,"num_write":1186,"num_write_kb":48861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"264'216","reported_seq":826,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.756202+0000","last_change":"2024-04-17T05:46:45.756202+0000","last_active":"2024-04-17T05:46:45.756202+0000","last_peered":"2024-04-17T05:46:45.756202+0000","last_clean":"2024-04-17T05:46:45.756202+0000","last_became_active":"2024-04-17T04:36:27.381836+0000","last_became_peered":"2024-04-17T04:36:27.381836+0000","last_unstale":"2024-04-17T05:46:45.756202+0000","last_undegraded":"2024-04-17T05:46:45.756202+0000","last_fullsized":"2024-04-17T05:46:45.756202+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'216","last_scrub_stamp":"2024-04-17T05:46:45.755911+0000","last_deep_scrub":"264'216","last_deep_scrub_stamp":"2024-04-17T05:46:45.755911+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.755911+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":200,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:22:01.103569+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":839,"num_read_kb":666,"num_write":325,"num_write_kb":173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8079,"num_omap_keys":32,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"264'371","reported_seq":1061,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:38.914547+0000","last_change":"2024-04-17T05:46:38.914661+0000","last_active":"2024-04-17T05:46:38.914547+0000","last_peered":"2024-04-17T05:46:38.914547+0000","last_clean":"2024-04-17T05:46:38.914547+0000","last_became_active":"2024-04-17T04:36:25.384669+0000","last_became_peered":"2024-04-17T04:36:25.384669+0000","last_unstale":"2024-04-17T05:46:38.914547+0000","last_undegraded":"2024-04-17T05:46:38.914547+0000","last_fullsized":"2024-04-17T05:46:38.914547+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'371","last_scrub_stamp":"2024-04-17T05:46:38.914310+0000","last_deep_scrub":"264'371","last_deep_scrub_stamp":"2024-04-17T05:46:38.914310+0000","last_clean_scrub_stamp":"2024-04-17T05:46:38.914310+0000","objects_scrubbed":6,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:28.802337+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2189648,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":654,"num_read_kb":34085,"num_write":1471,"num_write_kb":38063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"264'135","reported_seq":758,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:27.000580+0000","last_change":"2024-04-17T05:46:27.000580+0000","last_active":"2024-04-17T05:46:27.000580+0000","last_peered":"2024-04-17T05:46:27.000580+0000","last_clean":"2024-04-17T05:46:27.000580+0000","last_became_active":"2024-04-17T04:36:27.386438+0000","last_became_peered":"2024-04-17T04:36:27.386438+0000","last_unstale":"2024-04-17T05:46:27.000580+0000","last_undegraded":"2024-04-17T05:46:27.000580+0000","last_fullsized":"2024-04-17T05:46:27.000580+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'135","last_scrub_stamp":"2024-04-17T05:46:27.000319+0000","last_deep_scrub":"264'135","last_deep_scrub_stamp":"2024-04-17T05:46:27.000319+0000","last_clean_scrub_stamp":"2024-04-17T05:46:27.000319+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:56:15.623855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":578,"num_read_kb":478,"num_write":213,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"264'335","reported_seq":1015,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:53.675343+0000","last_change":"2024-04-17T05:46:53.675677+0000","last_active":"2024-04-17T05:46:53.675343+0000","last_peered":"2024-04-17T05:46:53.675343+0000","last_clean":"2024-04-17T05:46:53.675343+0000","last_became_active":"2024-04-17T04:36:25.379376+0000","last_became_peered":"2024-04-17T04:36:25.379376+0000","last_unstale":"2024-04-17T05:46:53.675343+0000","last_undegraded":"2024-04-17T05:46:53.675343+0000","last_fullsized":"2024-04-17T05:46:53.675343+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'335","last_scrub_stamp":"2024-04-17T05:46:53.675099+0000","last_deep_scrub":"264'335","last_deep_scrub_stamp":"2024-04-17T05:46:53.675099+0000","last_clean_scrub_stamp":"2024-04-17T05:46:53.675099+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:38:40.955125+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534609,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":546,"num_read_kb":42711,"num_write":1290,"num_write_kb":45238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"264'133","reported_seq":748,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:42.655975+0000","last_change":"2024-04-17T05:46:42.656040+0000","last_active":"2024-04-17T05:46:42.655975+0000","last_peered":"2024-04-17T05:46:42.655975+0000","last_clean":"2024-04-17T05:46:42.655975+0000","last_became_active":"2024-04-17T04:36:27.373377+0000","last_became_peered":"2024-04-17T04:36:27.373377+0000","last_unstale":"2024-04-17T05:46:42.655975+0000","last_undegraded":"2024-04-17T05:46:42.655975+0000","last_fullsized":"2024-04-17T05:46:42.655975+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'133","last_scrub_stamp":"2024-04-17T05:46:42.655782+0000","last_deep_scrub":"264'133","last_deep_scrub_stamp":"2024-04-17T05:46:42.655782+0000","last_clean_scrub_stamp":"2024-04-17T05:46:42.655782+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:07:25.917924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":524,"num_read_kb":430,"num_write":203,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"264'300","reported_seq":943,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.230338+0000","last_change":"2024-04-17T05:46:48.230447+0000","last_active":"2024-04-17T05:46:48.230338+0000","last_peered":"2024-04-17T05:46:48.230338+0000","last_clean":"2024-04-17T05:46:48.230338+0000","last_became_active":"2024-04-17T04:36:25.360940+0000","last_became_peered":"2024-04-17T04:36:25.360940+0000","last_unstale":"2024-04-17T05:46:48.230338+0000","last_undegraded":"2024-04-17T05:46:48.230338+0000","last_fullsized":"2024-04-17T05:46:48.230338+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'300","last_scrub_stamp":"2024-04-17T05:46:48.230130+0000","last_deep_scrub":"264'300","last_deep_scrub_stamp":"2024-04-17T05:46:48.230130+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.230130+0000","objects_scrubbed":6,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:29:19.387099+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262289,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":548,"num_write":1115,"num_write_kb":22113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"264'110","reported_seq":764,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:48.512660+0000","last_change":"2024-04-17T05:46:48.512719+0000","last_active":"2024-04-17T05:46:48.512660+0000","last_peered":"2024-04-17T05:46:48.512660+0000","last_clean":"2024-04-17T05:46:48.512660+0000","last_became_active":"2024-04-17T04:36:27.377509+0000","last_became_peered":"2024-04-17T04:36:27.377509+0000","last_unstale":"2024-04-17T05:46:48.512660+0000","last_undegraded":"2024-04-17T05:46:48.512660+0000","last_fullsized":"2024-04-17T05:46:48.512660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'110","last_scrub_stamp":"2024-04-17T05:46:48.512459+0000","last_deep_scrub":"264'110","last_deep_scrub_stamp":"2024-04-17T05:46:48.512459+0000","last_clean_scrub_stamp":"2024-04-17T05:46:48.512459+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:19:19.000329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":441,"num_write":157,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"264'302","reported_seq":961,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:59.670232+0000","last_change":"2024-04-17T05:46:59.670336+0000","last_active":"2024-04-17T05:46:59.670232+0000","last_peered":"2024-04-17T05:46:59.670232+0000","last_clean":"2024-04-17T05:46:59.670232+0000","last_became_active":"2024-04-17T04:36:25.352983+0000","last_became_peered":"2024-04-17T04:36:25.352983+0000","last_unstale":"2024-04-17T05:46:59.670232+0000","last_undegraded":"2024-04-17T05:46:59.670232+0000","last_fullsized":"2024-04-17T05:46:59.670232+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'302","last_scrub_stamp":"2024-04-17T05:46:59.670021+0000","last_deep_scrub":"264'302","last_deep_scrub_stamp":"2024-04-17T05:46:59.670021+0000","last_clean_scrub_stamp":"2024-04-17T05:46:59.670021+0000","objects_scrubbed":6,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:43:52.914846+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171414,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":456,"num_read_kb":4352,"num_write":1098,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"264'202","reported_seq":847,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:31.463713+0000","last_change":"2024-04-17T05:46:31.463772+0000","last_active":"2024-04-17T05:46:31.463713+0000","last_peered":"2024-04-17T05:46:31.463713+0000","last_clean":"2024-04-17T05:46:31.463713+0000","last_became_active":"2024-04-17T04:36:27.397892+0000","last_became_peered":"2024-04-17T04:36:27.397892+0000","last_unstale":"2024-04-17T05:46:31.463713+0000","last_undegraded":"2024-04-17T05:46:31.463713+0000","last_fullsized":"2024-04-17T05:46:31.463713+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'202","last_scrub_stamp":"2024-04-17T05:46:31.463487+0000","last_deep_scrub":"264'202","last_deep_scrub_stamp":"2024-04-17T05:46:31.463487+0000","last_clean_scrub_stamp":"2024-04-17T05:46:31.463487+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:59:24.787961+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":814,"num_read_kb":650,"num_write":309,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"264'317","reported_seq":938,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:10.663532+0000","last_change":"2024-04-17T05:46:10.663532+0000","last_active":"2024-04-17T05:46:10.663532+0000","last_peered":"2024-04-17T05:46:10.663532+0000","last_clean":"2024-04-17T05:46:10.663532+0000","last_became_active":"2024-04-17T04:37:03.631125+0000","last_became_peered":"2024-04-17T04:37:03.631125+0000","last_unstale":"2024-04-17T05:46:10.663532+0000","last_undegraded":"2024-04-17T05:46:10.663532+0000","last_fullsized":"2024-04-17T05:46:10.663532+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'317","last_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_deep_scrub":"264'317","last_deep_scrub_stamp":"2024-04-17T05:46:10.663219+0000","last_clean_scrub_stamp":"2024-04-17T05:46:10.663219+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:02.001315+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":443,"num_read_kb":16015,"num_write":1067,"num_write_kb":15162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6,3],"acting":[2,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"264'96","reported_seq":694,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:34.604944+0000","last_change":"2024-04-17T05:46:34.605089+0000","last_active":"2024-04-17T05:46:34.604944+0000","last_peered":"2024-04-17T05:46:34.604944+0000","last_clean":"2024-04-17T05:46:34.604944+0000","last_became_active":"2024-04-17T04:36:27.401343+0000","last_became_peered":"2024-04-17T04:36:27.401343+0000","last_unstale":"2024-04-17T05:46:34.604944+0000","last_undegraded":"2024-04-17T05:46:34.604944+0000","last_fullsized":"2024-04-17T05:46:34.604944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'96","last_scrub_stamp":"2024-04-17T05:46:34.604692+0000","last_deep_scrub":"264'96","last_deep_scrub_stamp":"2024-04-17T05:46:34.604692+0000","last_clean_scrub_stamp":"2024-04-17T05:46:34.604692+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:59.288725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":398,"num_read_kb":327,"num_write":140,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"264'360","reported_seq":1087,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:55.647400+0000","last_change":"2024-04-17T05:46:55.647491+0000","last_active":"2024-04-17T05:46:55.647400+0000","last_peered":"2024-04-17T05:46:55.647400+0000","last_clean":"2024-04-17T05:46:55.647400+0000","last_became_active":"2024-04-17T04:36:25.332370+0000","last_became_peered":"2024-04-17T04:36:25.332370+0000","last_unstale":"2024-04-17T05:46:55.647400+0000","last_undegraded":"2024-04-17T05:46:55.647400+0000","last_fullsized":"2024-04-17T05:46:55.647400+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'360","last_scrub_stamp":"2024-04-17T05:46:55.647216+0000","last_deep_scrub":"264'360","last_deep_scrub_stamp":"2024-04-17T05:46:55.647216+0000","last_clean_scrub_stamp":"2024-04-17T05:46:55.647216+0000","objects_scrubbed":4,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:23:13.323803+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":746,"num_read_kb":12134,"num_write":1256,"num_write_kb":21911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"264'101","reported_seq":696,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.414667+0000","last_change":"2024-04-17T05:46:37.414725+0000","last_active":"2024-04-17T05:46:37.414667+0000","last_peered":"2024-04-17T05:46:37.414667+0000","last_clean":"2024-04-17T05:46:37.414667+0000","last_became_active":"2024-04-17T04:36:27.388061+0000","last_became_peered":"2024-04-17T04:36:27.388061+0000","last_unstale":"2024-04-17T05:46:37.414667+0000","last_undegraded":"2024-04-17T05:46:37.414667+0000","last_fullsized":"2024-04-17T05:46:37.414667+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'101","last_scrub_stamp":"2024-04-17T05:46:37.414492+0000","last_deep_scrub":"264'101","last_deep_scrub_stamp":"2024-04-17T05:46:37.414492+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.414492+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:50:18.807121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":342,"num_write":140,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"264'357","reported_seq":1003,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:33.855024+0000","last_change":"2024-04-17T05:46:33.855258+0000","last_active":"2024-04-17T05:46:33.855024+0000","last_peered":"2024-04-17T05:46:33.855024+0000","last_clean":"2024-04-17T05:46:33.855024+0000","last_became_active":"2024-04-17T04:36:25.372807+0000","last_became_peered":"2024-04-17T04:36:25.372807+0000","last_unstale":"2024-04-17T05:46:33.855024+0000","last_undegraded":"2024-04-17T05:46:33.855024+0000","last_fullsized":"2024-04-17T05:46:33.855024+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'357","last_scrub_stamp":"2024-04-17T05:46:33.854698+0000","last_deep_scrub":"264'357","last_deep_scrub_stamp":"2024-04-17T05:46:33.854698+0000","last_clean_scrub_stamp":"2024-04-17T05:46:33.854698+0000","objects_scrubbed":3,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:35:38.333183+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":501,"num_read_kb":657,"num_write":1252,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"264'222","reported_seq":912,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:44.665956+0000","last_change":"2024-04-17T05:46:44.666020+0000","last_active":"2024-04-17T05:46:44.665956+0000","last_peered":"2024-04-17T05:46:44.665956+0000","last_clean":"2024-04-17T05:46:44.665956+0000","last_became_active":"2024-04-17T04:36:27.370585+0000","last_became_peered":"2024-04-17T04:36:27.370585+0000","last_unstale":"2024-04-17T05:46:44.665956+0000","last_undegraded":"2024-04-17T05:46:44.665956+0000","last_fullsized":"2024-04-17T05:46:44.665956+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'222","last_scrub_stamp":"2024-04-17T05:46:44.665753+0000","last_deep_scrub":"264'222","last_deep_scrub_stamp":"2024-04-17T05:46:44.665753+0000","last_clean_scrub_stamp":"2024-04-17T05:46:44.665753+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:05.646503+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1107,"num_read_kb":978,"num_write":366,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"264'420","reported_seq":1168,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.230796+0000","last_change":"2024-04-17T05:46:45.230919+0000","last_active":"2024-04-17T05:46:45.230796+0000","last_peered":"2024-04-17T05:46:45.230796+0000","last_clean":"2024-04-17T05:46:45.230796+0000","last_became_active":"2024-04-17T04:36:25.385774+0000","last_became_peered":"2024-04-17T04:36:25.385774+0000","last_unstale":"2024-04-17T05:46:45.230796+0000","last_undegraded":"2024-04-17T05:46:45.230796+0000","last_fullsized":"2024-04-17T05:46:45.230796+0000","mapping_epoch":23,"log_start":"264'400","ondisk_log_start":"264'400","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'420","last_scrub_stamp":"2024-04-17T05:46:45.230572+0000","last_deep_scrub":"264'420","last_deep_scrub_stamp":"2024-04-17T05:46:45.230572+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.230572+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":400,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:12.917895+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058993,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":894,"num_read_kb":32480,"num_write":1416,"num_write_kb":42891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6,3],"acting":[7,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"264'352","reported_seq":975,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.744514+0000","last_change":"2024-04-17T05:46:41.744578+0000","last_active":"2024-04-17T05:46:41.744514+0000","last_peered":"2024-04-17T05:46:41.744514+0000","last_clean":"2024-04-17T05:46:41.744514+0000","last_became_active":"2024-04-17T04:36:27.384908+0000","last_became_peered":"2024-04-17T04:36:27.384908+0000","last_unstale":"2024-04-17T05:46:41.744514+0000","last_undegraded":"2024-04-17T05:46:41.744514+0000","last_fullsized":"2024-04-17T05:46:41.744514+0000","mapping_epoch":25,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'352","last_scrub_stamp":"2024-04-17T05:46:41.744313+0000","last_deep_scrub":"264'352","last_deep_scrub_stamp":"2024-04-17T05:46:41.744313+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.744313+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:08:33.007856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1378,"num_read_kb":1081,"num_write":577,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"264'314","reported_seq":1000,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:25.458853+0000","last_change":"2024-04-17T05:46:25.458952+0000","last_active":"2024-04-17T05:46:25.458853+0000","last_peered":"2024-04-17T05:46:25.458853+0000","last_clean":"2024-04-17T05:46:25.458853+0000","last_became_active":"2024-04-17T04:36:25.354797+0000","last_became_peered":"2024-04-17T04:36:25.354797+0000","last_unstale":"2024-04-17T05:46:25.458853+0000","last_undegraded":"2024-04-17T05:46:25.458853+0000","last_fullsized":"2024-04-17T05:46:25.458853+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'314","last_scrub_stamp":"2024-04-17T05:46:25.458647+0000","last_deep_scrub":"264'314","last_deep_scrub_stamp":"2024-04-17T05:46:25.458647+0000","last_clean_scrub_stamp":"2024-04-17T05:46:25.458647+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:15:02.295132+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7281800,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":526,"num_read_kb":46303,"num_write":1296,"num_write_kb":25061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"264'322","reported_seq":1053,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:23.325081+0000","last_change":"2024-04-17T05:46:23.325152+0000","last_active":"2024-04-17T05:46:23.325081+0000","last_peered":"2024-04-17T05:46:23.325081+0000","last_clean":"2024-04-17T05:46:23.325081+0000","last_became_active":"2024-04-17T04:37:03.627798+0000","last_became_peered":"2024-04-17T04:37:03.627798+0000","last_unstale":"2024-04-17T05:46:23.325081+0000","last_undegraded":"2024-04-17T05:46:23.325081+0000","last_fullsized":"2024-04-17T05:46:23.325081+0000","mapping_epoch":36,"log_start":"264'300","ondisk_log_start":"264'300","created":25,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'322","last_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_deep_scrub":"264'322","last_deep_scrub_stamp":"2024-04-17T05:46:23.324895+0000","last_clean_scrub_stamp":"2024-04-17T05:46:23.324895+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:22:02.515429+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1428,"num_read_kb":1160,"num_write":605,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"264'328","reported_seq":971,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:49.276500+0000","last_change":"2024-04-17T05:46:49.276594+0000","last_active":"2024-04-17T05:46:49.276500+0000","last_peered":"2024-04-17T05:46:49.276500+0000","last_clean":"2024-04-17T05:46:49.276500+0000","last_became_active":"2024-04-17T04:36:25.361077+0000","last_became_peered":"2024-04-17T04:36:25.361077+0000","last_unstale":"2024-04-17T05:46:49.276500+0000","last_undegraded":"2024-04-17T05:46:49.276500+0000","last_fullsized":"2024-04-17T05:46:49.276500+0000","mapping_epoch":23,"log_start":"264'300","ondisk_log_start":"264'300","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'328","last_scrub_stamp":"2024-04-17T05:46:49.276270+0000","last_deep_scrub":"264'328","last_deep_scrub_stamp":"2024-04-17T05:46:49.276270+0000","last_clean_scrub_stamp":"2024-04-17T05:46:49.276270+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:32.984779+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":466,"num_read_kb":49597,"num_write":1231,"num_write_kb":34889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"264'297","reported_seq":933,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:45.292701+0000","last_change":"2024-04-17T05:46:45.292805+0000","last_active":"2024-04-17T05:46:45.292701+0000","last_peered":"2024-04-17T05:46:45.292701+0000","last_clean":"2024-04-17T05:46:45.292701+0000","last_became_active":"2024-04-17T04:36:25.344082+0000","last_became_peered":"2024-04-17T04:36:25.344082+0000","last_unstale":"2024-04-17T05:46:45.292701+0000","last_undegraded":"2024-04-17T05:46:45.292701+0000","last_fullsized":"2024-04-17T05:46:45.292701+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'297","last_scrub_stamp":"2024-04-17T05:46:45.292505+0000","last_deep_scrub":"264'297","last_deep_scrub_stamp":"2024-04-17T05:46:45.292505+0000","last_clean_scrub_stamp":"2024-04-17T05:46:45.292505+0000","objects_scrubbed":8,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:01:54.845561+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2129194,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":413,"num_read_kb":7661,"num_write":1087,"num_write_kb":19549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"264'244","reported_seq":853,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:37.670960+0000","last_change":"2024-04-17T05:46:37.670960+0000","last_active":"2024-04-17T05:46:37.670960+0000","last_peered":"2024-04-17T05:46:37.670960+0000","last_clean":"2024-04-17T05:46:37.670960+0000","last_became_active":"2024-04-17T04:36:27.378692+0000","last_became_peered":"2024-04-17T04:36:27.378692+0000","last_unstale":"2024-04-17T05:46:37.670960+0000","last_undegraded":"2024-04-17T05:46:37.670960+0000","last_fullsized":"2024-04-17T05:46:37.670960+0000","mapping_epoch":25,"log_start":"264'200","ondisk_log_start":"264'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'244","last_scrub_stamp":"2024-04-17T05:46:37.670697+0000","last_deep_scrub":"264'244","last_deep_scrub_stamp":"2024-04-17T05:46:37.670697+0000","last_clean_scrub_stamp":"2024-04-17T05:46:37.670697+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":200,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:31:58.583622+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":744,"num_write":397,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"264'277","reported_seq":930,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:41.851613+0000","last_change":"2024-04-17T05:46:41.851831+0000","last_active":"2024-04-17T05:46:41.851613+0000","last_peered":"2024-04-17T05:46:41.851613+0000","last_clean":"2024-04-17T05:46:41.851613+0000","last_became_active":"2024-04-17T04:36:25.353180+0000","last_became_peered":"2024-04-17T04:36:25.353180+0000","last_unstale":"2024-04-17T05:46:41.851613+0000","last_undegraded":"2024-04-17T05:46:41.851613+0000","last_fullsized":"2024-04-17T05:46:41.851613+0000","mapping_epoch":23,"log_start":"264'200","ondisk_log_start":"264'200","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'277","last_scrub_stamp":"2024-04-17T05:46:41.851309+0000","last_deep_scrub":"264'277","last_deep_scrub_stamp":"2024-04-17T05:46:41.851309+0000","last_clean_scrub_stamp":"2024-04-17T05:46:41.851309+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:22:42.210177+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":447,"num_read_kb":20933,"num_write":1123,"num_write_kb":35444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"264'198","reported_seq":829,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-04-17T05:46:29.314199+0000","last_change":"2024-04-17T05:46:29.314199+0000","last_active":"2024-04-17T05:46:29.314199+0000","last_peered":"2024-04-17T05:46:29.314199+0000","last_clean":"2024-04-17T05:46:29.314199+0000","last_became_active":"2024-04-17T04:36:27.378102+0000","last_became_peered":"2024-04-17T04:36:27.378102+0000","last_unstale":"2024-04-17T05:46:29.314199+0000","last_undegraded":"2024-04-17T05:46:29.314199+0000","last_fullsized":"2024-04-17T05:46:29.314199+0000","mapping_epoch":25,"log_start":"264'100","ondisk_log_start":"264'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'198","last_scrub_stamp":"2024-04-17T05:46:29.313797+0000","last_deep_scrub":"264'198","last_deep_scrub_stamp":"2024-04-17T05:46:29.313797+0000","last_clean_scrub_stamp":"2024-04-17T05:46:29.313797+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:53:17.398694+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":802,"num_read_kb":641,"num_write":301,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5626,"num_read_kb":4012,"num_write":2163,"num_write_kb":796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6748,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1282,"ondisk_log_size":1282,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19802,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":13277,"num_read_kb":13075,"num_write":16007,"num_write_kb":4700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4334,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704512,"data_stored":39604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1780,"ondisk_log_size":1780,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":255,"ondisk_log_size":255,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1458963,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":18010,"num_read_kb":37614,"num_write":15043,"num_write_kb":11218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":628236,"num_omap_keys":1518,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2038,"ondisk_log_size":2038,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1652,"num_read_kb":1963,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87012,"num_read_kb":70914,"num_write":35136,"num_write_kb":17737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43882,"num_omap_keys":181,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3970,"ondisk_log_size":3970,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187841250,"num_objects":443,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":34833,"num_read_kb":1864137,"num_write":77494,"num_write_kb":2131308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253607936,"data_stored":253607936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3747,"ondisk_log_size":3747,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":150,"num_read_kb":221363,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":145,"num_read_kb":210051,"num_write":161,"num_write_kb":147579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575708,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349100,"kb_used_data":38884,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":94022740,"statfs":{"total":96636764160,"available":96279285760,"internally_reserved":0,"allocated":39817216,"data_stored":37809825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":317651402},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575709,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":359832,"kb_used_data":39888,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94012008,"statfs":{"total":96636764160,"available":96268296192,"internally_reserved":0,"allocated":40845312,"data_stored":38885655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":327612871},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575709,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206104,"kb_used_data":58384,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94165736,"statfs":{"total":96636764160,"available":96425713664,"internally_reserved":0,"allocated":59785216,"data_stored":57737844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575711,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148100,"kb_used_data":36412,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94223740,"statfs":{"total":96636764160,"available":96485109760,"internally_reserved":0,"allocated":37285888,"data_stored":35330686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575709,"num_pgs":130,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298456,"kb_used_data":42448,"kb_used_omap":1,"kb_used_meta":255998,"kb_avail":94073384,"statfs":{"total":96636764160,"available":96331145216,"internally_reserved":0,"allocated":43466752,"data_stored":41436434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":262142407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575713,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159304,"kb_used_data":56000,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94212536,"statfs":{"total":96636764160,"available":96473636864,"internally_reserved":0,"allocated":57344000,"data_stored":55396653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575711,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183784,"kb_used_data":36064,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94188056,"statfs":{"total":96636764160,"available":96448569344,"internally_reserved":0,"allocated":36929536,"data_stored":34946873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575709,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327488,"kb_used_data":39544,"kb_used_omap":1,"kb_used_meta":287934,"kb_avail":94044352,"statfs":{"total":96636764160,"available":96301416448,"internally_reserved":0,"allocated":40493056,"data_stored":38470704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294844869},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35614720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26390528,"data_stored":26390528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36356096,"data_stored":36356096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":33927168,"data_stored":33927168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22581248,"data_stored":22581248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":38694912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31379456,"data_stored":31379456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28663808,"data_stored":28663808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":6418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":6388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":4578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":4617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T05:47:32.315 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-17T05:47:32.640 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-17T05:47:32.640 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-17T05:47:32.641 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-17T05:47:32.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:32.726 INFO:tasks.ceph.osd.0:Stopped 2024-04-17T05:47:32.726 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-17T05:47:32.726 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:32.822 INFO:tasks.ceph.osd.1:Stopped 2024-04-17T05:47:32.822 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-17T05:47:32.822 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:32.911 INFO:tasks.ceph.osd.2:Stopped 2024-04-17T05:47:32.912 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-17T05:47:32.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:36.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-17T05:47:39.014 INFO:tasks.ceph.osd.3:Stopped 2024-04-17T05:47:39.015 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-17T05:47:39.015 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:42.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-17T05:47:42.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-17T05:47:45.117 INFO:tasks.ceph.osd.4:Stopped 2024-04-17T05:47:45.118 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-17T05:47:45.118 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:45.212 INFO:tasks.ceph.osd.5:Stopped 2024-04-17T05:47:45.212 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-17T05:47:45.213 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:47.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-17T05:47:47.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-17T05:47:51.314 INFO:tasks.ceph.osd.6:Stopped 2024-04-17T05:47:51.315 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-17T05:47:51.315 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:53.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-17T05:47:53.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-17T05:47:57.416 INFO:tasks.ceph.osd.7:Stopped 2024-04-17T05:47:57.417 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-17T05:47:57.417 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-17T05:47:57.418 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:57.448 INFO:tasks.ceph.mgr.y:Stopped 2024-04-17T05:47:57.448 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-17T05:47:57.448 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:57.476 INFO:tasks.ceph.mgr.x:Stopped 2024-04-17T05:47:57.477 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-17T05:47:57.477 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-17T05:47:57.477 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:57.518 INFO:tasks.ceph.mon.a:Stopped 2024-04-17T05:47:57.518 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-17T05:47:57.518 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:57.576 INFO:tasks.ceph.mon.c:Stopped 2024-04-17T05:47:57.576 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-17T05:47:57.576 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T05:47:57.599 INFO:tasks.ceph.mon.b:Stopped 2024-04-17T05:47:57.599 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-17T05:47:57.600 DEBUG:teuthology.orchestra.run.smithi088:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2024-04-17T05:47:57.634 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T05:47:57.634 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-17T05:47:57.965 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T05:47:57.965 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-17T05:47:58.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T05:47:58.157 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-17T05:47:58.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-17T05:47:58.315 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T05:47:58.315 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-17T05:47:58.479 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T05:47:58.479 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-17T05:47:58.727 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T05:47:58.727 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-17T05:47:58.897 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T05:47:58.897 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-17T05:47:59.047 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T05:47:59.047 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-17T05:47:59.195 INFO:tasks.ceph:Archiving mon data... 2024-04-17T05:47:59.196 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/data/mon.a.tgz 2024-04-17T05:47:59.196 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-04-17T05:47:59.203 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.pbfhyMAviq 2024-04-17T05:47:59.203 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pbfhyMAviq 2024-04-17T05:47:59.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.pbfhyMAviq 2024-04-17T05:47:59.454 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.pbfhyMAviq is 527KB 2024-04-17T05:47:59.515 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.pbfhyMAviq 2024-04-17T05:47:59.520 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/data/mon.c.tgz 2024-04-17T05:47:59.520 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-04-17T05:47:59.565 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.qttwBii7KI 2024-04-17T05:47:59.565 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.qttwBii7KI 2024-04-17T05:47:59.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.qttwBii7KI 2024-04-17T05:47:59.817 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.qttwBii7KI is 550KB 2024-04-17T05:47:59.879 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.qttwBii7KI 2024-04-17T05:47:59.885 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/data/mon.b.tgz 2024-04-17T05:47:59.886 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-04-17T05:47:59.892 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.Hc2Wn43uEc 2024-04-17T05:47:59.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Hc2Wn43uEc 2024-04-17T05:48:00.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.Hc2Wn43uEc 2024-04-17T05:48:00.150 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.Hc2Wn43uEc is 550KB 2024-04-17T05:48:00.211 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.Hc2Wn43uEc 2024-04-17T05:48:00.216 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-17T05:48:00.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T05:48:00.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-17T05:48:00.333 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T05:48:00.338 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-17T05:48:00.339 INFO:tasks.ceph:Archiving crash dumps... 2024-04-17T05:48:00.341 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/remote/smithi088/crash 2024-04-17T05:48:00.342 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T05:48:00.356 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/remote/smithi154/crash 2024-04-17T05:48:00.357 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-17T05:48:00.393 INFO:tasks.ceph:Compressing logs... 2024-04-17T05:48:00.394 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T05:48:00.402 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T05:48:00.423 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113129.log 2024-04-17T05:48:00.444 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62798.log 2024-04-17T05:48:00.444 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.113129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108309.log 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62798.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113129.log.gz 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32099.log 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.108309.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.108309.log.gz 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.32099.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.62798.log.gz 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.0.135014.log 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32099.log.gz 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48556.log 2024-04-17T05:48:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.135014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55710.log 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31642.log 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55710.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.48556.log.gz 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr: 20.3% -- replaced with /var/log/ceph/ceph-client.admin.55710.log.gz 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101206.log 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125906.log 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31642.log.gz 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.101206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135737.log 2024-04-17T05:48:00.446 INFO:teuthology.orchestra.run.smithi088.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.101206.log.gz 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.125906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35767.log 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44376.log 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.0.125906.log.gz 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135737.log: /var/log/ceph/ceph-client.0.35767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135737.log.gz 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr: 43.5% -- replaced with /var/log/ceph/ceph-client.0.35767.log.gz 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77602.log 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44376.log.gz 2024-04-17T05:48:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116387.log 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mgr.y.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.77602.log.gz 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56358.log 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.116387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126496.log 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56358.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.116387.log.gz 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.126496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133576.log 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.0.126496.log.gz 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-17T05:48:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47376.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136023.log 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.133576.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32242.log 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr: 80.6%/var/log/ceph/ceph-client.admin.136023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135945.log 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.0.133576.log.gz 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0%/var/log/ceph/ceph-client.0.32242.log: -- replaced with /var/log/ceph/ceph-client.admin.136023.log.gz 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89403.log 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103566.log 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135945.log.gz 2024-04-17T05:48:00.449 INFO:teuthology.orchestra.run.smithi088.stderr: 95.9% -- replaced with /var/log/ceph/ceph-client.0.32242.log.gz 2024-04-17T05:48:00.450 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.89403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.124136.log 2024-04-17T05:48:00.450 INFO:teuthology.orchestra.run.smithi088.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.89403.log.gz 2024-04-17T05:48:00.450 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.103566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31319.log 2024-04-17T05:48:00.450 INFO:teuthology.orchestra.run.smithi088.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.103566.log.gz 2024-04-17T05:48:00.450 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41966.log 2024-04-17T05:48:00.450 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.124136.log: /var/log/ceph/ceph-client.admin.31319.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.113104.log 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31319.log.gz 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73472.log 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.0.124136.log.gz 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41966.log.gz 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.113104.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55735.log 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113104.log.gz 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31851.log 2024-04-17T05:48:00.451 INFO:teuthology.orchestra.run.smithi088.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.73472.log.gz 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81732.log 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55735.log: 19.9% -- replaced with /var/log/ceph/ceph-client.admin.55735.log.gz 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31851.log.gz 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107107.log 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86453.log 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.81732.log.gz/var/log/ceph/ceph-client.admin.107107.log: 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32195.log 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.107107.log.gz 2024-04-17T05:48:00.452 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135997.log 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.86453.log: /var/log/ceph/ceph-client.admin.32195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98256.log 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32195.log.gz 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135997.log: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.86453.log.gz 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135997.log.gz 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50377.log 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.98256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66155.log 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr: 85.8%/var/log/ceph/ceph-client.admin.50377.log: -- replaced with /var/log/ceph/ceph-client.admin.98256.log.gz 2024-04-17T05:48:00.453 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136153.log 2024-04-17T05:48:00.454 INFO:teuthology.orchestra.run.smithi088.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.50377.log.gz 2024-04-17T05:48:00.454 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110079.log 2024-04-17T05:48:00.454 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97666.log 2024-04-17T05:48:00.454 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136153.log.gz 2024-04-17T05:48:00.454 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2%/var/log/ceph/ceph-client.admin.110079.log: -- replaced with /var/log/ceph/ceph-client.admin.66155.log.gz 2024-04-17T05:48:00.464 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-17T05:48:00.464 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25627.log 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25483.log 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.25627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25555.log 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25627.log.gz 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25579.log 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25483.log.gz 2024-04-17T05:48:00.465 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25555.log.gz 2024-04-17T05:48:00.466 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-17T05:48:00.466 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T05:48:00.466 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25579.log: /var/log/ceph/ceph-mgr.x.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25579.log.gz 2024-04-17T05:48:00.466 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25459.log 2024-04-17T05:48:00.466 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T05:48:00.466 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25603.log 2024-04-17T05:48:00.467 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25459.log.gz 2024-04-17T05:48:00.469 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-17T05:48:00.469 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25603.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-17T05:48:00.469 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25603.log.gz 2024-04-17T05:48:00.470 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25531.log 2024-04-17T05:48:00.470 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.4.log: 90.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T05:48:00.470 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25507.log 2024-04-17T05:48:00.470 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25531.log.gz 2024-04-17T05:48:00.472 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.25507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25507.log.gz 2024-04-17T05:48:00.472 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31448.log 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.110079.log.gz 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.97666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47966.log 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.97666.log.gz 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107697.log 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31448.log: /var/log/ceph/ceph-client.admin.47966.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81142.log 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31448.log.gz 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr: 82.6%/var/log/ceph/ceph-client.admin.107697.log: -- replaced with /var/log/ceph/ceph-client.admin.47966.log.gz 2024-04-17T05:48:00.492 INFO:teuthology.orchestra.run.smithi088.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.107697.log.gz 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85273.log 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135711.log 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81142.log: /var/log/ceph/ceph-client.admin.85273.log: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.56358.log.gz 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.81142.log.gz 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135815.log 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.85273.log.gz 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31866.log 2024-04-17T05:48:00.493 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135711.log: /var/log/ceph/ceph-client.admin.135815.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135815.log.gz 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.135711.log.gz 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90586.log 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31866.log.gz 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32073.log 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.90586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31438.log 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr: 85.2%/var/log/ceph/ceph-client.admin.32073.log: -- replaced with /var/log/ceph/ceph-client.admin.90586.log.gz 2024-04-17T05:48:00.494 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39606.log 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32073.log.gz 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31438.log.gz 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129446.log 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82912.log 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.39606.log.gz 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.129446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118169.log 2024-04-17T05:48:00.495 INFO:teuthology.orchestra.run.smithi088.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.0.129446.log.gz 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127676.log 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.82912.log: /var/log/ceph/ceph-client.admin.118169.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.82912.log.gz 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.118169.log.gz 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62154.log 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.127676.log: /var/log/ceph/ceph-client.admin.62154.log: 84.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84683.log 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.0.127676.log.gz 2024-04-17T05:48:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.a.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62154.log.gz 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.0.135014.log.gz 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36935.log 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.84683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44426.log 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.84683.log.gz 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.36935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58290.log 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr: 43.5% -- replaced with /var/log/ceph/ceph-client.0.36935.log.gz 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72292.log 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44426.log: /var/log/ceph/ceph-client.admin.58290.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.44426.log.gz 2024-04-17T05:48:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32832.log 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65563.log 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72292.log: /var/log/ceph/ceph-client.0.32832.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.72292.log.gz 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr: 43.5% -- replaced with /var/log/ceph/ceph-client.0.32832.log.gz 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52787.log 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94126.log 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65563.log: /var/log/ceph/ceph-client.admin.52787.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.65563.log.gz 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.52787.log.gz 2024-04-17T05:48:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40786.log 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51557.log 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.94126.log: /var/log/ceph/ceph-client.admin.40786.log: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.94126.log.gz 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.40786.log.gz 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100026.log 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96486.log 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51557.log: /var/log/ceph/ceph-client.admin.100026.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.51557.log.gz 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr: 85.2% 93.2% -- replaced with /var/log/ceph/ceph-client.admin.58290.log.gz 2024-04-17T05:48:00.499 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100026.log.gz 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102976.log 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40196.log 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.96486.log: 82.2%/var/log/ceph/ceph-client.admin.102976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110669.log 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96486.log.gz 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135633.log 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.110669.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.40196.log.gz 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82322.log 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.110669.log.gz 2024-04-17T05:48:00.500 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.102976.log.gz 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135633.log.gz 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50327.log 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.82322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134418.log 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92946.log 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50327.log.gz 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.134418.log: 81.4% -- replaced with /var/log/ceph/ceph-client.0.134418.log.gz 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.82322.log.gz 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94716.log 2024-04-17T05:48:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.92946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53377.log 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.92946.log.gz 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.94716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78192.log 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.94716.log.gz 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112489.log 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136049.log 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78192.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.53377.log.gz/var/log/ceph/ceph-client.admin.112489.log: 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.78192.log.gz 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.112489.log.gz 2024-04-17T05:48:00.502 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72882.log 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113719.log 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136049.log.gz 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.72882.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.121251.log 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.113719.log: 86.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120662.log 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72882.log.gz 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.121251.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113719.log.gz 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.121251.log.gz 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.120662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64974.log 2024-04-17T05:48:00.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49146.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.120662.log.gz 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136101.log 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr: 81.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85863.log 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz/var/log/ceph/ceph-client.admin.64974.log: 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.64974.log.gz 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136101.log.gz 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111309.log 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.85863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77012.log 2024-04-17T05:48:00.504 INFO:teuthology.orchestra.run.smithi088.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.85863.log.gz/var/log/ceph/ceph-client.admin.111309.log: 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91766.log 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.111309.log.gz 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104747.log 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77012.log: /var/log/ceph/ceph-client.admin.91766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131216.log 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr: 82.0% 82.5% -- replaced with /var/log/ceph/ceph-client.admin.91766.log.gz 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77012.log.gz 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.104747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43196.log 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.104747.log.gz 2024-04-17T05:48:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.131216.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.0.131216.log.gz/var/log/ceph/ceph-client.admin.43196.log: gzip 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31865.log 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.43196.log.gz 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98846.log 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.31865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111259.log 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31865.log.gz 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.98846.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.98846.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32219.log 2024-04-17T05:48:00.506 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.111259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111259.log.gz 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93536.log 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.32219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57646.log 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32219.log.gz 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.93536.log: /var/log/ceph/ceph-client.admin.57646.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.93536.log.gz 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58934.log 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54653.log: 58.8% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-04-17T05:48:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108899.log 2024-04-17T05:48:00.512 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58934.log: 93.4% 93.1% -- replaced with /var/log/ceph/ceph-client.admin.58934.log.gz 2024-04-17T05:48:00.512 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57646.log.gz 2024-04-17T05:48:00.512 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67927.log 2024-04-17T05:48:00.512 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.108899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95896.log 2024-04-17T05:48:00.512 INFO:teuthology.orchestra.run.smithi088.stderr: 85.2%/var/log/ceph/ceph-client.admin.67927.log: -- replaced with /var/log/ceph/ceph-client.admin.108899.log.gz 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr: 24.8% -- replaced with /var/log/ceph/ceph-client.admin.67927.log.gz 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60866.log 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.95896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31850.log 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.95896.log.gz 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128856.log 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31850.log.gz 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.admin.log 2024-04-17T05:48:00.513 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.128856.log: 84.1% -- replaced with /var/log/ceph/ceph-client.0.128856.log.gz 2024-04-17T05:48:00.514 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25596.log 2024-04-17T05:48:00.514 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ops-log-ceph-client.admin.log: 94.2% -- replaced with /var/log/ceph/ops-log-ceph-client.admin.log.gz 2024-04-17T05:48:00.514 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-17T05:48:00.514 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.25596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25596.log.gz 2024-04-17T05:48:00.514 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71702.log 2024-04-17T05:48:00.514 INFO:teuthology.orchestra.run.smithi154.stderr: 95.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-17T05:48:00.521 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31863.log 2024-04-17T05:48:00.522 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71702.log: 81.8% 95.3% -- replaced with /var/log/ceph/ceph-client.admin.60866.log.gz 2024-04-17T05:48:00.522 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71702.log.gz 2024-04-17T05:48:00.523 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132986.log 2024-04-17T05:48:00.523 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31863.log.gz 2024-04-17T05:48:00.523 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95306.log 2024-04-17T05:48:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.132986.log: 84.5% -- replaced with /var/log/ceph/ceph-client.0.132986.log.gz 2024-04-17T05:48:00.524 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92356.log 2024-04-17T05:48:00.525 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.95306.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.95306.log.gz 2024-04-17T05:48:00.525 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136255.log 2024-04-17T05:48:00.526 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.92356.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.92356.log.gz 2024-04-17T05:48:00.527 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38896.log 2024-04-17T05:48:00.528 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136255.log.gz 2024-04-17T05:48:00.528 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135919.log 2024-04-17T05:48:00.529 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38896.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.38896.log.gz 2024-04-17T05:48:00.529 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113079.log 2024-04-17T05:48:00.530 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135919.log.gz 2024-04-17T05:48:00.531 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60222.log 2024-04-17T05:48:00.531 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.113079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113079.log.gz 2024-04-17T05:48:00.531 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135971.log 2024-04-17T05:48:00.533 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60222.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60222.log.gz 2024-04-17T05:48:00.534 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31636.log 2024-04-17T05:48:00.534 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135971.log.gz 2024-04-17T05:48:00.535 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55104.log 2024-04-17T05:48:00.535 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31636.log.gz 2024-04-17T05:48:00.536 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101796.log 2024-04-17T05:48:00.537 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55104.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.55104.log.gz 2024-04-17T05:48:00.537 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111899.log 2024-04-17T05:48:00.538 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.101796.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.101796.log.gz 2024-04-17T05:48:00.538 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100616.log 2024-04-17T05:48:00.539 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.111899.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.111899.log.gz 2024-04-17T05:48:00.539 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52737.log 2024-04-17T05:48:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.100616.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.100616.log.gz 2024-04-17T05:48:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114309.log 2024-04-17T05:48:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52737.log.gz 2024-04-17T05:48:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127086.log 2024-04-17T05:48:00.542 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.114309.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.114309.log.gz 2024-04-17T05:48:00.543 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84093.log 2024-04-17T05:48:00.544 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.127086.log: 84.1% -- replaced with /var/log/ceph/ceph-client.0.127086.log.gz 2024-04-17T05:48:00.544 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91176.log 2024-04-17T05:48:00.545 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.84093.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.84093.log.gz 2024-04-17T05:48:00.545 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67336.log 2024-04-17T05:48:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.91176.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.91176.log.gz 2024-04-17T05:48:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70522.log 2024-04-17T05:48:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67336.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.67336.log.gz 2024-04-17T05:48:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57002.log 2024-04-17T05:48:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70522.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.70522.log.gz 2024-04-17T05:48:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135893.log 2024-04-17T05:48:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57002.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.57002.log.gz 2024-04-17T05:48:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41991.log 2024-04-17T05:48:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135893.log.gz 2024-04-17T05:48:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135607.log 2024-04-17T05:48:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41991.log.gz 2024-04-17T05:48:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25441.log 2024-04-17T05:48:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135607.log.gz 2024-04-17T05:48:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-17T05:48:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.25441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25441.log.gz 2024-04-17T05:48:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97076.log 2024-04-17T05:48:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32047.log 2024-04-17T05:48:00.571 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.97076.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.97076.log.gz 2024-04-17T05:48:00.571 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2024-04-17T05:48:00.572 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.32047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32047.log.gz 2024-04-17T05:48:00.573 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-17T05:48:00.573 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31441.log 2024-04-17T05:48:00.574 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2024-04-17T05:48:00.574 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33416.log 2024-04-17T05:48:00.575 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31441.log.gz 2024-04-17T05:48:00.575 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132396.log 2024-04-17T05:48:00.576 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.33416.log: 80.8% -- replaced with /var/log/ceph/ceph-client.0.33416.log.gz 2024-04-17T05:48:00.576 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34593.log 2024-04-17T05:48:00.577 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.132396.log: 84.3% -- replaced with /var/log/ceph/ceph-client.0.132396.log.gz 2024-04-17T05:48:00.578 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79372.log 2024-04-17T05:48:00.578 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.34593.log: 81.4% -- replaced with /var/log/ceph/ceph-client.0.34593.log.gz 2024-04-17T05:48:00.579 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83502.log 2024-04-17T05:48:00.579 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79372.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.79372.log.gz 2024-04-17T05:48:00.580 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115206.log 2024-04-17T05:48:00.581 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.83502.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.83502.log.gz 2024-04-17T05:48:00.581 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31444.log 2024-04-17T05:48:00.582 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.115206.log: 84.9% -- replaced with /var/log/ceph/ceph-client.0.115206.log.gz 2024-04-17T05:48:00.582 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32147.log 2024-04-17T05:48:00.585 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31444.log.gz 2024-04-17T05:48:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64092.log 2024-04-17T05:48:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.32147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32147.log.gz 2024-04-17T05:48:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36351.log 2024-04-17T05:48:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.64092.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.64092.log.gz 2024-04-17T05:48:00.586 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31659.log 2024-04-17T05:48:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.36351.log: 43.5% -- replaced with /var/log/ceph/ceph-client.0.36351.log.gz 2024-04-17T05:48:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68569.log 2024-04-17T05:48:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31659.log.gz 2024-04-17T05:48:00.588 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136205.log 2024-04-17T05:48:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.68569.log: 84.7% -- replaced with /var/log/ceph/ceph-client.0.68569.log.gz 2024-04-17T05:48:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2024-04-17T05:48:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136205.log.gz 2024-04-17T05:48:00.590 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31435.log 2024-04-17T05:48:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55196.log: 57.3% -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2024-04-17T05:48:00.597 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31650.log 2024-04-17T05:48:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31435.log.gz 2024-04-17T05:48:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118758.log 2024-04-17T05:48:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31650.log.gz 2024-04-17T05:48:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105337.log 2024-04-17T05:48:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.118758.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.118758.log.gz 2024-04-17T05:48:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102386.log 2024-04-17T05:48:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.105337.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.105337.log.gz 2024-04-17T05:48:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-17T05:48:00.599 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.102386.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.102386.log.gz 2024-04-17T05:48:00.600 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41376.log 2024-04-17T05:48:00.609 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55769.log 2024-04-17T05:48:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41376.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.41376.log.gz 2024-04-17T05:48:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53967.log 2024-04-17T05:48:00.612 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55769.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.55769.log.gz 2024-04-17T05:48:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78782.log 2024-04-17T05:48:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53967.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.53967.log.gz 2024-04-17T05:48:00.637 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74652.log 2024-04-17T05:48:00.637 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78782.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.78782.log.gz 2024-04-17T05:48:00.638 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31295.log 2024-04-17T05:48:00.639 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74652.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.74652.log.gz 2024-04-17T05:48:00.648 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31199.log 2024-04-17T05:48:00.649 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31295.log.gz 2024-04-17T05:48:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88223.log 2024-04-17T05:48:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31199.log.gz 2024-04-17T05:48:00.665 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54937.log 2024-04-17T05:48:00.665 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.88223.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.88223.log.gz 2024-04-17T05:48:00.673 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31648.log 2024-04-17T05:48:00.673 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54937.log: 50.2% -- replaced with /var/log/ceph/ceph-client.admin.54937.log.gz 2024-04-17T05:48:00.681 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69235.log 2024-04-17T05:48:00.681 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31648.log.gz 2024-04-17T05:48:00.688 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136127.log 2024-04-17T05:48:00.692 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.69235.log: 92.1% -- replaced with /var/log/ceph/ceph-client.0.69235.log.gz 2024-04-17T05:48:00.697 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-04-17T05:48:00.697 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136127.log.gz 2024-04-17T05:48:00.705 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31867.log 2024-04-17T05:48:00.713 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69202.log 2024-04-17T05:48:00.713 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31867.log.gz 2024-04-17T05:48:00.721 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67979.log 2024-04-17T05:48:00.721 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69202.log: 50.3% -- replaced with /var/log/ceph/ceph-client.admin.69202.log.gz 2024-04-17T05:48:00.729 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46196.log 2024-04-17T05:48:00.729 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.67979.log: 84.7% -- replaced with /var/log/ceph/ceph-client.0.67979.log.gz 2024-04-17T05:48:00.737 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119347.log 2024-04-17T05:48:00.737 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46196.log: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.46196.log.gz 2024-04-17T05:48:00.745 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52147.log 2024-04-17T05:48:00.745 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.119347.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.119347.log.gz 2024-04-17T05:48:00.753 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25714.log 2024-04-17T05:48:00.753 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52147.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.52147.log.gz 2024-04-17T05:48:00.761 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-17T05:48:00.761 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.25714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25714.log.gz 2024-04-17T05:48:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66745.log 2024-04-17T05:48:00.780 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.audit.log: 94.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-17T05:48:00.792 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105927.log 2024-04-17T05:48:00.793 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66745.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.66745.log.gz 2024-04-17T05:48:00.805 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71112.log 2024-04-17T05:48:00.806 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.105927.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.105927.log.gz 2024-04-17T05:48:00.821 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.124726.log 2024-04-17T05:48:00.821 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71112.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.71112.log.gz 2024-04-17T05:48:00.833 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-04-17T05:48:00.834 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.124726.log: 84.7% -- replaced with /var/log/ceph/ceph-client.0.124726.log.gz 2024-04-17T05:48:00.849 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75832.log 2024-04-17T05:48:00.849 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42016.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-04-17T05:48:00.865 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31436.log 2024-04-17T05:48:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75832.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.75832.log.gz 2024-04-17T05:48:00.875 INFO:teuthology.orchestra.run.smithi088.stderr: 94.4% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-04-17T05:48:00.893 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44401.log 2024-04-17T05:48:00.893 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31436.log.gz 2024-04-17T05:48:00.904 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50352.log 2024-04-17T05:48:00.905 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44401.log.gz 2024-04-17T05:48:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135685.log 2024-04-17T05:48:00.921 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50352.log.gz 2024-04-17T05:48:00.932 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31651.log 2024-04-17T05:48:00.933 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135685.log.gz 2024-04-17T05:48:00.945 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31658.log 2024-04-17T05:48:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31651.log.gz 2024-04-17T05:48:00.957 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31451.log 2024-04-17T05:48:00.957 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31658.log.gz 2024-04-17T05:48:00.973 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89993.log 2024-04-17T05:48:00.973 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31451.log.gz 2024-04-17T05:48:00.984 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123535.log 2024-04-17T05:48:00.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.89993.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.89993.log.gz 2024-04-17T05:48:00.997 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/ceph.client.0.log 2024-04-17T05:48:00.997 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.123535.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.123535.log.gz 2024-04-17T05:48:01.008 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115798.log 2024-04-17T05:48:01.009 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/valgrind/ceph.client.0.log: 62.0% -- replaced with /var/log/ceph/valgrind/ceph.client.0.log.gz 2024-04-17T05:48:01.021 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125316.log 2024-04-17T05:48:01.023 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.115798.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115798.log.gz 2024-04-17T05:48:01.033 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76422.log 2024-04-17T05:48:01.033 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.125316.log: 80.3% -- replaced with /var/log/ceph/ceph-client.0.125316.log.gz 2024-04-17T05:48:01.045 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69825.log 2024-04-17T05:48:01.045 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76422.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.76422.log.gz 2024-04-17T05:48:01.065 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31270.log 2024-04-17T05:48:01.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.69825.log: 80.6% -- replaced with /var/log/ceph/ceph-client.0.69825.log.gz 2024-04-17T05:48:01.076 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2024-04-17T05:48:01.077 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31270.log.gz 2024-04-17T05:48:01.101 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130626.log 2024-04-17T05:48:01.102 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49736.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2024-04-17T05:48:01.117 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88813.log 2024-04-17T05:48:01.118 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.130626.log: 84.2% -- replaced with /var/log/ceph/ceph-client.0.130626.log.gz 2024-04-17T05:48:01.120 INFO:teuthology.orchestra.run.smithi088.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-17T05:48:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135789.log 2024-04-17T05:48:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.88813.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.88813.log.gz 2024-04-17T05:48:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135867.log 2024-04-17T05:48:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135789.log.gz 2024-04-17T05:48:01.124 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50967.log 2024-04-17T05:48:01.124 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135867.log.gz 2024-04-17T05:48:01.124 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59578.log 2024-04-17T05:48:01.125 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50967.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.50967.log.gz 2024-04-17T05:48:01.125 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54866.log 2024-04-17T05:48:01.127 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59578.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.59578.log.gz 2024-04-17T05:48:01.128 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135659.log 2024-04-17T05:48:01.128 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54866.log: 60.2% -- replaced with /var/log/ceph/ceph-client.admin.54866.log.gz 2024-04-17T05:48:01.129 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55219.log 2024-04-17T05:48:01.129 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135659.log.gz 2024-04-17T05:48:01.130 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128266.log 2024-04-17T05:48:01.131 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55219.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55219.log.gz 2024-04-17T05:48:01.131 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117580.log 2024-04-17T05:48:01.131 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.128266.log: 80.8% -- replaced with /var/log/ceph/ceph-client.0.128266.log.gz 2024-04-17T05:48:01.132 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45016.log 2024-04-17T05:48:01.133 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.117580.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.117580.log.gz 2024-04-17T05:48:01.134 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31631.log 2024-04-17T05:48:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45016.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.45016.log.gz 2024-04-17T05:48:01.135 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122946.log 2024-04-17T05:48:01.136 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31631.log.gz 2024-04-17T05:48:01.136 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38294.log 2024-04-17T05:48:01.138 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.122946.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.122946.log.gz 2024-04-17T05:48:01.138 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135763.log 2024-04-17T05:48:01.139 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38294.log: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.38294.log.gz 2024-04-17T05:48:01.140 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136075.log 2024-04-17T05:48:01.140 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135763.log.gz 2024-04-17T05:48:01.141 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31843.log 2024-04-17T05:48:01.142 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136075.log.gz 2024-04-17T05:48:01.142 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74062.log 2024-04-17T05:48:01.143 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31843.log.gz 2024-04-17T05:48:01.143 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87043.log 2024-04-17T05:48:01.144 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74062.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.74062.log.gz 2024-04-17T05:48:01.144 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109489.log 2024-04-17T05:48:01.145 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.87043.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.87043.log.gz 2024-04-17T05:48:01.145 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130036.log 2024-04-17T05:48:01.146 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.109489.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.109489.log.gz 2024-04-17T05:48:01.146 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35183.log 2024-04-17T05:48:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.130036.log: 81.2% -- replaced with /var/log/ceph/ceph-client.0.130036.log.gz 2024-04-17T05:48:01.148 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34006.log 2024-04-17T05:48:01.148 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.35183.log: 43.9% -- replaced with /var/log/ceph/ceph-client.0.35183.log.gz 2024-04-17T05:48:01.148 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87633.log 2024-04-17T05:48:01.149 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.34006.log: 43.9% -- replaced with /var/log/ceph/ceph-client.0.34006.log.gz 2024-04-17T05:48:01.149 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120073.log 2024-04-17T05:48:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.87633.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.87633.log.gz 2024-04-17T05:48:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61510.log 2024-04-17T05:48:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.120073.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.120073.log.gz 2024-04-17T05:48:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136179.log 2024-04-17T05:48:01.154 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61510.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.61510.log.gz 2024-04-17T05:48:01.155 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104157.log 2024-04-17T05:48:01.155 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136179.log.gz 2024-04-17T05:48:01.156 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31397.log 2024-04-17T05:48:01.157 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.104157.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.104157.log.gz 2024-04-17T05:48:01.157 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75242.log 2024-04-17T05:48:01.158 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31397.log.gz 2024-04-17T05:48:01.158 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99436.log 2024-04-17T05:48:01.159 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75242.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.75242.log.gz 2024-04-17T05:48:01.159 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108287.log 2024-04-17T05:48:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.99436.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.99436.log.gz 2024-04-17T05:48:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32171.log 2024-04-17T05:48:01.161 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.108287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108287.log.gz 2024-04-17T05:48:01.161 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80552.log 2024-04-17T05:48:01.162 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.32171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32171.log.gz 2024-04-17T05:48:01.162 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31371.log 2024-04-17T05:48:01.163 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80552.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.80552.log.gz 2024-04-17T05:48:01.163 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55144.log 2024-04-17T05:48:01.164 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31371.log.gz 2024-04-17T05:48:01.164 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31842.log 2024-04-17T05:48:01.165 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55144.log.gz 2024-04-17T05:48:01.165 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42606.log 2024-04-17T05:48:01.166 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31842.log.gz 2024-04-17T05:48:01.166 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31345.log 2024-04-17T05:48:01.167 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42606.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.42606.log.gz 2024-04-17T05:48:01.167 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55167.log 2024-04-17T05:48:01.168 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31345.log.gz 2024-04-17T05:48:01.168 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111284.log 2024-04-17T05:48:01.185 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55167.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131806.log 2024-04-17T05:48:01.186 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.111284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111284.log.gz 2024-04-17T05:48:01.186 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55167.log.gz 2024-04-17T05:48:01.187 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135841.log 2024-04-17T05:48:01.187 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.131806.log: 80.7% -- replaced with /var/log/ceph/ceph-client.0.131806.log.gz 2024-04-17T05:48:01.188 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46786.log 2024-04-17T05:48:01.188 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.135841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135841.log.gz 2024-04-17T05:48:01.188 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31447.log 2024-04-17T05:48:01.189 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46786.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.46786.log.gz 2024-04-17T05:48:01.189 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63442.log 2024-04-17T05:48:01.190 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.31447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31447.log.gz 2024-04-17T05:48:01.190 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-04-17T05:48:01.194 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63442.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.63442.log.gz 2024-04-17T05:48:01.195 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136231.log 2024-04-17T05:48:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43786.log: 80.7% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-04-17T05:48:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106517.log 2024-04-17T05:48:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.136231.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.136231.log.gz 2024-04-17T05:48:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121840.log 2024-04-17T05:48:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.106517.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.106517.log.gz 2024-04-17T05:48:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79962.log 2024-04-17T05:48:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.121840.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.121840.log.gz 2024-04-17T05:48:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20619.log 2024-04-17T05:48:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79962.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.79962.log.gz 2024-04-17T05:48:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45606.log 2024-04-17T05:48:01.201 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.tmp-client.admin.20619.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20619.log.gz 2024-04-17T05:48:01.202 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32123.log 2024-04-17T05:48:01.202 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45606.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.45606.log.gz 2024-04-17T05:48:01.203 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.32123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32123.log.gz 2024-04-17T05:48:01.242 INFO:teuthology.orchestra.run.smithi154.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-17T05:48:01.959 INFO:teuthology.orchestra.run.smithi088.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-17T05:48:03.786 INFO:teuthology.orchestra.run.smithi088.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-17T05:48:04.282 INFO:teuthology.orchestra.run.smithi088.stderr: 92.3% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-04-17T05:48:08.194 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-17T05:48:09.020 INFO:teuthology.orchestra.run.smithi088.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-17T05:48:09.757 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-17T05:48:09.928 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-17T05:48:10.897 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-17T05:48:12.837 INFO:teuthology.orchestra.run.smithi088.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-17T05:48:13.723 INFO:teuthology.orchestra.run.smithi154.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-17T05:48:13.724 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-04-17T05:48:13.724 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m13.282s 2024-04-17T05:48:13.724 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m41.805s 2024-04-17T05:48:13.724 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m1.550s 2024-04-17T05:48:19.636 INFO:teuthology.orchestra.run.smithi088.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-17T05:48:19.638 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-17T05:48:19.638 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m19.233s 2024-04-17T05:48:19.638 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m55.342s 2024-04-17T05:48:19.638 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m2.092s 2024-04-17T05:48:19.638 INFO:tasks.ceph:Archiving logs... 2024-04-17T05:48:19.639 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/remote/smithi088/log 2024-04-17T05:48:19.640 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T05:48:26.157 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/remote/smithi154/log 2024-04-17T05:48:26.158 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T05:48:31.699 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T05:48:31.714 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T05:48:31.715 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T05:48:31.718 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T05:48:31.784 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T05:48:31.795 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T05:48:31.796 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T05:48:31.798 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T05:48:31.812 INFO:teuthology.orchestra.run.smithi088.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T05:48:31.813 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================== 2024-04-17T05:48:31.813 INFO:teuthology.orchestra.run.smithi088.stdout:*hv01.front.sepi 67.205.162.81 3 u 17 64 377 0.100 -2.000 0.279 2024-04-17T05:48:31.813 INFO:teuthology.orchestra.run.smithi088.stdout:+hv02.front.sepi 63.231.80.2 3 u 66 64 377 0.095 +0.254 0.168 2024-04-17T05:48:31.813 INFO:teuthology.orchestra.run.smithi088.stdout:+hv03.front.sepi 74.6.168.72 3 u 7 64 377 0.119 -1.135 0.151 2024-04-17T05:48:31.813 INFO:teuthology.orchestra.run.smithi088.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-17T05:48:31.815 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2024-04-17T05:48:31.815 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2024-04-17T05:48:31.816 INFO:teuthology.orchestra.run.smithi154.stdout:*hv01.front.sepi 67.205.162.81 3 u 131 128 377 0.161 -4.596 1.041 2024-04-17T05:48:31.816 INFO:teuthology.orchestra.run.smithi154.stdout:+hv02.front.sepi 63.231.80.2 3 u 69 128 377 0.187 -2.830 1.142 2024-04-17T05:48:31.816 INFO:teuthology.orchestra.run.smithi154.stdout:+hv03.front.sepi 74.6.168.72 3 u 124 128 377 0.181 -2.562 1.370 2024-04-17T05:48:31.816 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-04-17T05:48:31.817 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T05:48:31.828 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T05:48:31.828 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T05:48:31.856 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T05:48:31.890 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T05:48:31.923 INFO:teuthology.task.internal:Duration was 4954.385903 seconds 2024-04-17T05:48:31.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T05:48:31.956 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T05:48:31.956 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T05:48:31.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T05:48:31.996 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T05:48:31.996 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-04-17T05:48:31.996 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T05:48:32.049 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-04-17T05:48:32.050 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T05:48:32.060 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T05:48:32.060 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T05:48:32.098 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T05:48:32.125 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T05:48:32.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T05:48:32.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T05:48:32.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T05:48:32.237 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T05:48:32.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T05:48:32.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T05:48:32.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T05:48:32.277 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T05:48:32.294 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T05:48:32.302 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-04-17T05:48:32.314 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-04-17T05:48:32.610 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T05:48:32.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:48:32.616 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T05:48:32.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:48:32.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T05:48:32.670 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T05:48:32.671 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/remote/smithi088 2024-04-17T05:48:32.671 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T05:48:32.696 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000/remote/smithi154 2024-04-17T05:48:32.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T05:48:32.720 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T05:48:32.721 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T05:48:32.734 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T05:48:32.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T05:48:32.778 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T05:48:32.778 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T05:48:32.801 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T05:48:32.802 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T05:48:32.804 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T05:48:32.808 INFO:teuthology.orchestra.run.smithi088.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 05:48 /home/ubuntu/cephtest 2024-04-17T05:48:32.813 INFO:teuthology.orchestra.run.smithi154.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 05:48 /home/ubuntu/cephtest 2024-04-17T05:48:32.815 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T05:48:32.832 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T05:48:32.899 INFO:teuthology.nuke:Checking targets against current locks 2024-04-17T05:48:32.920 DEBUG:teuthology.nuke:shortname: smithi088 2024-04-17T05:48:32.921 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T05:48:32.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 04:16:19.046875', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC+yz+tDPlkQjjbMRn3xAZhK5N+VOMBKaH64bHOAUTAGGaLPAgA17cnBwc6d1kNP1Ykyi6t9SWDZvp5vwZJxKLc='} 2024-04-17T05:48:32.948 DEBUG:teuthology.nuke:shortname: smithi154 2024-04-17T05:48:32.948 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T05:48:32.972 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-16_21:40:02-rgw-squid-distro-default-smithi/7659000', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 04:16:19.045559', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLipxCHy1pCsU745cAdHzITMYpbdtcaqlJswFTMKVxe3yg9Jd2rj6c+vI3v5PDaXqLk74gpu1memTOkr/Ex6wfY='} 2024-04-17T05:48:32.998 INFO:teuthology.orchestra.console.smithi088:Power off 2024-04-17T05:48:32.998 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T05:48:33.028 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-04-17T05:48:33.028 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:33.046 INFO:teuthology.orchestra.console.smithi154:Power off 2024-04-17T05:48:33.047 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T05:48:33.058 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-17T05:48:33.070 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-04-17T05:48:33.070 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:33.093 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-17T05:48:37.060 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:37.094 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:37.172 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-17T05:48:37.208 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-17T05:48:41.175 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:41.209 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:41.286 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-17T05:48:41.320 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-04-17T05:48:41.421 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-04-17T05:48:45.289 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T05:48:45.400 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-17T05:48:45.501 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-04-17T05:48:45.607 INFO:teuthology.run:Summary data: description: rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} duration: 4954.385902643204 failure_reason: 'Command failed (s3 tests against rgw) on smithi088 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m ''not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3''"' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ca2c46e2ffa94cc5b9f7c43ac858b932 status: fail success: false 2024-04-17T05:48:45.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T05:48:45.693 INFO:teuthology.run:FAIL